Implementing Lua Scripting and Visual Scripting for Game Engine (Part 2)

At last, I finished the second part of this post. This is the second part of “Implementing Lua Scripting and Visual Scripting for Game Engine”. Check part 1.

This part will cover the hardest part and will be hard to explain. If you don’t get the first idea when you read it (me too! Damn! I’m bad at writing), bear with me and try to read again.

Visual Scripting

Struct Editor + Visual Scripting

The Visual Script editor is created on top of Maya using Qt and Python, so there is no C++ code in here. Even we created this on top of Maya, the editor doesn’t really depend on Maya library. So in other words, we can make the editor stand-alone. As I mentioned before, we have 2 types of Visual Script editor, one is for a struct and another one is for an entire scene. Once again, the difference is the scope of the scripting.

For implementing visual script components, we used base Qt Classes for drawing. the Qt has several classes that can be used, such as QGraphicsScene, QGraphicsItem, QGraphicsPathItem, and QGraphicsProxyWidget. QGraphicsScene is used as a scene, or you can think this as a canvas. QGraphicsItem is a base class for a thing that is drawn in a canvas/scene. QGraphicsPathItem is a specialized class from Qt for drawing a line. And last, QGraphicsProxyWidget is a proxy class for drawing basic form widget, such as text field, toggle, into a canvas/scene.

All those classes, we used to build new classes for our visual scripting purpose. These are classes that we created and the functionality of each class.

• NodeScene, the class acts as a canvas to draw all nodes and lines. The class is inherited from QGraphicsScene.
• NodeItem. This is a node class for creating a node in a scene. This class contains some basic node information like name of a node, type of connection and also list of parameter/attribute. This class is inherited from QGraphicsItem.
• AttributeNodeItem. The node has its parameter, either input or output or both. This parameter represents AttributeNodeItem, which is drawn under NodeItem. This class is inherited from also from QGraphicsItem.
• NodeSocket. An instance of this class handle connection between two nodes or two attributes. This class is also inherited from QGraphicsItem.
• NodeLine. The name is pretty obvious, this will connect two NodeSockets. This class is inherited from QGraphicsPathItem.
• GraphicsEditLine and GraphicsCheckbox. Both classes are used to draw the text input and checkbox — which is usually part of Qt form — in NodeScene or QGraphicsScene in general.

We created functions to parse to and from JSON file for all the custom classes. This makes loading and saving are really easy to implement. Other than that, we use the functions to also make Node can be defined or made by a single JSON file. For instance, take some arithmetic operation, such as float multiplication. I can define that as JSON format like below.

{
   "Name" : "a * b",
   "Conn" : "none",
   "Func" : "({1}*{2})",
   "Attrs" : [
      {
         "Name" : "res",
         "Type" : "float",
         "Conn" : "out"
      },
      {
         "Name" : "a",
         "Type" : "float",
         "Conn" : "in",
         "Constant" : true
      },
      {
         "Name" : "b",
         "Type" : "float",
         "Conn" : "in",
         "Constant" : true
      }
   ]
}

In the editor, it’ll look like this.

It is a very simple node. By seeing it’s visual, it can be easy to see the relation between JSON attributes and its visual. While “Func” in a JSON file is used to represent the node when converting to Lua script that will be explained later. If you notice there are some expressions in “Func” ( “{1}” and “{2}” ), the expression means the value for attribute’s value by index.

Also with the JSON functions, we created for the editor, we did create a bunch of node templates, from simple math to object movement and Physics support. Take an example of Physics raycast node in JSON format and its visual.

{
	"Name" : "RaycastLine",
	"Conn" : "both",
	"Func" : "Physics.RaycastLine(l_getGameContext(), {0}, {1}, {2}, {3})",
	"Attrs" : [
		{
			"Name" : "from",
			"Type" : "Object",
			"Conn" : "in"
		},
		{
			"Name" : "origin",
			"Type" : "vector",
			"Conn" : "in"
		},
		{
			"Name" : "direction",
			"Type" : "vector",
			"Conn" : "in"
		},
		{
			"Name" : "length",
			"Type" : "float",
			"Conn" : "in",
			"Constant" : true
		}
		
	]
}

Lua Script Integration

In this section, I’ll explain how to generate Lua script from the visual script editor, how the Lua script running in the C++ engine, and some tricks I used to handle access object between C++ game engine and Lua script.

First, let me tell you why don’t we use visual script JSON file and parse it into C++ game engine and let the engine handle the visual script without converting to Lua extension.  The first reason is because the engine we used is already integrated with Lua extension, so it’s not good if we don’t leverage its existence. The second reason is by building the script engine for the visual script is not doable for one-third of semester plus we have another class to worry (Damn, student excuse!). It also would give us headache and become sleepless zombie if we forced to do that. The last reason is more about file size and memory when load the JSON file itself. Obviously Lua script generated is smaller than JSON representation of whole visual script. Also I believe we can make Lua script as binary file and load that into the game engine with help from Lua-C extension. All those reasons are my excuses. On other hand, by implementing its own visual script engine itself makes it easy to us to implement more advanced features, like debugging the visual script.

To implement a generator for the visual script is relatively easy. We just need to traverse all node from the root node to all node by following the flow line (thick white line). There is an exception to our generator, it can’t handle cycle in the flow, so basically it should be a DAG (directed acyclic graph). Each time it stops on a node, it will grab the “Func” parameter and process that based on the parameter and put that into the generated Lua script. An example of the generated Lua script is below. It’s generated from the image of the visual script before.

function Update(struct)
	if (struct.isShooting) then
		if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 16)) then
		else
			Animation.SetAnimation(l_getGameContext(), struct.id, 16, true)
		end
	else
		if (Vector.LengthSquared(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))) > 0.01) then
			SceneNode.SetPosition(l_getGameContext(), struct.id, Vector.Add(SceneNode.GetPosition(l_getGameContext(), struct.id), Vector.MultScalar(Vector.Normalize(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))), (g_delta_time*struct.Speed))))
			SceneNode.TurnInDir(l_getGameContext(), struct.id, Vector.Normalize(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))))
			if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 18)) then
			else
				Animation.SetAnimation(l_getGameContext(), struct.id, 18, true)
			end
		else
			if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 17)) then
			else
				Animation.SetAnimation(l_getGameContext(), struct.id, 17, true)
			end
		end
	end
end

Implementing Lua into a game engine — regardless that Lua script already embedded to the engine — is not that hard. What we need a Lua library or API that works with C++. Hopefully, there is a good (I think it’s an official one) API that already implemented (see more on https://www.lua.org/pil/24.html). I won’t talk about how to implement the Lua script API and how that works in C or C++ code. But I will talk about some tricks I used to make it work with our visual script and our struct.

Unfortunately, the API doesn’t a reference to a parameter from C++ into Lua, it passes a value of a parameter. Also, it only handles primitive types, like string, number, int, and boolean so we can’t actually pass just a defined struct from C++ to Lua.  We need to create a table in Lua and put each member of a struct into the table. For our struct, besides generating the declaration of struct into a header file (see part 1), we also generate some helpful function to push the struct into Lua. This one is an example for our Soldier.

void Soldier::pushLuaStruct(lua_State* l, Soldier& s) 
{
	lua_newtable(l);

	PE_PUSH_STRING_TO_LUA(l, "StructName", Soldier::getName())
	PE_PUSH_STRING_TO_LUA(l, "Name", s.m_Name)
	PE_PUSH_NUMBER_TO_LUA(l, "Speed", s.m_Speed)
	PE_PUSH_NUMBER_TO_LUA(l, "Health", s.m_Health)
	PE_PUSH_STRING_TO_LUA(l, "Weapon", s.m_Weapon)
	PE_PUSH_NUMBER_TO_LUA(l, "Mana", s.m_Mana)
	PE_PUSH_PEUUID_TO_LUA(l, "Enemy", s.m_Enemy)
	PE_PUSH_NUMBER_TO_LUA(l, "Happiness", s.m_Happiness)
	PE_PUSH_BOOL_TO_LUA(l, "isShooting", s.m_isShooting)
	PE_PUSH_NUMBER_TO_LUA(l, "target_x", s.m_target_x)
	PE_PUSH_NUMBER_TO_LUA(l, "target_y", s.m_target_y)
	PE_PUSH_NUMBER_TO_LUA(l, "target_z", s.m_target_z)
}

PE_PUSH_<TYPE>_TO_LUA is a macro I created to simplify generating C++ file.

Then, we pass the struct into a Lua script. Then script start changing the struct. But like I mentioned, it only passes the value, it doesn’t matter how the script changing it, the engine still doesn’t know what is changed. We need the struct back after the script deals with it. To do that it’s needed to create a wrapper inside Lua script for a function and return the struct back after changing it.

function Update_Wrapper(struct)
	if(Update ~= nil) then
		Update(struct)
	end
	return struct
end

That’ how the struct pushed and pulled between C++ and Lua. Unfortunately, the struct itself is not representing the real object in the game world. It is something that we can use to modify the real object though. Another problem is how to access and modify the real object’s attributes. There is no easy way passing an instance of a class in C++ to Lua and access all attributes of the class freely (You already knew the way I pass the struct to C++). Technically, there are some ways to do it, but I’ll mention two basic and simple ways to do it. The first one is by passing the memory address of the object, but we need to make some helper functions in Lua that help to get or push attribute from an instance of C++ class. The other way is by passing a unique id from C++, and still, it needs some helper functions to access real object’s attributes. Basically, the engine has table that will point a unique id to real object. The unique id can be anything, it can be a string or integer or four integers or any struct. It has no significant advantages or disadvantages between two ways. Personally, I think using unique id is safer than passing the object pointer, since it’s not exposing the real memory address, and if the engine have some memory management complex functions, it’s still guaranteed that the id is never changed.

As I mentioned, the struct is attached to the real object in the game, so we can put the object id as an attribute of the struct itself (“struct.id”). After exposing the unique id through the struct, we need to make some functions to help access its attribute. This is an example of a function to get the position of an object.

SceneNode.GetPosition(l_getGameContext(), struct.id)

Function “l_getGameContext()” is a global function to get the “context” or instance of the game. Underneath the function, the implementation in C++ would be like this.

int l_getSceneNodePosition(lua_State* luaVM)
{
	// Get the parameter of a function
	PE::GameContext *pContext = (PE::GameContext*) (lua_touserdata(luaVM, -2)); // get first parameter *GameContext
	PEUUID uid = LuaGlue::readPEUUID(luaVM, -1); // get second parameter "struct.id"
	lua_pop(luaVM, 2);

	PE::Components::SceneNode* pSceneNode = getSceneNodeFromUID(pContext, uid); // searching the object on the table

	Vector3 result;
	if (pSceneNode) {
		result = pSceneNode->m_base.getPos(); // read attribute from the object
	}

	pushVector(luaVM, result); // return a result to Lua

	return 1;
}

I know it’s a very tedious process if you have a lot of attributes of the object that you really want to expose to Lua. But this is a learning process, I don’t mind to get messy for this.

I think that’s it for now. It’s too much to handle (both to read and to write). I’ll make the last part to discuss some future improvements and also the conclusion of the current system we developed.

Advertisement