Sunday, February 18, 2007

Wrapping C++ classes in Lua

Lua is a script language frequently used in game development. Typically the game engine is written in C/C++ and the game itself is defined in Lua. Meaning setting up behaviour, placement of objects like powerups, doors, enemies etc is done in Lua. That way the game can be quickly tweaked without having to go through a time consuming re-compile.

But I am not going to go in depth about the different usage of Lua, nor explain the language itself. An online book at the Lua web page does that very well already. I will here assume you already have some basic knowledge about Lua.

However finding any documentation on how to wrap a C++ class so that it can be used in Lua is difficult. One could of course use one of the ready made bridges. But here we are going to look at how to do it yourself.

The tricky part is deciding on how to do it. Because Lua is such a flexible language there is really a lot of ways you could achieve it.

The naive approach

First lets look at several options, what works and doesn't. My first approach was to light user data to store a pointer to my C++ class. I will use a Sprite class and related classes as examples here as that was the type of classes I was wrapping.

static int newSprite(lua_State *L)
{
    int n = lua_gettop(L);  // Number of arguments
    if (n != 4)
        return luaL_error(L, "Got %d arguments expected 4", n);
  
    double x = luaL_checknumber (L, 1);      
    double y = luaL_checknumber (L, 2);
    double dir = luaL_checknumber (L, 3);      
    double speed = luaL_checknumber (L, 4);

    Sprite *s = new Sprite(Point2(x, y), dir, speed);
    
    lua_pushlightuserdata(L, s);

  return 1;
}

The code snippet above shows a naive implementation of this approach. Unfortunately it doesn't work. The problem is that a light user data is just a simple pointer. Lua does not store any information with it. For instance a metatable which we could use to define the methods the class supports.

An approach with limited functionality

The next approach would be to use user data. Unlike light user data it can store a reference to a metatable.

static int newSprite(lua_State *L)
{
    int n = lua_gettop(L);  // Number of arguments
    if (n != 4)
        return luaL_error(L, "Got %d arguments expected 4", n);
  
    // Allocate memory for a pointer to to object
    Sprite **s = (Sprite **)lua_newuserdata(L, sizeof(Sprite *));  

    double x = luaL_checknumber (L, 1);      
    double y = luaL_checknumber (L, 2);
    double dir = luaL_checknumber (L, 3);      
    double speed = luaL_checknumber (L, 4);

    *s = new Sprite(Point2(x, y), dir, speed);
    
    lua_getglobal(L, "Sprite"); // Use global table 'Sprite' as metatable
    lua_setmetatable(L, -2);       
    
  return 1;
}

For us to be able to use sprite like this we need to register class first. Basically we need to create a table Sprite which contains all the methods that our user data should support.

// Show registration of class
static const luaL_Reg gSpriteFuncs[] = {
  // Creation
  {"new", newSprite},
  {"position", position},
  {"nextPosition", nextPosition},    
  {"setPosition", setPosition},  
  {"render", render},      
  {"update", update},          
  {"collision", collision},   
  {"move", move},    
  {"accelerate", accelerate},      
  {"rotate", rotate},  
  {NULL, NULL}
};

void registerSprite(lua_State *L)
{
  luaL_register(L, "Sprite", gSpriteFuncs);  
  lua_pushvalue(L,-1);
  lua_setfield(L, -2, "__index");    
}

This will allow us to create instances of Sprite and call methods on it in Lua like this:

-- Create an instance of sprite an call some methods
local sprite = Sprite.new(x, y, dir, speed)
sprite:setPosition(x,y)
sprite:render()

The final approach

In most cases this approach is sufficient but it has one major limitation. It does not support inheritance. You can change the methods of Sprite in Lua but that will change the behavior of all instances of Sprite. What you would want to do is to be able to change method on just the instance and then use that instance as a prototype for new Sprite instances, effectively creating a class inheritance system.

To do this we need to change the instance into being a table. How do we access our C++ object then? Simple, we just store the pointer to it as user data in one of the field of the table. You might think that this time light user data will be sufficient. However the problem is that only user data is informed of garbage collection, not tables or light user data. So if you want to delete the C++ object when corresponding lua table is garbage collected you need to use user data.

So we then arrive at our final solution. We will store a pointer to our C++ object as user data on the key __self in the table that represents our instance. __self is an arbitrary selected name. It could be anything. We will not register Sprite as as the metatable for our instance but instead register the first argument to the new function as it. This will allow us to support inheritance. Further the garbage collection function will be register on a separate table which will be used as metatable only for the user data. This is to allow it to be garbage collected.

static int newSprite(lua_State *L)
{
    int n = lua_gettop(L);  // Number of arguments
    if (n != 5)
        return luaL_error(L, "Got %d arguments expected 5 (class, x, y, dir, speed)", n);
    // First argument is now a table that represent the class to instantiate        
    luaL_checktype(L, 1, LUA_TTABLE);   
    
    lua_newtable(L);      // Create table to represent instance

    // Set first argument of new to metatable of instance
    lua_pushvalue(L,1);       
    lua_setmetatable(L, -2);

    // Do function lookups in metatable
    lua_pushvalue(L,1);
    lua_setfield(L, 1, "__index");  

    // Allocate memory for a pointer to to object
    Sprite **s = (Sprite **)lua_newuserdata(L, sizeof(Sprite *));  

    double x = luaL_checknumber (L, 2);      
    double y = luaL_checknumber (L, 3);
    double dir = luaL_checknumber (L, 4);      
    double speed = luaL_checknumber (L, 5);

    *s = new Sprite(Point2(x, y), dir, speed);
    
    // Get metatable 'Lusion.Sprite' store in the registry
    luaL_getmetatable(L, "Lusion.Sprite");

    // Set user data for Sprite to use this metatable
    lua_setmetatable(L, -2);       
    
    // Set field '__self' of instance table to the sprite user data
    lua_setfield(L, -2, "__self");  
    
  return 1;
}

We can now work with sprite instances in Lua like this:

-- Instantiate from Sprite class
local sprite = Sprite:new(x, y, dir, speed)
sprite:setPosition(x,y)
sprite:render()

-- Add method to instance as use it as class
function sprite:doSomething()
  print("do something")
end

local derived = sprite:new(x, y, dir, speed)
derived:render()
derived:doSomething() -- This is now a legal operation

There are still a couple of loose ends. We haven't showed how the methods are registered with this new solution nor how we access C++ object pointer in methods. But this is fairly straight forward as I will show.

void registerSprite(lua_State *L)
{  
  // Register metatable for user data in registry
  luaL_newmetatable(L, "Lusion.Sprite");
  luaL_register(L, 0, gDestroySpriteFuncs);      
  luaL_register(L, 0, gSpriteFuncs);      
  lua_pushvalue(L,-1);
  lua_setfield(L,-2, "__index");  
  
  // Register the base class for instances of Sprite
  luaL_register(L, "Sprite", gSpriteFuncs);  
}

We can then implement a method of Sprite like this

static int setSpeed(lua_State *L)
{
  int n = lua_gettop(L);  // Number of arguments
  
  if (n == 2) {
    Sprite* sprite = checkSprite(L);
    assert(sprite != 0);
    real speed = luaL_checknumber (L, 2);
    sprite->setSpeed(speed);
  }
  else
    luaL_error(L, "Got %d arguments expected 2 (self, speed)", n);
  
  return 0;
}

To extract the pointer to the C++ object and make sure it is of the correct type we use the following code:

Sprite* checkSprite(lua_State* L, int index)
{
  void* ud = 0;
  luaL_checktype(L, index, LUA_TTABLE);
  lua_getfield(L, index, "__self");
  ud = luaL_checkudata(L, index, "Lusion.Sprite");
  luaL_argcheck(L, ud != 0, "`Lusion.Sprite' expected");  
  
  return *((Sprite**)ud);      
}

The only thing left is dealing with garbage collection but I leave that as an exercise. You should already have the basic idea of how to deal with it. Please note that I have not tested the exact same code as written here so why the principles are correct there might be minor errors in the code. In my own code I have separated more of the code into separate functions since the code for creating an instance is almost identical for any class, as well as the code for extracting the __self pointer.

Conclusion

While using a bridge might be better for bigger projects I think it is good to know for yourself exactly what goes on under the hood and when you do it yourself you can more easily fine tune what you export and not and in which way. Typically you would want the Lua interface to be simpler and more limited than the C++ interface to your classes.

6 comments:

a said...

Thanks for your great tutorial. I liked it a lot.

Aaron said...

Thanks a tone for the tutorial. The first part is working out for me, very slick. I'm confused about the final approach however. It would help if you posted the complete code (sorry if you have and I just can't find it). For example what is stored in gDestroySpriteFuncs. I assume a wrapper to ~Sprite() but it's name is plural, is there more to it? Also why does gSpriteFuncs need to be registered to 0 as well as "Sprite,"? Help would be super appreciated. Thanks in advance.

Adam Smith said...

Yeah sorry Aaron I left out a few parts. gDestroySpriteFuncs contains registration for the functions for garbage collection. I am sorry I can't post my full code. It is because it is a bigger system with lots of helper functions, so there would be much more code to read to understand anything. In this blog I just tried to extract the important parts. Anyway here is some code:

// __gc
static int destroySprite(lua_State* L)
{
Sprite* sprite = 0;

checkUserData(L, "Lusion.Sprite", sprite);
sprite->release();
return 0;
}

// functions that will show up in our Lua environment
static const luaL_Reg gDestroySpriteFuncs[] = {
{"__gc", destroySprite},
{NULL, NULL}
};

The reason why you register several places is because you want the methods available both on "Lusion.Sprite" and "Sprite". "Sprite" is what the users will see. It is essentially the class object. They can write "Sprite.new" e.g. "Lusion.Sprite" however is used as the metatable for each instance of sprites you create. So it needs the gDestroySpriteFuncs so it can handle garbage collection of each instance. We don't need to handle garbage collection of "Sprite" object because there is no class objects in C++.

I must admit I don't remember the details that well anymore so take what I say with a grain of salt. In fact I think this last method is too complicated, that I started using a simpler approach instead. I don't care about supporting inheritance now.

You can usually accomplish what you need with embedding / aggregation. Since Lua is ducktyped you don't really need to think in terms of inheritance hierarchies the way you do in C++.

There is a lot to be said about how I use lua now, but I'll leave that for another blogpost :-)

MarkusPfundstein said...

Hi,

nice read my friend. Helped me a lot. I just have one question to the second method.

When i register my sprite** metatable with

Sprite**sprite=(Sprite**)lua_newuserdata(L, sizeof(Sprite*))
Sprite*sprite=new Sprite( ... )

don't i have to delete **sprite later?

I created a function which deletes the object itself like this:

static int delete_object(lua_State *lua_script){
Sprite** obj=(Sprite**)lua_topointer(lua_script, 1);
printf("destroy: *%p\t**%p\n", obj, *obj);
delete (*obj);

return 1;
}

which works well with deleting the object. But what is with the pointer to the meta table? Is this a job for the lua gc?

Hope to get an answer :-)

Bye
Markus

http://howtomakeitinamsterdam.wordpress.com

Adam Smith said...

Markus, memory allocated using lua_newuserdata() is managed by lua. Lua will garbage collect it when necessary. What you need to do is register a callback function for when garbage collection of your Sprite ** pointer happens. That is what the destroySprite() functions discussed above is for. Here you explicitly free the Sprite object you allocated.

In my example code in answer to Aaron you see:
sprite->release();

rather than:
delete sprite;

The reason for this is because I am reference counting my sprite objects. So even if lua doesn't know about a particular sprite anymore it can still be held by other parts of the system.

I might have misunderstood but I believe your explicit delete function is the wrong thing to do. It seems to me like you are trying to delete yourself memory managed by lua. I would expect this to cause trouble the instance lua tries to garbage collect an object you already deleted.

lklkl said...

Hello. Thank you for the tutorial. But why are you just calling checkSprite(L) when the function for checking sprites need 2 arguments? What should I pass for the second one? thanks.