Lua is a script language frequently used in game development. Typically the game engine is written in C/C++ and the game itself is defined in Lua. Meaning setting up behaviour, placement of objects like powerups, doors, enemies etc is done in Lua. That way the game can be quickly tweaked without having to go through a time consuming re-compile.
But I am not going to go in depth about the different usage of Lua, nor explain the language itself. An online book at the Lua web page does that very well already. I will here assume you already have some basic knowledge about Lua.
However finding any documentation on how to wrap a C++ class so that it can be used in Lua is difficult. One could of course use one of the ready made bridges. But here we are going to look at how to do it yourself.
The tricky part is deciding on how to do it. Because Lua is such a flexible language there is really a lot of ways you could achieve it.
The naive approach
First lets look at several options, what works and doesn't. My first approach was to light user data to store a pointer to my C++ class. I will use a Sprite class and related classes as examples here as that was the type of classes I was wrapping.
The code snippet above shows a naive implementation of this approach. Unfortunately it doesn't work. The problem is that a light user data is just a simple pointer. Lua does not store any information with it. For instance a metatable which we could use to define the methods the class supports.
An approach with limited functionality
The next approach would be to use user data. Unlike light user data it can store a reference to a metatable.
For us to be able to use sprite like this we need to register class first. Basically we need to create a table Sprite which contains all the methods that our user data should support.
This will allow us to create instances of Sprite and call methods on it in Lua like this:
The final approach
In most cases this approach is sufficient but it has one major limitation. It does not support inheritance. You can change the methods of Sprite in Lua but that will change the behavior of all instances of Sprite. What you would want to do is to be able to change method on just the instance and then use that instance as a prototype for new Sprite instances, effectively creating a class inheritance system.
To do this we need to change the instance into being a table. How do we access our C++ object then? Simple, we just store the pointer to it as user data in one of the field of the table. You might think that this time light user data will be sufficient. However the problem is that only user data is informed of garbage collection, not tables or light user data. So if you want to delete the C++ object when corresponding lua table is garbage collected you need to use user data.
So we then arrive at our final solution. We will store a pointer to our C++ object as user data on the key __self in the table that represents our instance. __self is an arbitrary selected name. It could be anything. We will not register Sprite as as the metatable for our instance but instead register the first argument to the new function as it. This will allow us to support inheritance. Further the garbage collection function will be register on a separate table which will be used as metatable only for the user data. This is to allow it to be garbage collected.
We can now work with sprite instances in Lua like this:
There are still a couple of loose ends. We haven't showed how the methods are registered with this new solution nor how we access C++ object pointer in methods. But this is fairly straight forward as I will show.
We can then implement a method of Sprite like this
To extract the pointer to the C++ object and make sure it is of the correct type we use the following code:
The only thing left is dealing with garbage collection but I leave that as an exercise. You should already have the basic idea of how to deal with it. Please note that I have not tested the exact same code as written here so why the principles are correct there might be minor errors in the code. In my own code I have separated more of the code into separate functions since the code for creating an instance is almost identical for any class, as well as the code for extracting the __self pointer.
While using a bridge might be better for bigger projects I think it is good to know for yourself exactly what goes on under the hood and when you do it yourself you can more easily fine tune what you export and not and in which way. Typically you would want the Lua interface to be simpler and more limited than the C++ interface to your classes.