Using a 3D engine from Common Lisp with the foreign function interface

I have been using Okra for a couple of months now after having worked on it for a few weeks. Although "a couple of months" should be seen in the perspective of someone with a full-time job, a wife and a kid.

While researching avenues to do games and graphics programming from CL and while working on Okra I often had this Usenet thread from comp.lang.lisp on my mind: C++ to CLOS mapping.

Very disheartening.

However, I'm currently prototyping a mobile game using Okra which has procedurally generated textured terrain, animated models walking on that terrain and mouse picking down to the triangle level.

Let me be the first to say that I would love to have native games, graphics, physics and input libraries for CL but they are simple either not available or not mature enough. So that leaves either writing those myself in CL or talking to the outside world. Since I'm mainly interested in games programming and not engine building the latter choice was the obvious one for me, not in the least since it would take far less time (weeks versus months) to get decent results on the screen.

So here are some of my observations in no specific order:

Abstractions over the raw FFI calls to C++ are quickly made and once you have those the programming experience is suffiently 'Lispy' to be pleasant.

Performance is good enough for prototyping and even simple games. You will not see cutting edge AAA titles released with Okra in the near future though.

New minor versions of Ogre are handled well by the automatic bindings generator. Okra was started when Ogre was at 1.6.2 and it is at 1.6.4 now and I had no problems following the upgrades. Keep in mind that Ogre is a very well written and stable library. I do expect the step to 1.7 to be more problematic.

I would have spent far more time doing the explorative programming projects in C++ than it took me doing them in CL. Additionally they would have been far less fun. This is also due to me being even less versed in C++ than I am in Common Lisp.

Portability between Linux and Windows has not been much of a problem. I can move pretty effortlessly between different combinations of Linux / Windows Vista / Windows XP and CLISP / Clozure CL / SBCL. I did have to turn to #lisp once for a problem with SBCL on Linux (thanks pkhuong!). My main problems have been hardware related: my machine at work I use to test now and then has an old on-board Intel 910 that barfs on some Ogre material scripts which use shaders.

That's it for now, I'll add more points when I remember them :)


Wow, I hadn't seen that thread before.

Using Lisp all the way down would be nice, but people need to be able to choose their battles. For me, as an inexperienced Lisper, that battle would be to create working software for the end users. With more experience and success, I could move on to aid CL library development, but it would be a fool's errand for me to start that way.

Beginners need to get working software out the door and have positive results early on, and I have no doubt wrappers can play a vital part in that. I use C code from within Ruby gems very often, and since it currently buys my beer, I won't be crying into it. Here's to working software over abandoned toys!

After you mulled over that thread, I think it's great that you had the balls to wrap Ogre with Okra! I'm looking forward to hearing more.
Sunday, 25 October, 2009

After a little fiddling, I have simple-okra running on OS X Snow Leopard. I hacked around with a Makefile, not knowing anything about cmake - some notes:

-include Carbon/Carbon.h, as Ogre uses Carbon. Carbon use means 32-bit, so... -arch i386 -m32, to match the Ogre prebuilt SDK.

Ogre OS X takes the framework approach and the resulting dylib paths will be bundle relative. Copy the Ogre frameworks from the SDK Dependencies directory into ~/Library/Frameworks for a dev fix.

I also added a darwin foreign library name in src-bindings/ogre-lib/ogre-lib.lisp, and changed simple-okra.lisp to load-op okra.
Wednesday, 28 October, 2009

Thanks for writing down your experiences with OS X and glad to hear it took only a little fiddling.

CMake is a utility that auto-generates Makefiles or project files for different platforms and IDEs and greatly simplifies cross-platform development. However I do not have access to OS X so I can't really help in that department.

Any chance you could e-mail the changes you made as a comprehensive diff?
Wednesday, 28 October, 2009

A Git diff is on the way!
Wednesday, 28 October, 2009

Have you documented what you did with the bindings generator somewhere? It seems like something that could be worthwhile to make available separately? If I understood it correctly at a quick glance, you use doxygen to extract information from the C++ source files, use that to create C wrapper functions and build your FFI on top of those C wrapper functions?

Would you estimate it to be a lot of work to adapt it to some other C++ lib?
Sunday, 22 November, 2009

I haven't documented what I did with the bindings generator (BG) and I should. There might a post to the Okra mailing list but is down at the moment so I can't check.

You are basically correct. I let Doxygen generate XML output of Ogre's header files and from those the BG creates the C wrappers and the CFFI code. (It also generates some extra files for CMake, ASDF and packages.) Doxygen's options should also be explored further to see whether the BG can be simplified.

It might be worthwhile to make available seperately... I'm not sure. It will be a lot of work to generalise it and I will not do it in the near future (the next 12 months) and while it has been in my mind now and then I have no serious plans for it. So go ahead ;-) Feel free to make a project of it.

Adapting it to another C++ will be easier and I've been planning to do that with Bullet but also not in the near future (the next 3 to 6 months). It will certainly be a good project to improve the BG (since it suffers very much from the first version syndrome) and could eventually lead to a more general C++ BG.
Monday, 23 November, 2009

Thanks for this blog!

I've been working to build okra in OS X with Ogre 1.7.1 and SBCL on an Intel CoreDuo. I've built and run the Ogre library and examples using CMake and XCode.

Ive been getting errors that look similar to these:

../libs/aerique-okra-bce814f/libokra/src/ogre-mesh.cpp:40: error: expected unqualified-id before '&' token
../libs/aerique-okra-bce814f/libokra/src/ogre-node.cpp:458: error: 'class Ogre::Node' has no member named 'getMaterial'
../libs/aerique-okra-bce814f/libokra/src/ogre-movable-object.cpp:217: error: 'UserDefinedObject' has not been declared

My guess is these are new features added from 1.6.1 but the missing getMaterial looks fairly necessary. The first error above seems to be referring to Material::LodDistanceList for which I cant find a definition in the okra source files. Any help would be much appreciated.
Tuesday, 03 August, 2010