After the last hastily constructed stream-of-consciousness post, I feel like I didn't explain some things very well.
I mentioned that I have been failing for about 10 years. This isn't completely accurate, as I have both learned a lot, and have been able to re-use that knowledge and a lot of mathematics and code in future attempts. Writing a game engine as part of a small team is difficult, and this is expected.
I have been writing games and game engines for 25 years. Why? Because it's fun, and an endless journey of knowledge. I am less interested in making games, and more interested in the design of game engines. A game engine is interesting to me because it requires discipline in many fields of study, and each implementation is different. The thing is, a game engine is a piece of software that manages the data flow for a particular game, or a particular category of games. It is nothing more than a set of choices someone made for you in order to write games in a particular way. Any given game engine could be productive or counter-productive in creating your game. Even using a general purpose game engine like Unity and Unreal is a trade-off, and for a significant game, you'll find you still have to work around or reimplement core engine features at the 11th hour to get your game shipped.
I mentioned I work in a small team writing game engine code. Yes, there are three other developers working with me to write a game engine in Common Lisp, which is a different project than the engine mentioned in the previous post, and serves different game developer needs. Half of the people in this team are currently part of a games studio that professionally use Unity and have released real games, both as part of the studio, and individually, and the reason why they want to make an engine in Common Lisp, is because of the numerous shortcomings of that engine. Even with millions of dollars, and an endless army of developers, a particular game engine still may not work for you, and could be more of a hindrance than starting from scratch, especially if you already have experience in the black arts.
As mentioned, I do enjoy working out the math and architectural decisions involved with a complicated piece of software such as a game engine, far more than making games. It is why I have made several (perhaps a dozen) game engines in Common Lisp – that's what I find fun. Ocassionally, I get a good idea for a game, and I stop to try using one of these to execute that idea. This latest attempt was trying to use an engine designed for one particular type of game for another, so it is no wonder it wasn't suited for the performance (and some feature) characteristics required.
Even if one has an engine particularly suited for the type of game they want to make, making games is hard, and requires lots of discipline in many different fields, not just maths and computer science. Content is king, and asset creation accounts for a lot of the work, in addition to all the game logic and making it all well-balanced. This can only come after a seemingly never-ending tweak, play-test, tweak feedback loop in most situations, for a moderately sized game. This large cost in writing a game is one reason why lots of people reach for a ready-made engine instead of doing things themselves, and there is no harm in that.
While there are some promising engines and tools which can be built upon for Common Lisp, they all have not been battle-tested, or are otherwise not very usable out-of-box for a sizable game idea. This has led me and a few others to try changing that, slowly but surely. Common Lisp is an excellent platform for a game engine, despite what some may think. Common Lisp can be extremely performant, but ultimate runtime performance is not usually required for games these days. Good game design is about finding a balance between the CPU and GPU, and with concurrency and the very little work most games have to do on the CPU relative to that of the GPU, unless there is a complex, non-discrete physics system involved, or tens of thousands of nodes in your scene, it really isn't a problem. If it ever is, you can shift around work between the two processors in a lot of cases.
Where Common Lisp really shines for making games, is in the interactivity the whole language is designed around. Generic functions, while much slower than regular functions, is a cost we're usually willing to pay. Macros, and designing all of the DSLs a game requires for describing data, is dead-simple in Common Lisp. Hot code reloading, is one of my favorite features. Being able to recompile individual functions, DSLs, etc, as a game is running, without requiring custom support from a game engine editor, is the biggest time-saver for me.
For example, I could write a DSL to describe an entity along with all of its properties, and any children and their properties, and so on. Then I could create an instance of this sub-tree and stick it somewhere in the game world. Maybe I'll add 100 instances, each at different locations. Then I could go back to the DSL and decide that I want them all to have an additional child node, so I add it, hit a button, and just like that they're all updated in the game world. Similarly, if one of their textures doesn't look quite right, I could recompile another DSL that describes the texture, to have all uses of it updated in the game world. This workflow is very welcome after coming from a language that forces you to stop the game, recompile everything, restart the game, and get back to the place you were in the game. After all that is said and done, it is very difficult to know if your changes made an impact for the better - often times you are making small color adjustments, or otherwise hard to notice, but better nonetheless, shader program adjustments.
These are some of the reasons why I love Common Lisp, and why I love making game engines and games in Common Lisp. Just because I got burnt out for a bit, doesn't mean I'm done or have given up. This is a lifelong journey of mine, because I find it fun and a pleasure to work with such a dynamic, interactive, and fast when it needs to be language.
People like Nicolas Hafner (Shinmera), Chris Bagley (Baggers), and Pavel Korolev (borodust) are inspirations, who have also devoted themselves to game development in Common Lisp, with great success. I wish them all the best of luck, and my thanks for giving me the will to continue after so many years. I would also like to sincerely thank all of the people who have been supportive over the years, and special thanks to those few of you who have sponsored me. Thank you, everyone!