Thursday, 22 November 2007

The Importance of building a Data Driven Engine

During the design phase of an application, game or project very strong consideration should be given to building a data driven engine. Essentially, it is an engine where some or all of the data that it uses is external to it, perhaps in configuration files, XML files or a database.
What follows is a brief look at why developing such a thing is a really good idea.


  • No compiling...

When you make changes to the data, you do not need to compile them into the engine. The opposite of this is where data is "hard coded" into the engine, so changes to the data, even small changes, require the whole engine to be recompiled. This is extremely unefficient and time consuming and so should be avoided.

  • Flexible...

Within the framework of the engine, this will prove to be a highly flexible approach. By and large because it is much easier to change the data that the engine uses than to implement code within the engine, it helps with iterative testing, e.g. for establishing balancing rules within a game.

  • My own experience...

My own experience with hobbiest games development (generally I don't build games professionally, instead it is applications, web sites and utilities on a day to day basis) has taught me a lot about what one lone developer can achieve. Having a fantastic vision doesn't mean it can be translated into reality, but having a data driven engine will help massively. I could not have build Necrotech on my own a few years ago without this approach.

  • Vs Design patterns...

Im also relearning this very lesson with my latest project Dungeon Creep. It is an RPG based on the principles of procedural content generation and is in its early stages. It can build dungeon layouts and the player can walk around them. This is in Macromedia Flash, by the way.
I'm currently working on the inventory system which works in a similar way to Diablo and Two Worlds in that the items are also randomly generated. For example, a knife and a heavy knife share the base stats of a knife but the heavy knife weighs more, is slower and inflicts more damage when it hits.
I implemented this originally using a polymorphic design pattern which was fine. The basic idea is that items can have modifiers stacked on them (in the above example, "heavy" is such a modifier) with some rules controlling which modifiers can be stacked on what (e.g. you will not get a "Sharp" Staff). This worked very well and made it easy to add lots of modifiers and was even flexible enough for me to expand it into materials (e.g. a Steel Sword, where Steel is a special type of modifier). However, now I have about twenty of these in the item engine, it is starting to become hard to manage.

Moving this out to a data driven approach that loads them from XML files makes it much easier to get an overall view on the whole data set. I had to write some loader and conversion code to get this working but now I'm back in control of what is going on. So design patterns are extremely useful but should be used in conjunction with a data driven engine and not as a replacement.

  • It could mean life or death...

...for the project you're working on. Essentially, it can be the difference between completing a project and not, especially if it is a large or ambitious project, or the team building it is very small. Take every advantage you can to minimise the time involved in creating the project and also the stress.


  • Performance could suffer

Anything that isn't hardcoded directly into the game engine code itself will not perform as well. However, this performance disadvantage can normally be removed by simlpy reading the external data into internal data structures as part of the main engine initialisation process. So it isn't as big a deal as you might expect.

No comments: