All-purpose script parser         Load your own custom data from Ogre-style scripts

Custom Script Parser?

In many games, object data is stored in external files for easy adjustment and management. For example, in a racing game, each car might have it's own configuration file that specifies it's top speed, turning radius, etc. Many programmers use XML or a binary format to store this information.

If you've ever written a .material script for Ogre, you probably know that Ogre's C-style script syntax is very intuitive and compact. Now you can store any kind of data you want in Ogre C-style scripts, and and quickly and easily load them with this parser.

Like Ogre's .material scripts, this will automatically parse all your scripts when a resource group is loaded. Since all scripts are pre-parsed, the system can access scripts not by filename, but by script name; in other words, you can include multiple scripts per file and still access them individually.

This script parser was designed to be very lightweight and fast; it's only a few KBs of code, and can parse around 10 MB per second on an average PC (if your average script is ~1 KB, you can load around 10,000 scripts in one second).

Using the Parser

Using this script parser is very easy. Simply initialize the system by creating a ConfigScriptLoader instance. You don't even have to keep a pointer to the instance because ConfigScriptLoader is a singleton class:

new ConfigScriptLoader();

Make sure you create a ConfigScriptLoader class before loading any resource groups, otherwise it will miss them and your scripts won't get loaded.

Now, after loading your resource groups, the ConfigScriptLoader will have automatically parsed your scripts for you! All you need to do now is request the desired script, and process it's contents. Getting a script is simple:

ConfigNode *rootNode;
 rootNode = ConfigScriptLoader::getSingleton().getConfigScript("entity", "Crate");

ConfigScriptLoader::getConfigScript() will retrieve the root node of the specified script, which you can use to access any of it's data.

You may notice that the getConfigScript() requires two names; one for the script type, and another for the actual name. For example, the above code would load this script:

entity Crate{
   position 100 50 200
   rotation 0 0 0
   scale 1 1 1
 }

You may be wondering what file this script is coming from. The fact is, it doesn't matter! As long as your scripts have a .object extension (and this can be changed if you modify line 22 of ConfigScript.cpp), their filename is completely irrelevant. Since you access scripts by script name rather than file name, it makes no difference where the script is located. This is a big advantage since this allows you to organize your scripts any way you want without worrying about your game not being able to find it.

Now that you have a pointer to the root ConfigNode, you can access any of the data it contains. For example, to access the Y position coordinate (50) given in the above script example, you first find the "position" node, then access it's 2nd value (as you can see it contains 3 values in all: 100, 50, and 200).

float positionY = rootNode->findChild("position")->getValueF(1);

1 is used to access the 2nd position value since std::vector is 0-based (like any other C array). getValueF() is a variation of getValue() that automatically converts the value to a float. Other variations are included for doubles, ints, etc.

Note: When you shut down your application, don't forget to delete the ConfigScriptLoader instance - Ogre won't do this for you, so failing to do so will result in a memory leak. Since it's a singleton class, you can always delete it using this code:

delete ConfigScriptLoader::getSingletonPtr();

Source Files



ConfigScript.h

//This code is public domain - you can do whatever you want with it
//Original author: John Judnich
#ifndef _CONFIGSCRIPT_H__
#define _CONFIGSCRIPT_H__

#include <OgreScriptLoader.h>
#include <OgreStringConverter.h>
#include <hash_map>
#include <vector>

class ConfigNode;

class ConfigScriptLoader: public Ogre::ScriptLoader
{
public:
    ConfigScriptLoader();
    ~ConfigScriptLoader();

    inline static ConfigScriptLoader &getSingleton() { return *singletonPtr; }
    inline static ConfigScriptLoader *getSingletonPtr() { return singletonPtr; }

    Ogre::Real getLoadingOrder() const;
    const Ogre::StringVector &getScriptPatterns() const;

    ConfigNode *getConfigScript(const Ogre::String &type, const Ogre::String &name);

    void parseScript(Ogre::DataStreamPtr &stream, const Ogre::String &groupName);
    
private:
    static ConfigScriptLoader *singletonPtr;

    Ogre::Real mLoadOrder;
    Ogre::StringVector mScriptPatterns;

    stdext::hash_map<Ogre::String, ConfigNode*> scriptList;
    
    //Parsing
    char *parseBuff, *parseBuffEnd, *buffPtr;
    size_t parseBuffLen;

    enum Token
    {
        TOKEN_Text,
        TOKEN_NewLine,
        TOKEN_OpenBrace,
        TOKEN_CloseBrace,
        TOKEN_EOF,
    };

    Token tok, lastTok;
    Ogre::String tokVal, lastTokVal;
    char *lastTokPos;

    void _parseNodes(ConfigNode *parent);
    void _nextToken();
    void _prevToken();
};

class ConfigNode
{
public:
    ConfigNode(ConfigNode *parent, const Ogre::String &name = "untitled");
    ~ConfigNode();

    inline void setName(const Ogre::String &name)
    {
        this->name = name;
    }

    inline Ogre::String &getName()
    {
        return name;
    }

    inline void addValue(const Ogre::String &value)
    {
        values.push_back(value);
    }

    inline void clearValues()
    {
        values.clear();
    }

    inline std::vector<Ogre::String> &getValues()
    {
        return values;
    }

    inline const Ogre::String &getValue(unsigned int index = 0)
    {
        assert(index < values.size());
        return values[index];
    }

    inline float getValueF(unsigned int index = 0)
    {
        assert(index < values.size());
        return Ogre::StringConverter::parseReal(values[index]);
    }

    inline double getValueD(unsigned int index = 0)
    {
        assert(index < values.size());

        std::istringstream str(values[index]);
        double ret = 0;
        str >> ret;
        return ret;
    }

    inline int getValueI(unsigned int index = 0)
    {
        assert(index < values.size());
        return Ogre::StringConverter::parseInt(values[index]);
    }
    
    ConfigNode *addChild(const Ogre::String &name = "untitled", bool replaceExisting = false);
    ConfigNode *findChild(const Ogre::String &name, bool recursive = false);

    inline std::vector<ConfigNode*> &getChildren()
    {
        return children;
    }

    inline ConfigNode *getChild(unsigned int index = 0)
    {
        assert(index < children.size());
        return children[index];
    }
    
    void setParent(ConfigNode *newParent);

    inline ConfigNode *getParent()
    {
        return parent;
    }

private:
    Ogre::String name;
    std::vector<Ogre::String> values;
    std::vector<ConfigNode*> children;
    ConfigNode *parent;

    int lastChildFound;  //The last child node's index found with a call to findChild()

    std::vector<ConfigNode*>::iterator _iter;
    bool _removeSelf;
};

#endif


ConfigScript.cpp

#include "ConfigScript.h"
#include "Exception.h"

#include <OgreScriptLoader.h>
#include <OgreScriptLoader.h>
#include <OgreResourceGroupManager.h>
using namespace Ogre;

#include <vector>
#include <hash_map>
using namespace std;
using namespace stdext;

ConfigScriptLoader *ConfigScriptLoader::singletonPtr = NULL;

ConfigScriptLoader::ConfigScriptLoader()
{
    //Init singleton
    if (singletonPtr)
        EXCEPTION("Multiple ConfigScriptManager objects are not allowed", "ConfigScriptManager::ConfigScriptManager()");
    singletonPtr = this;

    //Register as a ScriptLoader
    mLoadOrder = 100.0f;
    mScriptPatterns.push_back("*.object");
    ResourceGroupManager::getSingleton()._registerScriptLoader(this);
}

ConfigScriptLoader::~ConfigScriptLoader()
{
    singletonPtr = NULL;

    //Delete all scripts
    stdext::hash_map<String, ConfigNode*>::iterator i;
    for (i = scriptList.begin(); i != scriptList.end(); i++){
        delete i->second;
    }
    scriptList.clear();

    //Unregister with resource group manager
    if (ResourceGroupManager::getSingletonPtr())
        ResourceGroupManager::getSingleton()._unregisterScriptLoader(this);
}

Real ConfigScriptLoader::getLoadingOrder() const
{
    return mLoadOrder;
}

const StringVector &ConfigScriptLoader::getScriptPatterns() const
{
    return mScriptPatterns;
}

ConfigNode *ConfigScriptLoader::getConfigScript(const String &type, const String &name)
{
    stdext::hash_map<String, ConfigNode*>::iterator i;

    String key = type + ' ' + name;
    i = scriptList.find(key);

    //If found..
    if (i != scriptList.end())
        return i->second;
    else
        return NULL;
}

void ConfigScriptLoader::parseScript(DataStreamPtr &stream, const String &groupName)
{
    //Copy the entire file into a buffer for fast access
    parseBuffLen = stream->size();
    parseBuff = new char[parseBuffLen];
    buffPtr = parseBuff;
    stream->read(parseBuff, parseBuffLen);
    parseBuffEnd = parseBuff + parseBuffLen;

    //Close the stream (it's no longer needed since everything is in parseBuff)
    stream->close();

    //Get first token
    _nextToken();
    if (tok == TOKEN_EOF)
        return;
    
    //Parse the script
    _parseNodes(0);

    if (tok == TOKEN_CloseBrace)
        EXCEPTION("Parse Error: Closing brace out of place", "ConfigScript::load()");
    
    //Delete the buffer
    delete[] parseBuff;
}

void ConfigScriptLoader::_nextToken()
{
    lastTok = tok;
    lastTokVal = tokVal;
    lastTokPos = buffPtr;

    //EOF token
    if (buffPtr >= parseBuffEnd){
        tok = TOKEN_EOF;
        return;
    }

    //(Get next character)
    int ch = *buffPtr++;
    while (ch == ' ' || ch == 9){    //Skip leading spaces / tabs
        ch = *buffPtr++;
    }

    //Newline token
    if (ch == '\r' || ch == '\n'){
        do {
            ch = *buffPtr++;
        } while ((ch == '\r' || ch == '\n') && buffPtr < parseBuffEnd);
        buffPtr--;

        tok = TOKEN_NewLine;
        return;
    }

    //Open brace token
    else if (ch == '{'){
        tok = TOKEN_OpenBrace;
        return;
    }

    //Close brace token
    else if (ch == '}'){
        tok = TOKEN_CloseBrace;
        return;
    }

    //Text token
    if (ch < 32 || ch > 122)    //Verify valid char
        EXCEPTION("Parse Error: Invalid character", "ConfigScript::load()");

    tokVal = "";
    tok = TOKEN_Text;
    do {
        //Skip comments
        if (ch == '/'){
            int ch2 = *buffPtr;
            
            //C++ style comment (//)
            if (ch2 == '/'){
                buffPtr++;
                do {
                    ch = *buffPtr++;
                } while (ch != '\r' && ch != '\n' && buffPtr < parseBuffEnd);
                
                tok = TOKEN_NewLine;
                return;
            }
        }

        //Add valid char to tokVal
        tokVal += ch;

        //Next char
        ch = *buffPtr++;
    } while (ch > 32 && ch <= 122 && buffPtr < parseBuffEnd);
    buffPtr--;

    return;
}

void ConfigScriptLoader::_prevToken()
{
    tok = lastTok;
    tokVal = lastTokVal;
    buffPtr = lastTokPos;
}

void ConfigScriptLoader::_parseNodes(ConfigNode *parent)
{
    typedef std::pair<String, ConfigNode*> ScriptItem;

    while (1) {
        switch (tok){
            //Node
            case TOKEN_Text:
                //Add the new node
                ConfigNode *newNode;
                if (parent)
                    newNode = parent->addChild(tokVal);
                else
                    newNode = new ConfigNode(0, tokVal);

                //Get values
                _nextToken();
                while (tok == TOKEN_Text){
                    newNode->addValue(tokVal);
                    _nextToken();
                }

                //Add root nodes to scriptList
                if (!parent){
                    String key;

                    if (newNode->getValues().empty())
                        key = newNode->getName() + ' ';
                    else
                        key = newNode->getName() + ' ' + newNode->getValues().front();

                    scriptList.insert(ScriptItem(key, newNode));
                }

                //Skip any blank spaces
                while (tok == TOKEN_NewLine)
                    _nextToken();
                
                //Add any sub-nodes
                if (tok == TOKEN_OpenBrace){
                    //Parse nodes
                    _nextToken();
                    _parseNodes(newNode);
                    
                    //Skip blank spaces
                    while (tok == TOKEN_NewLine)
                        _nextToken();

                    //Check for matching closing brace
                    if (tok != TOKEN_CloseBrace)
                        EXCEPTION("Parse Error: Expecting closing brace", "ConfigScript::load()");
                } else {
                    //If it's not a opening brace, back up so the system will parse it properly
                    _prevToken();
                }
                                
                break;

            //Out of place brace
            case TOKEN_OpenBrace:
                EXCEPTION("Parse Error: Opening brace out of plane", "ConfigScript::load()");
                break;

            //Return if end of nodes have been reached
            case TOKEN_CloseBrace:
                return;

            //Return if reached end of file
            case TOKEN_EOF:
                return;
        }
    
        //Next token
        _nextToken();
    };
}

ConfigNode::ConfigNode(ConfigNode *parent, const String &name)
{
    ConfigNode::name = name;
    ConfigNode::parent = parent;
    _removeSelf = true;    //For proper destruction
    lastChildFound = -1;

    //Add self to parent's child list (unless this is the root node being created)
    if (parent != NULL){
        parent->children.push_back(this);
        _iter = --(parent->children.end());
    }
}

ConfigNode::~ConfigNode()
{
    //Delete all children
    std::vector<ConfigNode*>::iterator i;
    for (i = children.begin(); i != children.end(); i++){
        ConfigNode *node = *i;
        node->_removeSelf = false;
        delete node;
    }
    children.clear();

    //Remove self from parent's child list
    if (_removeSelf && parent != NULL)
        parent->children.erase(_iter);
}

ConfigNode *ConfigNode::addChild(const String &name, bool replaceExisting)
{
    if (replaceExisting) {
        ConfigNode *node = findChild(name, false);
        if (node)
            return node;
    }
    return new ConfigNode(this, name);
}

ConfigNode *ConfigNode::findChild(const String &name, bool recursive)
{
    int indx, prevC, nextC;
    int childCount = (int)children.size();

    if (lastChildFound != -1){
        //If possible, try checking the nodes neighboring the last successful search
        //(often nodes searched for in sequence, so this will boost search speeds).
        prevC = lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
        nextC = lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
        for (indx = prevC; indx <= nextC; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }

        //If not found that way, search for the node from start to finish, avoiding the
        //already searched area above.
        for (indx = nextC + 1; indx < childCount; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
        for (indx = 0; indx < prevC; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
    } else {
        //Search for the node from start to finish
        for (indx = 0; indx < childCount; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
    }

    //If not found, search child nodes (if recursive == true)
    if (recursive){
        for (indx = 0; indx < childCount; ++indx){
            children[indx]->findChild(name, recursive);
        }
    }

    //Not found anywhere
    return NULL;
}

void ConfigNode::setParent(ConfigNode *newParent)
{
    //Remove self from current parent
    parent->children.erase(_iter);

    //Set new parent
    parent = newParent;

    //Add self to new parent
    parent->children.push_back(this);
    _iter = --(parent->children.end());
}

Porting Code

Our project uses Ogre on the client but not on the server. If you need to fetch resource files on the server side, you may try to port this code. The above code assumes the input reader does onto convert CR-LF to CR, which happens on Windows systems. It also make the assumption, "we read the whole file and then process the resulting buffer." This ignores the issue of CR-LF and also ignores the normal buffered input stream available in C++. And there is a problem with the order of getting new tokens that breaks on some files.

There is also a mistake related to the last character in the file being a }
_parseNodes. Before the "//Check for matching closing brace" it skips white space, but then the check for TOKEN_NewLine fails. The order there should be reversed.

Non Ogre Version

Here is a version of the code that is not dependent at all on Ogre. You will have to modify it for you use and remove the TDL_ hooks, and it depends on boost. You can remove the boost parts easily. It also has a few corrections to the above code.

The class ConfigLoaderEntity is at the bottom. ConfigLoaderCategory is nearly the same.

#ifndef _TDL_CONFIG_LOADER_H__
#define _TDL_CONFIG_LOADER_H__
 
#pragma once

#include <hash_map>
#include <vector>

namespace TDLConfigLoaders
{
	class TDLConfigNode;

	// The base class of loaders that read Ogre style script files to get configuration and settings for TDL.
	class ConfigLoader
	{
	public:
		static void loadAllFiles();
		static void scanLoadFiles(ConfigLoader * c);

		ConfigLoader(std::string fileEnding, float loadOrder = 100.0f);
		virtual ~ConfigLoader();
 
		std::string m_fileEnding;
 
		// For a line like
		// entity animals/dog
		// {
		//    ...
		// }
		// The type is "entity" and the name is "animals/dog"
		// Or if anima/dog was not there then name is ""
		virtual TDLConfigNode *getConfigScript(const std::string &type, const std::string &name);
 
		virtual void parseScript(std::ifstream &stream) = 0;

 
	protected:

		float m_LoadOrder;
		// like "*.object"
 
		stdext::hash_map<std::string, TDLConfigNode*> m_scriptList;
  
		enum Token
		{
			TOKEN_Text,
			TOKEN_NewLine,
			TOKEN_OpenBrace,
			TOKEN_CloseBrace,
			TOKEN_EOF,
		};
 
		Token tok, lastTok;
		std::string tokVal, lastTokVal;
 
		void _parseNodes(std::ifstream &stream, TDLConfigNode *parent);
		void _nextToken(std::ifstream &stream);
		void _skipNewLines(std::ifstream &stream);

		virtual void clearScriptList();
	};
 
	class TDLConfigNode
	{
	public:
		TDLConfigNode(TDLConfigNode *parent, const std::string &name = "untitled");
		~TDLConfigNode();
 
		inline void setName(const std::string &name)
		{
			this->m_name = name;
		}
 
		inline std::string &getName()
		{
			return m_name;
		}
 
		inline void addValue(const std::string &value)
		{
			m_values.push_back(value);
		}
 
		inline void clearValues()
		{
			m_values.clear();
		}
 
		inline std::vector<std::string> &getValues()
		{
			return m_values;
		}
 
		inline const std::string &getValue(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return m_values[index];
		}
 
		inline float getValueF(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return (float)::atof(m_values[index].c_str());
		}
 
		inline double getValueD(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return ::atof(m_values[index].c_str());
		}

		inline int getValueI(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return ::atoi(m_values[index].c_str());
		}
 
		inline btScalar getValueBtScalar(unsigned int index = 0)
		{
			return parseBtScalar(m_values[index]);
		}
 
		inline btVector3 getValueV3(unsigned int index = 0)
		{
			assert(index < m_values.size() - 3);
			return btVector3(parseBtScalar(m_values[index]), parseBtScalar(m_values[index + 1]), parseBtScalar(m_values[index + 2]));
		}

		inline btQuaternion getValueYPR(unsigned int index = 0)
		{
			assert(index < m_values.size() - 3);
			return btQuaternion(parseBtDegrees(m_values[index]), parseBtDegrees(m_values[index + 1]), parseBtDegrees(m_values[index + 2]));
		}

		inline double parseDouble(std::string s)
		{
			double d;
			int rc = sscanf(s.c_str(), "%lf", &d);
			if(rc == 1)
			{
				return d;
			}
			assert(false);
			return 1.0;
		}
 
		inline btScalar parseBtScalar(std::string s)
		{
			double d;
			int rc = sscanf(s.c_str(), "%lf", &d);
			if(rc == 1)
			{
				return btScalar(d);
			}
			assert(false);
			return btScalar(1.0);
		}

		// s is like 90.0 degrees. returns btScalar radians.
		inline btScalar parseBtDegrees(std::string s)
		{
			return btRadians(parseBtScalar(s));
		}
 
		// s is like 1.7 radians. returns btScalar radians.
		inline btScalar parseBtRadians(std::string s)
		{
			return parseBtScalar(s);
		}
 
		TDLConfigNode *addChild(const std::string &name = "untitled", bool replaceExisting = false);
		TDLConfigNode *findChild(const std::string &name, bool recursive = false);
 
		inline std::vector<TDLConfigNode*> &getChildren()
		{
			return m_children;
		}
 
		inline TDLConfigNode *getChild(unsigned int index = 0)
		{
			assert(index < m_children.size());
			return m_children[index];
		}
 
		void setParent(TDLConfigNode *newParent);
 
		inline TDLConfigNode *getParent()
		{
			return m_parent;
		}
 
	private:
		std::string m_name;
		std::vector<std::string> m_values;
		std::vector<TDLConfigNode*> m_children;
		TDLConfigNode *m_parent;
 
		int m_lastChildFound;  //The last child node's index found with a call to findChild()
 
		std::vector<TDLConfigNode*>::iterator _iter;
		bool _removeSelf;
	};

}
 
#endif

#include <vector>
#include <hash_map>
#include <exception>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
#include "ConfigLoaders\ConfigLoaderCategory.h"
#include "ConfigLoaders\ConfigLoaderEntity.h"

#include "boost/filesystem.hpp"
#include "boost/filesystem/operations.hpp" 
#include "boost/filesystem/fstream.hpp"

using namespace std;
using namespace stdext;
using namespace TDLConfigLoaders;
using namespace boost::filesystem;  
using namespace boost::system;


void ConfigLoader::loadAllFiles()
{
	ConfigLoaderCategory * cat = new ConfigLoaderCategory();
	ConfigLoaderEntity * ent = new ConfigLoaderEntity();

	scanLoadFiles(cat);
	scanLoadFiles(ent);
}

void ConfigLoader::scanLoadFiles(ConfigLoader * c)
{
	for ( boost::filesystem::recursive_directory_iterator end, dir("../content"); 
       dir != end; ++dir ) {
		   //TDL_Log("%s", (*dir).string().c_str()); 
		   path p(*dir);
		   if(p.extension() == c->m_fileEnding)
		   {
			   std::ifstream in((*dir).string().c_str(), ios::binary);
			   c->parseScript(in);
		   }
	}
}

ConfigLoader::ConfigLoader(std::string fileEnding, float loadOrder)
{ 
    //Register as a ScriptLoader
    m_LoadOrder = loadOrder;
	m_fileEnding = fileEnding;
}
 
ConfigLoader::~ConfigLoader()
{
    clearScriptList();

}

void ConfigLoader::clearScriptList()
{
    stdext::hash_map<std::string, TDLConfigNode *>::iterator i;
    for (i = m_scriptList.begin(); i != m_scriptList.end(); i++)
	{
        delete i->second;
    }
    m_scriptList.clear();
}
 
TDLConfigNode *ConfigLoader::getConfigScript(const std::string &type, const std::string &name)
{
    stdext::hash_map<std::string, TDLConfigNode*>::iterator i;
 
    std::string key = type + ' ' + name;
    i = m_scriptList.find(key);
 
    //If found..
    if (i != m_scriptList.end())
	{
        return i->second;
	}
    else
	{
        return NULL;
	}
}

void ConfigLoader::parseScript(std::ifstream &stream)
{ 
    //Get first token
    _nextToken(stream);
    if (tok == TOKEN_EOF)
	{
	    stream.close();
        return;
	}
 
    //Parse the script
    _parseNodes(stream, 0);
 
    stream.close();
}
 
void ConfigLoader::_nextToken(std::ifstream &stream)
{
    lastTok = tok;
    lastTokVal = tokVal;
 
    //EOF token
	if (stream.eof())
	{
        tok = TOKEN_EOF;
        return;
    }
 
    //(Get next character)
	int ch = stream.get();
    while ((ch == ' ' || ch == 9) && !stream.eof())
	{    //Skip leading spaces / tabs
        ch = stream.get();
    }

	if (stream.eof())
	{
        tok = TOKEN_EOF;
        return;
    }
 
    //Newline token
    if (ch == '\r' || ch == '\n')
	{
        do 
		{
            ch = stream.get();
		} while ((ch == '\r' || ch == '\n') && !stream.eof());

		stream.unget();
 
        tok = TOKEN_NewLine;
        return;
    }
 
    //Open brace token
    else if (ch == '{')
	{
        tok = TOKEN_OpenBrace;
        return;
    }
 
    //Close brace token
    else if (ch == '}')
	{
        tok = TOKEN_CloseBrace;
        return;
    }
 
    //Text token
    if (ch < 32 || ch > 122)    //Verify valid char
	{
        throw "Parse Error: Invalid character, ConfigLoader::load()";
	}
 
    tokVal = "";
    tok = TOKEN_Text;
    do 
	{
        //Skip comments
        if (ch == '/')
		{
			int ch2 = stream.peek();
 
            //C++ style comment (//)
            if (ch2 == '/')
			{
				stream.get();
                do 
				{
                    ch = stream.get();
                } while (ch != '\r' && ch != '\n' && !stream.eof());
 
                tok = TOKEN_NewLine;
                return;
            }
        }
 
        //Add valid char to tokVal
        tokVal += (char)ch;
 
        //Next char
        ch = stream.get();

    } while (ch > 32 && ch <= 122 && !stream.eof());

	stream.unget();
 
    return;
}
 
void ConfigLoader::_skipNewLines(std::ifstream &stream)
{
	while (tok == TOKEN_NewLine)
	{
		_nextToken(stream);
	}
}

void ConfigLoader::_parseNodes(std::ifstream &stream, TDLConfigNode *parent)
{
    typedef std::pair<std::string, TDLConfigNode*> ScriptItem;
 
    while (true) 
	{
        switch (tok)
		{
            //Node
            case TOKEN_Text:
                //Add the new node
                TDLConfigNode *newNode;
                if (parent)
				{
                    newNode = parent->addChild(tokVal);
				}
                else
				{
                    newNode = new TDLConfigNode(0, tokVal);
				}
 
                //Get values
                _nextToken(stream);
                while (tok == TOKEN_Text)
				{
                    newNode->addValue(tokVal);
                    _nextToken(stream);
                }
 
                //Add root nodes to scriptList
                if (!parent){
                    std::string key;
 
                    if (newNode->getValues().empty())
					{
                        key = newNode->getName() + ' ';
					}
                    else
					{
                        key = newNode->getName() + ' ' + newNode->getValues().front();
					}
 
                    m_scriptList.insert(ScriptItem(key, newNode));
                }
 
                _skipNewLines(stream);
 
                //Add any sub-nodes
                if (tok == TOKEN_OpenBrace)
				{
                    //Parse nodes
                    _nextToken(stream);
                    _parseNodes(stream, newNode);
 
                    //Check for matching closing brace
                    if (tok != TOKEN_CloseBrace)
					{
                        throw "Parse Error: Expecting closing brace";
					}

					_skipNewLines(stream);
                } 
 
                break;
 
            //Out of place brace
            case TOKEN_OpenBrace:
                throw "Parse Error: Opening brace out of plane";
                break;
 
            //Return if end of nodes have been reached
            case TOKEN_CloseBrace:
                return;
 
            //Return if reached end of file
            case TOKEN_EOF:
                return;

			case TOKEN_NewLine:
				_nextToken(stream);
				break;
        }
    };
}
 
TDLConfigNode::TDLConfigNode(TDLConfigNode *parent, const std::string &name)
{
    m_name = name;
    m_parent = parent;
    _removeSelf = true;    //For proper destruction
    m_lastChildFound = -1;
 
    //Add self to parent's child list (unless this is the root node being created)
    if (parent != NULL)
	{
        m_parent->m_children.push_back(this);
        _iter = --(m_parent->m_children.end());
    }
}
 
TDLConfigNode::~TDLConfigNode()
{
    //Delete all children
    std::vector<TDLConfigNode*>::iterator i;
    for (i = m_children.begin(); i != m_children.end(); i++)
	{
        TDLConfigNode *node = *i;
        node->_removeSelf = false;
        delete node;
    }
    m_children.clear();
 
    //Remove self from parent's child list
    if (_removeSelf && m_parent != NULL)
	{
        m_parent->m_children.erase(_iter);
	}
}
 
TDLConfigNode *TDLConfigNode::addChild(const std::string &name, bool replaceExisting)
{
    if (replaceExisting) 
	{
        TDLConfigNode *node = findChild(name, false);
        if (node)
		{
            return node;
		}
    }
    return new TDLConfigNode(this, name);
}
 
TDLConfigNode *TDLConfigNode::findChild(const std::string &name, bool recursive)
{
    int indx, prevC, nextC;
    int childCount = (int)m_children.size();
 
    if (m_lastChildFound != -1)
	{
        //If possible, try checking the nodes neighboring the last successful search
        //(often nodes searched for in sequence, so this will boost search speeds).
        prevC = m_lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
        nextC = m_lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
        for (indx = prevC; indx <= nextC; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) 
			{
                m_lastChildFound = indx;
                return node;
            }
        }
 
        //If not found that way, search for the node from start to finish, avoiding the
        //already searched area above.
        for (indx = nextC + 1; indx < childCount; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
        for (indx = 0; indx < prevC; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
    } 
	else 
	{
        //Search for the node from start to finish
        for (indx = 0; indx < childCount; ++indx){
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
    }
 
    //If not found, search child nodes (if recursive == true)
    if (recursive)
	{
        for (indx = 0; indx < childCount; ++indx)
		{
            m_children[indx]->findChild(name, recursive);
        }
    }
 
    //Not found anywhere
    return NULL;
}
 
void TDLConfigNode::setParent(TDLConfigNode *newParent)
{
    //Remove self from current parent
    m_parent->m_children.erase(_iter);
 
    //Set new parent
    m_parent = newParent;
 
    //Add self to new parent
    m_parent->m_children.push_back(this);
    _iter = --(m_parent->m_children.end());
}

#ifndef _CONFIG_LOADER_ENTITY_H__
#define _CONFIG_LOADER_ENTITY_H__
 
#pragma once

#include <hash_map>
#include <vector>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
 
namespace TDLConfigLoaders
{
	class TDLConfigNode;

	// Any entity in the game.  Include simple unanimated models, animated models, NPCs.
	// Inlcudes the mesh, LODs textures, animations, hit points, defense, attack, armor values.
	// And anything else.
	// This class also knows how to create a ScenarioPart (so far only SPGenericProp).
	// To create the Ogre graphical entity see COnfigLoaderGraphics in the client part of the solution.
	class ConfigLoaderEntity: public ConfigLoader
	{
	public:
		ConfigLoaderEntity();
		virtual ~ConfigLoaderEntity();
 
		inline static ConfigLoaderEntity &getSingleton() { return *singletonPtr; }
		inline static ConfigLoaderEntity *getSingletonPtr() { return singletonPtr; }
   
		virtual void parseScript(std::ifstream &stream);

		// If you give a uid it makes the part with that uid, else generate a next uid automatically.
		static TDLScenario::ScenarioPart * makeSPForEntity(std::string entityName, btVector3 & pos, btQuaternion & rot, int uid = -1);
		
		// Loading Ogre graphics for an entity is up in TDL client side code, ConfigLoadersGR.
 
	private:
		static ConfigLoaderEntity *singletonPtr;

	};

}
 
#endif

#include <hash_map>
#include <vector>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
#include "ConfigLoaders\ConfigLoaderEntity.h"

#include "Scenario\ScenarioPart.h"
#include "Scenario\Parts\SPGenericProp.h"

using namespace std;
using namespace stdext;

using namespace TDL;
using namespace TDLScenario;
using namespace TDLConfigLoaders;
 
ConfigLoaderEntity * ConfigLoaderEntity::singletonPtr = NULL;
 
ConfigLoaderEntity::ConfigLoaderEntity() :
	ConfigLoader(".entity")
{
    //Init singleton
    if (singletonPtr)
	{
        throw "Multiple ConfigLoaderEntity objects are not allowed, ConfigLoaderEntity::ConfigLoaderEntity()";
	}
    singletonPtr = this;
}
 
ConfigLoaderEntity::~ConfigLoaderEntity()
{
    singletonPtr = NULL; 
}

void ConfigLoaderEntity::parseScript(std::ifstream &stream)
{
	ConfigLoader::parseScript(stream);
}

ScenarioPart * ConfigLoaderEntity::makeSPForEntity(std::string entityName, btVector3 & pos, btQuaternion & rot, int uid)
{
	TDLConfigNode * rootN = getSingleton().getConfigScript("entity", entityName);
	
	if(rootN == NULL)
	{
		TDL_Log("ConfigLoaderEntity::makeSPForEntity: WARNING - Entity not found: %s", entityName.c_str());
		return NULL;
	}

	TDLConfigNode * classN = rootN->findChild("class");
	assert(classN != NULL);
	TDLConfigNode * lodN = rootN->findChild("lod");
	assert(lodN != NULL);
	float nearLOD = lodN->getValueF(0);
	float farLOD = lodN->getValueF(1);

	TDLConfigNode * sizeN = rootN->findChild("size");
	assert(sizeN != NULL);
	btVector3 size = sizeN->getValueV3();

	if(classN->getValue() == "prop")
	{
		if(uid == -1)
		{
			uid = ScenarioPartNextAutoUID++;
		}

		SPGenericProp * sp = new SPGenericProp(uid, entityName, pos, rot, size / 2.0, nearLOD, farLOD);
		return sp;
	}
	else
	{
		TDL_Log("WARNING - Entity class not found: %s", classN->getValue().c_str());
	}
	return NULL;
}


Bugfixed version by scrawl


This is a bugfixed version of the above ogre-independent parser (it had a bug in the _parseNodes functions which didn't call _nextToken after parsing a node, thus "hanging" at the closing brace and failing to add further nodes - this caused the behaviour that nodes with more than one child only had their first child parsed). This code was tested on GCC.

Besides, a few other changes have been made:

- changed stdext::hash_map to std::map to compile on all compilers.

- added a getAllConfigScripts method for convenience.

- fixed some includes.

ConfigLoader.hpp

#ifndef SH_CONFIG_LOADER_H__
#define SH_CONFIG_LOADER_H__

#include <map>
#include <vector>
#include <cassert>
#include <string>
 
namespace sh
{
    class ConfigNode;

	class ConfigLoader
	{
	public:
		static void loadAllFiles(ConfigLoader* c, const std::string& path);

		ConfigLoader(const std::string& fileEnding);
		virtual ~ConfigLoader();

		std::string m_fileEnding;

		// For a line like
		// entity animals/dog
		// {
		//    ...
		// }
		// The type is "entity" and the name is "animals/dog"
		// Or if animal/dog was not there then name is ""
		virtual ConfigNode *getConfigScript (const std::string &name);

		virtual std::map <std::string, ConfigNode*> getAllConfigScripts ();

		virtual void parseScript(std::ifstream &stream);


	protected:

		float m_LoadOrder;
		// like "*.object"

		std::map <std::string, ConfigNode*> m_scriptList;

		enum Token
		{
			TOKEN_Text,
			TOKEN_NewLine,
			TOKEN_OpenBrace,
			TOKEN_CloseBrace,
			TOKEN_EOF,
		};

		Token tok, lastTok;
		std::string tokVal, lastTokVal;

		void _parseNodes(std::ifstream &stream, ConfigNode *parent);
		void _nextToken(std::ifstream &stream);
		void _skipNewLines(std::ifstream &stream);

		virtual void clearScriptList();
	};

	class ConfigNode
	{
	public:
		ConfigNode(ConfigNode *parent, const std::string &name = "untitled");
		~ConfigNode();

		inline void setName(const std::string &name)
		{
			this->m_name = name;
		}

		inline std::string &getName()
		{
			return m_name;
		}

		inline void addValue(const std::string &value)
		{
			m_values.push_back(value);
		}

		inline void clearValues()
		{
			m_values.clear();
		}

		inline std::vector<std::string> &getValues()
		{
			return m_values;
		}

		inline const std::string &getValue(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return m_values[index];
		}

		ConfigNode *addChild(const std::string &name = "untitled", bool replaceExisting = false);
		ConfigNode *findChild(const std::string &name, bool recursive = false);

		inline std::vector<ConfigNode*> &getChildren()
		{
			return m_children;
		}

		inline ConfigNode *getChild(unsigned int index = 0)
		{
			assert(index < m_children.size());
			return m_children[index];
		}

		void setParent(ConfigNode *newParent);
 
		inline ConfigNode *getParent()
		{
			return m_parent;
		}

	private:
		std::string m_name;
		std::vector<std::string> m_values;
		std::vector<ConfigNode*> m_children;
		ConfigNode *m_parent;

		int m_lastChildFound;  //The last child node's index found with a call to findChild()

		std::vector<ConfigNode*>::iterator _iter;
		bool _removeSelf;
	};
 
}
 
#endif

#include "ConfigLoader.hpp"

#include <vector>
#include <map>
#include <exception>
#include <fstream>

#include <boost/filesystem.hpp>

namespace sh
{
	void ConfigLoader::loadAllFiles(ConfigLoader* c, const std::string& path)
	{
		for ( boost::filesystem::recursive_directory_iterator end, dir(path); dir != end; ++dir )
		{
			boost::filesystem::path p(*dir);
			if(p.extension() == c->m_fileEnding)
			{
				std::ifstream in((*dir).path().string().c_str(), std::ios::binary);
				c->parseScript(in);
			}
		}
	}

	ConfigLoader::ConfigLoader(const std::string& fileEnding)
	{
		//Register as a ScriptLoader
		m_fileEnding = fileEnding;
	}

	ConfigLoader::~ConfigLoader()
	{
		clearScriptList();

	}

	void ConfigLoader::clearScriptList()
	{
		std::map <std::string, ConfigNode *>::iterator i;
		for (i = m_scriptList.begin(); i != m_scriptList.end(); i++)
		{
			delete i->second;
		}
		m_scriptList.clear();
	}

	ConfigNode *ConfigLoader::getConfigScript(const std::string &name)
	{
		std::map <std::string, ConfigNode*>::iterator i;

		std::string key = name;
		i = m_scriptList.find(key);

		//If found..
		if (i != m_scriptList.end())
		{
			return i->second;
		}
		else
		{
			return NULL;
		}
	}

	std::map <std::string, ConfigNode*> ConfigLoader::getAllConfigScripts ()
	{
		return m_scriptList;
	}

	void ConfigLoader::parseScript(std::ifstream &stream)
	{
		//Get first token
		_nextToken(stream);
		if (tok == TOKEN_EOF)
		{
			stream.close();
			return;
		}

		//Parse the script
		_parseNodes(stream, 0);

		stream.close();
	}

	void ConfigLoader::_nextToken(std::ifstream &stream)
	{
		lastTok = tok;
		lastTokVal = tokVal;

		//EOF token
		if (stream.eof())
		{
			tok = TOKEN_EOF;
			return;
		}

		//(Get next character)
		int ch = stream.get();
		if (ch == -1)
		{
			tok = TOKEN_EOF;
			return;
		}
		while ((ch == ' ' || ch == 9) && !stream.eof())
		{    //Skip leading spaces / tabs
			ch = stream.get();
		}

		if (stream.eof())
		{
			tok = TOKEN_EOF;
			return;
		}

		//Newline token
		if (ch == '\r' || ch == '\n')
		{
			do
			{
				ch = stream.get();
			} while ((ch == '\r' || ch == '\n') && !stream.eof());

			stream.unget();

			tok = TOKEN_NewLine;
			return;
		}

		//Open brace token
		else if (ch == '{')
		{
			tok = TOKEN_OpenBrace;
			return;
		}

		//Close brace token
		else if (ch == '}')
		{
			tok = TOKEN_CloseBrace;
			return;
		}

		//Text token
		if (ch < 32 || ch > 122)    //Verify valid char
		{
			throw std::runtime_error("Parse Error: Invalid character, ConfigLoader::load()");
		}

		tokVal = "";
		tok = TOKEN_Text;
		do
		{
			//Skip comments
			if (ch == '/')
			{
				int ch2 = stream.peek();

				//C++ style comment (//)
				if (ch2 == '/')
				{
					stream.get();
					do
					{
						ch = stream.get();
					} while (ch != '\r' && ch != '\n' && !stream.eof());

					tok = TOKEN_NewLine;
					return;
				}
			}

			//Add valid char to tokVal
			tokVal += (char)ch;

			//Next char
			ch = stream.get();

		} while (ch > 32 && ch <= 122 && !stream.eof());

		stream.unget();

		return;
	}

	void ConfigLoader::_skipNewLines(std::ifstream &stream)
	{
		while (tok == TOKEN_NewLine)
		{
			_nextToken(stream);
		}
	}

	void ConfigLoader::_parseNodes(std::ifstream &stream, ConfigNode *parent)
	{
		typedef std::pair<std::string, ConfigNode*> ScriptItem;

		while (true)
		{
			switch (tok)
			{
				//Node
				case TOKEN_Text:
					//Add the new node
					ConfigNode *newNode;
					if (parent)
					{
						newNode = parent->addChild(tokVal);
					}
					else
					{
						newNode = new ConfigNode(0, tokVal);
					}

					//Get values
					_nextToken(stream);
					while (tok == TOKEN_Text)
					{
						newNode->addValue(tokVal);
						_nextToken(stream);
					}

					//Add root nodes to scriptList
					if (!parent){
						std::string key;

						if (newNode->getValues().empty())
						{
							key = newNode->getName() + ' ';
						}
						else
						{
							key = newNode->getName() + ' ' + newNode->getValues().front();
						}

						m_scriptList.insert(ScriptItem(key, newNode));
					}

					_skipNewLines(stream);

					//Add any sub-nodes
					if (tok == TOKEN_OpenBrace)
					{
						//Parse nodes
						_nextToken(stream);
						_parseNodes(stream, newNode);
						//Check for matching closing brace
						if (tok != TOKEN_CloseBrace)
						{
							throw std::runtime_error("Parse Error: Expecting closing brace");
						}
						_nextToken(stream);
						_skipNewLines(stream);
					}

					break;

				//Out of place brace
				case TOKEN_OpenBrace:
					throw std::runtime_error("Parse Error: Opening brace out of plane");
					break;

				//Return if end of nodes have been reached
				case TOKEN_CloseBrace:
					return;

				//Return if reached end of file
				case TOKEN_EOF:
					return;

				case TOKEN_NewLine:
					_nextToken(stream);
					break;
			}
		};
	}

	ConfigNode::ConfigNode(ConfigNode *parent, const std::string &name)
	{
		m_name = name;
		m_parent = parent;
		_removeSelf = true;    //For proper destruction
		m_lastChildFound = -1;

		//Add self to parent's child list (unless this is the root node being created)
		if (parent != NULL)
		{
			m_parent->m_children.push_back(this);
			_iter = --(m_parent->m_children.end());
		}
	}

	ConfigNode::~ConfigNode()
	{
		//Delete all children
		std::vector<ConfigNode*>::iterator i;
		for (i = m_children.begin(); i != m_children.end(); i++)
		{
			ConfigNode *node = *i;
			node->_removeSelf = false;
			delete node;
		}
		m_children.clear();

		//Remove self from parent's child list
		if (_removeSelf && m_parent != NULL)
		{
			m_parent->m_children.erase(_iter);
		}
	}

	ConfigNode *ConfigNode::addChild(const std::string &name, bool replaceExisting)
	{
		if (replaceExisting)
		{
			ConfigNode *node = findChild(name, false);
			if (node)
			{
				return node;
			}
		}
		return new ConfigNode(this, name);
	}

	ConfigNode *ConfigNode::findChild(const std::string &name, bool recursive)
	{
		int indx, prevC, nextC;
		int childCount = (int)m_children.size();

		if (m_lastChildFound != -1)
		{
			//If possible, try checking the nodes neighboring the last successful search
			//(often nodes searched for in sequence, so this will boost search speeds).
			prevC = m_lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
			nextC = m_lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
			for (indx = prevC; indx <= nextC; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name)
				{
					m_lastChildFound = indx;
					return node;
				}
			}

			//If not found that way, search for the node from start to finish, avoiding the
			//already searched area above.
			for (indx = nextC + 1; indx < childCount; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
			for (indx = 0; indx < prevC; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
		}
		else
		{
			//Search for the node from start to finish
			for (indx = 0; indx < childCount; ++indx){
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
		}

		//If not found, search child nodes (if recursive == true)
		if (recursive)
		{
			for (indx = 0; indx < childCount; ++indx)
			{
				m_children[indx]->findChild(name, recursive);
			}
		}

		//Not found anywhere
		return NULL;
	}

	void ConfigNode::setParent(ConfigNode *newParent)
	{
		//Remove self from current parent
		m_parent->m_children.erase(_iter);

		//Set new parent
		m_parent = newParent;

		//Add self to new parent
		m_parent->m_children.push_back(this);
		_iter = --(m_parent->m_children.end());
	}
}


Alias: All-purpose_script_parser