Custom Script Parser?

In many games, object data is stored in external files for easy adjustment and management. For example, in a racing game, each car might have it's own configuration file that specifies it's top speed, turning radius, etc. Many programmers use XML or a binary format to store this information.

If you've ever written a .material script for Ogre, you probably know that Ogre's C-style script syntax is very intuitive and compact. Now you can store any kind of data you want in Ogre C-style scripts, and and quickly and easily load them with this parser.

Like Ogre's .material scripts, this will automatically parse all your scripts when a resource group is loaded. Since all scripts are pre-parsed, the system can access scripts not by filename, but by script name; in other words, you can include multiple scripts per file and still access them individually.

This script parser was designed to be very lightweight and fast; it's only a few KBs of code, and can parse around 10 MB per second on an average PC (if your average script is ~1 KB, you can load around 10,000 scripts in one second).

Using the Parser

Using this script parser is very easy. Simply initialize the system by creating a ConfigScriptLoader instance. You don't even have to keep a pointer to the instance because ConfigScriptLoader is a singleton class:

new ConfigScriptLoader();

Make sure you create a ConfigScriptLoader class before loading any resource groups, otherwise it will miss them and your scripts won't get loaded.

Now, after loading your resource groups, the ConfigScriptLoader will have automatically parsed your scripts for you! All you need to do now is request the desired script, and process it's contents. Getting a script is simple:

ConfigNode *rootNode;
 rootNode = ConfigScriptLoader::getSingleton().getConfigScript("entity", "Crate");

ConfigScriptLoader::getConfigScript() will retrieve the root node of the specified script, which you can use to access any of it's data.

You may notice that the getConfigScript() requires two names; one for the script type, and another for the actual name. For example, the above code would load this script:

entity Crate{
   position 100 50 200
   rotation 0 0 0
   scale 1 1 1
 }

You may be wondering what file this script is coming from. The fact is, it doesn't matter! As long as your scripts have a .object extension (and this can be changed if you modify line 22 of ConfigScript.cpp), their filename is completely irrelevant. Since you access scripts by script name rather than file name, it makes no difference where the script is located. This is a big advantage since this allows you to organize your scripts any way you want without worrying about your game not being able to find it.

Now that you have a pointer to the root ConfigNode, you can access any of the data it contains. For example, to access the Y position coordinate (50) given in the above script example, you first find the "position" node, then access it's 2nd value (as you can see it contains 3 values in all: 100, 50, and 200).

float positionY = rootNode->findChild("position")->getValueF(1);

1 is used to access the 2nd position value since std::vector is 0-based (like any other C array). getValueF() is a variation of getValue() that automatically converts the value to a float. Other variations are included for doubles, ints, etc.

Note: When you shut down your application, don't forget to delete the ConfigScriptLoader instance - Ogre won't do this for you, so failing to do so will result in a memory leak. Since it's a singleton class, you can always delete it using this code:

delete ConfigScriptLoader::getSingletonPtr();

Source Files



ConfigScript.h

//This code is public domain - you can do whatever you want with it
//Original author: John Judnich
#ifndef _CONFIGSCRIPT_H__
#define _CONFIGSCRIPT_H__

#include <OgreScriptLoader.h>
#include <OgreStringConverter.h>
#include <hash_map>
#include <vector>

class ConfigNode;

class ConfigScriptLoader: public Ogre::ScriptLoader
{
public:
    ConfigScriptLoader();
    ~ConfigScriptLoader();

    inline static ConfigScriptLoader &getSingleton() { return *singletonPtr; }
    inline static ConfigScriptLoader *getSingletonPtr() { return singletonPtr; }

    Ogre::Real getLoadingOrder() const;
    const Ogre::StringVector &getScriptPatterns() const;

    ConfigNode *getConfigScript(const Ogre::String &type, const Ogre::String &name);

    void parseScript(Ogre::DataStreamPtr &stream, const Ogre::String &groupName);
    
private:
    static ConfigScriptLoader *singletonPtr;

    Ogre::Real mLoadOrder;
    Ogre::StringVector mScriptPatterns;

    stdext::hash_map<Ogre::String, ConfigNode*> scriptList;
    
    //Parsing
    char *parseBuff, *parseBuffEnd, *buffPtr;
    size_t parseBuffLen;

    enum Token
    {
        TOKEN_Text,
        TOKEN_NewLine,
        TOKEN_OpenBrace,
        TOKEN_CloseBrace,
        TOKEN_EOF,
    };

    Token tok, lastTok;
    Ogre::String tokVal, lastTokVal;
    char *lastTokPos;

    void _parseNodes(ConfigNode *parent);
    void _nextToken();
    void _prevToken();
};

class ConfigNode
{
public:
    ConfigNode(ConfigNode *parent, const Ogre::String &name = "untitled");
    ~ConfigNode();

    inline void setName(const Ogre::String &name)
    {
        this->name = name;
    }

    inline Ogre::String &getName()
    {
        return name;
    }

    inline void addValue(const Ogre::String &value)
    {
        values.push_back(value);
    }

    inline void clearValues()
    {
        values.clear();
    }

    inline std::vector<Ogre::String> &getValues()
    {
        return values;
    }

    inline const Ogre::String &getValue(unsigned int index = 0)
    {
        assert(index < values.size());
        return values[index];
    }

    inline float getValueF(unsigned int index = 0)
    {
        assert(index < values.size());
        return Ogre::StringConverter::parseReal(values[index]);
    }

    inline double getValueD(unsigned int index = 0)
    {
        assert(index < values.size());

        std::istringstream str(values[index]);
        double ret = 0;
        str >> ret;
        return ret;
    }

    inline int getValueI(unsigned int index = 0)
    {
        assert(index < values.size());
        return Ogre::StringConverter::parseInt(values[index]);
    }
    
    ConfigNode *addChild(const Ogre::String &name = "untitled", bool replaceExisting = false);
    ConfigNode *findChild(const Ogre::String &name, bool recursive = false);

    inline std::vector<ConfigNode*> &getChildren()
    {
        return children;
    }

    inline ConfigNode *getChild(unsigned int index = 0)
    {
        assert(index < children.size());
        return children[index];
    }
    
    void setParent(ConfigNode *newParent);

    inline ConfigNode *getParent()
    {
        return parent;
    }

private:
    Ogre::String name;
    std::vector<Ogre::String> values;
    std::vector<ConfigNode*> children;
    ConfigNode *parent;

    int lastChildFound;  //The last child node's index found with a call to findChild()

    std::vector<ConfigNode*>::iterator _iter;
    bool _removeSelf;
};

#endif


ConfigScript.cpp

#include "ConfigScript.h"
#include "Exception.h"

#include <OgreScriptLoader.h>
#include <OgreScriptLoader.h>
#include <OgreResourceGroupManager.h>
using namespace Ogre;

#include <vector>
#include <hash_map>
using namespace std;
using namespace stdext;

ConfigScriptLoader *ConfigScriptLoader::singletonPtr = NULL;

ConfigScriptLoader::ConfigScriptLoader()
{
    //Init singleton
    if (singletonPtr)
        EXCEPTION("Multiple ConfigScriptManager objects are not allowed", "ConfigScriptManager::ConfigScriptManager()");
    singletonPtr = this;

    //Register as a ScriptLoader
    mLoadOrder = 100.0f;
    mScriptPatterns.push_back("*.object");
    ResourceGroupManager::getSingleton()._registerScriptLoader(this);
}

ConfigScriptLoader::~ConfigScriptLoader()
{
    singletonPtr = NULL;

    //Delete all scripts
    stdext::hash_map<String, ConfigNode*>::iterator i;
    for (i = scriptList.begin(); i != scriptList.end(); i++){
        delete i->second;
    }
    scriptList.clear();

    //Unregister with resource group manager
    if (ResourceGroupManager::getSingletonPtr())
        ResourceGroupManager::getSingleton()._unregisterScriptLoader(this);
}

Real ConfigScriptLoader::getLoadingOrder() const
{
    return mLoadOrder;
}

const StringVector &ConfigScriptLoader::getScriptPatterns() const
{
    return mScriptPatterns;
}

ConfigNode *ConfigScriptLoader::getConfigScript(const String &type, const String &name)
{
    stdext::hash_map<String, ConfigNode*>::iterator i;

    String key = type + ' ' + name;
    i = scriptList.find(key);

    //If found..
    if (i != scriptList.end())
        return i->second;
    else
        return NULL;
}

void ConfigScriptLoader::parseScript(DataStreamPtr &stream, const String &groupName)
{
    //Copy the entire file into a buffer for fast access
    parseBuffLen = stream->size();
    parseBuff = new char[parseBuffLen];
    buffPtr = parseBuff;
    stream->read(parseBuff, parseBuffLen);
    parseBuffEnd = parseBuff + parseBuffLen;

    //Close the stream (it's no longer needed since everything is in parseBuff)
    stream->close();

    //Get first token
    _nextToken();
    if (tok == TOKEN_EOF)
        return;
    
    //Parse the script
    _parseNodes(0);

    if (tok == TOKEN_CloseBrace)
        EXCEPTION("Parse Error: Closing brace out of place", "ConfigScript::load()");
    
    //Delete the buffer
    delete[] parseBuff;
}

void ConfigScriptLoader::_nextToken()
{
    lastTok = tok;
    lastTokVal = tokVal;
    lastTokPos = buffPtr;

    //EOF token
    if (buffPtr >= parseBuffEnd){
        tok = TOKEN_EOF;
        return;
    }

    //(Get next character)
    int ch = *buffPtr++;
    while (ch == ' ' || ch == 9){    //Skip leading spaces / tabs
        ch = *buffPtr++;
    }

    //Newline token
    if (ch == '\r' || ch == '\n'){
        do {
            ch = *buffPtr++;
        } while ((ch == '\r' || ch == '\n') && buffPtr < parseBuffEnd);
        buffPtr--;

        tok = TOKEN_NewLine;
        return;
    }

    //Open brace token
    else if (ch == '{'){
        tok = TOKEN_OpenBrace;
        return;
    }

    //Close brace token
    else if (ch == '}'){
        tok = TOKEN_CloseBrace;
        return;
    }

    //Text token
    if (ch < 32 || ch > 122)    //Verify valid char
        EXCEPTION("Parse Error: Invalid character", "ConfigScript::load()");

    tokVal = "";
    tok = TOKEN_Text;
    do {
        //Skip comments
        if (ch == '/'){
            int ch2 = *buffPtr;
            
            //C++ style comment (//)
            if (ch2 == '/'){
                buffPtr++;
                do {
                    ch = *buffPtr++;
                } while (ch != '\r' && ch != '\n' && buffPtr < parseBuffEnd);
                
                tok = TOKEN_NewLine;
                return;
            }
        }

        //Add valid char to tokVal
        tokVal += ch;

        //Next char
        ch = *buffPtr++;
    } while (ch > 32 && ch <= 122 && buffPtr < parseBuffEnd);
    buffPtr--;

    return;
}

void ConfigScriptLoader::_prevToken()
{
    tok = lastTok;
    tokVal = lastTokVal;
    buffPtr = lastTokPos;
}

void ConfigScriptLoader::_parseNodes(ConfigNode *parent)
{
    typedef std::pair<String, ConfigNode*> ScriptItem;

    while (1) {
        switch (tok){
            //Node
            case TOKEN_Text:
                //Add the new node
                ConfigNode *newNode;
                if (parent)
                    newNode = parent->addChild(tokVal);
                else
                    newNode = new ConfigNode(0, tokVal);

                //Get values
                _nextToken();
                while (tok == TOKEN_Text){
                    newNode->addValue(tokVal);
                    _nextToken();
                }

                //Add root nodes to scriptList
                if (!parent){
                    String key;

                    if (newNode->getValues().empty())
                        key = newNode->getName() + ' ';
                    else
                        key = newNode->getName() + ' ' + newNode->getValues().front();

                    scriptList.insert(ScriptItem(key, newNode));
                }

                //Skip any blank spaces
                while (tok == TOKEN_NewLine)
                    _nextToken();
                
                //Add any sub-nodes
                if (tok == TOKEN_OpenBrace){
                    //Parse nodes
                    _nextToken();
                    _parseNodes(newNode);
                    
                    //Skip blank spaces
                    while (tok == TOKEN_NewLine)
                        _nextToken();

                    //Check for matching closing brace
                    if (tok != TOKEN_CloseBrace)
                        EXCEPTION("Parse Error: Expecting closing brace", "ConfigScript::load()");
                } else {
                    //If it's not a opening brace, back up so the system will parse it properly
                    _prevToken();
                }
                                
                break;

            //Out of place brace
            case TOKEN_OpenBrace:
                EXCEPTION("Parse Error: Opening brace out of plane", "ConfigScript::load()");
                break;

            //Return if end of nodes have been reached
            case TOKEN_CloseBrace:
                return;

            //Return if reached end of file
            case TOKEN_EOF:
                return;
        }
    
        //Next token
        _nextToken();
    };
}

ConfigNode::ConfigNode(ConfigNode *parent, const String &name)
{
    ConfigNode::name = name;
    ConfigNode::parent = parent;
    _removeSelf = true;    //For proper destruction
    lastChildFound = -1;

    //Add self to parent's child list (unless this is the root node being created)
    if (parent != NULL){
        parent->children.push_back(this);
        _iter = --(parent->children.end());
    }
}

ConfigNode::~ConfigNode()
{
    //Delete all children
    std::vector<ConfigNode*>::iterator i;
    for (i = children.begin(); i != children.end(); i++){
        ConfigNode *node = *i;
        node->_removeSelf = false;
        delete node;
    }
    children.clear();

    //Remove self from parent's child list
    if (_removeSelf && parent != NULL)
        parent->children.erase(_iter);
}

ConfigNode *ConfigNode::addChild(const String &name, bool replaceExisting)
{
    if (replaceExisting) {
        ConfigNode *node = findChild(name, false);
        if (node)
            return node;
    }
    return new ConfigNode(this, name);
}

ConfigNode *ConfigNode::findChild(const String &name, bool recursive)
{
    int indx, prevC, nextC;
    int childCount = (int)children.size();

    if (lastChildFound != -1){
        //If possible, try checking the nodes neighboring the last successful search
        //(often nodes searched for in sequence, so this will boost search speeds).
        prevC = lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
        nextC = lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
        for (indx = prevC; indx <= nextC; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }

        //If not found that way, search for the node from start to finish, avoiding the
        //already searched area above.
        for (indx = nextC + 1; indx < childCount; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
        for (indx = 0; indx < prevC; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
    } else {
        //Search for the node from start to finish
        for (indx = 0; indx < childCount; ++indx){
            ConfigNode *node = children[indx];
            if (node->name == name) {
                lastChildFound = indx;
                return node;
            }
        }
    }

    //If not found, search child nodes (if recursive == true)
    if (recursive){
        for (indx = 0; indx < childCount; ++indx){
            children[indx]->findChild(name, recursive);
        }
    }

    //Not found anywhere
    return NULL;
}

void ConfigNode::setParent(ConfigNode *newParent)
{
    //Remove self from current parent
    parent->children.erase(_iter);

    //Set new parent
    parent = newParent;

    //Add self to new parent
    parent->children.push_back(this);
    _iter = --(parent->children.end());
}

Porting Code

Our project uses Ogre on the client but not on the server. If you need to fetch resource files on the server side, you may try to port this code. The above code assumes the input reader does onto convert CR-LF to CR, which happens on Windows systems. It also make the assumption, "we read the whole file and then process the resulting buffer." This ignores the issue of CR-LF and also ignores the normal buffered input stream available in C++. And there is a problem with the order of getting new tokens that breaks on some files.

There is also a mistake related to the last character in the file being a }
_parseNodes. Before the "//Check for matching closing brace" it skips white space, but then the check for TOKEN_NewLine fails. The order there should be reversed.

Non Ogre Version

Here is a version of the code that is not dependent at all on Ogre. You will have to modify it for you use and remove the TDL_ hooks, and it depends on boost. You can remove the boost parts easily. It also has a few corrections to the above code.

The class ConfigLoaderEntity is at the bottom. ConfigLoaderCategory is nearly the same.

#ifndef _TDL_CONFIG_LOADER_H__
#define _TDL_CONFIG_LOADER_H__
 
#pragma once

#include <hash_map>
#include <vector>

namespace TDLConfigLoaders
{
	class TDLConfigNode;

	// The base class of loaders that read Ogre style script files to get configuration and settings for TDL.
	class ConfigLoader
	{
	public:
		static void loadAllFiles();
		static void scanLoadFiles(ConfigLoader * c);

		ConfigLoader(std::string fileEnding, float loadOrder = 100.0f);
		virtual ~ConfigLoader();
 
		std::string m_fileEnding;
 
		// For a line like
		// entity animals/dog
		// {
		//    ...
		// }
		// The type is "entity" and the name is "animals/dog"
		// Or if anima/dog was not there then name is ""
		virtual TDLConfigNode *getConfigScript(const std::string &type, const std::string &name);
 
		virtual void parseScript(std::ifstream &stream) = 0;

 
	protected:

		float m_LoadOrder;
		// like "*.object"
 
		stdext::hash_map<std::string, TDLConfigNode*> m_scriptList;
  
		enum Token
		{
			TOKEN_Text,
			TOKEN_NewLine,
			TOKEN_OpenBrace,
			TOKEN_CloseBrace,
			TOKEN_EOF,
		};
 
		Token tok, lastTok;
		std::string tokVal, lastTokVal;
 
		void _parseNodes(std::ifstream &stream, TDLConfigNode *parent);
		void _nextToken(std::ifstream &stream);
		void _skipNewLines(std::ifstream &stream);

		virtual void clearScriptList();
	};
 
	class TDLConfigNode
	{
	public:
		TDLConfigNode(TDLConfigNode *parent, const std::string &name = "untitled");
		~TDLConfigNode();
 
		inline void setName(const std::string &name)
		{
			this->m_name = name;
		}
 
		inline std::string &getName()
		{
			return m_name;
		}
 
		inline void addValue(const std::string &value)
		{
			m_values.push_back(value);
		}
 
		inline void clearValues()
		{
			m_values.clear();
		}
 
		inline std::vector<std::string> &getValues()
		{
			return m_values;
		}
 
		inline const std::string &getValue(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return m_values[index];
		}
 
		inline float getValueF(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return (float)::atof(m_values[index].c_str());
		}
 
		inline double getValueD(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return ::atof(m_values[index].c_str());
		}

		inline int getValueI(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return ::atoi(m_values[index].c_str());
		}
 
		inline btScalar getValueBtScalar(unsigned int index = 0)
		{
			return parseBtScalar(m_values[index]);
		}
 
		inline btVector3 getValueV3(unsigned int index = 0)
		{
			assert(index < m_values.size() - 3);
			return btVector3(parseBtScalar(m_values[index]), parseBtScalar(m_values[index + 1]), parseBtScalar(m_values[index + 2]));
		}

		inline btQuaternion getValueYPR(unsigned int index = 0)
		{
			assert(index < m_values.size() - 3);
			return btQuaternion(parseBtDegrees(m_values[index]), parseBtDegrees(m_values[index + 1]), parseBtDegrees(m_values[index + 2]));
		}

		inline double parseDouble(std::string s)
		{
			double d;
			int rc = sscanf(s.c_str(), "%lf", &d);
			if(rc == 1)
			{
				return d;
			}
			assert(false);
			return 1.0;
		}
 
		inline btScalar parseBtScalar(std::string s)
		{
			double d;
			int rc = sscanf(s.c_str(), "%lf", &d);
			if(rc == 1)
			{
				return btScalar(d);
			}
			assert(false);
			return btScalar(1.0);
		}

		// s is like 90.0 degrees. returns btScalar radians.
		inline btScalar parseBtDegrees(std::string s)
		{
			return btRadians(parseBtScalar(s));
		}
 
		// s is like 1.7 radians. returns btScalar radians.
		inline btScalar parseBtRadians(std::string s)
		{
			return parseBtScalar(s);
		}
 
		TDLConfigNode *addChild(const std::string &name = "untitled", bool replaceExisting = false);
		TDLConfigNode *findChild(const std::string &name, bool recursive = false);
 
		inline std::vector<TDLConfigNode*> &getChildren()
		{
			return m_children;
		}
 
		inline TDLConfigNode *getChild(unsigned int index = 0)
		{
			assert(index < m_children.size());
			return m_children[index];
		}
 
		void setParent(TDLConfigNode *newParent);
 
		inline TDLConfigNode *getParent()
		{
			return m_parent;
		}
 
	private:
		std::string m_name;
		std::vector<std::string> m_values;
		std::vector<TDLConfigNode*> m_children;
		TDLConfigNode *m_parent;
 
		int m_lastChildFound;  //The last child node's index found with a call to findChild()
 
		std::vector<TDLConfigNode*>::iterator _iter;
		bool _removeSelf;
	};

}
 
#endif

#include <vector>
#include <hash_map>
#include <exception>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
#include "ConfigLoaders\ConfigLoaderCategory.h"
#include "ConfigLoaders\ConfigLoaderEntity.h"

#include "boost/filesystem.hpp"
#include "boost/filesystem/operations.hpp" 
#include "boost/filesystem/fstream.hpp"

using namespace std;
using namespace stdext;
using namespace TDLConfigLoaders;
using namespace boost::filesystem;  
using namespace boost::system;


void ConfigLoader::loadAllFiles()
{
	ConfigLoaderCategory * cat = new ConfigLoaderCategory();
	ConfigLoaderEntity * ent = new ConfigLoaderEntity();

	scanLoadFiles(cat);
	scanLoadFiles(ent);
}

void ConfigLoader::scanLoadFiles(ConfigLoader * c)
{
	for ( boost::filesystem::recursive_directory_iterator end, dir("../content"); 
       dir != end; ++dir ) {
		   //TDL_Log("%s", (*dir).string().c_str()); 
		   path p(*dir);
		   if(p.extension() == c->m_fileEnding)
		   {
			   std::ifstream in((*dir).string().c_str(), ios::binary);
			   c->parseScript(in);
		   }
	}
}

ConfigLoader::ConfigLoader(std::string fileEnding, float loadOrder)
{ 
    //Register as a ScriptLoader
    m_LoadOrder = loadOrder;
	m_fileEnding = fileEnding;
}
 
ConfigLoader::~ConfigLoader()
{
    clearScriptList();

}

void ConfigLoader::clearScriptList()
{
    stdext::hash_map<std::string, TDLConfigNode *>::iterator i;
    for (i = m_scriptList.begin(); i != m_scriptList.end(); i++)
	{
        delete i->second;
    }
    m_scriptList.clear();
}
 
TDLConfigNode *ConfigLoader::getConfigScript(const std::string &type, const std::string &name)
{
    stdext::hash_map<std::string, TDLConfigNode*>::iterator i;
 
    std::string key = type + ' ' + name;
    i = m_scriptList.find(key);
 
    //If found..
    if (i != m_scriptList.end())
	{
        return i->second;
	}
    else
	{
        return NULL;
	}
}

void ConfigLoader::parseScript(std::ifstream &stream)
{ 
    //Get first token
    _nextToken(stream);
    if (tok == TOKEN_EOF)
	{
	    stream.close();
        return;
	}
 
    //Parse the script
    _parseNodes(stream, 0);
 
    stream.close();
}
 
void ConfigLoader::_nextToken(std::ifstream &stream)
{
    lastTok = tok;
    lastTokVal = tokVal;
 
    //EOF token
	if (stream.eof())
	{
        tok = TOKEN_EOF;
        return;
    }
 
    //(Get next character)
	int ch = stream.get();
    while ((ch == ' ' || ch == 9) && !stream.eof())
	{    //Skip leading spaces / tabs
        ch = stream.get();
    }

	if (stream.eof())
	{
        tok = TOKEN_EOF;
        return;
    }
 
    //Newline token
    if (ch == '\r' || ch == '\n')
	{
        do 
		{
            ch = stream.get();
		} while ((ch == '\r' || ch == '\n') && !stream.eof());

		stream.unget();
 
        tok = TOKEN_NewLine;
        return;
    }
 
    //Open brace token
    else if (ch == '{')
	{
        tok = TOKEN_OpenBrace;
        return;
    }
 
    //Close brace token
    else if (ch == '}')
	{
        tok = TOKEN_CloseBrace;
        return;
    }
 
    //Text token
    if (ch < 32 || ch > 122)    //Verify valid char
	{
        throw "Parse Error: Invalid character, ConfigLoader::load()";
	}
 
    tokVal = "";
    tok = TOKEN_Text;
    do 
	{
        //Skip comments
        if (ch == '/')
		{
			int ch2 = stream.peek();
 
            //C++ style comment (//)
            if (ch2 == '/')
			{
				stream.get();
                do 
				{
                    ch = stream.get();
                } while (ch != '\r' && ch != '\n' && !stream.eof());
 
                tok = TOKEN_NewLine;
                return;
            }
        }
 
        //Add valid char to tokVal
        tokVal += (char)ch;
 
        //Next char
        ch = stream.get();

    } while (ch > 32 && ch <= 122 && !stream.eof());

	stream.unget();
 
    return;
}
 
void ConfigLoader::_skipNewLines(std::ifstream &stream)
{
	while (tok == TOKEN_NewLine)
	{
		_nextToken(stream);
	}
}

void ConfigLoader::_parseNodes(std::ifstream &stream, TDLConfigNode *parent)
{
    typedef std::pair<std::string, TDLConfigNode*> ScriptItem;
 
    while (true) 
	{
        switch (tok)
		{
            //Node
            case TOKEN_Text:
                //Add the new node
                TDLConfigNode *newNode;
                if (parent)
				{
                    newNode = parent->addChild(tokVal);
				}
                else
				{
                    newNode = new TDLConfigNode(0, tokVal);
				}
 
                //Get values
                _nextToken(stream);
                while (tok == TOKEN_Text)
				{
                    newNode->addValue(tokVal);
                    _nextToken(stream);
                }
 
                //Add root nodes to scriptList
                if (!parent){
                    std::string key;
 
                    if (newNode->getValues().empty())
					{
                        key = newNode->getName() + ' ';
					}
                    else
					{
                        key = newNode->getName() + ' ' + newNode->getValues().front();
					}
 
                    m_scriptList.insert(ScriptItem(key, newNode));
                }
 
                _skipNewLines(stream);
 
                //Add any sub-nodes
                if (tok == TOKEN_OpenBrace)
				{
                    //Parse nodes
                    _nextToken(stream);
                    _parseNodes(stream, newNode);
 
                    //Check for matching closing brace
                    if (tok != TOKEN_CloseBrace)
					{
                        throw "Parse Error: Expecting closing brace";
					}

					_skipNewLines(stream);
                } 
 
                break;
 
            //Out of place brace
            case TOKEN_OpenBrace:
                throw "Parse Error: Opening brace out of plane";
                break;
 
            //Return if end of nodes have been reached
            case TOKEN_CloseBrace:
                return;
 
            //Return if reached end of file
            case TOKEN_EOF:
                return;

			case TOKEN_NewLine:
				_nextToken(stream);
				break;
        }
    };
}
 
TDLConfigNode::TDLConfigNode(TDLConfigNode *parent, const std::string &name)
{
    m_name = name;
    m_parent = parent;
    _removeSelf = true;    //For proper destruction
    m_lastChildFound = -1;
 
    //Add self to parent's child list (unless this is the root node being created)
    if (parent != NULL)
	{
        m_parent->m_children.push_back(this);
        _iter = --(m_parent->m_children.end());
    }
}
 
TDLConfigNode::~TDLConfigNode()
{
    //Delete all children
    std::vector<TDLConfigNode*>::iterator i;
    for (i = m_children.begin(); i != m_children.end(); i++)
	{
        TDLConfigNode *node = *i;
        node->_removeSelf = false;
        delete node;
    }
    m_children.clear();
 
    //Remove self from parent's child list
    if (_removeSelf && m_parent != NULL)
	{
        m_parent->m_children.erase(_iter);
	}
}
 
TDLConfigNode *TDLConfigNode::addChild(const std::string &name, bool replaceExisting)
{
    if (replaceExisting) 
	{
        TDLConfigNode *node = findChild(name, false);
        if (node)
		{
            return node;
		}
    }
    return new TDLConfigNode(this, name);
}
 
TDLConfigNode *TDLConfigNode::findChild(const std::string &name, bool recursive)
{
    int indx, prevC, nextC;
    int childCount = (int)m_children.size();
 
    if (m_lastChildFound != -1)
	{
        //If possible, try checking the nodes neighboring the last successful search
        //(often nodes searched for in sequence, so this will boost search speeds).
        prevC = m_lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
        nextC = m_lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
        for (indx = prevC; indx <= nextC; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) 
			{
                m_lastChildFound = indx;
                return node;
            }
        }
 
        //If not found that way, search for the node from start to finish, avoiding the
        //already searched area above.
        for (indx = nextC + 1; indx < childCount; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
        for (indx = 0; indx < prevC; ++indx)
		{
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
    } 
	else 
	{
        //Search for the node from start to finish
        for (indx = 0; indx < childCount; ++indx){
            TDLConfigNode *node = m_children[indx];
            if (node->m_name == name) {
                m_lastChildFound = indx;
                return node;
            }
        }
    }
 
    //If not found, search child nodes (if recursive == true)
    if (recursive)
	{
        for (indx = 0; indx < childCount; ++indx)
		{
            m_children[indx]->findChild(name, recursive);
        }
    }
 
    //Not found anywhere
    return NULL;
}
 
void TDLConfigNode::setParent(TDLConfigNode *newParent)
{
    //Remove self from current parent
    m_parent->m_children.erase(_iter);
 
    //Set new parent
    m_parent = newParent;
 
    //Add self to new parent
    m_parent->m_children.push_back(this);
    _iter = --(m_parent->m_children.end());
}

#ifndef _CONFIG_LOADER_ENTITY_H__
#define _CONFIG_LOADER_ENTITY_H__
 
#pragma once

#include <hash_map>
#include <vector>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
 
namespace TDLConfigLoaders
{
	class TDLConfigNode;

	// Any entity in the game.  Include simple unanimated models, animated models, NPCs.
	// Inlcudes the mesh, LODs textures, animations, hit points, defense, attack, armor values.
	// And anything else.
	// This class also knows how to create a ScenarioPart (so far only SPGenericProp).
	// To create the Ogre graphical entity see COnfigLoaderGraphics in the client part of the solution.
	class ConfigLoaderEntity: public ConfigLoader
	{
	public:
		ConfigLoaderEntity();
		virtual ~ConfigLoaderEntity();
 
		inline static ConfigLoaderEntity &getSingleton() { return *singletonPtr; }
		inline static ConfigLoaderEntity *getSingletonPtr() { return singletonPtr; }
   
		virtual void parseScript(std::ifstream &stream);

		// If you give a uid it makes the part with that uid, else generate a next uid automatically.
		static TDLScenario::ScenarioPart * makeSPForEntity(std::string entityName, btVector3 & pos, btQuaternion & rot, int uid = -1);
		
		// Loading Ogre graphics for an entity is up in TDL client side code, ConfigLoadersGR.
 
	private:
		static ConfigLoaderEntity *singletonPtr;

	};

}
 
#endif

#include <hash_map>
#include <vector>

#include "TDLCommon.h"
#include "ConfigLoaders\ConfigLoader.h"
#include "ConfigLoaders\ConfigLoaderEntity.h"

#include "Scenario\ScenarioPart.h"
#include "Scenario\Parts\SPGenericProp.h"

using namespace std;
using namespace stdext;

using namespace TDL;
using namespace TDLScenario;
using namespace TDLConfigLoaders;
 
ConfigLoaderEntity * ConfigLoaderEntity::singletonPtr = NULL;
 
ConfigLoaderEntity::ConfigLoaderEntity() :
	ConfigLoader(".entity")
{
    //Init singleton
    if (singletonPtr)
	{
        throw "Multiple ConfigLoaderEntity objects are not allowed, ConfigLoaderEntity::ConfigLoaderEntity()";
	}
    singletonPtr = this;
}
 
ConfigLoaderEntity::~ConfigLoaderEntity()
{
    singletonPtr = NULL; 
}

void ConfigLoaderEntity::parseScript(std::ifstream &stream)
{
	ConfigLoader::parseScript(stream);
}

ScenarioPart * ConfigLoaderEntity::makeSPForEntity(std::string entityName, btVector3 & pos, btQuaternion & rot, int uid)
{
	TDLConfigNode * rootN = getSingleton().getConfigScript("entity", entityName);
	
	if(rootN == NULL)
	{
		TDL_Log("ConfigLoaderEntity::makeSPForEntity: WARNING - Entity not found: %s", entityName.c_str());
		return NULL;
	}

	TDLConfigNode * classN = rootN->findChild("class");
	assert(classN != NULL);
	TDLConfigNode * lodN = rootN->findChild("lod");
	assert(lodN != NULL);
	float nearLOD = lodN->getValueF(0);
	float farLOD = lodN->getValueF(1);

	TDLConfigNode * sizeN = rootN->findChild("size");
	assert(sizeN != NULL);
	btVector3 size = sizeN->getValueV3();

	if(classN->getValue() == "prop")
	{
		if(uid == -1)
		{
			uid = ScenarioPartNextAutoUID++;
		}

		SPGenericProp * sp = new SPGenericProp(uid, entityName, pos, rot, size / 2.0, nearLOD, farLOD);
		return sp;
	}
	else
	{
		TDL_Log("WARNING - Entity class not found: %s", classN->getValue().c_str());
	}
	return NULL;
}


Bugfixed version by scrawl


This is a bugfixed version of the above ogre-independent parser (it had a bug in the _parseNodes functions which didn't call _nextToken after parsing a node, thus "hanging" at the closing brace and failing to add further nodes - this caused the behaviour that nodes with more than one child only had their first child parsed). This code was tested on GCC.

Besides, a few other changes have been made:

- changed stdext::hash_map to std::map to compile on all compilers.

- added a getAllConfigScripts method for convenience.

- fixed some includes.

ConfigLoader.hpp

#ifndef SH_CONFIG_LOADER_H__
#define SH_CONFIG_LOADER_H__

#include <map>
#include <vector>
#include <cassert>
#include <string>
 
namespace sh
{
    class ConfigNode;

	class ConfigLoader
	{
	public:
		static void loadAllFiles(ConfigLoader* c, const std::string& path);

		ConfigLoader(const std::string& fileEnding);
		virtual ~ConfigLoader();

		std::string m_fileEnding;

		// For a line like
		// entity animals/dog
		// {
		//    ...
		// }
		// The type is "entity" and the name is "animals/dog"
		// Or if animal/dog was not there then name is ""
		virtual ConfigNode *getConfigScript (const std::string &name);

		virtual std::map <std::string, ConfigNode*> getAllConfigScripts ();

		virtual void parseScript(std::ifstream &stream);


	protected:

		float m_LoadOrder;
		// like "*.object"

		std::map <std::string, ConfigNode*> m_scriptList;

		enum Token
		{
			TOKEN_Text,
			TOKEN_NewLine,
			TOKEN_OpenBrace,
			TOKEN_CloseBrace,
			TOKEN_EOF,
		};

		Token tok, lastTok;
		std::string tokVal, lastTokVal;

		void _parseNodes(std::ifstream &stream, ConfigNode *parent);
		void _nextToken(std::ifstream &stream);
		void _skipNewLines(std::ifstream &stream);

		virtual void clearScriptList();
	};

	class ConfigNode
	{
	public:
		ConfigNode(ConfigNode *parent, const std::string &name = "untitled");
		~ConfigNode();

		inline void setName(const std::string &name)
		{
			this->m_name = name;
		}

		inline std::string &getName()
		{
			return m_name;
		}

		inline void addValue(const std::string &value)
		{
			m_values.push_back(value);
		}

		inline void clearValues()
		{
			m_values.clear();
		}

		inline std::vector<std::string> &getValues()
		{
			return m_values;
		}

		inline const std::string &getValue(unsigned int index = 0)
		{
			assert(index < m_values.size());
			return m_values[index];
		}

		ConfigNode *addChild(const std::string &name = "untitled", bool replaceExisting = false);
		ConfigNode *findChild(const std::string &name, bool recursive = false);

		inline std::vector<ConfigNode*> &getChildren()
		{
			return m_children;
		}

		inline ConfigNode *getChild(unsigned int index = 0)
		{
			assert(index < m_children.size());
			return m_children[index];
		}

		void setParent(ConfigNode *newParent);
 
		inline ConfigNode *getParent()
		{
			return m_parent;
		}

	private:
		std::string m_name;
		std::vector<std::string> m_values;
		std::vector<ConfigNode*> m_children;
		ConfigNode *m_parent;

		int m_lastChildFound;  //The last child node's index found with a call to findChild()

		std::vector<ConfigNode*>::iterator _iter;
		bool _removeSelf;
	};
 
}
 
#endif

#include "ConfigLoader.hpp"

#include <vector>
#include <map>
#include <exception>
#include <fstream>

#include <boost/filesystem.hpp>

namespace sh
{
	void ConfigLoader::loadAllFiles(ConfigLoader* c, const std::string& path)
	{
		for ( boost::filesystem::recursive_directory_iterator end, dir(path); dir != end; ++dir )
		{
			boost::filesystem::path p(*dir);
			if(p.extension() == c->m_fileEnding)
			{
				std::ifstream in((*dir).path().string().c_str(), std::ios::binary);
				c->parseScript(in);
			}
		}
	}

	ConfigLoader::ConfigLoader(const std::string& fileEnding)
	{
		//Register as a ScriptLoader
		m_fileEnding = fileEnding;
	}

	ConfigLoader::~ConfigLoader()
	{
		clearScriptList();

	}

	void ConfigLoader::clearScriptList()
	{
		std::map <std::string, ConfigNode *>::iterator i;
		for (i = m_scriptList.begin(); i != m_scriptList.end(); i++)
		{
			delete i->second;
		}
		m_scriptList.clear();
	}

	ConfigNode *ConfigLoader::getConfigScript(const std::string &name)
	{
		std::map <std::string, ConfigNode*>::iterator i;

		std::string key = name;
		i = m_scriptList.find(key);

		//If found..
		if (i != m_scriptList.end())
		{
			return i->second;
		}
		else
		{
			return NULL;
		}
	}

	std::map <std::string, ConfigNode*> ConfigLoader::getAllConfigScripts ()
	{
		return m_scriptList;
	}

	void ConfigLoader::parseScript(std::ifstream &stream)
	{
		//Get first token
		_nextToken(stream);
		if (tok == TOKEN_EOF)
		{
			stream.close();
			return;
		}

		//Parse the script
		_parseNodes(stream, 0);

		stream.close();
	}

	void ConfigLoader::_nextToken(std::ifstream &stream)
	{
		lastTok = tok;
		lastTokVal = tokVal;

		//EOF token
		if (stream.eof())
		{
			tok = TOKEN_EOF;
			return;
		}

		//(Get next character)
		int ch = stream.get();
		if (ch == -1)
		{
			tok = TOKEN_EOF;
			return;
		}
		while ((ch == ' ' || ch == 9) && !stream.eof())
		{    //Skip leading spaces / tabs
			ch = stream.get();
		}

		if (stream.eof())
		{
			tok = TOKEN_EOF;
			return;
		}

		//Newline token
		if (ch == '\r' || ch == '\n')
		{
			do
			{
				ch = stream.get();
			} while ((ch == '\r' || ch == '\n') && !stream.eof());

			stream.unget();

			tok = TOKEN_NewLine;
			return;
		}

		//Open brace token
		else if (ch == '{')
		{
			tok = TOKEN_OpenBrace;
			return;
		}

		//Close brace token
		else if (ch == '}')
		{
			tok = TOKEN_CloseBrace;
			return;
		}

		//Text token
		if (ch < 32 || ch > 122)    //Verify valid char
		{
			throw std::runtime_error("Parse Error: Invalid character, ConfigLoader::load()");
		}

		tokVal = "";
		tok = TOKEN_Text;
		do
		{
			//Skip comments
			if (ch == '/')
			{
				int ch2 = stream.peek();

				//C++ style comment (//)
				if (ch2 == '/')
				{
					stream.get();
					do
					{
						ch = stream.get();
					} while (ch != '\r' && ch != '\n' && !stream.eof());

					tok = TOKEN_NewLine;
					return;
				}
			}

			//Add valid char to tokVal
			tokVal += (char)ch;

			//Next char
			ch = stream.get();

		} while (ch > 32 && ch <= 122 && !stream.eof());

		stream.unget();

		return;
	}

	void ConfigLoader::_skipNewLines(std::ifstream &stream)
	{
		while (tok == TOKEN_NewLine)
		{
			_nextToken(stream);
		}
	}

	void ConfigLoader::_parseNodes(std::ifstream &stream, ConfigNode *parent)
	{
		typedef std::pair<std::string, ConfigNode*> ScriptItem;

		while (true)
		{
			switch (tok)
			{
				//Node
				case TOKEN_Text:
					//Add the new node
					ConfigNode *newNode;
					if (parent)
					{
						newNode = parent->addChild(tokVal);
					}
					else
					{
						newNode = new ConfigNode(0, tokVal);
					}

					//Get values
					_nextToken(stream);
					while (tok == TOKEN_Text)
					{
						newNode->addValue(tokVal);
						_nextToken(stream);
					}

					//Add root nodes to scriptList
					if (!parent){
						std::string key;

						if (newNode->getValues().empty())
						{
							key = newNode->getName() + ' ';
						}
						else
						{
							key = newNode->getName() + ' ' + newNode->getValues().front();
						}

						m_scriptList.insert(ScriptItem(key, newNode));
					}

					_skipNewLines(stream);

					//Add any sub-nodes
					if (tok == TOKEN_OpenBrace)
					{
						//Parse nodes
						_nextToken(stream);
						_parseNodes(stream, newNode);
						//Check for matching closing brace
						if (tok != TOKEN_CloseBrace)
						{
							throw std::runtime_error("Parse Error: Expecting closing brace");
						}
						_nextToken(stream);
						_skipNewLines(stream);
					}

					break;

				//Out of place brace
				case TOKEN_OpenBrace:
					throw std::runtime_error("Parse Error: Opening brace out of plane");
					break;

				//Return if end of nodes have been reached
				case TOKEN_CloseBrace:
					return;

				//Return if reached end of file
				case TOKEN_EOF:
					return;

				case TOKEN_NewLine:
					_nextToken(stream);
					break;
			}
		};
	}

	ConfigNode::ConfigNode(ConfigNode *parent, const std::string &name)
	{
		m_name = name;
		m_parent = parent;
		_removeSelf = true;    //For proper destruction
		m_lastChildFound = -1;

		//Add self to parent's child list (unless this is the root node being created)
		if (parent != NULL)
		{
			m_parent->m_children.push_back(this);
			_iter = --(m_parent->m_children.end());
		}
	}

	ConfigNode::~ConfigNode()
	{
		//Delete all children
		std::vector<ConfigNode*>::iterator i;
		for (i = m_children.begin(); i != m_children.end(); i++)
		{
			ConfigNode *node = *i;
			node->_removeSelf = false;
			delete node;
		}
		m_children.clear();

		//Remove self from parent's child list
		if (_removeSelf && m_parent != NULL)
		{
			m_parent->m_children.erase(_iter);
		}
	}

	ConfigNode *ConfigNode::addChild(const std::string &name, bool replaceExisting)
	{
		if (replaceExisting)
		{
			ConfigNode *node = findChild(name, false);
			if (node)
			{
				return node;
			}
		}
		return new ConfigNode(this, name);
	}

	ConfigNode *ConfigNode::findChild(const std::string &name, bool recursive)
	{
		int indx, prevC, nextC;
		int childCount = (int)m_children.size();

		if (m_lastChildFound != -1)
		{
			//If possible, try checking the nodes neighboring the last successful search
			//(often nodes searched for in sequence, so this will boost search speeds).
			prevC = m_lastChildFound-1; if (prevC < 0) prevC = 0; else if (prevC >= childCount) prevC = childCount-1;
			nextC = m_lastChildFound+1; if (nextC < 0) nextC = 0; else if (nextC >= childCount) nextC = childCount-1;
			for (indx = prevC; indx <= nextC; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name)
				{
					m_lastChildFound = indx;
					return node;
				}
			}

			//If not found that way, search for the node from start to finish, avoiding the
			//already searched area above.
			for (indx = nextC + 1; indx < childCount; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
			for (indx = 0; indx < prevC; ++indx)
			{
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
		}
		else
		{
			//Search for the node from start to finish
			for (indx = 0; indx < childCount; ++indx){
				ConfigNode *node = m_children[indx];
				if (node->m_name == name) {
					m_lastChildFound = indx;
					return node;
				}
			}
		}

		//If not found, search child nodes (if recursive == true)
		if (recursive)
		{
			for (indx = 0; indx < childCount; ++indx)
			{
				m_children[indx]->findChild(name, recursive);
			}
		}

		//Not found anywhere
		return NULL;
	}

	void ConfigNode::setParent(ConfigNode *newParent)
	{
		//Remove self from current parent
		m_parent->m_children.erase(_iter);

		//Set new parent
		m_parent = newParent;

		//Add self to new parent
		m_parent->m_children.push_back(this);
		_iter = --(m_parent->m_children.end());
	}
}


Alias: All-purpose_script_parser

<HR>
Creative Commons Copyright -- Some rights reserved.


THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.

BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

1. Definitions

  • "Collective Work" means a work, such as a periodical issue, anthology or encyclopedia, in which the Work in its entirety in unmodified form, along with a number of other contributions, constituting separate and independent works in themselves, are assembled into a collective whole. A work that constitutes a Collective Work will not be considered a Derivative Work (as defined below) for the purposes of this License.
  • "Derivative Work" means a work based upon the Work or upon the Work and other pre-existing works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which the Work may be recast, transformed, or adapted, except that a work that constitutes a Collective Work will not be considered a Derivative Work for the purpose of this License. For the avoidance of doubt, where the Work is a musical composition or sound recording, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered a Derivative Work for the purpose of this License.
  • "Licensor" means the individual or entity that offers the Work under the terms of this License.
  • "Original Author" means the individual or entity who created the Work.
  • "Work" means the copyrightable work of authorship offered under the terms of this License.
  • "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
  • "License Elements" means the following high-level license attributes as selected by Licensor and indicated in the title of this License: Attribution, ShareAlike.

2. Fair Use Rights

Nothing in this license is intended to reduce, limit, or restrict any rights arising from fair use, first sale or other limitations on the exclusive rights of the copyright owner under copyright law or other applicable laws.

3. License Grant

Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

  • to reproduce the Work, to incorporate the Work into one or more Collective Works, and to reproduce the Work as incorporated in the Collective Works;
  • to create and reproduce Derivative Works;
  • to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission the Work including as incorporated in Collective Works;
  • to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission Derivative Works.
  • For the avoidance of doubt, where the work is a musical composition:
    • Performance Royalties Under Blanket Licenses. Licensor waives the exclusive right to collect, whether individually or via a performance rights society (e.g. ASCAP, BMI, SESAC), royalties for the public performance or public digital performance (e.g. webcast) of the Work.
    • Mechanical Rights and Statutory Royalties. Licensor waives the exclusive right to collect, whether individually or via a music rights society or designated agent (e.g. Harry Fox Agency), royalties for any phonorecord You create from the Work ("cover version") and distribute, subject to the compulsory license created by 17 USC Section 115 of the US Copyright Act (or the equivalent in other jurisdictions).
    • Webcasting Rights and Statutory Royalties. For the avoidance of doubt, where the Work is a sound recording, Licensor waives the exclusive right to collect, whether individually or via a performance-rights society (e.g. SoundExchange), royalties for the public digital performance (e.g. webcast) of the Work, subject to the compulsory license created by 17 USC Section 114 of the US Copyright Act (or the equivalent in other jurisdictions).


The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats. All rights not expressly granted by Licensor are hereby reserved.

4. Restrictions

The license granted in Section 3 above is expressly made subject to and limited by the following restrictions:

  • You may distribute, publicly display, publicly perform, or publicly digitally perform the Work only under the terms of this License, and You must include a copy of, or the Uniform Resource Identifier for, this License with every copy or phonorecord of the Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Work that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Work itself to be made subject to the terms of this License. If You create a Collective Work, upon notice from any Licensor You must, to the extent practicable, remove from the Collective Work any credit as required by clause 4(c), as requested. If You create a Derivative Work, upon notice from any Licensor You must, to the extent practicable, remove from the Derivative Work any credit as required by clause 4(c), as requested.
  • You may distribute, publicly display, publicly perform, or publicly digitally perform a Derivative Work only under the terms of this License, a later version of this License with the same License Elements as this License, or a Creative Commons iCommons license that contains the same License Elements as this License (e.g. Attribution-ShareAlike 2.5 Japan). You must include a copy of, or the Uniform Resource Identifier for, this License or other license specified in the previous sentence with every copy or phonorecord of each Derivative Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Derivative Works that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder, and You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Derivative Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Derivative Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Derivative Work itself to be made subject to the terms of this License.
  • If you distribute, publicly display, publicly perform, or publicly digitally perform the Work or any Derivative Works or Collective Works, You must keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or (ii) if the Original Author and/or Licensor designate another party or parties (e.g. a sponsor institute, publishing entity, journal) for attribution in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; the title of the Work if supplied; to the extent reasonably practicable, the Uniform Resource Identifier, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work; and in the case of a Derivative Work, a credit identifying the use of the Work in the Derivative Work (e.g., "French translation of the Work by Original Author," or "Screenplay based on original Work by Original Author"). Such credit may be implemented in any reasonable manner; provided, however, that in the case of a Derivative Work or Collective Work, at a minimum such credit will appear where any other comparable authorship credit appears and in a manner at least as prominent as such other comparable authorship credit.

5. Representations, Warranties and Disclaimer

UNLESS OTHERWISE AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE MATERIALS, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU.

6. Limitation on Liability.

EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. Termination

  • This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Derivative Works or Collective Works from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License.
  • Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.

8. Miscellaneous

  • Each time You distribute or publicly digitally perform the Work or a Collective Work, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License.
  • Each time You distribute or publicly digitally perform a Derivative Work, Licensor offers to the recipient a license to the original Work on the same terms and conditions as the license granted to You under this License.
  • If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.
  • No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent.
  • This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You.