Intermediate Tutorial 3         Selecting Entities and Query Masks



This tutorial will continue from the previous one. Do not start with the usual base code. Instead, start up exactly where Intermediate Tutorial 2 left off. This tutorial will be covering the basics of selecting entities in the scene. It will also cover ways of restricting what is selectable.

The full source for this tutorial is here.

Note: There is also source available that uses the BaseApplication framework and Ogre 1.7 here.

Note: There is also source available that uses the BaseApplication framework, Ogre 1.7, and the builtin SdkTrays overlay system here.



This tutorial assumes that you already know how to set up an Ogre project and compile it successfully. Knowledge of the topics from previous tutorials is also assumed.

This tutorial is a continuation of Intermediate Tutorial 2. The base code for this tutorial is here.

Highlighting a Selected Entity

The first thing we want to do is make it possible to highlight a selected entity. In your own serious applications, you will most likely want to use something like projective decals or changing the color of the entity's mesh, but we will use the basic bounding box visualization provided by Ogre.

Setting up this bounding box is easy. When the left mouse button is pressed, we will check to see if there was already a previous entity selected. Add the following to the if statement that checks for the left mouse button in mousePressed:

if (mCurObject)

That will turn off the bounding box for any previously selected entity. Now add the following code right before the return statement:

if (mCurObject)

This will turn on the bounding box for any entity that was just selected. That's it. Compile and run the application. You should now see the current entity highlighted with a wireframe box.



Now we want to modify the code so that we can also create ninjas. We will have a "Robot Mode" and a "Ninja Mode". We will use the spacebar to switch between them.

First, let's add a boolean variable that will keep track of what mode we are in.

bool bRobotMode;

Now initialize the variable to true so that we start in Robot Mode.


We need to make sure that the right kind of entity is created based on our current mode. Find the following in mousePressed:

Entity* ent = mSceneMgr->createEntity("robot.mesh");

We will replace this with code that takes our current mode into account.

Ogre::Entity* ent;
if (bRobotMode)
  ent = mSceneMgr->createEntity("robot.mesh");
  ent = mSceneMgr->createEntity("ninja.mesh");

Now we need to toggle the mode when the spacebar is pressed. Add the following to keyPressed:

if(ke.key == OIS::KC_SPACE)
  bRobotMode = !bRobotMode;

The last thing we'll do is simply for convenience. We already have a CEGUI label that tells us the terrain is building, but we don't use it for anything after that. Let's have it display what mode we are in so the user can know what they will place next.

Find this line in frameRenderingQueued:


And change it to this:

if (mRobotMode)
  terrainLabel->setText("Robot Mode");
  terrainLabel->setText("Ninja Mode");

That's all there is to it. Compile and run the application. You should be able to switch between placing robots and ninjas. The current mode should also be displayed in the label.

Selecting Entities

We are now going to use ray scene queries to select entities in our scene. The result of our query will be a RaySceneQueryResult, which is an iterable collection of RaySceneQueryResultEntry structs.

This struct contains three variables. The distance variable tells us how far down the ray our object is. The movable and worldFragment variables tell us what kind of object was hit by the ray. With the old terrain system, testing for world fragments was how we would do raycasts against the terrain. The new terrain system makes this method obsolete. But the movable variable is still used to determine when a ray has hit any entity attached to a scene node (notice that this includes things like the camera). If an entity was hit, then movable will contain a MovableObject. Usually, we will get the scene node for our MovableObject by calling getParentSceneNode or getParentNode.

Let's look at an example to help clarify these concepts. The example would print out everything that was discovered by a ray scene query.

// Do not add this to your project
RaySceneQueryResult& result = mRayScnQuery->execute();
RaySceneQueryResult::iterator it;
for (it = result.begin(); it != result.end(); it++)
  if (it->worldFragment)
    Vector3 location = it->worldFragment->singleIntersection;
    std::cout << "WorldFragment: (" << 
      location.x << ", " << location.y << ", " << location.z << ")" << std::endl;
  else if (it->movable)
    std::cout << "MovableObject: " << it->movable->getName() << std::endl;

Using the new terrain system, we would not get any worldFragment results, even when clicking on the terrain. It's been left it in simply for completeness. But we would still get MovableObject results if our ray hit one of the entities we've placed in our scene.

Now let's actually add these concepts into our application. To get started, let's add a few more variables to our class. Add the following to the header:

bool mMovableFound;

Ogre::RaySceneQuery* mRayScnQuery;

Then add these initializations:


We now have to reorganize our mousePressed function. First, find the code that starts with this line:

Ogre::TerrainGroup::RayResult terrainResult =

And ends with this:


We're going to wrap this in an if statement that checks to see if mMovableFound is false.

if (!mMovableFound)
  Ogre::TerrainGroup::RayResult terrainResult =

Now we need to get into setting up and using our ray scene query. Add the following right before the if statement we just added:

CEGUI::Vector2f mousePos = 
Ogre::Ray mouseRay =
    mousePos.d_x / float(me.state.width),
    mousePos.d_y / float(me.state.height));


Ogre::RaySceneQueryResult& result = mRaySceneQuery->execute();
Ogre::RaySceneQueryResult::iterator it = result.begin();

mMovableFound = false;

The first thing we do is give our mouseRay to our ray scene query. It requires this to do the raycast. Next, we tell it to return the results sorted by distance. Otherwise, they would come back in a seemingly random order for performance reasons.

Then we actually execute the query and save a reference to the result. We also get an iterator pointing to the first position in the collection. The last thing we do is "reset" our flag that keeps track of whether or not we've found a MovableObject.

Now we have to look through the results of our query.

for ( ; it != result.end(); it++)
  mMovableFound =
    it->movable &&
    it->movable->getName() != "" &&
    it->movable->getName() != "PlayerCam";

  if (mMovableFound)
    mCurObject = it->movable->getParentSceneNode();

For each RaySceneQueryResult, we first check to make sure it is one of our robot/ninja entities. The camera is also attached to a scene node, so we want to ignore that result, and if you run tests on your current scene, you'll notice there appears to be other entities with no name floating around your scene. We must also discard these entities as invalid (they are not terrain elements either, try clicking on the sky and you'll still get some results).

Next, we update our currently selected object if we have found a valid movable object. After that, we break from the loop, because we've found what we're looking for.

Finally, we should make sure to clean up the RaySceneQuery. Add the following to ~BasicApp:


That is actually all we have to do. That's pretty impressive! Imagine trying to accomplish all of this by calculating all of these rays and manipulating all of the matrices involved.

Compile and run the application. You should be able to reselect entities that you've already placed into the world, while still being able to create new entities when clicking on empty terrain.

Selecting Specific Types of Entities

That last major part of this tutorial is going to deal with masking our selection. This means only selecting certain entities. We want to be able to select robots only when we're in Robot Mode and select ninjas only when we're in Ninja Mode. This is actually very easy to do if we understand a little bit about binary operations.

The first thing we are going to do is add an enum to our class. Add the following to the public section of our header:

enum QueryFlags
  ROBOT_MASK = 1 << 0,
  NINJA_MASK = 1 << 1

The technique we are going to use is called bitmasking. It is a very widely used concept in programming. It makes use of the bitwise operators that most languages provide for changing individual bits in a byte (or at least they simulate it). Wikipedia has a pretty good introduction to the topic here. It is something worth understanding well.

The technique allows us to pack a lot of information into a small amount of data. What we're doing with the enum is declaring two values using the bitwise "shift" operator. This takes a binary number and shifts the bits by a certain number of places. So our ROBOT_MASK value is going to be the number one shifted zero places to the left, and our NINJA_MASK is going to be the number one shifted one place to the left.

ROBOT_MASK = 0000 0001
NINJA_MASK = 0000 0010

These will be used to uniquely identify each type of entity. The next step is to set the query flags for our entities. Update the entity creation code in mousePressed so that it looks like this:

if (mRobotMode)
  ent = mSceneMgr->createEntity("robot.mesh");
  ent = mSceneMgr->createEntity("ninja.mesh");

The only thing we've changed is the addition of the setQueryFlags methods. This method takes the enum values we set up. The last thing we need to do is to tell our ray scene query when to use each mask. Add the following to mousePressed right after we call setSortByDistance:

mRayScnQuery->setQueryMask(mRobotMode ? ROBOT_MODE : NINJA_MASK);

That is all there is to it. Compile and run your application. You should only be able to select an entity when you are in the corresponding mode.

This is the end of the main tutorial. The rest is supplemental information about further making use of bitmasks with Ogre. It is some very useful information, but it will not add anything new to the code.

More on Masks

We are going to describe some other useful techniques for working with these masks. The code from this point forth will not be added to our project, because most of it requires adding more to the scene. But these techniques should give you a number of ideas on how to customize the application we've already built. Especially once you've learned more about building your scene from later tutorials.

Query Type Masks

There are many other kinds of movable object that can used in your raycasts. For instance, if you were using a BillboardSet or a ParticleSystem, then the raycasts we've been using would ignore those elements in your scene. You may or may not want things to work that way. This is where query type masks come in.

A query type mask allows you to decide what type of objects are to be included in your query. By default, the ray scene query has its mask set to ENTITY_TYPE_MASK. This only returns entities you've placed in your scene. To recognize effects like the particle system, we would use the following:


The scene manager defines six query type mask values:

WORLD_GEOMETRY_TYPE_MASK   // returns world geometry
ENTITY_TYPE_MASK           // returns entities (default)
FX_TYPE_MASK               // returns BillboardSets/ParticleSystems.
STATICGEOMETRY_TYPE_MASK   // returns static geometry
LIGHT_TYPE_MASK            // returns lights
USER_TYPE_MASK_LIMIT       // user type mask limit

Combining Masks

It is often useful to combine masks so we can select objects from more than one group with the same query. This is easily accomplished by using another bitwise operator. The bitwise | (OR) operator allows us to combine masks. Let's say we had a set of query flags like this:

enum QueryFlags
  NPC = 1 << 0,
  ENEMY = 1 << 1,
  ITEM = 1 << 2,

If we wanted to to select both NPCs and ITEMS we could set the mask like this:

mRayScnQuery->setQueryMask(NPC | ITEM);

Now mRayScnQuery would return movable objects with either NPC or ITEM masks. You can start to see why these bitmasks are so useful. If you have a particular combination that you use a lot, then you can even assign it a new name in your enum for convenience:

enum QueryFlags
  NPC = 1 << 0,
  ENEMY = 1 << 1,
  ITEM = 1 << 2,

Negation of a Mask

The bitwise operator ~ flips every bit in the number. This is bitwise negation. It is equivalent to or'ing together every mask except the one you are negating. Here's an example:


This is exactly equivalent to our previous example with bitwise OR. The ~ENEMY mask would have all of its bits on except for ENEMY. For our example, that is exactly the same as NPC | ITEM.

Finally, we can combine these bitwise operations. What if we wanted to return everything except NPCs and ITEMs? We could do this by combining bitwise negation and bitwise OR.

mRayScnQuery->setQueryMask(~(NPC | ITEM));

One last trick you can do with negation is create a query that returns every kind of movable object. The trick is to negate the number zero. By doing this you get the mask of all ones. This is functionally equivalent to mask you'd get by using bitwise OR on every mask in your enum.


In our example, this is equivalent to:

mRayScnQuery->setQueryMask(NPC | ENEMY | ITEM);

Note: To avoid confusion, it should be pointed out that the binary numbers ~0 and NPC | ENEMY | ITEM would not be equivalent. It is just that they will have the equivalent effect in our example because we are not using any of the places where they are different. The actual values would be:

NPC | ENEMY | ITEM = 0000 0000 0000 0111
~0                 = 1111 1111 1111 1111

The important part is that they match for the lowest three positions, because that is all our example was using.

If a lot of these bitmasking examples have been confusing, then you should seriously consider doing some supplemental reading along with these tutorials. If you have a solid handle on the mathematics, it will help you immensely as a programmer. And if you try and get by without it, then you will be constantly struggling with trying to understand two subjects at once. Make your life easier, study some math.


This tutorial was a continuation of the previous tutorial about using raycasts in your scene. We introduced the full ray scene query process for building a raycast and searching through the results. We used these queries to reselect entities we had already placed into our scene.

This lead us to controlling what entities would be selected. The problem was solved by the use of bitmasks. Bitmasking is a widely-used technique that allows us to make use of the concept of individual bits to keep track of data instead of always using full bytes. We learned about a series of bitwise operations which allow us to quickly create complicated bit patterns representing collections of entities to be used in our raycasts.



  1. Choose some of the techniques introduced in the More About Masks section and update your scene to use them in any way you choose.


  1. Since we are creating multiples types of objects now, we should be using the concept of a Factory to properly create our Entities and SceneNodes. Implement this design pattern.


  1. One bug in our current design is that no matter where you select an entity, our application will consider it like you are holding the model from the very bottom. To make this more clear, just try it in your application. Grab a model by the top and then move the mouse slightly. If your cursor is still over some terrain, then the model will fly far across the terrain. This is because it is trying to place the bottom of the model where your cursor is. Fix this issue.
  2. Our application only allows two types of entities. In a real scene editor, you would want to be able to choose from a customizable list of entities. Use CEGUI to create GUI elements that allow you to chose from a number of different entities to place onto the terrain.
  3. After you've completed the previous exercise, then allow the user to use the GUI to select what types of entities are active. You will be using techniqes that are covered in the More About Masks section after the main tutorial.


  1. Add the ability to select multiple objects by holding the Ctrl key and clicking on a number of different entities. Also make it so that moving one of these selected entities moves the entire group.
  2. Many strategy games allow you to create persistent groups of entities that can be selected automatically and moved together. Implement this feature in your program using CEGUI.

Full Source

The full source for this tutorial is here.


Intermediate Tutorial 4

Alias: Intermediate_Tutorial_3