HLSL, OGRE, and in between

What is this guide?

While OGRE is a great open source 3d engine, your control over its capabilities is limited by you knowledge of how it operates.

At higher levels of operation, things are easy to understand, even without understanding what’s happening deep inside. But eventually, you will reach a point where you want more complex operations to take place. Some of these operations revolve around the way things are rendered.

Such operation can be designed using shaders. Shaders (in a very broad, simplified description) are “mini programs”, sent to the GPU with instructions.

There are three primary languages for writing shaders:

  1. HLSL ( DirectX )
  2. GLSL ( OpenGL )
  3. CG ( OpenGL & DirectX capable, made by NVIDIA )

Originally, shaders were written only in Assembler (made by Satan), which we can call “machine language”. Assembler is the language all other code is converted to before it becomes 0 or 1. As such, it’s not a very fun language to work with, and the primary reason why the three others exist.

While HLSL, GLSL and CG are very similar, each one has its specifics. It’s important, as advised by many, to “pick one and stick with it”.

OGRE, like many other 3d engines, is capable of utilizing such programs.

As an OGRE developer, experienced or fresh out of tutorials, you have two options concerning shaders:

  1. Using a pre-made shader.
  2. creating a shader by yourself.

While the first option is predictably easier, if your shader ever breaks down or stop behaving as you thought it would, you are stuck.

This guide (concentrating on HLSL) will teach you what shaders are, how to understand them, and how to create your own.

Additionally, the second subject of this guide is how to integrate your code with OGRE. The ‘in between’ part, is the 'OGRE material scripts'.

Intended audience

Primary audience

Newcomers in computer graphics field who already grasped to basics of the engine but yet to fully understand (or at all) how to write material scripts.

This guide will explain in detail and reconstruct many widely used terms and techniques in order to understand them.

Secondary audience

More experienced OGRE programmers who want to understand and create their own shaders. This group might find some of the explanations or operations in the guide redundant.


Q: “What do I need to know?”

You need to have at least basic understanding of the OGRE engine (scene nodes, basic managers, entities etc... half way through OGRE wiki tutorials to begin with) since we will not write any Ogre code here. Whatever you decide to do within the bounds of OGRE and how (other than loading material) is not my concern.

This guide also requires you to be fluent in your own native language, preferably a C style language (I will not explain what a struct or a matrix is, only how it looks in HLSL)

Q: “Do I need anything other than a working OGRE application?”


NVIDIA Fx Composer http://developer.nvidia.com/object/fx_composer_home.html

Fx Composer is a tool that compiles shader code, render and allow tweaks on the fly. We will edit the code in FXC and ‘export’ it to OGRE.

Highly recommended:

Any simple OGRE Script editor


YAOSE (Yet Another Ogre Script Editor)


other questions

Q: “what shader model are we learning?”

We will use the up-to version 3 (D3D9) shader model. That means no geometry shaders for now.

Q: “Why Fx composer? Can I use something else? Why not just use my shaders directly in OGRE?”

  1. I chose Fx composer because its very intuitive interface and broad range of supported languages. the other reason is that the way you write hlsl shaders in it (fx file format) is very similar to OGRE material scripts.
  2. There are alternatives such as ATI’s render monkey, but ATI doesn’t seem to give RM much thought or support. In addition, RM interface is not very friendly. If you wish to use this guide, you should stick to FXC, since most of the guidelines will be very FXC specific.
  3. Why not drop it directly into OGRE? FXC is built for the purpose of developing shaders, you can see your result right away, and when errors occur, you don’t have to dig log files.

Q: “Why YAOSE? There are lots of other OGRE script editors”

True, YOASE very name is evident enough. I chose YAOSE because it’s eaten-by-zombies-brain-dead easy to use. Its best feature is the context specific code completion.

YAOSE has its share of problems – if you so choose, pick a different editor. CODE::BLOCKS users have syntax highlighting and code completion by default for OGRE scripts, but not context specific.

Q: “Why HLSL?”

My decision making process regarding shading language can be visualized as such:

As I mentioned earlier, pick one and stick with it.


Why are shaders so obscure and scary? Is it really hard to learn?

When learning HLSL or another shading language, there are two main difficulties:

  1. Adjustment issue: shaders have some specific logics that will not feel natural right away even for experienced programmers. Other than that, when creating shaders you see only the “links” between the operations. Much of the work flow is done behind the curtain, making it somewhat difficult to comprehend at first.
  2. Reading material: generally speaking, shaders are “tools for other tools”. Shaders usually provide “micro-management” inside a 3d engine. As such, they integrate differently with each engine, while remaining only half relevant to the engine documentation. Most basic shader teaching guides reside in dark corners of 3d engine documentation and consist of a basic introduction, basic design and basic integration with the specific engine. The other side of this very short spectrum consists of very specific, very professional articles that won’t make much sense to beginners.

All in all, these two points make it hard to find a good, relevant reading material that rely on basic existing knowledge, and expand upon accumulated understanding.

“Who are you? Are you qualified?”

The community praises JaJDoo:

Image CrazyEddie: “grow up … idiot.”

Image Jacmoe: ” Your comment smells like.....Open Source paranoia“

Image “Kojack: Your phong shader (the first one) has a bug”

One of my famous posts:

''“So I found this shader… knowing next to nothing about shaders, I’m not surprised that the result is a twitchy half ball of blackness. help?”

Who else would you turn to??

The big bang

So, we cleared the basics out of the way, now we can start causing ourselves some brain-damage!