History: SoC2011 Modern Illumination Techniques
Source of version: 3
Copy to clipboard
!Project Proposal
My proposal focuses on implementing several modern illumination techniques, that are either related
to, or benefit from deferred shading. Most of these have been used in several shipped games, while a
couple are still considered 'bleeding edge'. My list of features is prioritized, and I tried to make sure
that each milestone delivers an usable and thoroughly documented piece of code.
!!Light Pre-pass
This technique is an alternative to deferred shading, promising lower memory bandwidth requirements
and a somewhat higher material system flexibility, at the expense of adding another geometry
pass(which is relatively cheap). It was first introduced by Wolfgang Engel via his blog{FOOTNOTE()}http://diaryofagraphicsprogrammer.blogspot.com/2008/03/light-pre-pass-renderer.html{FOOTNOTE}, and later
developed trough several articles in the ShaderX and Gpu Pro series.
The rendering process starts by rendering normals and depth information into a slim G-Buffer(the
normals can be stored in polar coordinates, which leaves 16 bits for depth; this should be enough
precision, and requires a single RGBA8 render target). Next, for each light in the scene,
N*L*light_color and RV^n(for the phong model) are rendered into a light buffer(also a single RGBA8
target). A second geometry pass is performed(while keeping the Z-Buffer from the first pass, thus there
is no over-draw), and the lighting equation is reconstructed, using the material info.
This technique has been used in several shipped games, and a rather impressive demo, involving a large
number of light sources, can be found at [2]. A larger performance gain is achieved when some post-
processing effects like motion blurred or precomputed ambient occlusion are performed(and thus
would require a larger G-Buffer in a classical deferred shading approach).
Implementation wise, this requires a minor modification to the already existent deferred shading demo.
!!Inferred Lighting [3]
This builds on the previous technique and tries to use smaller resolution buffers. The main issue that
arises is that rope-like artifacts appear on object edges(due to sampling issues). The proposed solution
is using another buffer that stores both linear depth and a semi-unique ID of that surface(two adjacent
surfaces shouldn't have the same ID). This is used in the final material pass to bias the bilinear filtering
that would occur due to sampling a smaller light buffer.
The reduction in bandwidth necessity can be significant(a double sized G-Buffer is offset by a 4 or
even 16 times smaller light buffer). Another important issue is that performance can be easily traded for
visual accuracy(we simply need to use smaller or larger light/G buffer). To the extent of my knowledge, this has not yet been used in a shipped game.
!!__Screen Space Directional Occlusion__ [4]
This is build on SSAO, and takes only requires the depth and normal at each pixel(which is great
considering that's what's stored in the G-Buffer in the previous techniques). The same sampling pattern
used for SSAO can be used here as well, the main difference being that an environment map is sampled
to account for the incoming radiance, rather than computing a general occlusion term. This technique
can improve visual results in situations where lighting conditions vary(especially in color).
This can be easily extended to add one bounce global illumination.
!!__Light Propagation Volumes__ [5]
This is a global illumination approximation technique, that can be used for low frequency lighting
situations.
The first step is partitioning the scene in a grid. Normally, the grid would have varying cell size, but for
my implementation I will only implement a fixed size grid.
The scene is then rendered into a set of Reflective Shadow Maps[6]. Generating RSMs is fairly straight
forward(we add a world-space position, a normal and a flux channel to the depth channel in used for a
regular shadow map). Each pixel in a RSM is considered a virtual point light(VPL). Each VPL's
position is then determined, and converted to a spherical harmonics representation[7]. The VPL's in
each cell are then 'accumulated'(we perform a summation in SH space). Low frequency direct
illumination can be approximated as well by creating VPL's from environment maps or area lights.
Next the depth and surface normal at each point is used to reconstruct a coarse approximation of the
scene's geometry(surface information stored in the RSMs is also used; additional information can be
obtained by using depth peeling). This geometry information is used per cell to obtain occlusion data
for each incoming direction. This is also converted into SH space and accumulated.
Implementation Details: The whole technique can be implemented as a single compositor. The grids
will consist of 3D textures and a custom rendering pass will be used to inject the RSMs into it. Light
propagation is computed in a fixed number of passes, using ping-pong buffers(in each pass each cell
looks at it's six neighbors). The RSMs can be computed by a custom shadow mapping algorithm. Two
open-source stand-alone implementations exist at [8] and [9].
!!__Demos__
The plan is to write two demos. The first would showcase the deferred rendering techniques. This
means that the user can choose between standard deferred shading, deferred lighting and inferred
lighting. The demo should be based on rendering a high amount of lights, and I intend to make the
exact number variable trough a slider. In the case of inferred lighting the user can also choose the G-
Buffer and light buffer size.
The second demo will use deferred lighting, and will showcase the global illumination techniques
presented here+SSAO. The user will be able to choose between SSAO, SSDO or no ambient occlusion.
He will also be able to activate/deactivate LPV and choose the grid size and the number of passes used for light propagation.
The main benefit this project would bring Ogre is novelty. Very few engines implement the global
illumination techniques presented here(and none of them is open-source), so Ogre's implementation
could become a reference. Not to mention that global illumination is receiving a lot of interest lately.
Furthermore, these techniques complement each other nicely, and work well with Ogre's existing post-
processing effects(especially SSAO). Both light pre-pass rendering and inferred lighting supply
normals and depths for every pixel, which are then needed by SSDO(and SSAO) and LPV. LPV works
well for low frequency indirect lighting, while SSDO works well for higher frequencies.
!__Schedule__
The only other commitment I have this summer are my third year exams, and I'll finish those around
the 10th
of June. That gives me roughly 10 weeks of actual work.
• __Preparation period__ – Find out what the community expects from this project. Get familiar
with the inner workings of the compositor framework.
• ''Weeks 1-2(June 12th'')– Implement and optimize Light Pre-Pass and Inferred lighting
• ''Week 3(June 26th)''Implement Depth Peeling(useful for both LPV and SSDO)
• ''Week 4(July 3rd)''Implement SSDO
• ''Week 5-6(July 10th)''Implement RSMs(I want to implement these so that they can be used without LPV)
• ''Week 7(July 24th)''Implement reconstructing the scene geometry from the various depth and
normal information available; add optional depth peeling
• ''Week 8(July 31st)''Finalize LPV
• ''Week 9(August 7th)'' Analyze possibilities to add direct lighting and glossy reflections to the LPV compositor
• ''Week 10(August 14th)''Document everything. Make sure to note possible areas for
improvement.
!!Some further development ideas:
• SSDO can easily be extended to accommodate one bounce global illumination
• LPV can be optimized by allowing cascading grids(recursively subdivided; think of a octree)
• The SH representation from LPV can be used as a basis for further development in SSDO(instead of accessing a cube-map for each direction, we could use the per-cell
representation)
• Depth Peeling can be used for transparent objects in any deferred technique
Why I'm The Person For This Project
I am a computer science student, in my third year, and the courses I followed span from object oriented
programming to software engineering, to computer graphics and computer architectures. I consider
myself to be well prepared in all of these fields, especially since I have taken a special interest in them.
I'm also quite a talkative and sociable person, and enjoy working with others. I am currently involved
in a large scale virtual reality project that uses an Ogre based client. You can find more about my work
from my blog[10] or my projects page[11].
Why OGRE?
I have a great interest in computer graphics and engine design, and hope to work in the field in the near
future and has helped me improve in both fields. I also learned a lot from Ogre, both in coding style
and in terms of engine design(I often find myself skimming trough open source engines' doxygen
documentation to see how different modules work together).
Anything Else
My proposal might look a little ambitious, or unfeasible, but I believe that I have the needed skills and
motivation to implement it. Plus, I don't have anything planned until early September :)
{FOOTNOTEAREA()}{FOOTNOTEAREA}
[1]http://diaryofagraphicsprogrammer.blogspot.com/2008/03/light-pre-pass-renderer.html
[2]http://www.confettispecialfx.com/river-of-lights#more-92
[3]http://graphics.cs.uiuc.edu/~kircher/inferred/inferred_lighting_paper.pdf
[4]http://www.mpi-inf.mpg.de/~ritschel/Papers/SSDO.pdf
[5]http://www6.incrysis.com/Light_Propagation_Volumes.pdf
[6]Reflective Shadow Maps
[7]http://www.ppsloan.org/publications/StupidSH36.pdf
[8]http://lee.fov120.com/lpv.zip
[9]http://blog.blackhc.net/wp-content/uploads/2010/07/LPVPrototype.zip
[10]http://andreiradu.blogspot.com
[11]http://graphics.cs.pub.ro/~andreiradu