Repository
Project Thread

Project Proposal

My proposal focuses on implementing several modern illumination techniques, that are either related to, or benefit from deferred shading. Most of these have been used in several shipped games, while a
couple are still considered 'bleeding edge'. My list of features is prioritized, and I tried to make sure that each milestone delivers an usable and thoroughly documented piece of code.

Light Pre-pass

This technique is an alternative to deferred shading, promising lower memory bandwidth requirements and a somewhat higher material system flexibility, at the expense of adding another geometry pass(which is relatively cheap). It was first introduced by Wolfgang Engel via his blog1 , and after developed trough several articles in the ShaderX and Gpu Pro series.
The rendering process starts by rendering normals and depth information into a slim G-Buffer(the normals can be stored in polar coordinates, which leaves 16 bits for depth; this should be enough
precision, and requires a single RGBA8 render target; we use a 16 bit floating point render target for clarity). Next, for each light in the scene, N*L*light_color and RV^n(for the phong model) are rendered into a light buffer(also a single RGBA8 target, or a RGBA16F to support HDR). A second geometry pass is performed(while keeping the Z-Buffer from the first pass, thus there
is no over-draw), and the lighting equation is reconstructed, using the material info.
This technique has been used in several shipped games, and a rather impressive demo, involving a large number of light sources, can be found at 2 . A larger performance gain is achieved when some post-processing effects like motion blurred or precomputed ambient occlusion are performed(and thus would require a larger G-Buffer in a classical deferred shading approach).
In my implementation, the alpha channel of the light buffer contains the specular term raised at a light-specific value, and multiplied by NL and attenuation. The specular term(NH) is recalculated at merging by division with the luminance of the diffuse channel.

Inferred Lighting 3

This builds on the previous technique and tries to use smaller resolution buffers. The main issue that arises is that rope-like artifacts appear on object edges(due to sampling issues). The proposed solution is using another buffer that stores both linear depth and a semi-unique ID of that surface(two adjacent surfaces shouldn't have the same ID). This is used in the final material pass to bias the bilinear filtering that would occur due to sampling a smaller light buffer.
The main purpose for this is reducing the cost of illumination, since lighting is performed at a lower resolution.I used a light buffer with 0.75 the height and 0.75 the width of the viewport, so a ~44% reduction in lighting calculations. Artifacts are present along curve surfaces, and lighting detail on pixel sized objects or highlights is lost(temporal aliasing occurs due to this as well).
Another important issue is that performance can be easily traded for visual accuracy(we simply need to use smaller or larger light/G buffer). To the extent of my knowledge, this has not yet been used in a shipped game.

Reflective Shadow Maps

Reflective shadow maps are used, in this implementation, as a form of instant radiosity. The purpose is to provide one bounce global illumination for diffuse surfaces. The technique starts with the idea that every point that would give one bounce GI is captured by the light's shadowmap. So we store extra information in the shadowmap(normal and flux), and generate Virtual Point Lights(VPLs) based on that. We then use these pointlights to light the scene. Since several hundred point lights are needed to get good results, using a deferred technique is mandatory.

Normally the RSM would also contain world-space position, and the flux would contain the albedo multiplied by the attenuation(both from distance and from spotlight cutoff). However, OGRE doesn't currently allow Shadow Textures to be MRTs, so the world-space position was dropped(it's computed at VPL generation from depth), and the flux only contains the albedo(attenuation is factored in at VPL creation).

VPLs use two attenuation factors: first a squared distance attenuation is applied. Then the dot product between the VPLs normal and the surface normal is factored in.
VPLs are splatted as unit spheres that are scaled and deformed(they reach a hemisphere shape) in the vertex shader. They are rendered using instancing(the difference in framerate between using/not using instancing is almost two orders of magnitude; less than one fps to over 50 on my configuration).

Light Propagation Volumes 4

This is a global illumination approximation technique, that can be used for low frequency lighting situations. The first step is partitioning the scene in a grid. Normally, the grid would have varying cell size, but for my implementation I will only implement a fixed size grid. The scene is then rendered into a set of Reflective Shadow Maps Each VPL's position is then determined, and converted to a spherical harmonics representation5 . The VPL's in each cell are then 'accumulated'(we perform a summation in SH space). Low frequency direct illumination can be approximated as well by creating VPL's from environment maps or area lights.
Next the depth and surface normal at each point is used to reconstruct a coarse approximation of the scene's geometry(surface information stored in the RSMs is also used; additional information can be
obtained by using depth peeling). This geometry information is used per cell to obtain occlusion data for each incoming direction. This is also converted into SH space and accumulated.
Implementation Details: The whole technique can be implemented as a single compositor. The grids will consist of flattened 3D textures(2D textures that contain several slices) and a custom rendering pass will be used to inject the RSMs into it(sample the RSM and render the point lights directly in SH coordinates). Light propagation is computed in a fixed number of passes, using ping-pong buffers(in each pass each cell looks at it's six neighbors).
Two open-source stand-alone implementations exist at 6 and 7 .

Demos

The demo extends the deferred shading demo with a drop-down that selects the lighting technique used. The options will be: forward rendering, deferred shading, deferred lighting, inferred lighting, spherical harmonics lighting and light propagation volumes. Additionally, when selecting deferred shading, deferred lighting or inferred lighting, there will be an option to activate Reflective Shadow Maps. Debugging outputs are also extended to allow the viewing of the light buffer, for deferred lighting and inferred lighting, and object IDs for inferred lighting.

Schedule

The only other commitment I have this summer are my third year exams, and I'll finish those around the 10th of June. That gives me roughly 10 weeks of actual work.

• Preparation period – Find out what the community expects from this project. Get familiar <br />with the inner workings of the compositor framework.
• Weeks 1-2(June 12th)– Implement and optimize Light Pre-Pass and Inferred lighting
>• Week 3(June 26th)Implement Depth Peeling(useful for both LPV and SSDO)
• Week 4(July 3rd)Implement SSDO
• Week 5-6(July 10th)Implement RSMs(I want to implement these so that they can be used without LPV)
• Week 7(July 24th)Implement reconstructing the scene geometry from the various depth and <br />normal information available; add optional depth peeling<br />• Week 8(July 31st)Finalize LPV
• Week 9(August 7th) Analyze possibilities to add direct lighting and glossy reflections to the LPV compositor
• Week 10(August 14th)Document everything. Make sure to note possible areas for <br />improvement.

Some further development ideas:

• SSDO can easily be extended to accommodate one bounce global illumination
• LPV can be optimized by allowing cascading grids(recursively subdivided; think of a octree)
• The SH representation from LPV can be used as a basis for further development in SSDO(instead of accessing a cube-map for each direction, we could use the per-cell <br />representation)
• Depth Peeling can be used for transparent objects in any deferred technique
Why I'm The Person For This Project<br />I am a computer science student, in my third year, and the courses I followed span from object oriented <br />programming to software engineering, to computer graphics and computer architectures. I consider <br />myself to be well prepared in all of these fields, especially since I have taken a special interest in them. <br />I'm also quite a talkative and sociable person, and enjoy working with others. I am currently involved <br />in a large scale virtual reality project that uses an Ogre based client. You can find more about my work <br />from my blog8 or my projects page9 .<br />Why OGRE?<br />I have a great interest in computer graphics and engine design, and hope to work in the field in the near <br />future and has helped me improve in both fields. I also learned a lot from Ogre, both in coding style <br />and in terms of engine design(I often find myself skimming trough open source engines' doxygen <br />documentation to see how different modules work together). <br />Anything Else<br />My proposal might look a little ambitious, or unfeasible, but I believe that I have the needed skills and <br />motivation to implement it. Plus, I don't have anything planned until early September + The only other commitment I have this summer are my third year exams, and I'll finish those around the 10th of June. That gives me roughly 10 weeks of actual work.

- <HR>

THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.

BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

1. Definitions

• "Collective Work" means a work, such as a periodical issue, anthology or encyclopedia, in which the Work in its entirety in unmodified form, along with a number of other contributions, constituting separate and independent works in themselves, are assembled into a collective whole. A work that constitutes a Collective Work will not be considered a Derivative Work (as defined below) for the purposes of this License.
• "Derivative Work" means a work based upon the Work or upon the Work and other pre-existing works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which the Work may be recast, transformed, or adapted, except that a work that constitutes a Collective Work will not be considered a Derivative Work for the purpose of this License. For the avoidance of doubt, where the Work is a musical composition or sound recording, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered a Derivative Work for the purpose of this License.
• "Licensor" means the individual or entity that offers the Work under the terms of this License.
• "Original Author" means the individual or entity who created the Work.
• "Work" means the copyrightable work of authorship offered under the terms of this License.
• "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
• "License Elements" means the following high-level license attributes as selected by Licensor and indicated in the title of this License: Attribution, ShareAlike.

2. Fair Use Rights

Nothing in this license is intended to reduce, limit, or restrict any rights arising from fair use, first sale or other limitations on the exclusive rights of the copyright owner under copyright law or other applicable laws.

3. License Grant

Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

• to reproduce the Work, to incorporate the Work into one or more Collective Works, and to reproduce the Work as incorporated in the Collective Works;
• to create and reproduce Derivative Works;
• to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission the Work including as incorporated in Collective Works;
• to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission Derivative Works.
• For the avoidance of doubt, where the work is a musical composition:
  • Performance Royalties Under Blanket Licenses. Licensor waives the exclusive right to collect, whether individually or via a performance rights society (e.g. ASCAP, BMI, SESAC), royalties for the public performance or public digital performance (e.g. webcast) of the Work.
  • Mechanical Rights and Statutory Royalties. Licensor waives the exclusive right to collect, whether individually or via a music rights society or designated agent (e.g. Harry Fox Agency), royalties for any phonorecord You create from the Work ("cover version") and distribute, subject to the compulsory license created by 17 USC Section 115 of the US Copyright Act (or the equivalent in other jurisdictions).
  • Webcasting Rights and Statutory Royalties. For the avoidance of doubt, where the Work is a sound recording, Licensor waives the exclusive right to collect, whether individually or via a performance-rights society (e.g. SoundExchange), royalties for the public digital performance (e.g. webcast) of the Work, subject to the compulsory license created by 17 USC Section 114 of the US Copyright Act (or the equivalent in other jurisdictions).

The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats. All rights not expressly granted by Licensor are hereby reserved.

4. Restrictions

The license granted in Section 3 above is expressly made subject to and limited by the following restrictions:

• You may distribute, publicly display, publicly perform, or publicly digitally perform the Work only under the terms of this License, and You must include a copy of, or the Uniform Resource Identifier for, this License with every copy or phonorecord of the Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Work that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Work itself to be made subject to the terms of this License. If You create a Collective Work, upon notice from any Licensor You must, to the extent practicable, remove from the Collective Work any credit as required by clause 4(c), as requested. If You create a Derivative Work, upon notice from any Licensor You must, to the extent practicable, remove from the Derivative Work any credit as required by clause 4(c), as requested.
• You may distribute, publicly display, publicly perform, or publicly digitally perform a Derivative Work only under the terms of this License, a later version of this License with the same License Elements as this License, or a Creative Commons iCommons license that contains the same License Elements as this License (e.g. Attribution-ShareAlike 2.5 Japan). You must include a copy of, or the Uniform Resource Identifier for, this License or other license specified in the previous sentence with every copy or phonorecord of each Derivative Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Derivative Works that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder, and You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Derivative Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Derivative Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Derivative Work itself to be made subject to the terms of this License.
• If you distribute, publicly display, publicly perform, or publicly digitally perform the Work or any Derivative Works or Collective Works, You must keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or (ii) if the Original Author and/or Licensor designate another party or parties (e.g. a sponsor institute, publishing entity, journal) for attribution in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; the title of the Work if supplied; to the extent reasonably practicable, the Uniform Resource Identifier, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work; and in the case of a Derivative Work, a credit identifying the use of the Work in the Derivative Work (e.g., "French translation of the Work by Original Author," or "Screenplay based on original Work by Original Author"). Such credit may be implemented in any reasonable manner; provided, however, that in the case of a Derivative Work or Collective Work, at a minimum such credit will appear where any other comparable authorship credit appears and in a manner at least as prominent as such other comparable authorship credit.

5. Representations, Warranties and Disclaimer

UNLESS OTHERWISE AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE MATERIALS, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU.

6. Limitation on Liability.

EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. Termination

• This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Derivative Works or Collective Works from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License.
• Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.

8. Miscellaneous

• Each time You distribute or publicly digitally perform the Work or a Collective Work, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License.
• Each time You distribute or publicly digitally perform a Derivative Work, Licensor offers to the recipient a license to the original Work on the same terms and conditions as the license granted to You under this License.
• If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.
• No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent.
• This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You.