History: TindalosNotes

            Welcome to the ChangeLog and porting notes for __OGRE 2.0.0 "Tindalos"__ - work in progress

{QUOTE(replyto="Frank Belknap Long")}"They are lean and athirst!" he shrieked... "All the evil in the universe was concentrated in their lean, hungry bodies. Or had they bodies? I saw them only for a moment, I cannot be certain." - Frank Belknap Long, "The Hounds of Tindalos"{QUOTE}

{maketoc}

!!Updates -- IMPORTANT 2014/12/06
Original plan for 2.0 is ready and currently available as branch 2.0 from the repository; labelled "CTP" (Community Technology Preview). We never managed to package it as a downloadable version; but it's quite useful and stable.
It's the most similar version to 1.x but with a new, more powerful Compositor, and much faster Frustum culling and Scene node updates.

After discussion with the rest of the team, what belongs here to "Ogre 2.1" is now referred as "Ogre 2.0 Final"; which is a deep refactor in API usage:
* It removes support for D3D9 (See http://www.yosoygames.com.ar/wp/2014/04/dx9-may-be-dropped-in-ogre-2-0-final/)
* Refactored how materials are handled. (See http://www.yosoygames.com.ar/wp/2014/05/a-glimpse-of-whats-comming-to-ogre-2-0-final/)
* New material system, much friendlier and easier to use.

For updates on progress follow my board on [https://trello.com/b/59pirbgD/ogre-2-0-final/|Trello]. It is much easier to maintain than a Wiki and is up to date. JIRA is nice but it's better for tracking bugs, it isn't quick for keep track of progress. The trello board contains only my personal tasks required to bring Ogre 2.0 Final to completion.

This page will be kept as is as history record.

Matías (aka. dark_sylinc)

!!ChangeLog
* Not started

!!Porting notes
* None yet

!!Annotated roadmap
* SceneManager redesign
** Very large world support (deal with precision issues at large coordinates)
** Support for rendering a single scene using multiple SceneManagers 
** Significant decoupling of SM roles & responsibilities
** SceneNode optional, can integrate objects directly with external state system (Ogre == View in MVC)
** Asynchronous updates, support multithreading where Model is updated at different rate to View
* Construct Camera & Light via MovableObjectFactory instances

dark_sylinc notes:
After much consideration, it appears that it would be best to follow Blender's strategy towards their redesign, where they released several alpha/unstable versions with official numbering (the 2.5x branch) until around 2.53 it became stable enough & with enough features to become production ready.
Their redesign was done rather in incremental steps than complete write from scratch.
Each version down below will lack some features (since they'll become incompatible) but it should be significally faster/stabler/with more features than 1.x

!!Missing stuff that needs to be designed for 2.x:
* Design idiom (material syntax & code interface) for binding shaders using constant buffers for GL & D3D11 instead of constant registers
* Design Compute Shaders interface (must work within Compositor manager for integrationg with rendering pipeline, and outside of it for regular GPGPU)
* Resource System replacement. Should be scalable, load in background, easier to work with; and can "define itself" so that exporters and other DCC tools can read the Resource config and ease the importing process of assets into Ogre content.

!!2.0 - Cache misses and ground base
!!!Scene Manager
{FANCYTABLE( head=" Task | Status ")}
Create all SoA counterparts (SoA_Vector3; SoA_Matrix4) | __DONE__
Create small unit tests for SoA math | __DONE__
Implement fsel (conditional move) | __DONE__
Replace Node::mChildren from ChildNodeMap to a sorted vector and use std::lower_bound for search. | __DONE__ (1)
No mutables in Node (makes a lot harder to detect race conditions when accessing supposedly read-only data at the same time) | __DONE__
Remove Node::mNeedParentUpdate, Node::mNeedChildUpdate, Node::mCachedTransformOutOfDate except for debug asserts & implement Node::_getDerivedPosition + Node::_getDerivedPositionUpdated | __DONE__
Refactor Node::mInitialPosition, Node::mInitialOrientation, Node::mInitialScale: They're only used on keyframed Nodes (i.e. barely used) | __DONE__
Create flexibility levels (virtual_l0, virtual_l1, virtual_l2). Set Node::get & setPosition to the levels specified in the slides | __DONE__
Disable (temporarily) OctreeSceneManager & co. Create a null HighLevelCull that will keep all nodes in one Chunk (grouped per render queue ID, grouped by hierarchy depth for breath first) | __DONE__
  ** Must handle growing of the Chunks. | __DONE__
  ** Create MemoryManager that will control how memory is assigned to SoA_Vector3 AND to Nodes (so that Nodes are also contiguous, but not SoA) | __DONE__
  ** Until CompositorManager is refactored, all nodes will be grouped on the same render queue ID. | __DONE__
  ** Modify Node::_update to be used only on demand's Node::_getDerivedPositionUpdated | __DONE__
  ** Replace Node::_update with a function called UpdateAllTransforms() that interates on all nodes chunks. Shouldn't be virtual. Will be called by SceneManager for the moment. | __DONE__
     *** Write UpdateAllTransforms() code with parallelization in mind! | __DONE__
Thread Manager (can be delayed to 2.1) | __DONE__
Tool to split submeshes into multiple Entities (useful when a submesh must go in a different render queue) | __POSTPONED__
Render queues go from [0; user defined), no gaps. In order to be more flexible & array friendly. (Remove currently fixed "[10; 108)" range) | __POSTPONED__
Adapt Aabb according to LightType and update if position changes | __DONE__
All entities are unique by ID, not by name. An incrementing counter (i.e. mId = ++mIdCount) is recomended | __DONE__
Entity names may be allowed, but use fixed strings. Not variable-length (2) | __DONE__
Implement compositor manager | __DONE__
Each compositor has multiple input textures (superior method than "input previous"; just defined "texture input 0" avoids duplicating resources as a bonus) as well as global texture names | __DONE__
Each compositor has multiple (named) output targets | __DONE__
Make Frustum::isVisible SoA and parallel | __DONE__
Null LowLevelCuller for the moment | __DONE__
Implement UpdateAllAnimations() | __DONE__
Group all bones by hierarchy for breath first. | __DONE__
Do SoA in all Bones | __DONE__
Integrate OgreAnimation into OgreCore | __DONE__
Integrate OgreAnimation into HW VTF instancing | __DONE__
Integrate OgreAnimation into Entity & SubEntity | __POSTPONED__
Restore LOD and allow switching between LOD techniques | __DONE__

{FANCYTABLE}

(1) Not using sorting, but a better approach.
* See http://www.yosoygames.com.ar/wp/2013/06/ogre-2-0-o1-insertion-removal-and-contiguous-memory-iteration/ Blog post

(2) Interesting reads:
* http://home.comcast.net/~tom_forsyth/blog.wiki.html#[[A%20sprintf%20that%20isn%27t%20as%20ugly]] Fixed length strings
* http://bitsquid.blogspot.com.ar/2011/06/strings-redux.html Hashed strings
Update: Hashed strings ended up being used for unique names. But where names are optional (ie. Entity) an Id is used; the names are variable-length, but empty by default.
!!!RenderSystem
* Replace "create read only -> lock" pattern for "initialize on creation" pattern in order to ease D3D11 development.

!!!Missing broken stuff
*Intersection & PlaneBoundedVolumeList queries
*--TagPoints-- __UPDATE 2.1:__ v2 TagPoints are working v1 TagPoints still broken and wontfix. Checkout TagPointListener workaround: http://www.ogre3d.org/forums/viewtopic.php?f=4&t=84650#p522055.
*--Shadows in Terrain component-- __UPDATE 2.1:__ Done in 2.1-pso branch using new Terra system and compute shaders.
*Volume component

Samples:
*Character: Crashes when withdrawing weapon as TagPoints aren't supported yet. __UPDATE 2.1:__ see TagPoints above.
*DeferredShading: Needs to be ported to the new compositor.
*--SSAO: Compositor scripts need to be ported.-- __UPDATE 2.1:__ __DONE__. See new SSAO sample.

__The only "hard" sample to port could be ''DeferredShading''.__
!!2.1a
!!!--Phase One: Setting up base infrastructure--
* --Implement UpdateAllAnimations()--
** --Modify the Mesh format to store bones in the same way (but store 1 float only "XYZXYZ" for cross platform and easy parsing. Do the swizzle conversion when loading if needed) __DONE__--
** --Write update code with parallelization in mind! __DONE__--
** --TagPoints are now SceneNodes. (TagPoints have lots of issues) __DONE__--
!!2.1b - Features back
!!!Phase Two: Restoring advanced culling & compositing abilities
* Implement HighLevelCull grid as in the slides.
** Can merge all RenderQueueIDs that can be ever be requested by current compositor chain, so HighLevelCull can know which ones it should send
** Can workout compositor dependencies (i.e. Stream Out & shadow passes) __DONE__
** Keyword hint on cull list reuse (so Compo Mgr decides whether to create two identical lists of culled objects in 2 different threads for each compositor pass, or process them sequentially in one thread and reuse the list).
** Define compositor syntax to work with Stream Out.
* Add MetaData to Mesh format
** Metadata: Support bounding geometry for sw occlusion culling
** Metadata: Number ID system for keyframed animation, to allow embedding sound cue information, particle effects trigger, etc.
* Implement low level depth-only sw rasterizer for occlusion culling
* Get rid of "debug visualizers/debug renderables" in Frustum and Nodes (which are adding unnecessary dependencies and don't belong there)

__-- At this point Ogre 2.1b is similar to 1.x, however it should be much faster & efficient. --__

!!2.3 - Rendering features
!!!RenderSystem
* --Removal of all fixed function state managing out of Scene Manager & RenderSystem __DONE__--
* --Put all state changing code into one or multiple state blocks. __DONE__--
* --Add support for Depth textures (including HW PCF) __DONE__--
** --GL, DX9 (through INTZ fourcc), and DX11 __DONE__--
* Add HW Gamma keyword toggle on the material (in 1.x if the texture name is not defined, gamma can't be turned on through scripts)
* Add ".texture" files where textures may be defined for certain particularities (i.e. change of format, load for gamma correction) before all materials are parsed.
* --Add normal compression. __DONE__--
* --Interface to pass data through texture registers (VTF for DX9) __DONE__--
** --HW Skinning through texture registers __DONE__--
** Morph Targets through through texture registers
* Replacement of "camera relative rendering" hack in favour of true working in camera space matrices.
** Binding to pass HW Skinning matrices in local space rather than world space.
* --Add Compute Shaders (GPGPU) to system. May be managed from CompositorManager (i.e. for rendering) or manually (for pure GPGPU) __DONE__--
** Implement bokeh demo using atomic counters
** --Implement Light Indexed Deferred demo. __Considered DONE (via Forward3D)__--
** Implement Real Time Global Ilumination demo.

!!2.4 - Flexibility, tools and integration
* Remove manually setting VertexElement::mOffset as it induces to commong mistakes (may remove the variable itself if caching proves to be unnecessary), because this value can be deduced from declaration order.
* Vertex format enhancements (custom flexible vlayout files)

!!2.5-2.9
TODO
* Add next gen demos.

!!3.0
!!!First stable release

!!Talks about it
* http://www.ogre3d.org/forums/viewtopic.php?t=30250|"Tindalos" (Ogre v2.0) SceneManager redesign forum thread
* http://www.ogre3d.org/forums/viewtopic.php?f=25&t=75459 |Ogre 2.x engine