This is the start of the Non-Photorealistic Rendering series of articles.
The other articles are:

• Edge techniques
• Artistic media overlay

---

Overview

Non-photorealistic (NPR) techniques, as presented here, is about creating/rendering content without photo-realism as the goal. It's about controlling your style and creating content with a specific look in mind. The techniques and approaches listed here are so far focusing on reproducing artistic styles, a common goal of NPR artists.

There is some disagreement over what to call NPR. Suggestions I like are "expressive graphics" or "artistic rendering," but as I first encountered this topic under the title of NPR, I've decided to keep that phrase for now.

Common Challenges

---

When you look at a model and think of representing it without trying to mimic the real appearance, usually the task falls into one of two broad categories of techniques:

• Edge Differentiation

You want the edges of the model to be distinct so that objects on screen are visually separate entities. Edges can be used in any number of ways to add flavor to the representation, but usually the goal is to tell the viewer's eyes how to tell objects apart.

• Internal Shading

If edges are the outlines, internal shading is using crayons to color in between the lines. Fills don't have to work around edges, but usually need to avoid the edges since the design of internal techniques isn't usually suited for drawing things near the edge of a surface, especially true of fill techniques that aren't aware if they are near edges.

These two processes can be aware of each other or interact on many levels and don't necessarily have to be separate. However, because the challenges are very distinct, they are usually tackled with different approaches that by design become separate and distinct.

Post-Processing vs. Direct Techniques

---

There are two broad categories of approaches that entail completely different ways of using the information from the 3D scene. Some prefer to distinguish the two as 2D or 3D techniques, but as I will propose later in this article, it's possible to use data with 3D significance in a 2D process.

• Post-Processing will typically use a series of images created from rendering passes and then execute algorithms that use these values as a basis to generate the final rendered image. Post-processing can be used as a total solution or as a supplement. It's usually more straight-forward since you're working with data that exists in the form of a 2D image, but not always as capable of acting in a manner that is aware of how the data exists in the 3D scene.
• Direct techniques mean you're using shaders that are written to produce output for the final image. Although the hardware is producing the output during the process of rendering the 3D scene, the result will usually look as though it was done with awareness of how the shapes would exist in 2D. Direct processes have the advantage of control. You can assign a different technique/material to every object and thus gain different behavior where desired. In addition, because the direct techniques will be working directly with 3D data, it's more straight-forward to use 3D relationships in producing the final rendered image.

Which is more powerful? Notice that both approaches strive to operate in a way that will produce an image reflecting the information that would be readily available to the other. Direct processes don't always know what their neighbor pixels are doing. Post-processing can no longer work with the raw 3D data. The most powerful technique will use them both to do what the other does poorly and to provide information that can be readily obtained in the 3D stage in a well-packaged format to the 2D stage. Using render layers in Blender 3D's compositor are the best example I can provide.

The Mind of the Expressive Rendering Artist

---

Humans have been striving to portray things since time immemorial. The arts have had innumerable schools in so many cultures even since recorded history began. When looking for a good technique to go for, if you create one that can reproduce a traditional artistic style, you can create a render that doesn't just get recognized by our eyes but speaks to our hands. Not only can we see what it is, but we can feel how it's drawn. For a program to have that kind of impact is amazing. That is what I consider the most powerful aspect of expressive rendering. In context of this discussion, I have to use the phrase expressive rendering, because that's what it's really about.

So you have a model. Now, a model describes many valuable things that a human knows through spatial intelligence. To mimic a technique for transforming that into a 2D image, you have to follow yourself step by step through the process of how you would do it with your hands. What information are you looking at? What part of the information is describing the thing you want your algorithm to know? How can you either extract this data from the 3D data or provide it separately? What aspect of the object would your hand be trying to recreate? How can you describe this relationship mathematically or how can you measure this relationship on the model? These are the questions that will take you from knowing what your hand knows to being able to tell a computer how to mimic it.

Basic Techniques

Here are some tricks that I have successfully implemented in Ogre and how they work:

Screen Coordinate Texture Fetching for Stroke Drawing

---

This is a very basic technique that is very helpful to start building more complex techniques. The idea is to use your knowledge of where a pixel is on screen to do a texture fetch from a simple 2D texture that stores how to draw "strokes." The texture is sized/shaped to match the screen so that the point on the screen corresponds to the correct texel. I call these textures "stroke textures," and they are incredibly valuable in that you can have one stroke texture and use it to create a very good shade that doesn't require the shader code to be aware of what's going on in neighboring pixels. The relationship to neighboring pixels is implicitly correct in that the pixels are fetching from the same texture, which already contains this information of how to look on screen.

Edge envelopes

---

In order to either draw the edges or just have a readily available surface for creating edges, the most common technique is having several copies of the mesh displaced slightly but with reversed normals. The displaced mesh will protrude on one side of the regular mesh. Because the faces are back-facing, most of them will be culled from the render, but the ones that protrude and are not occluded by the front of the regular mesh will be rendered as non-lit poloygons. You can apply any material you want to them, but for black lines, leaving them unlit is the most common solution.

A much, much streamlined version of this technique (and I'm fairly certain this is what's behind the beautiful sumi-e inspired edges in Okami) is to scale the original mesh along its normals (inflating it slightly) and then reversing the normals. This requires only one set of extra vertices to achieve roughly the same geometric description as the previous technique.

Because the inflated mesh has back-facing normals, only the half of the mesh where you are looking into its interior will be rendered. Because the original mesh will render it's front half, it will occlude the normal-flipped mesh everywhere except on the edges or where the normal of the mesh crosses from positive to negative (a mathematical description of the phenomenon). If you render just the inflated, normal-flipped mesh, you will be looking into the concave interior of it, and this is why I call it "edge envelopes." You render an envelope that sticks out at the edges of the front-facing mesh, creating very convenient surfaces to draw edges on.

The beauty of these two techniques is that they don't actually perform edge detection. The edge display is implicit in culling and z-buffering, so there is no shader required. This is how PS2 hardware, with no shaders, was able to draw the very beautiful edges in Okami.

Normal-Driven Internal Shading

---

If you are sketching something by hand start shading the object very diligently, notice how you attempt to follow the surface contours so that the anisotropy (directional quality of the lines you use to shade) will describe the direction of the surface as much as the shading amount describes the lighting on the surface.

Ever looked at a contour map that uses tangent lines to express topology? If you could drop the z-component of the surface normal (or not, I've made techniques that work either way) you would be left with a vector that points away from the surface in 2D. If you shade perpendicular to that vector, you will create lines that flow along the topology in the same step as doing lighting, making a very distinct 3D appearance.

The Ogre head at the top of the page is done with an algorithm that gets the 2D normal and uses that to drive the application of a stroke texture. The correct anisotropy is implicit in how the texture is fetched. Thus the shader, even with no knowledge of what adjacent pixels are doing, is able to apply the right amount of shade in the correct direction (ironic that the shader has no knowledge of direction since it's only shading a single pixel) and produce a very nice 3D shade.

<HR>

THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.

BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

1. Definitions

• "Collective Work" means a work, such as a periodical issue, anthology or encyclopedia, in which the Work in its entirety in unmodified form, along with a number of other contributions, constituting separate and independent works in themselves, are assembled into a collective whole. A work that constitutes a Collective Work will not be considered a Derivative Work (as defined below) for the purposes of this License.
• "Derivative Work" means a work based upon the Work or upon the Work and other pre-existing works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which the Work may be recast, transformed, or adapted, except that a work that constitutes a Collective Work will not be considered a Derivative Work for the purpose of this License. For the avoidance of doubt, where the Work is a musical composition or sound recording, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered a Derivative Work for the purpose of this License.
• "Licensor" means the individual or entity that offers the Work under the terms of this License.
• "Original Author" means the individual or entity who created the Work.
• "Work" means the copyrightable work of authorship offered under the terms of this License.
• "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
• "License Elements" means the following high-level license attributes as selected by Licensor and indicated in the title of this License: Attribution, ShareAlike.

2. Fair Use Rights

Nothing in this license is intended to reduce, limit, or restrict any rights arising from fair use, first sale or other limitations on the exclusive rights of the copyright owner under copyright law or other applicable laws.

3. License Grant

Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

• to reproduce the Work, to incorporate the Work into one or more Collective Works, and to reproduce the Work as incorporated in the Collective Works;
• to create and reproduce Derivative Works;
• to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission the Work including as incorporated in Collective Works;
• to distribute copies or phonorecords of, display publicly, perform publicly, and perform publicly by means of a digital audio transmission Derivative Works.
• For the avoidance of doubt, where the work is a musical composition:
  • Performance Royalties Under Blanket Licenses. Licensor waives the exclusive right to collect, whether individually or via a performance rights society (e.g. ASCAP, BMI, SESAC), royalties for the public performance or public digital performance (e.g. webcast) of the Work.
  • Mechanical Rights and Statutory Royalties. Licensor waives the exclusive right to collect, whether individually or via a music rights society or designated agent (e.g. Harry Fox Agency), royalties for any phonorecord You create from the Work ("cover version") and distribute, subject to the compulsory license created by 17 USC Section 115 of the US Copyright Act (or the equivalent in other jurisdictions).
  • Webcasting Rights and Statutory Royalties. For the avoidance of doubt, where the Work is a sound recording, Licensor waives the exclusive right to collect, whether individually or via a performance-rights society (e.g. SoundExchange), royalties for the public digital performance (e.g. webcast) of the Work, subject to the compulsory license created by 17 USC Section 114 of the US Copyright Act (or the equivalent in other jurisdictions).

The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats. All rights not expressly granted by Licensor are hereby reserved.

4. Restrictions

The license granted in Section 3 above is expressly made subject to and limited by the following restrictions:

• You may distribute, publicly display, publicly perform, or publicly digitally perform the Work only under the terms of this License, and You must include a copy of, or the Uniform Resource Identifier for, this License with every copy or phonorecord of the Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Work that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Work itself to be made subject to the terms of this License. If You create a Collective Work, upon notice from any Licensor You must, to the extent practicable, remove from the Collective Work any credit as required by clause 4(c), as requested. If You create a Derivative Work, upon notice from any Licensor You must, to the extent practicable, remove from the Derivative Work any credit as required by clause 4(c), as requested.
• You may distribute, publicly display, publicly perform, or publicly digitally perform a Derivative Work only under the terms of this License, a later version of this License with the same License Elements as this License, or a Creative Commons iCommons license that contains the same License Elements as this License (e.g. Attribution-ShareAlike 2.5 Japan). You must include a copy of, or the Uniform Resource Identifier for, this License or other license specified in the previous sentence with every copy or phonorecord of each Derivative Work You distribute, publicly display, publicly perform, or publicly digitally perform. You may not offer or impose any terms on the Derivative Works that alter or restrict the terms of this License or the recipients' exercise of the rights granted hereunder, and You must keep intact all notices that refer to this License and to the disclaimer of warranties. You may not distribute, publicly display, publicly perform, or publicly digitally perform the Derivative Work with any technological measures that control access or use of the Work in a manner inconsistent with the terms of this License Agreement. The above applies to the Derivative Work as incorporated in a Collective Work, but this does not require the Collective Work apart from the Derivative Work itself to be made subject to the terms of this License.
• If you distribute, publicly display, publicly perform, or publicly digitally perform the Work or any Derivative Works or Collective Works, You must keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or (ii) if the Original Author and/or Licensor designate another party or parties (e.g. a sponsor institute, publishing entity, journal) for attribution in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; the title of the Work if supplied; to the extent reasonably practicable, the Uniform Resource Identifier, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work; and in the case of a Derivative Work, a credit identifying the use of the Work in the Derivative Work (e.g., "French translation of the Work by Original Author," or "Screenplay based on original Work by Original Author"). Such credit may be implemented in any reasonable manner; provided, however, that in the case of a Derivative Work or Collective Work, at a minimum such credit will appear where any other comparable authorship credit appears and in a manner at least as prominent as such other comparable authorship credit.

5. Representations, Warranties and Disclaimer

UNLESS OTHERWISE AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE MATERIALS, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU.

6. Limitation on Liability.

EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. Termination

• This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Derivative Works or Collective Works from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License.
• Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.

8. Miscellaneous

• Each time You distribute or publicly digitally perform the Work or a Collective Work, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License.
• Each time You distribute or publicly digitally perform a Derivative Work, Licensor offers to the recipient a license to the original Work on the same terms and conditions as the license granted to You under this License.
• If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.
• No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent.
• This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You.