{"id":16131350,"url":"https://github.com/calinou/tesseract-renderer-design","last_synced_at":"2025-04-06T14:32:23.255Z","repository":{"id":54700360,"uuid":"45566557","full_name":"Calinou/tesseract-renderer-design","owner":"Calinou","description":"Tesseract renderer design document, originally written by Lee \"eihrul\" Salzman","archived":false,"fork":false,"pushed_at":"2022-02-01T23:35:49.000Z","size":18,"stargazers_count":14,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-29T12:34:56.682Z","etag":null,"topics":["cube2","cube2-engine","deferred-rendering","tesseract","tesseract-game"],"latest_commit_sha":null,"homepage":"http://tesseract.gg","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Calinou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-11-04T20:49:48.000Z","updated_at":"2024-08-26T17:07:28.000Z","dependencies_parsed_at":"2022-08-14T00:30:28.966Z","dependency_job_id":null,"html_url":"https://github.com/Calinou/tesseract-renderer-design","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Calinou%2Ftesseract-renderer-design","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Calinou%2Ftesseract-renderer-design/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Calinou%2Ftesseract-renderer-design/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Calinou%2Ftesseract-renderer-design/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Calinou","download_url":"https://codeload.github.com/Calinou/tesseract-renderer-design/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247495578,"owners_count":20948077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cube2","cube2-engine","deferred-rendering","tesseract","tesseract-game"],"created_at":"2024-10-09T22:24:55.660Z","updated_at":"2025-04-06T14:32:23.222Z","avatar_url":"https://github.com/Calinou.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Tesseract Rendering Pipeline\n\n**This article is a brief overview of the rendering pipeline in\n[Tesseract](http://tesseract.gg), an open-source FPS game and level-creation\nsystem.** It assumes basic knowledge of modern rendering techniques and is\nmeant to point out particular design decisions rather than explain in detail\nhow the renderer works.\n\n## Platform\n\nThe Tesseract renderer ideally targets OpenGL 3.0 or greater, using only Core\nprofile functionality and extensions, as its graphics API and using SDL 2.0 as a\nplatform abstraction layer across Windows, Linux, BSD, and OS X. However, it\nalso functions on OpenGL 2.1 where extensions are available to emulate Core\nprofile functionality. Mobile GPUs with OpenGL ES are, at the moment, not\ntargeted. GLSL compatibility is provided across various incompatible versions\nusing a small preprocessor-based prelude of macros abstracting texture access\nand attribute/interpolant/output specification that would otherwise be\nincompatible across GLSL versions.\n\n## Motivation\n\nThe main design goal of Tesseract is both to support high numbers of dynamically\nshadowmapped omnidirectional point-lights and minimize the number of rendering\npasses used with respect to its predecessor engine, Sauerbraten. While improved\nvisuals over Sauerbraten were somewhat desired, it was more important to make\nthe rendering process more dynamic with at least comparable visual fidelity\nwhile still emphasizing performance/throughput, although at a higher baseline\nthan Sauerbraten required. So while other potential design choices might have\nresulted in further visual improvements, they were ultimately discarded in the\nservice of more reasonable performance.\n\nSauerbraten made many redundant geometry passes per frame for effects such as\nglow, reflections and refractions, and shadowing that multiplied the cost of\ngeometry intensive levels and complex material shaders. So, where possible,\nTesseract instead chooses methods that factor rendering costs, such as deferred\nshading which allows complex material shaders to be evaluated only once per\nlighting pass, or screen-space effects for things such as reflection that reuse\nrendering results rather than require more subsequent rendering passes.\n\nFurther, Sauerbraten was reliant on precomputed lightmapping techniques to\nhandle lighting of the world. This approach posed several problems. Dynamic\nentities in the world could not fully participate in lighting and so never\nlooked quite \"right\" due to mismatches between dynamic and static lighting\ntechniques. Lightmap generation took significant amounts of time, making light\nplacement a painful guess-and-check process for mappers, creating a mismatch\nbetween the ease of instantly creating the level geometry and the\nnot-so-instant-process of lighting it. Finally, storage of these lightmaps\nbecame a concern, so low-precision lightmaps were usually chosen, at the cost of\nappearance to reduce storage requirements. Tesseract instead chooses to use a\nfully dynamic lighting engine to resolve the mismatch between lighting of\ndynamic and static entities while making better trade-offs between appearance\nand storage requirements.\n\nCertain features of Sauerbraten's renderer that were otherwise functional such\nas occlusion culling, decal rendering, and particle rendering have mostly been\ninherited from Sauerbraten and are not extensively detailed in this document, if\nat all. For more information about Sauerbraten in general, see [Sauerbraten's\nhomepage](http://sauerbraten.org).\n\n## Shadows\n\nTesseract's shadowing setup is built around observations originally made while\nimplementing omnidirectional shadowmapping in the\n[DarkPlaces](http://icculus.org/twilight/darkplaces/) engine.\n\nThe first observation is that by use of the texture gather (a.k.a. fetch4)\nfeature of modern GPUs, it is possible to implement a weighted box PCF filter of\nsufficient visual quality that generally performs better than all other\ncompeting shadowmap filters while also requiring less bandwidth-hungry shadowmap\nformats, especially if 16bpp depth formats are used. This is advantageous over\ncompeting methods such as variance shadowmaps or exponential shadowmaps that\nprefer high-precision floating-point texture formats or prefiltering with\nseparable blurs that can become quite costly when shadowmaps are atlased into a\nlarger texture as well as suffering from light bleeding artifacts that plain old\nPCF does not have any problems with. A final benefit of relying only on plain\nold depth textures and PCF is that depth-only renders are generally accelerated\non modern GPUs and so provide a speedup for rendering the shadowmaps in the\nfirst place over other techniques before they are ever sampled.\n\nFor general information about PCF filters, see [this\npage](http://www.gdcvault.com/play/10092/Efficient-PCF-Shadow-Map).\n\nIt was later discovered that this same weighted box filter could be approximated\nwith the native bilinear shadowmap filter (originally limited to Nvidia hardware\nunder the \"UltraShadow\" moniker, but now basically present on all DirectX 10\nhardware when using a shadow sampler in combination with linear filtering) so\nthat no texture gather functionality is even required, and allowing further\nperformance enhancements. The particular approximation avoids use of\ndivision/renormalizing blending weights while only causing a slight sharpening\nof the filter result that is almost indistinguishable from the aforementioned\nweighted box filter. This method in general, though, allows the (approximated)\nNxN weighted box filters to be implemented in about (N+1)/2*(N+1)/2 taps. The\ndefault shadowmap filter provides a 3×3 weighted box filter using only 4 native\nbilinear taps, providing a good balance between performance and quality.\n\nThe final 3×3 filter utilizing native bilinear shadow taps contains some\nnon-obvious voodoo and was largely found by experimenting with fast\napproximations for renormalizing filter weights in the weighted box filter.\nUltimately it was discovered that just the seed value for iteration via\n[Newton's method](http://en.wikipedia.org/wiki/Newton's_method) was more than\nsufficient to compute filter weights and did not significantly impact the look\nof the result. Texture rectangles are also used where possible instead of\nnormalized 2D textures to avoid some extra texture coordinate math. The filter\n(with the unoptimized yet more precise box filter in comments) is listed here\nfor posterity's sake:\n\n```cpp\n#define shadowval(center, xoff, yoff) float(shadow2DRect(shadowatlas, center + vec3(xoff, yoff, 0.0)))\nfloat filtershadow(vec3 shadowtc)\n{\n    vec2 offset = fract(shadowtc.xy - 0.5);\n    vec3 center = shadowtc;\n    //center.xy -= offset;\n    //vec4 size = vec4(offset + 1.0, 2.0 - offset), weight = vec4(2.0 - 1.0 / size.xy, 1.0 / size.zw - 1.0);\n    //return (1.0/9.0) * dot(size.zxzx * size.wwyy,\n    //    vec4(shadowval(center, weight.zw),\n    //         shadowval(center, weight.xw),\n    //         shadowval(center, weight.zy),\n    //         shadowval(center, weight.xy)));\n    center.xy -= offset*0.5;\n    vec4 size = vec4(offset + 1.0, 2.0 - offset);\n    return (1.0/9.0) * dot(size.zxzx * size.wwyy,\n        vec4(shadowval(center, -0.5, -0.5),\n                shadowval(center, 1.0, -0.5),\n                shadowval(center, -0.5, 1.0),\n                shadowval(center, 1.0, 1.0)));\n}\n```\n\nThis idea is extended to larger filter radiuses but is not shown here.\n\nAfter experimenting with different projection setups for omnidirectional shadows\nsuch as tetrahedral (4 faces) or dual-parabolic (2 faces), it was found that the\nordinary cubemap (6 faces) layout was best as the larger number of smaller\nfrustums actually provides better opportunities for culling and caching of faces\nwhile providing the least amount of projection distortion. However, for\nmulti-tap shadowmap filters, the native cubemap format is insufficient for\neasily computing the locations of neighboring taps. Also, despite texture arrays\nallowing for batching of many shadowmaps during a single rendering pass, they do\nnot allow adequate control of sizing of individual shadowmaps and their\npartitions.\n\nFor further information about the basics of rendering cubemap shadowmaps, see\npage 42+ of [this\nPDF](https://http.download.nvidia.com/developer/presentations/2004/GPU_Jackpot/Shadow_Mapping.pdf).\n\nBoth of these problems may be addressed by unrolled cubemaps, or rather, by\nemulating cubemaps within a 2D texture by manually computing the offset of each\n\"cubemap\" face within an atlas texture. The face offset needs only to be\ncomputed once and then any number of filter taps can be cheaply computed based\non that offset. The perpective projection of each frustum must be slightly wider\nthan the necessary 90 degree field-of-view, to allow the filter taps to sample\nsome texels outside of the actual frustum bounds without crossing any face\nboundaries. A filter with an N texel radius needs a face border of at least that\nmany texels to account for such out-of-bounds taps.\n\nFurther, it becomes trivial to support custom layouts based on modifying the\nunrolled lookup algorithm, or to allow other types of shadowmap projections to\nco-exist with the unrolled cubemaps in a single texture atlas. Yet another\nadvantage of the cubemap approach in general, not limited to unrolled cubemaps,\nis that rather than sampling omnidirectional shadows frustum-by-frustum\n(requiring as many as 6 frustums) as some other past engines do and needing\ncomplicated multi-pass stenciling techniques limit overdraw, the omnidirectoinal\nshadowmap may be sampled in a single draw pass over all affected pixels.\n\nInitially, this emulation was done by use of a cubemap (known as a \"Visual\nShadow Depth Cube Texture\" or VSDCT) to implement the face offset lookup to\nindirect into the texture atlas. Later, an equally efficient sequence of simple,\ncoherent branches was discovered that obviated the need for any lookup texture\nand removed precision issues inherent in the lookup texture strategy. The lookup\nfunction that provided the best balance of performance across Nvidia and AMD\nGPUs is listed here:\n\n```cpp\nvec3 getshadowtc(vec3 dir, vec4 shadowparams, vec2 shadowoffset)\n{\n    vec3 adir = abs(dir);\n    float m = max(adir.x, adir.y);\n    vec2 mparams = shadowparams.xy / max(adir.z, m);\n    vec4 proj;\n    if(adir.x \u003e adir.y) proj = vec4(dir.zyx, 0.0); else proj = vec4(dir.xzy, 1.0);\n    if(adir.z \u003e m) proj = vec4(dir, 2.0);\n    return vec3(proj.xy * mparams.x + vec2(proj.w, step(proj.z, 0.0)) * shadowparams.z + shadowoffset, mparams.y + shadowparams.w);\n}\n```\n\nThis function overall maps a world-space light-to-surface vector to texture\ncoordinates within the shadowmap atlas. A useful trick is used in the first few\nlines for computing a depth for the shadowmap comparison - the maximum linear\ndepth along the 3 axial projections is ultimately the linear depth for the\ncubemap face that will be later selected - and allows the depth computation to\nhappen before the resulting projection is found via branching giving slightly\nbetter pipelining here. Note that this lookup function assumes a slightly\nnon-standard orientation for the rendering of cubemap faces that avoids the need\nto flip some coordinates relative to the native cubemap face orientations. It\notherwise lays out the faces in a 3×2 grid. Various math has been baked into\nuniforms and passed into this function to transform to post-perspective depth\nfrom linear depth for the later actual shadowmap test in the filtershadow\nfunction. The function is only listed here to give an idea of the performance of\nthe unrolled cubemap lookup, so reader beware, it is not quite plug-and-play and\nsome investigation of the engine source code is required for more details. The\nresult of this lookup function is then passed into the filtershadow function\nlisted above. These two little functions are rather important and inspired\nTesseract's design; they represent its beating heart and make large numbers of\nomnidirectional shadowed lights possible.\n\nAll of the shadowmaps affecting a single frame are further aggregated into one\ngiant shadowmap atlas, currently 4096×4096 using 16bpp depth texture format.\nThis better decouples the shadowmap generation and lighting phases and allows\nlookups for any number of shadowmaps to be easily performed in a single batch or\nmany shading passes. Various types of shadowmaps are stored in the atlas:\nunrolled cubemap shadowmaps for point lights, a simple perspective projection\nfor spotlights, and cascaded shadowmaps for directional sunlight.\n\nFor cascaded shadowmaps for sunlight, Tesseract uses an enhanced parallel-split\nscheme with rotationally invariant world-space bounding boxes rounded to stable\ncoordinate increments for each split as originally detailed for Dice's Frostbite\nengine. This allows for somewhat less waste of available shadowmap resolution\nthan the standard view-parallel split scheme as well as combats temporally\ninstability/shadow swim that would otherwise occur. For further information, see\n[this page](https://web.archive.org/web/20121105134010/http://dice.se:80/publications/title-shadows-decals-d3d10-techniques-from-frostbite/).\n\n\"Caching is the new culling.\" Lights can often have large radiuses that pass\nthrough walls and other such occluders, often making occlusion culling or\nview-frustum culling of light volumes ineffective. As an alternative to never\nthe less greatly reduce shadowmap rendering costs for such lights, the shadowmap\natlas caches shadowmaps from frame to frame, down to the granularity of\nindividual cubemap faces, if no moving objects are present in the shadowmap.\nLights in Tesseract usually only affect static world geometry, at least when\nindividual cubemap faces are considered, so the majority of shadowmapped lights\nare not more expensive than unshadowed lights, adding only the cost of the\nshadowmap lookup and filtering itself. To further optimize the rendering of\nshadows for static geometry, for each frustum of each light, an optimal mesh is\ngenerated of all triangles contained only within that frustum and omitting all\nbackfacing triangles. To avoid moving textures around within the atlas, cached\nshadowmaps attempt to retain their placement from the previous frame within the\natlas. To combat fragmentation, if the atlas becomes overly full, cached\nshadowmaps are occasionally evicted from a quadrant window of the atlas that\nprogresses through the atlas from frame to frame.\n\n## Deferred shading and the g-buffer\n\nAfter evaluating many alternatives, given the small range of materials used in\nTesseract maps, it was decided that deferred shading, in contrast to competing\nmethods such as light pre-pass or light-indexed, was the most sensible method\nfor the actual shading/lighting step. Deferred shading provides other benefits\nsuch as easy blending of materials in the g-buffer before the actual shading\nstep takes place.\n\nFurther, by use of tiled approaches to deferred shading, the cost of sampling\nthe g-buffer can be largely amortized, to the extent that Tesseract's renderer\nis, in fact, compute bound by the cost of evaluating the actual per-light\nlighting equation on lights that pass culling/rejection tests, rather than bound\nby bandwidth or culling costs as other deferred renderers and related research\nclaim to be.\n\nFor further information about the trade-offs involved in various deferred\nrendering schemes, see [this\npage](http://c0de517e.blogspot.com/2011/01/mythbuster-deferred-rendering.html)\nor [this\none](http://software.intel.com/en-us/articles/deferred-rendering-for-current-and-future-rendering-pipelines)\nor [this one](http://aras-p.info/blog/2012/03/27/tiled-forward-shading-links/).\n\nTesseract breaks the screen up into a grid of 10x10 tiles aligned to pixel group\nboundaries. Lights are then inserted into per-tile lists by computing a 2D\nbounding box of affected tiles. Finally, lights are batched into groups of at\nmost 8 lights of equivalent type (shadowed or unshadowed, point or spot light)\nwhich are then evaluated per-tile in a single draw call. As many calls to the\ntile shader are made as necessary to exhaust all the lights in the per-tile\nlist. Other lighting effects such as sunlight, ambient lighting, or global\nillumination is also optionally applied by the tile shader.\n\nIt was found that beyond coarse light culling and per-tile bucketing, more\ncomplicated schemes in the fragment shader for culling lights, possibly\ninvolving dynamic light lists, yield little to no actual speedups when measured\nagainst simpler rejection tests that involve static uniforms. The bandwidth\ncosts of accessing dynamic light lists tend to actually exceed the costs of\naccessing the g-buffer in a tiled renderer thus motivating a simpler tile\nshader. As even using uniform buffers to store light parameters imposes a cost\nsimilar to a texture access, it is strongly preferred to use statically indexed\nuniforms to supply light parameters to minimize the cost of iterating through\nlight parameters in the tile shader.\n\nFor information about computing accurate screen-space bounding rectangles for\npoint light sources, see [this page](http://www.terathon.com/code/scissor.html).\n\nThe actual g-buffer is composed a depth24-stencil8 depth buffer, an RGBA8 with\ndiffuse/albedo in RGB and specular strength in A, an RGBA8 with world-space\nnormal in RGB and either a scalar glow value or an alpha transparency or a\nmulti-purpose anti-aliasing mask/depth hash value in A. When rendering\ntransparenct objects, an extra packed-HDR (or RGBA8 if required) texture is used\nwith additive/emissive light in RGB. On some platforms, a further RGBA8 texture\nis used to store a piece-wise encoded linear depth where directly accessing from\nthe depth buffer texture is either slow or buggy. RGBA8 textures are used for\nall layers of the g-buffer both to support older GPUs that can't accept multiple\nrender targets of varying formats and because RGBA8 textures provide a good\ntrade-off between size and encoding flexibility.\n\nWorld-space RGB8 normals are chosen as they are both temporally stable (no\nframe-to-frame jitter artifacts) and require almost no encode/decode costs like\nother eye-space normal encodings might. The additive/emissive layer allows for\neasy handling of environment maps or reflection effects. Overall, this layout\nprovides a reasonably compact g-buffer while handling the range of materials\nused by Tesseract maps. For more information about g-buffer normal encodings,\nsee [this page](http://aras-p.info/texts/CompactNormalStorage.html).\n\n## Mesh rendering\n\nWhere possible, Tesseract utilizes support for half-precision floating point in\nmodern GPUs to reduce the memory footprint of mesh vertex buffers. Instead of\nthe usual tangent and bitangent representation, Tesseract utilizes quaternion\ntangent frames (QTangents) for compressed tangent-space specification of mesh\ntriangles, which combine well with the existing use of dual-quaternion skinning\nfor animated meshes. For more information about QTangents, see [this\npage](http://www.crytek.com/cryengine/presentations/spherical-skinning-with-dual-quaternions-and-qtangents).\n\n## Decal rendering\n\nBefore final shading, any decaling effects are applied to the scene by blending\ninto the g-buffer.\n\n## Material shading/Light accumulation\n\nThe shading is evaluated into a light accumulation buffer, containing the final\nshaded result, that preferably uses the R11G11B10F packed floating-point format.\nWhen the GPU hardware does not support packed float format or is otherwise buggy\n(as observed on some older AMD GPUs that do not properly implement blending of\npacked float format render targets), a fallback RGB10 fixed-point format is used\nthat is scaled to a 0..2 range to allow some overbright lighting and still\nprovide somewhat better precision than an LDR RGB8 format. For more information\nabout the packed floating-point format and its limitations, see [this OpenGL\nspecification](http://www.opengl.org/registry/specs/EXT/packed_float.txt).\n\nLinear-space lighting and sRGB textures have also been avoided here because,\nduring experimenting, it was found they unavoidably produce lighting values that\nquantize poorly in these lower precision formats, and the bandwidth cost of\nhigher-precision formats was ultimately not worth the perceived benefits. The\ngamma-space lighting curves and values are well understood by Sauerbraten\nmappers, providing for better fill lighting due to softer/less harsh lighting\nfalloff, interoperating better with a wealth of pre-existing textures optimized\nto look appealing under gamma-space lighting, and producing fewer banding\nartifacts with lower-precision HDR texture formats.\n\nWhile the lighting thus still happens in gamma-space, overbright lighting values\nare never the less supported and utilized, motivating later tonemapping and\nbloom steps.\n\nFor more information about the trade-offs involved in working in gamma-space vs.\nlinear-space, see [this page](http://filmicgames.com/archives/299).\n\n## Screen-space ambient obscurance\n\nTo help break up the monotony of those indoor areas of a map that may rely on\nambient lighting and to help reduce the burden of requiring lots of point lights\nto provide contrast in such places, Tesseract implements a form of SSAO.\n\nAfter the g-buffer has been filled, but before the shading step, the depth\nbuffer is downscaled to half-resolution and SSAO is computed, utilizing both the\ndownscaled depth and the normal layer of the g-buffer, into a another buffer\npacking both the noisy/unfiltered obscurance value and a copy of the depth in\neach texel. This buffer is then bilaterally filtered, efficiently sampling both\nthe obscurance and depths in a single tap due to the aforementioned packing\nscheme. The final resulting buffer is then used to affect sunlight and ambient\nlighting in the deferred shading step.\n\nIn particular, Tesseract makes use of the Alchemy Screen-Space Ambient\nObscurance algorithm detailed here: [this\npage](http://graphics.cs.williams.edu/papers/AlchemyHPG11/). Tesseract further\nincorpates improvements to the algorithm suggested\n[here](http://graphics.cs.williams.edu/papers/SAOHPG12/).\n\n## Global illumination\n\nOne of the primary motivations for including global illumination in Tesseract\nwas not so much to increase visual quality, but instead to actually increase\nperformance. While Tesseract can support a large number of shadowed lights,\neventually mappers with the best of intentions can defeat the best of engines.\nSo having some form of indirect/bounced lighting allows for light to get to\nnormally dark corners in a map that would otherwise require a lot of \"fill\"\nlights to brighten them up or otherwise rely on ugly/flat ambient lighting.\nTesseract provides a form of diffuse global illumination for the map's global,\ndirectional sunlight that can thus help to brighten up maps, without requiring a\nlot of point light entities, so long as a mapper is careful to allow sunlight\ninto the map interior.\n\nDiffuse global illumination is computed only for sunlight using the Radiance\nHints algorithm, which is similar to but distinct from Light Propagation\nVolumes. First, a reflective shadowmap is computed for the scene from the sun's\nperspective, storing both the depth and reflected surface color for any surface\nthe sunlight directly hits. Then using a particular random sampling scheme, the\nreflective shadowmap is gathered into a set of RGBA8 cascaded 3D textures\nstoring low-order spherical harmonics. 3D textures are used for both Radiance\nHints and LPV algorithms as they allow for cheap trilinear filtering of the\nspherical harmonics. However, Radiance Hints still differs from the LPV approach\nin that it gathers numerous samples from the reflective shadowmap in one shading\npass, rather than injecting seed values into a 3D grid and iteratively refining\nit, offering some performance and simplicity advantages for the case of\nsingle-bounce diffuse global illumination.\n\nDuring this process, an ambient occlusion term is also computed beyond what is\ndetailed by the basic Radiance Hints algorithm. Where possible, these 3D\ntextures are cached from frame to frame. These cascaded 3D textures are then\nsampled in the shading step to provide both the sunlight global illumination\neffect as well as using the ambient obscurance term to implement an\natmospheric/skylight effect.\n\nFor further information on Radiance Hints, see \"Real-Time Diffuse Global\nIllumination Using Radiance Hints\" at [this\nPDF](http://graphics.cs.aueb.gr/graphics/docs/papers/RadianceHintsPreprint.pdf)\nor [this page](http://graphics.cs.aueb.gr/graphics/research_illumination.html).\n\n## Transparency, reflection, and refraction\n\nSauerbraten supported an efficient alpha material for world geometry, where\nfirst only the depth of world geometry was rendered, and then finally shading of\nthe world geometry was rendered with alpha-blending enabled. This allowed only\nthe first layer, and optionally before that a back-facing layer, to be rendered\ncheaply with no depth-sorting involved and is essentially a limited and cheaper\nform of the more general-purpose depth-peeling approach. This was sufficient for\nmaking props like windows or similar glass structures in levels by mappers.\nThough there is the drawback that transparent layers can't be seen behind other\ntransparent layers, when used in moderation this drawback is not debilitating.\n\nTesseract expands upon this notion by shading transparent geometry in a separate\nlater pass from opaque geometry, though both are accumulated into the light\naccumulation buffer. This has both the above-mentioned benefits, as well as\nallowing transparencies to be easily shadowed and lit just like any other opaque\ngeometry and avoiding the need for a separate forward-renderer implementation.\nBecause transparent geometry is first rendered into the g-buffer as if it were\nopaque, there is no need to do a prior depth-only rendering pass to isolate the\nfront-most transparency layer like in Sauerbraten. Careful stenciling and\nscissoring is used to limit the actual shading step to only the necessary screen\npixels that will have transparent geometry blended over them. The A channel in\nthe normal layer of the g-buffer is used to store the alpha transparency value\nto control the blending output of this shading step.\n\nThis separate shading pass for transparent geometry also allows the light\naccumulation buffer from the previous opaque geometry pass to be easily\nresampled for screen-space reflection and refraction effects on materials like\ndistorted glass or water, providing for a greater range of reflective and\nrefractive materials than Sauerbraten was previously capable of. The emissive\nlayer of the g-buffer is used for handling the refractive/reflective component\nof a material's shading.\n\nRefraction effects are done by sampling the light accumulation buffer with added\ndistortion, limited by a mask of refracting surfaces to control bleed-in of\nthings outside the refraction area. For more information about the refraction\nmask technique, see\nhttp://http.developer.nvidia.com/GPUGems2/gpugems2_chapter19.html\n\nReflection poses some difficulty since a separate render pass for every\nreflecting plane was no longer desired and would anyway be too expensive given\nthe heavyweight deferred shading pipeline. A screen-space ray marching approach\nis instead used that in a small number of fixed steps walks through the depth\nbuffer until it hits a surface and then samples the light accumulation buffer at\nthis location. Some care is needed to fade out the reflections when either\nlooking directly at the reflecting surface or when it might otherwise sample\noutside of screen borders or valid reflection boundaries in general. This\napproach has some potential artifacts when objects are floating above reflective\nsurfaces, but on the other hand allows any number of reflective objects in the\nscene without requiring a separate render pass for every distinct reflection\nplane in the scene. For materials where screen-space reflections are inadequate,\nenvironment maps are instead used.\n\nFor further information on screen-space reflections, see\nhttp://www.crytek.com/cryengine/presentations/secrets-of-cryengine-3-graphics-technology\nor http://www.gamedev.net/blog/1323/entry-2254101-real-time-local-reflections/\n\n## Particle rendering\n\nBefore the tonemapping step, Tesseract does particle rendering. Unlike in\nSauerbraten, the depth buffer and shading results are more easily available\nwithout having to read back the frame buffer or using special-cased kluges to\navoid such read-backs, making effects like soft or refractive particles cheaper\nand simpler to implement.\n\n## Tonemapping and bloom\n\nThe light accumulation buffer is first quickly downscaled to approximately\n512×512. A high-pass filter is run over this and then separably blurred to yield\na bloom buffer that will be added to the lighting. This downscaled buffer (the\nnon-blurred one) is also converted to luma values and iteratively reduced down\nto 1×1 to compute an average luma for the scene. This average luma is slowly\naccumulated into a further 1×1 texture that allows for the scene's brightness\nkey to gradually adjust to changing viewing conditions. This accumulated average\nluma is fed back (via a vertex texture fetch in the tonemapping shader) into a\ntonemapping pass which maps the lighting into a displayable range. Note that the\ngamma-space lighting is converted temporarily into linear-space before\ntonemapping is applied and then converted back to gamma-space.\n\nTo better preserve color tones and in contrast to tonemapping operators that\nunfortunately tend to \"greyify\" a scene such as filmic tonemapping, Tesseract\nuses a simpler \"photographic\" tonemapping operator suggested by Emil Persson\na.k.a. Humus, but applied to luma. See [this forum\ntopic](http://beyond3d.com/showthread.php?t=60907) or [this\none](http://beyond3d.com/showthread.php?t=52747).\n\nFor more information about the trade-offs involved in various tonemapping\noperators, see [this page](http://filmicgames.com/archives/category/tonemapping)\nor [this\npage](http://mynameismjp.wordpress.com/2010/04/30/a-closer-look-at-tone-mapping/).\n\n## Generic post-processing\n\nBefore the final anti-aliasing and/or upscale step, any generic post-processing\neffects are applied. Currently this stage is not extensively utilized.\n\n## Anti-aliasing\n\nIn contrast to Sauerbraten's forward renderer, Tesseract's performance is\nstrongly impacted by resolution. Many schemes were evaluated for reducing\nshading costs, such as inferred lighting or interleaved rendering, but\nultimately they were more complicated and no more performant or visually\npleasing than simply rendering at reduced resolution and anti-aliasing the\nresult with a final upscale to desktop resolution. Since Tesseract relies upon\ndeferred shading, simply using MSAA by itself does not provide adequate\nperformance due to increasing memory bandwidth usage from large multisampled\ng-buffer textures, though Tesseract does, in fact, implement stand-alone MSAA in\nspite of deferred shading. To this end, Tesseract provides several forms of\npost-processing-centric anti-aliasing, though mostly in the service of\nimplementing one particular post-process anti-aliasing algorithm, Enhanced\nSubpixel Morphological Anti-Aliasing by Jorge Jiminez et al, otherwise known as\nSMAA.\n\nThe baseline SMAA 1× algorithm provides morphological anti-aliasing utilizing\nonly the output color buffer. While this algorithm is an improvement over\ncompetitors such as FXAA, it still suffers from some temporal instability\nvisible as frame-to-frame jitter/swim. To combat this, Tesseract implements\ntemporal anti-aliasing that combines with SMAA to provide the SMAA T2× mode. The\nSMAA T2× mode, and temporal anti-aliasing in general, however, are often\ninadequate when things move quickly on-screen. Temporal anti-aliasing reprojects\nthe rendering output of prior frames onto the current frame, and when the scene\nchanges quickly, this is often not possible, so the temporal anti-aliasing fails\nto anti-alias in such cases. The A channel of the g-buffer's normal layer is\nused to provide a mask of all pixels belonging to moving objects in the scene,\nas distinguished from static world geometry, instead of requiring a more costly\nvelocity buffer. Ultimately, only static geometry that is subject only to\ncamera-relative movement participates in the temporal anti-aliasing which allows\ncheap computation of per-pixel velocity vectors from the global camera\ntransforms without requiring storing object velocities.\n\nTo overcome the particular movement limitations of temporal anti-aliasing, SMAA\nalso provides several modes that combine with multisample anti-aliasing, SMAA\nS2× and SMAA 4×, utilizing 2× spatial multisampling to provide temporal\nstability. SMAA 4× further combines temporal anti-aliasing and 2× multisample\nanti-aliasing with the baseline morphological anti-aliasing to provide a level\nof post-process anti-aliasing that can rival MSAA 8× modes while using far less\nbandwidth (only requiring 2× MSAA textures) and being much faster.\n\nOverall, SMAA gracefully scales up and down both in terms of performance and\nvisual quality according to the user's tastes with its ability to incorporate\nall these disparate anti-aliasing methods, and while still interacting well with\ndeferred shading. For more information about SMAA, see [this\npage](http://www.iryoku.com/smaa/) and also for further improvements recently\nsuggested by Crytek see [this\npage](http://www.crytek.com/cryengine/presentations/cryengine-3-graphic-gems).\n\nTesseract's deferred MSAA implementation renders into multisampled g-buffer and\nlight accumulation textures. Before shading, an edge detection pass is run using\ninformation contained in the normal/depth hash layer of the g-buffer to fill the\nstencil buffer with an edge mask. The depth hash value, stored in the A channel\nof the normal layer of the g-buffer, is simply an 8 bit hash combining\ninformation about linear depth and material id that when combined with the world\nspace normal stored in the RGB channels of this same texture provides reasonable\nand cheap determination of pixels that only very occasionally mispredicts edges.\nSingle-sample shading is evaluated at internal/non-edge pixels, and multisample\nshading is evaluated at edge pixels. The tonemapping pass is able to run before\nthe MSAA resolve to properly support the multisampled SMAA modes; however, this\nis not done by default for stand-alone MSAA usage as it can decrease performance\nfor mostly imperceptible benefits in quality.\n\nFor more information about implementing MSAA in combination with deferred\nshading, see [this\npage](http://software.intel.com/en-us/articles/deferred-rendering-for-current-and-future-rendering-pipelines).\n\nAs a final low-quality but highly performant backup and comparison basis,\nTesseract provides an implementation of FXAA, though the baseline SMAA 1× is\nultimately preferred.\n\nFor higher quality upscaling of the anti-aliased result than the usual linear\nfiltering, Tesseract also provides a bicubic filter such as used for upscaling\nvideo. This can alleviate some blurring a linear filter would otherwise cause.\nFor more information about fast bicubic filtering, see [this\npage](http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter20.html).\n\n## Further information\n\n- **Homepage** - [Tesseract](http://tesseract.gg)\n- **Developer** - [Lee Salzman](http://sauerbraten.org/lee/)\n\n*Last revised November 7, 2013.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalinou%2Ftesseract-renderer-design","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcalinou%2Ftesseract-renderer-design","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalinou%2Ftesseract-renderer-design/lists"}