Add some visual examples and compress page

_guides/lost-knowledge-and-secrets.html

@@ -5,9 +5,7 @@ comments: true
 date: "2022-01-13"
 ---
 
-{% capture content %}When knowledge becomes common knowledge, the curse of time starts affecting it and eventually it may be lost. This is a page dedicated to such knowledge I possess or rediscovered through encountering the same problems. I can't guarantee that someone may hold a patent on the knowledge recorded here.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-
+{% capture content %}When knowledge becomes common knowledge, the curse of time starts affecting it and eventually it may be lost. This is a page dedicated to such knowledge I possess or rediscovered through encountering the same problems. I can't guarantee that someone does or doesn't hold a patent on the knowledge recorded here.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
 
 {% capture terminology_block %}
 {% capture list %}
@@ -54,46 +52,36 @@ date: "2022-01-13"
 {% endcapture %}{% include blocks/list.liquid content=list %}
 {% endcapture %}{% include blocks/details.liquid level=3 title="Terminology" content=terminology_block %}
 
-{% capture color_space_block %}
+{% capture block_cg %}
+{% capture block_cg_particles %}
 
-{% include blocks/heading.liquid level=2 content="Is lossless conversion between YUV and RGB possible?" %}
-{% capture content %}It depends on the matrix used to transfer to and from YUV, as well as the number of bits per channel. To be on the safe side, it should be assumed that converting to and from YUV will result in slight quality loss, which may be acceptable to you, but will result in accumulating error.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+{% capture block_cg_particles_maxcount %}{% capture content %}Many modern Particle simulation engines rely on user input for their maximum particle count, despite the maximum particle count being something that can be calculated ahead of time. All you need is the highest possible spawn rate (as particles per second), the minimum decay rate (as decay value per second), and the maximum life time (in seconds). Once you have these variables the following formulae will result in the maximum particle count:{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+{% capture particle_max_count_code %}particles = min(maxSpawnRate / minDecayRate, maxLifeTime * maxSpawnRate){% endcapture %}{% include blocks/code.liquid content=particle_max_count_code %}
+{% endcapture %}{% include blocks/details.liquid level=3 title="Calculate maximum Particle count ahead of time" content=block_cg_particles_maxcount %}
 
-{% endcapture %}{% include blocks/details.liquid level=1 title="Color Space Handling" content=color_space_block %}
+{% endcapture %}{% include blocks/details.liquid level=2 title="Particle Systems" content=block_cg_particles %}
+
+{% capture block_cg_colors %}
+
+{% capture block_cg_colors_hdr888 %}
+{% capture content %}This is a technique dating back to the days of DirectX 7 and was implemented differently from engine to engine - it was a wild west of ideas after all. The technique works by using the Alpha channel as a Multiplier, often a logarithmic or exponential multiplier. This allowed representing values above 255 somewhat safely, but using it reduced the precision significantly.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+
+{% capture content %}Thanks to advanced in computing performance we no longer have to rely on this technique as a 10/10/10/2 RGBA texture can safely represent values up to 1023 with no issue. It is still possible to use the Alpha channel as a multiplier, but with only 4 values for the multiplier it is significantly more limited. Both techniques, old and new, are still in use today.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+{% endcapture %}{% include blocks/details.liquid level=3 title="FakeHDR: Simulating HDR by abusing the Alpha channel of an 8/8/8/8 Texture" content=block_cg_colors_hdr888 %}
+
+{% capture block_cg_colors_yuvrgb %}{% capture content %}No, but it's basically visually lossless. Usually it does not matter for final encoding, but if you do this for some effect, consider staying in a higher precision format for the converted values. Alternatively you can also use temporal dithering to hide the imperfections, which is what some streaming and recording software does to improve their output quality.{% endcapture %}{% include blocks/paragraph.liquid content=content %}{% endcapture %}{% include blocks/details.liquid level=3 title="Is YUV/RGB Conversion lossless?" content=block_cg_colors_yuvrgb %}
+
+{% capture block_cg_colors_banding %}
+{% capture content %}Dithering - a technique that is somewhat lost to time, thanks to advances in computing power. Dithering used to be everywhere in older games, as the hardware as incapable of rendering what people wanted to do. It works by abusing a significant flaw of Human vision, namely that we aren't that good at identifying colors surrounded by other colors, and will often take a shortcut instead. Thanks to advances in computing power, we can now also do temporal dithering - a technique that monitors already use to make HDR more affordable (see "Frame Rate Control").{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+{% capture content %}You can also make use of this in your own games, either as a stylistic choice, or to enhance the visual fidelity of the output for any player. All you need is a higher precision input than output and a dithering function of your choice. My personal choice for spatial dithering is Bayer, seeded with some per-frame per-pixel random number to make it temporal. Take a look at {% include inline/link.liquid content="this ShaderToy example" url="https://www.shadertoy.com/view/7syGR1" %}, {% include inline/link.liquid content="this image comparison" url="https://imgsli.com/MTY2NTA2/7/8" %}, or {% include inline/link.liquid content="this image comparison" url="https://imgsli.com/MTY2NTEy/7/8" %} to see what Dithering can do for you.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
+{% endcapture %}{% include blocks/details.liquid level=3 title="Dithering: Avoiding color banding by tricking human vision" content=block_cg_colors_banding %}
+
+{% endcapture %}{% include blocks/details.liquid level=2 title="Color Handling" content=block_cg_colors %}
+{% endcapture %}{% include blocks/details.liquid level=1 title="Computer Graphics" content=block_cg open=true %}
 
 {% capture block_hosting %}
-
 {% capture block_hosting_trunas %}
-
 {% capture block_hosting_trunas_recordsize %}{% capture content %}The Record Size of a storage dataset in TrueNAS and FreeNAS matters a lot. It can be the difference between being able to transfer 2 GiB in a second, and taking multiple minutes to do so. While documentation is a bit sparse and fractured, through testing I was able to find out that the best record size is the default it has already selected. It offers the best mixture of performance for large and small writes, especially on faster storage.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
 {% capture content %}My tests went through many record sizes, halving and doubling from 128K outwards. Every halving resulted in large read/writes getting significantly slower, but small read/writes getting slightly faster. Every doubling resulting in large read/write getting slightly faster, but small read/writes getting significantly slower. The best value ended up being 128K, which managed to keep most of the NVMe performance as is (~80MiB/s in RND4K, ~2.5GiB/s in RND1M).{% endcapture %}{% include blocks/paragraph.liquid content=content %}{% endcapture %}{% include blocks/details.liquid level=3 title="Ideal Record Size" content=block_hosting_trunas_recordsize %}
-
 {% endcapture %}{% include blocks/details.liquid level=2 title="TrueNAS & FreeNAS" content=block_hosting_trunas %}
-
-{% endcapture %}{% include blocks/details.liquid level=1 title="Hosting" content=block_hosting %}
-
-
-{% capture particle_block %}
-
-{% include blocks/heading.liquid level=2 content="Calculate maximum Particle count ahead of time" %}
-{% capture content %}Many modern Particle simulation engines rely on user input for their maximum particle count, despite the maximum particle count being something that can be calculated ahead of time. All you need is the highest possible spawn rate (as particles per second), the minimum decay rate (as decay value per second), and the maximum life time (in seconds). Once you have these variables the following formulae will result in the maximum particle count:{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-{% capture particle_max_count_code %}particles = min(maxSpawnRate / minDecayRate, maxLifeTime * maxSpawnRate){% endcapture %}{% include blocks/code.liquid content=particle_max_count_code %}
-
-{% endcapture %}{% include blocks/details.liquid level=1 title="Particle Simulation" content=particle_block %}
-
-{% capture rendering_block %}
-
-{% include blocks/heading.liquid level=2 content="Is it possible to avoid Color Banding?" %}
-{% capture content %}Not while we work with finite space, but we can make it nearly invisible to a Human-like observer through Dithering. Dithering requires a higher precision input, so a 10/10/10/X texture for intermediate and 8/8/8/X texture for output would already improve quality. Most game engines opt for 16/16/16/X or 10/10/10/X for their internal render buffer, but many do not opt to dither to the target buffer.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-{% include blocks/heading.liquid level=2 content="Could Dithering provide higher quality?" %}
-{% capture content %}Yes with a Human-like observer (perceptual Quality), otherwise no. We can abuse the poor quality of Human vision through spatial and temporal Dithering, thus pretending to have extra precision that doesn't exist. Many older games and consoles abused this to draw more colors than they actually could render on screen. Often it is comined with sub-pixel rendering to provide further quality improvements.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-{% capture content %}With modern technology, this trick became much easier than before, to the point that Displays have this built-in now: FRC. After you hit a refresh rate of roughly 40 Hz, you can fake an extra bit of precision every additional 40 Hz. So a 120 Hz monitor with only 8 bits per channel could fake 10 bits per channel to a human-like observer, and they would very likely not notice it.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-{% include blocks/heading.liquid level=2 content="Can you store HDR in 8/8/8/8-bit?" %}
-{% capture content %}No, but some game engines did abuse this format for faking HDR. The 4th channel would contain a multiplier for the first three channels, however this came at a cost of precision and often reduced overall quality. As GPUs got more advanced, we now have a better HDR format when space is limited without any of the downsides: 10/10/10/2.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
-
-{% endcapture %}{% include blocks/details.liquid level=1 title="Rendering" content=rendering_block %}
+{% endcapture %}{% include blocks/details.liquid level=1 title="Hosting" content=block_hosting open=true %}