Update NVIDIA NVENC guides for StreamFX v0.11.1

This commit is contained in:
Michael Fabian 'Xaymar' Dirks
2022-02-18 04:55:56 +01:00
parent 2f670ca134
commit 8557e88155
2 changed files with 420 additions and 162 deletions
+333 -89
View File
@@ -5,102 +5,346 @@ title: "High Quality Recordings with NVIDIA NVENC (in OBS® Studio)"
comments: true
---
{% capture content %}With {% include inline/link.liquid url="https://streamfx.xaymar.com/" content="StreamFX" %}'s NVENC you can achieve the quality targets in one of two ways: Constant Quantization Parameters (CQP) or Variable Bitrate with Constant Quality (VBR-CQ). Both have their own advantages and downsides, and it depends entirely on your requirements which one you should use. Modern {% include inline/link.liquid url="https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new" content="NVIDIA GPUs (Pascal and newer)" %} also all support 4:4:4 and 4:2:0 encoding at most of the quality levels shown here with hardly any impact on framerate. Please note that due to API limitations, it is currently not possible to get zero-copy encoding for 4:4:4 content.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture content %}With {% include inline/link.liquid url="https://streamfx.xaymar.com/" content="StreamFX" %}'s NVENC you can achieve the quality targets in one of two ways: Constant Quantization Parameters (CQP) or Variable Bitrate with Constant Quality (VBR-CQ). Both have their own advantages and downsides, and it depends entirely on your requirements which one you should use. Modern {% include inline/link.liquid url="https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new" content="NVIDIA GPUs (Pascal and newer)" %} also all support 4:4:4 and 4:2:0 encoding at most of the quality levels shown here.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture content %}When using VBR-CQ, it should be noted that the quality you will end up with is directly related to the settings chosen, which will have a direct impact on framerate. This impact will be far higher than anything CQP could cause, so be warned of skipped frames from rendering lag if you don't have OBS Studio running as Administrator. However thanks to this additional work, VBR-CQ can exceed CQP quality while using less space - very similar to how CRF in x264 works!{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% include blocks/heading.liquid level=2 content="Constant Quantization Parameter (CQP)" %}
{% capture content %}Of the two methods available, CQP is by far the faster method, with hardly any noticable impact on other parts of the GPU. It also supports vastly higher framerates compared to the alternative method, thanks to NVIDIA optimizing their presets further. With the "Fastest (P1)" preset it is now possible to record 1920x1080 NV12 at nearly 460 FPS on an RTX 3080 or higher. Unfortunately none of my PCI-E SSDs, not even the PCI-E 4.0 from Samsung, managed to withstand the amount of data requested at once, so the exact upper limit is unknown.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% include blocks/heading.liquid level=2 content="True Lossless" %}
{% capture content %}Due to codec limitations, "True Lossless" is not possible with H.264. It is however possible to get extremely close with I-only encoding, which has next to no measurable error. Ensure that you have your Color Format set to I444, Color Range set to Full, and that your Color Range matches the content recorded.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture columns %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642026495_obs64_OpulentStandardLlama.png" caption="H.264" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642031553_obs64_ElementaryForkedAsianpiedstarling.png" caption="H.265" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% endcapture %}{% include blocks/columns.liquid content=columns columns=2 %}
{% capture content %}
<thead><tr>
<th>Option</th>
<th width="16%">Lossless</th>
<th width="16%">Near Lossless</th>
<th width="16%">Indistinguishable</th>
<th width="16%">High Quality</th>
<th width="16%">Acceptable Quality</th>
</tr>
</thead>
<tr>
<td>Preset</td>
<td colspan=5>Fastest (P1)</td>
</tr>
<tr>
<td>Tune</td>
<td>Lossless</td>
<td colspan=4>High Quality</td>
</tr>
<tr>
<th colspan=6>H264</th>
</tr>
<tr>
<td>Profile</td>
<td>High 4:4:4 Predictive</td>
<td colspan=4>High 4:4:4 Predictive (when using I444)<br> High (when using NV12)</td>
</tr>
<tr>
<td>Level</td>
<td colspan=5>Automatic</td>
</tr>
<tr>
<th colspan=6>H265</th>
</tr>
<tr>
<td>Profile</td>
<td colspan=5>Main</td>
</tr>
<tr>
<td>Tier</td>
<td colspan=5>High</td>
</tr>
<tr>
<td>Level</td>
<td colspan=5>Automatic</td>
</tr>
<tr>
<th colspan=6>Rate Control Options</th>
</tr>
<tr>
<td>Mode</td>
<td colspan=5>Constant Quantization Parameter</td>
</tr>
<tr>
<td>Multi-Pass</td>
<td colspan=5>Single Pass</td>
</tr>
<tr>
<td>Look Ahead</td>
<td colspan=5>0 frames</td>
</tr>
<tr>
<td>Adaptive I-Frames</td>
<td colspan=5>Disabled</td>
</tr>
<tr>
<td>Adaptive B-Frames</td>
<td colspan=5>Disabled</td>
</tr>
<tr>
<th colspan=6>Quantization Parameters</th>
</tr>
<tr>
<td>I-Frame QP<br>P-Frame QP<br>B-Frame QP</td>
<td>0</td>
<td>0 to 4</td>
<td>5 to 10</td>
<td>11 to 16</td>
<td>17 to 22</td>
</tr>
<tr>
<th colspan=6>Adaptive Quantization</th>
</tr>
<tr>
<td>Spatial Adaptive Quantization</td>
<td colspan=5>Disabled</td>
</tr>
<tr>
<td>Temporal Adaptive Quantization</td>
<td colspan=5>Disabled</td>
</tr>
<tr>
<th colspan=6>Other Options</th>
</tr>
<tr>
<td>Maximum B-Frames</td>
<td colspan=5><i>Any</i></td>
</tr>
<tr>
<td>B-Frame Reference Mode</td>
<td colspan=5><i>Any</i></td>
</tr>
<tr>
<td>Zero Latency</td>
<td colspan=5>Default</td>
</tr>
<tr>
<td>Weighted Prediction</td>
<td colspan=5>Default</td>
</tr>
<tr>
<td>Non-reference P-Frames</td>
<td colspan=5>Default</td>
</tr>
<tr>
<td>Reference Frames</td>
<td colspan=5>-1 frames</td>
</tr>
<tr>
<td>Low Delay Key-Frame Scale</td>
<td colspan=5>-1</td>
</tr>
<tr>
<th colspan=6>Key Frames</th>
</tr>
<tr>
<td>Interval Type</td>
<td>Frames</td>
<td colspan=4>Frames<br>Seconds</td>
</tr>
<tr>
<td>Interval</td>
<td>1 frame</td>
<td colspan=4>1 frame<br>0.5 seconds</td>
</tr>
<tr>
<th colspan=6>OBS Studio Advanced Video Settings</th>
</tr>
<tr>
<td>Color Format</td>
<td>I444</td>
<td colspan=4>I444 or NV12</td>
</tr>
<tr>
<td>Color Space</td>
<td colspan=5><i>Identical to Input (sRGB on PC, Bt.709/sRGB for Console)</i></td>
</tr>
<tr>
<td>Color Range</td>
<td colspan=5><i>Identical to Input (Full on PC, Partial on Console)</i></td>
</tr>
{% endcapture %}{% include blocks/table.liquid content=content %}
{% include blocks/heading.liquid level=2 content="Effective Lossless" %}
{% capture content %}Effective Lossless is not quite True Lossless, but close enough that it will almost never matter. Ensure that you have your Color Format set to I444, Color Range set to Full, and that your Color Range matches the content recorded.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture columns %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642026797_obs64_LeadingMetallicNurseshark.png" caption="H.264 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642041471_obs64_PeskyPuzzledAsiaticlesserfreshwaterclam.png" caption="H.264 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642031790_obs64_TeemingRecklessBullmastiff.png" caption="H.265 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032358_obs64_ShySpanishWolverine.png" caption="H.265 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% endcapture %}{% include blocks/columns.liquid content=columns columns=4 %}
{% include blocks/heading.liquid level=2 content="Variable Bitrate with Constant Quality (VBR-CQ)" %}
{% capture content %}The second method for recording is VBR-CQ, which is similar to x264's CRF but not a perfect match. While x264's CRF will improve with higher presets and lower CRF values, NVENC's VBR-CQ has a flat ceiling that simply can't be bypassed, no matter what settings are applied. Still, it has its uses for Indistinguishable, High quality and Acceptable Quality recordings. Performance wise it is far slower and has much more impact on the machine, so it should not be used for Single-PC setups.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% include blocks/heading.liquid level=2 content="Visually Lossless" %}
{% capture content %}Visually Lossless tries to get as close as possible to the point where Humans are unable to tell between original and encoded. Ensure that you have your Color Format set to I444, Color Range set to Full, and that your Color Range matches the content recorded.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture columns %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642026555_obs64_WellgroomedEarnestGroundbeetle.png" caption="H.264 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642028421_obs64_CulturedSolidEquine.png" caption="H.264 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642031992_obs64_TomatoOddGalapagossealion.png" caption="H.265 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032844_explorer_OutrageousDroopyEasternnewt.png" caption="H.265 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% endcapture %}{% include blocks/columns.liquid content=columns columns=4 %}
{% include blocks/heading.liquid level=2 content="High Quality" %}
{% capture content %}As we now have fallen below the ideal quality level where it matters, you can freely choose between I444 and NV12. Note that for H.264 encoding, you will need to change the 'Profile' to 'High' if you choose NV12. Ensure that you have your Color Format set to I444 or NV12, Color Range set to Full, and that your Color Range matches the content recorded.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture columns %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642026608_obs64_WhimsicalBareTomtit.png" caption="H.264 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642028483_obs64_HandyAnxiousAmericanalligator.png" caption="H.264 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032037_obs64_EachMoistStork.png" caption="H.265 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032889_obs64_SuperficialSeriousAmmonite.png" caption="H.265 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% endcapture %}{% include blocks/columns.liquid content=columns columns=4 %}
{% include blocks/heading.liquid level=2 content="Acceptable Quality" %}
{% capture content %}At this level you will likely be unable to reach reasonable quality, and can freely choose between I444 and NV12. Note that for H.264 encoding, you will need to change the 'Profile' to 'High' if you choose NV12. Ensure that you have your Color Format set to I444 or NV12, Color Range set to Full, and that your Color Range matches the content recorded.{% endcapture %}{% include blocks/paragraph.liquid content=content %}
{% capture columns %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642026667_obs64_SoftGraciousGlowworm.png" caption="H.264 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642028506_obs64_BuzzingPrimaryVervet.png" caption="H.264 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032071_obs64_FondArtisticArieltoucan.png" caption="H.265 CQP" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% capture column %}
{% include blocks/media.liquid type="image" url="https://cdn.xaymar.com/blog/2022/01/1642032923_obs64_PutridGrayZeren.png" caption="H.265 VBR-CQ" link=true %}
{% endcapture %}{% include blocks/column.liquid content=column %}
{% endcapture %}{% include blocks/columns.liquid content=columns columns=4 %}
{% capture content %}
<thead><tr>
<th>Option</th>
<th width="25%">Indistinguishable</th>
<th width="25%">High Quality</th>
<th width="25%">Acceptable Quality</th>
</tr>
</thead>
<tr>
<td>Preset</td>
<td colspan=3>Medium (P4) or slower</td>
</tr>
<tr>
<td>Tune</td>
<td colspan=3>High Quality</td>
</tr>
<tr>
<th colspan=4>H264</th>
</tr>
<tr>
<td>Profile</td>
<td colspan=3>High 4:4:4 Predictive (when using I444)<br> High (when using NV12)</td>
</tr>
<tr>
<td>Level</td>
<td colspan=3>Automatic</td>
</tr>
<tr>
<th colspan=4>H265</th>
</tr>
<tr>
<td>Profile</td>
<td colspan=3>Main</td>
</tr>
<tr>
<td>Tier</td>
<td colspan=3>High</td>
</tr>
<tr>
<td>Level</td>
<td colspan=3>Automatic</td>
</tr>
<tr>
<th colspan=4>Rate Control Options</th>
</tr>
<tr>
<td>Mode</td>
<td colspan=3>Variable Bitrate</td>
</tr>
<tr>
<td>Multi-Pass</td>
<td colspan=3>Two Pass at Quarter Resolution<br><i>or:</i> Two Pass at Full Resolution</td>
</tr>
<tr>
<td>Look Ahead</td>
<td colspan=3>32 frames</td>
</tr>
<tr>
<td>Adaptive I-Frames</td>
<td colspan=3>Enabled</td>
</tr>
<tr>
<td>Adaptive B-Frames</td>
<td colspan=3>Enabled</td>
</tr>
<tr>
<th colspan=4>Limits</th>
</tr>
<tr>
<td>Target Quality</td>
<td>1.00 to 10.99</td>
<td>11.00 to 17.99</td>
<td>17.00 to 22.99</td>
</tr>
<tr>
<td>Target Bitrate</td>
<td colspan=3>0 kbit/s</td>
</tr>
<tr>
<td>Maximum Bitrate</td>
<td colspan=3>0 kbit/s</td>
</tr>
<tr>
<td>Buffer Size</td>
<td colspan=3>0 kbit</td>
</tr>
<tr>
<th colspan=6>Quantization Parameters</th>
</tr>
<tr>
<td>Minimum QP<br>Maximum QP</td>
<td colspan=3>-1</td>
</tr>
<tr>
<td>I-Frame QP<br>P-Frame QP<br>B-Frame QP</td>
<td colspan=3>-1</td>
</tr>
<tr>
<th colspan=4>Adaptive Quantization</th>
</tr>
<tr>
<td>Spatial Adaptive Quantization</td>
<td colspan=3>Enabled</td>
</tr>
<tr>
<td>Spatial Adaptive Quantization Strength</td>
<td colspan=3>1 (Lowest) to 15 (Highest)</td>
</tr>
<tr>
<td>Temporal Adaptive Quantization</td>
<td colspan=3>Enabled</td>
</tr>
<tr>
<th colspan=4>Other Options</th>
</tr>
<tr>
<td>Maximum B-Frames</td>
<td colspan=3><i>As many as your GPU supports</i></td>
</tr>
<tr>
<td>B-Frame Reference Mode</td>
<td colspan=3>Each <i>if supported</i><br>Half <i>if supported</i><br>None <i>in any other case</i></td>
</tr>
<tr>
<td>Zero Latency</td>
<td colspan=3>Default</td>
</tr>
<tr>
<td>Weighted Prediction</td>
<td colspan=3>Default</td>
</tr>
<tr>
<td>Non-reference P-Frames</td>
<td colspan=3>Enabled</td>
</tr>
<tr>
<td>Reference Frames</td>
<td colspan=3><i>As many as your GPU supports</i></td>
</tr>
<tr>
<td>Low Delay Key-Frame Scale</td>
<td colspan=3>-1</td>
</tr>
<tr>
<th colspan=4>Key Frames</th>
</tr>
<tr>
<td>Interval Type</td>
<td colspan=3>Seconds</td>
</tr>
<tr>
<td>Interval</td>
<td colspan=3>1.00 seconds <i>or more</i></td>
</tr>
<tr>
<th colspan=6>OBS Studio Advanced Video Settings</th>
</tr>
<tr>
<td>Color Format</td>
<td colspan=3>I444 or NV12</td>
</tr>
<tr>
<td>Color Space</td>
<td colspan=3><i>Identical to Input (sRGB on PC, Bt.709/sRGB for Console)</i></td>
</tr>
<tr>
<td>Color Range</td>
<td colspan=3><i>Identical to Input (Full on PC, Partial on Console)</i></td>
</tr>
{% endcapture %}{% include blocks/table.liquid content=content %}
{% include blocks/heading.liquid level=2 content="Additional Information" %}
{% capture list %}
<li>Some of the settings for VBR-CQ can be freely adjusted in order to regain performance:
{% capture list2 %}
<li><i>Two Pass</i> can be disabled, but it will increase file size and may decrease quality.</li>
<li><i>Look Ahead</i> can be set as low as 16 in H264 and 4 in HEVC, but it will increase file size and may decrease quality.</li>
<li><i>Maximum B-Frames</i> can be freely adjusted, which may increase file size and may decrease quality.</li>
{% endcapture %}{% include blocks/list.liquid content=list2 %}
</li>
<li>You can freely adjust the Key Frame <i>Interval</i> for anything not requiring I-only.</li>
<li>On Pascal, Maxwell and Kepler GPUs, it may be necessary to set <i>Maximum B-Frames</i> to 0.</li>
<li>On some Hardware, it may be necessary to use the <i>Preset</i> "High Performance" over "Lossless High Performance".</li>
<li>When OBS Studio updates FFmpeg to a more recent version, it may be necessary to set the <i>Preset</i> and <i>Tune</i> via Custom Settings until StreamFX updates.</li>
<li><i>Multi-Pass</i> at Full Resolution is a new option, and has measurably increased quality in VBR-CQ. However, it is very expensive and should be the first option changed if performance is a problem.</li>
<li><i>Look Ahead</i> in VBR-CQ can be freely adjusted as needed, as long as it is not lower than the number of B-Frames and Reference Frames.</li>
<li><i>Adaptive B-Frames</i> is required for VBR-CQ, as B-Frames can both increase and decrease quality if used in the wrong place. </li>
<li>Stronger <i>Spatial Adaptive Quantization</i> will focus more bitrate towards visually complex areas, but may starve smoother areas of the bitrate it needs.</li>
<li>The Key Frames <i>Interval</i> can be freely adjusted for anything but Lossless, as I-only encoding is required to guarantee the least data loss.</li>
<li>The old presets were deprecated by NVIDIA and now map to the new P1-P7 presets, which come with massive performance or quality boosts. <i>High Quality</i> is now <i>Medium (P4)</i>, while <i>High Performance</i> is closer to <i>Faster (P2)</i>.</li>
{% endcapture %}{% include blocks/list.liquid content=list %}
{% capture content %}{% include inline/link.liquid url="../" content="Back to the Guide" %}{% endcapture %}{% include blocks/paragraph.liquid align="center" content=content %}