Add old WordPress content
This commit is contained in:
Michael Fabian 'Xaymar' Dirks
+70
@@ -0,0 +1,70 @@
|
||||
---
|
||||
title: "A Look Back: Creating a VST 2.x Plug-In from Nothing (Part 1)"
|
||||
category: Blog
|
||||
tags: ["VoiceFX", "VST", "VST 2.x", "WordPress Archive"]
|
||||
---
|
||||
|
||||
|
||||
<p class="block">When I started with <a href="https://blog.xaymar.com/projects/voicefx/" data-type="URL" data-id="https://blog.xaymar.com/projects/voicefx/" target="_blank" rel="noreferrer noopener">VoiceFX</a>, my original goal was to only support VST 3.x, as it was the most modern version of the SDK, and surely by now every important software had moved to it. Unfortunately I didn't account for the occasional big shot releasing a modern product with a relatively ancient version of the SDK - an SDK that no longer officially exists. So what do you do in this situation?</p>
|
||||
|
||||
<p class="block">You do what every other totally sane developer does and start a clean-room reverse engineering project for the now abandoned VST 2.x SDK, staying faithful to the law. And finally, after roughly 5 months of development, I managed to make it work. So how did I get there?</p><!--more-->
|
||||
|
||||
<h2 class="block">Establishing the Boundaries</h2>
|
||||
|
||||
<p class="block">When trying to stay within the law, you have to establish clear boundaries. Especially for clean-room reverse engineering, I had to go into quite a few lawsuits about it to figure out what is and is not allowed. In the end, I had to actually ask a lawyer for advice, which ended up with some costs, and the advice I got boiled down to this:</p>
|
||||
|
||||
<ol class="block">
|
||||
<li>Any work performed must be for the purpose of interoperability.</li>
|
||||
<li>Don't use any original material or third-party source material that is not clearly set apart from the original.</li>
|
||||
<li>Avoid the use of reverse engineering tools where possible.</li>
|
||||
</ol>
|
||||
|
||||
<p class="block">These rules sound simple, but they ended up making the project a living hell. I had to rely only on information that was clearly detached from the applications used, or not at all rely on these applications. A single misstep and you end up with a huge amount of legal issues. But what other option do I have when the actual SDK is now intentionally hidden by the creators, but still in use by commercial applications released today?</p>
|
||||
|
||||
<h2 class="block">Where is the Entry?</h2>
|
||||
|
||||
<p class="block">I have to begin somewhere, and for VST 2.x Plug-Ins and Hosts, it is the interface to the Plug-In itself. I needed to figure out the "entry point" from which everything starts, and what that entry point actually does, which meant I had to look into what existing VST 2.x Plug-Ins export. On Windows, I could hook into <code>GetProcAddress</code>, while on Linux, I could hook into <code>dlsym</code>.</p>
|
||||
|
||||
<p class="block">I ended up using the Windows function for the initial steps and after filtering out several thousand unrelated strings, I finally found something that looked relevant: a string with the content "<em>VSTPluginMain</em>". This string was often accompanied by "<em>MAIN</em>" and "<em>main_macho</em>", which I assume are for Windows and MacOS exclusive VST 2.x plugins only.</p>
|
||||
|
||||
<p class="block">I had the name of the entry point, but none of it's details - in common terms, I knew where the door was, but not where the knob was, what the key looked like, and how it would even open. But I had a start, a which is more than nothing, and gave me the necessary push required to move on after almost giving up from the mass of data.</p>
|
||||
|
||||
<h2 class="block">Lockpicking the Entry</h2>
|
||||
|
||||
<p class="block">While having an entry point is great, it still being locked is a problem - it needs to be unlocked for us to actually do anything. I could not rely on anything but the most basic tools at this point, as every information other tools may give me could be wrong. The only tool that I had available to unlock the entry point are the CPU registers, so the work began.</p>
|
||||
|
||||
<p class="block">On AMD64, you have the registers (R)AX, (R)BX, (R)CX, (R)DX, (R)SI, (R)DI, (R)BP, (R)SP, R8, R9, R10, R11, R12, R13, R14, R15, (R)IP, and (R)FLAGS. All of these have the potential to hold critical information, but some of these will actually contain useful information. I won't bore you with the details about x86/amd64 Assembly here, there's plenty of other resources out there that are easily found if you're actually into that.</p>
|
||||
|
||||
<figure class="block"><a href="https://cdn.xaymar.com/blog/2021/03/devenv_1Jn50r0LKs.png"><img src="https://cdn.xaymar.com/blog/2021/03/devenv_1Jn50r0LKs.png" alt="" class="wp-image-2162" /></a>
|
||||
<figcaption>Register debugging is not fun.</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block">The short form of testing is that I found two clear patterns that repeated every single time: (R)CX would point into some kind of <em>read-only executable</em> memory, while the value in (R)AX when returning would result in different crashes - except if the value is zero. So most likely I was looking at two pointers of some kind, one as the argument, one as the return value.</p>
|
||||
|
||||
<p class="block">That left the calling convention, which is where it got a bit difficult. I've never had a clear resource on what each calling convention actually does, but it seems that 64-bit has somewhat unified the world to stop creating additional standards for something so critical. I have no idea what the 32-bit calling convention would be, but my assumption is that it is either <code>stdcall</code> or <code>cdecl</code>, with the latter sounding more sane - time will tell.</p>
|
||||
|
||||
<h2 class="block">Magic Space</h2>
|
||||
|
||||
<p class="block">With the entry point unlocked, but without a clear idea of what it does, questions flooded into my mind. <em>Clearly the value I'm returning has some sort of meaning, so can I affect the behavior in other ways that just returning 0?</em> <em>What if I have a bunch of memory which is all filled with 0, but I return the pointer to that memory?</em> <em>How large does that memory have to be if that works?</em></p>
|
||||
|
||||
<p class="block">While it seemed simple to test, of course the reality turned out to be much harder. My initial idea of simply returning memory that shrinks every time it succeeds resulted in a size of 4 bytes - nowhere enough to store anything. There was clearly something going on with the first four bytes, but what? It had to be a magic number of some kind to clearly identify the structure.</p>
|
||||
|
||||
<p class="block">And so I wrote a script which would repeatedly launch a VST 2.x host application with a crafted plug-in, which on every iteration would try a new number to insert into the first four bytes. And lo and behold, after 1450406992 iterations, I found the exact string: <em>VstP</em>.</p>
|
||||
|
||||
<p class="block">I assume that this is short for Vst Plug-In, and it should have occured to me - before I wasted several kWh on this problem - to attempt to limit the possible values of each byte. But I now had the magic number, and thanks to it I had a rough estimate of the size of the memory to return: roughly 128 bytes or more.</p>
|
||||
|
||||
<h2 class="block">Structural Inspection</h2>
|
||||
|
||||
<p class="block">With the magic number in place, it was time to delve into what the structure actually contains aside from the first four bytes clearly being a magic number. The easiest way to check data integrity is by poisioning it intentionally, and that's exactly what I did: I started inserting 1 bit changes at random locations to figure out what does and does not cause problems.</p>
|
||||
|
||||
<p class="block">I didn't have to wait long to hit something. Precisely when I chose the bytes 8 though 15 I was hitting something critical that seems to be used right after loading the VST 2.x Plug-In. The length of this looked suspiciously like a pointer, so my first attempt was to place a function at this point, and my guess was right. I was now hitting a breakpoint in my newly created function.</p>
|
||||
|
||||
<p class="block">After more register investigation, I figured out that the function in total has 6 arguments, of which two are pointers (one pointing at the memory i returned from the entry point), one is at least 8 bits, another is at least 24 bits (most likely 32), another is either 16, 32 or 64, and one is a 32 bit float. Going by the <a rel="noreferrer noopener" href="https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-160" target="_blank">MSVC x64 calling convention</a>, I guessed at the rough order of parameters, which seems to be correct so far. Time will tell if I messed up.</p>
|
||||
|
||||
<h2 class="block">Difficulty Spike</h2>
|
||||
|
||||
<p class="block">At this point, the difficulty went from <em>Easy</em> to <em>Dark Souls but you only have the X button</em>. It was time to start logging the calls to the function which I dubbed "control", every single one that would happen. It did not take long for patterns to emerge on most VST 2.x Hosts, with the most common calls having the second parameter be 0x2D, 0x2F, 0x30, 0x31, 0x3A, 0x23, 0x0A, 0x0B, and finally 0x00 - I called this parameter the "opcode".</p>
|
||||
|
||||
<p class="block">Unfortunately after this much time I can't remember the exact details anymore. Some of them are extremely obvious, like 0x0A which sets the sample rate. Others were more complex and I only just recently figured out what they mean and how to use them correctly through the development of VoiceFX. </p>
|
||||
|
||||
<p class="block">However this part is already long enough, so I'll continue with this in the 2nd part in the future.</p>
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: "High Quality Streaming (with OBS Studio)"
|
||||
published: false
|
||||
---
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>Creating entertaining, educational or otherwise useful content while live is a tough job, but it is a rewarding job if you make it. And I'm here to help you get started producing high quality content the moment you set foot into the world of streaming, at least in terms of Audio and Video - I can't help you if your content itself is unwatchable.</p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:more -->
|
||||
<!--more-->
|
||||
<!-- /wp:more -->
|
||||
|
||||
<!-- wp:heading -->
|
||||
<h2>Setting up the Basics</h2>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>Everyone has to go back to the basics sometimes, some more than others, some less. It is important to sometimes take a step back and get a proper look at what you are actually doing, and if it is actually working as it should. Sometimes it doesn't, sometimes it does, and sometimes it's based on luck entirely. So let's try to strip out the "doesn't work" and "luck" part entirely, and ensure that you get high quality right from the basics.</p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:heading {"level":3} -->
|
||||
<h3>Good Audio on a Budget</h3>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>Audio is what you and your viewers hear, or what is converted into words on screen if you happen to have a subtitling software running for disabled viewers. It is one of three critical parts of streaming, and messing it up can drive away interested viewers the moment things go out of control. As a Creator you want to sound better than the masses that don't know how to do it right, not the same.</p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>One thing many people get wrong is that they assume that every piece of Hardware is the same, which has resulted in the annoying "48 kHz is enough!" argument that went way out of control. Not only does it not apply universally, it also requires good Hardware or Software resampling - and the former is not very common in cheap or integrated Hardware. So the next best thing that we can use is Software resampling, which will often sound better.</p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:image {"align":"right","id":2183,"sizeSlug":"thumbnail","linkDestination":"media"} -->
|
||||
<div class="wp-block-image"><figure class="alignright size-thumbnail"><a href="https://cdn.xaymar.com/blog/2021/04/rundll32_lK2mC1aylX.png"><img src="https://cdn.xaymar.com/blog/2021/04/rundll32_lK2mC1aylX.png" alt="" class="wp-image-2183"/></a><figcaption>96 kHz Sample Rate</figcaption></figure></div>
|
||||
<!-- /wp:image -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>If your Hardware doesn't have a good resampling chip, you should set it to it's native Sample Rate. For many devices this is either 96 kHz or 192 kHz, but often 96 kHz should already sound better than 48 kHz to a trained ear. On the flip side, stacking a good hardware resampling chip with a mediocre software resampling implementation actually degrades audio quality - watch out for that when you have good audio hardware.</p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p>Finally in order to complete the basic Audio set up, all that's left is setting up OBS Studio accordingly. That means setting the Sample Rate to 48 kHz or 44.1 kHz (depending on the streaming platform), and setting the correct number of channels - which is almost always Stereo now. If you used a higher Hardware Sample Rate, you will now have OBS Studio resampling your Audio single to the correct Sample Rate with a mediocre but not awful resampling algorithm. </p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:image {"align":"center","id":2184,"sizeSlug":"full","linkDestination":"media"} -->
|
||||
<figure class="block"><picture><a href="https://cdn.xaymar.com/blog/2021/04/obs64_WQLZxOUwrI.png"><img src="https://cdn.xaymar.com/blog/2021/04/obs64_WQLZxOUwrI.png" alt="" class="wp-image-2184"/></a><figcaption>OBS Studio should be set to 48 kHz Stereo</figcaption></figure></div>
|
||||
<!-- /wp:image -->
|
||||
|
||||
<!-- wp:heading {"level":3} -->
|
||||
<h3>Video for Everyone</h3>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p></p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p></p>
|
||||
<!-- /wp:paragraph -->
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: "Low Latency Streaming to Twitch"
|
||||
category: Blog
|
||||
tags: ["Tutorial", "Twitch", "OBS Studio", "StreamFX", "x264", "NVENC", "WordPress Archive"]
|
||||
---
|
||||
|
||||
<p class="block">If you've been following my social media for the past few years, or have read my <a rel="noreferrer noopener" href="https://blog.xaymar.com/guides/high-quality-recordings/" target="_blank">Recording</a> or <a rel="noreferrer noopener" href="https://blog.xaymar.com/2020/06/24/the-art-of-encoding-with-nvidia-turing-nvenc/" target="_blank">Streaming on NVIDIA Turing/Ampere</a> guides, I'm always chasing the next higher level of "perfection". And this time I was chasing the lowest possible latency on Twitch - and it appears that I have finally found it, after days of trying.</p><!--more-->
|
||||
|
||||
|
||||
<p class="block">My search began as usual, with a manual investigation into the problem - I didn't have infinite resources to throw at the problem, and I didn't want to increase Twitch's hosting costs either. Therefore I was stuck with manual testing, which worked out well enough in the end for me, and didn't take much time either. I started my testing with the Key-Frame Interval, going down from 5 Seconds to 1 Second in 1 Second interval. </p>
|
||||
|
||||
<p class="block">Oddly enough, I noticed that certain Intervals would end up "grouped together" into a higher latency. For example, 1 Second and 2 Seconds would end up being 3 Seconds of latency, while 3 Seconds and 4 Seconds would end up being 5 Seconds of latency, and 5 Seconds ended up being 7 Seconds of latency. Unable to make sense of it, I moved on to other settings. </p>
|
||||
|
||||
<p class="block">Look-Ahead was the first one to recieve the axe, and it made no difference - 30 Frames or 0 Frames, both had the same latency. Similar to this was Zero-Latency and B-Frames, neither made a difference. But one option did, and it is what every RTMP+HLS based service recommends against: Adaptive I-Frames, or Scene Cut in x264 terms.</p>
|
||||
|
||||
<p class="block">Both x264 and NVENC use this option to insert complete Key-Frames if they happen to be a better option, for example with a fade to black, or a complete cut to new content. Unfortunately this option results in a measurable and visible increase in latency between a 250 and 500ms - half a Second gone to unfortunate Key-Frame placement.</p>
|
||||
|
||||
<p class="block">This brought the latency ladder down to ~2.4 Seconds for a Key-Frame Interval of 1 and 2 Seconds, ~4.2 Seconds for an Interval of 3 and 4, and ~5.6 Seconds for an Interval of 5. Careful observers may have just noticed the same thing I did, which is that the latency appears to climb in ~1.5 Second intervals - and that's what I tested next. </p>
|
||||
|
||||
<p class="block">I set up the Key-Frame Interval to 1.5 Seconds, and then I watched the Video Stats...</p>
|
||||
|
||||
<figure class="block"><img src="https://cdn.xaymar.com/blog/2021/04/msedge_dRkkOGANII.png" alt="" class="wp-image-2197" width="425" height="409" />
|
||||
<figcaption>One Second Transport Latency</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block"><strong>I finally had it.</strong> Almost a perfectly flat 1 Second latency for the stream itself, though with all the Rendering, Encoding and Muxing buffering, it still ended up being roughly 2 Seconds. Which is still less than the lowest total latency I managed with Adaptive I-Frames being Enabled, and any other Key-Frame Interval. I repeated this test several times, and was able to achieve the same latency over several hours in Firefox, Chrome and Edge. I am a happy Twitch streamer now.</p>
|
||||
|
||||
<p class="block"><strong>Disclaimer:</strong> This discovery is not guaranteed to work everywhere, and may be unique to the Ingest server I used. It may even be different due to changing which GPU is used, which CPU is used, which RAM is used, how your network looks like, etc. <em>It is provided at no guarantee or warranty.</em></p>
|
||||
@@ -0,0 +1,100 @@
|
||||
---
|
||||
title: "AMD and the Curse of Conflicting Information"
|
||||
category: Blog
|
||||
tags: ["AMD", "Ryzen", "Overclocking", "Undervolting", "Curve Optimizer", "WordPress Archive"]
|
||||
---
|
||||
|
||||
<p class="block">Against better judgement to just wait, back in December 2020, I ordered a AMD Ryzen 9 5950X - and received possible one of the worst chips to be on the market. In Cinebench R23, it achieved a Single Core score of ca. 1550, with a Multi Core score of ca. 24040. This by itself doesn't look too bad, until you open Cinebench R20 and get ca. 580 in Single Core, with Multi Core just barely hitting the 9800 barrier.</p>
|
||||
|
||||
<p class="block">So I did what any person with this hardware would do, and searched for overclocking options.</p><!--more-->
|
||||
|
||||
<h2 class="block">Overclocking on AMD Ryzen</h2>
|
||||
|
||||
<p class="block">With Zen3, AMD has granted us overclockers more than enough options to edge out every possible point of performance we can possibly need at the current time. Some of them are exclusive to any other method, while other methods can stack to some degree. I'll only look at the main three here:</p>
|
||||
|
||||
<ol class="block">
|
||||
<li><strong>VCore:</strong> This is your classic undervolting option which can be used to offset the Voltage the CPU Cores actually get.</li>
|
||||
<li><strong>PBO:</strong> Automatically increases the Boost clock speed and voltage according to a built in Voltage/Multiplier curve, up to the limits specified by you or the Motherboard. Not exclusive with VCore.</li>
|
||||
<li><strong>Curve Optimizer:</strong> An offset applied to the indexing of the Voltage/Multiplier curve, with each step being 3-5mV according to AMD. Not exclusive with VCore, but appears to do the same.</li>
|
||||
</ol>
|
||||
|
||||
<h2 class="block">VCore vs Curve Optimizer</h2>
|
||||
|
||||
<p class="block">A fair number of resources out there state that VCore offsets and Curve Optimizer are exclusive. In a perfect ideal CPU sample, they are, but in reality, it's not that simple. In order to understand why, it is necessary to understand the basics of how PBO figures out what Multiplier it can actually run at. Most likely, AMD uses a Look-Up-Table (LUT) to figure out what Multiplier to use. Below is an example of such a LUT:</p>
|
||||
|
||||
<figure class="block">
|
||||
<table class="has-fixed-layout">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>mV</th>
|
||||
<th style="text-align: center;">300</th>
|
||||
<th style="text-align: center;">305</th>
|
||||
<th style="text-align: center;">310</th>
|
||||
<th style="text-align: center;">315</th>
|
||||
<th style="text-align: center;">320</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>Multiplier</strong></td>
|
||||
<td style="text-align: center;">37.500</td>
|
||||
<td style="text-align: center;">37.625</td>
|
||||
<td style="text-align: center;">37.750</td>
|
||||
<td style="text-align: center;">37.875</td>
|
||||
<td style="text-align: center;">38.000</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<figcaption>Example PBO Look-Up-Table</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block">For now, let's ignore the additional calculations for the allowed maximum mV and Multiplier, and just assume it is a perfect world. If the user has Curve Optimizer disabled, a lookup for which Multiplier would apply at +310mV would return 37.750. But if the Curve Optimizer is set to -10mV (-2 in the options), the Multiplier returned would be 38.000 - we offset the Multiplier lookup up by 2 in the table.</p>
|
||||
|
||||
<p class="block">And then there is VCore offsets, which sound like they should do the same, but in reality they don't. Unlike PBO and Curve Optimizer, VCore offset does not modify which multiplier we end up with, it only modifies the effective Voltage. So in the above example with PBO and Curve Optimizer at -2 with an offset of -25mV, we'd still have lookup for +320mV a Multiplier of 38.000, but now our Voltage offset is only +295mV.</p>
|
||||
|
||||
<p class="block">So now I hope you understand the difference between the two options and why they are not exclusive to each other. It is true that a fixed VCore will break PBO and Curve Optimizer, but offsets are perfectly fine. In fact, I wrote this blog post with PBO On, Curve Optimizer at -10, and a VCore offset of -37.5mV.</p>
|
||||
|
||||
<h2 class="block">Benchmarks</h2>
|
||||
|
||||
<p class="block">As usual, to verify my findings I ran a number of benchmarks. Tests were ran for 20 minutes each, so plenty of heat was available to reduce the score.</p>
|
||||
|
||||
<figure class="block">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th class="has-text-align-left" data-align="left">Settings</th>
|
||||
<th style="text-align: right;">CBR23 MC</th>
|
||||
<th style="text-align: right;">CBR23 SC</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td class="has-text-align-left" data-align="left">PBO Off, CO 0, VCoreD 0</td>
|
||||
<td style="text-align: right;">~24040</td>
|
||||
<td style="text-align: right;">~1550</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="has-text-align-left" data-align="left">PBO On, CO 0, VCoreD 0</td>
|
||||
<td style="text-align: right;">~26530</td>
|
||||
<td style="text-align: right;">~1542</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="has-text-align-left" data-align="left">PBO MB, CO 0, VCoreD 0</td>
|
||||
<td style="text-align: right;">~26670</td>
|
||||
<td style="text-align: right;">~1550</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="has-text-align-left" data-align="left">PBO MB, CO -10, VCoreD 0</td>
|
||||
<td style="text-align: right;">~27211</td>
|
||||
<td style="text-align: right;">~1555</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="has-text-align-left" data-align="left">PBO MB, CO -10, VCoreD -25mV</td>
|
||||
<td style="text-align: right;">~27400</td>
|
||||
<td style="text-align: right;">~1560</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</figure>
|
||||
|
||||
<p class="block">While I'm still far away from what other people manage, this is already very promising. Maybe I'll find a golden egg while searching for settings and can bring this bad bin to a decent state.</p>
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "What's coming in VoiceFX v0.3.0?"
|
||||
category: Blog
|
||||
tags: ["VoiceFX", "WordPress Archive"]
|
||||
---
|
||||
|
||||
<p class="block">With what is most likely the final beta version of <a rel="noreferrer noopener" href="https://blog.xaymar.com/projects/voicefx/" target="_blank">VoiceFX</a> 0.3.0 being released, it's time to talk about all the improvements and changes that VoiceFX had to go through to get here. Let's take a look at all the new and upgraded things in it.</p><!--more-->
|
||||
|
||||
<h3 class="block">The Plug-In has been renamed to VoiceFX</h3>
|
||||
|
||||
<figure class="block"><img src="https://cdn.xaymar.com/blog/2021/05/Adobe_Audition_SBVeGbfxzy.png" />
|
||||
<figcaption>Adobe Audition detecting VoiceFX</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block">This may come as a surprise to some, but the VST 3.x version was never actually called VoiceFX. It has now been renamed - along with an identifier change - to match the actual project name. This will break existing configurations/setups, so you will have to update accordingly, and no further renames are planned. This was mainly done to integrate other effects later down the line, and no longer potentially infringe on trademarks and branding guidelines.</p>
|
||||
|
||||
<h3 class="block">Drastically reduced the Load/Reset Times</h3>
|
||||
|
||||
<p class="block">If you have used VoiceFX before, you've probably noticed and complained about the extremely long time required to load and/or reset the plugin. I wasn't able to remove it entirely - still waiting on NVIDIA to answer my questions - but I was able to significantly reduce it. As an example, instead of having to wait close to 18 seconds for Stereo playback with VoiceFX on in Adobe Audition, it is now only 3.8 seconds - almost 5 times faster!</p>
|
||||
|
||||
<h3 class="block">Improved Buffering and Latency Detection</h3>
|
||||
|
||||
<p class="block">Due to the mixed bag that VST hosts are, I had to invest quite some time into a solution that was not just efficient, but also covered most of the possible bad situations. This led to a much more stable plugin with far better buffering (fixing Adobe Premiere Pro exporting), better latency detection and various other things. Many VST hosts are also now 100% compatible with this.</p>
|
||||
|
||||
<h3 class="block">Text File Logs for improved end-user Support</h3>
|
||||
|
||||
<figure class="block"><img src="https://cdn.xaymar.com/blog/2021/05/explorer_ldYlKjIW8I.png" />
|
||||
<figcaption>Generated Log Files</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block">With the mixed bag of VST Hosts that we have, I could no longer rely on the VST 3 and VST 2 standard as my only resource for problems users experienced. So I did what any Software does, and now write log files to a pre-defined location. You can find these at <code>%LOCALAPPDATA%\VoiceFX\logs</code>, and sending the last 2-3 <em>files</em> along with any request for support will drastically improve the chances of fixing the problem.</p>
|
||||
|
||||
<h3 class="block">Support for VST 2.x Hosts</h3>
|
||||
|
||||
<p class="block">With 0.3.0 comes the promised integration into many VST 2.x only hosts, available to any Tier 2 or higher Supporter of mine on Github and Patreon. This integration took quite a while as the VST 2.x API is practically undefined if you want to stay on the legal side, so quite a bit of time was spent reverse engineering an API I had no actual information about, other than it exists.</p>
|
||||
|
||||
<h3 class="block">Improved support for non-standard-compliant VST Hosts</h3>
|
||||
|
||||
<p class="block">As with any standard, there are always pseudo-implementations of it in Software and Hardware - and VST is no different. I've applied many safe-guards now to either make these VST Hosts work fine, or prevent them from crashing entirely, but there may still be a VST Host out there that does yet another thing differently from the actual standard. </p>
|
||||
|
||||
<h2 class="block">With every update comes...</h2>
|
||||
|
||||
<p class="block">... a lot of testing. I'm currently running a public beta test of the new VST 3.x version, as well as a Supporter-exclusive beta test of the VST 2.x version - both are available <a rel="noreferrer noopener" href="https://s.xaymar.com/discord" target="_blank">in Discord</a>. If you're less chatty you can still try out the new version linked on the <a href="https://s.xaymar.com/voicefx">official website</a> (automatically updated short link) for it. If you have an NVIDIA Tensor capable GPU (RTX 2060 or better), why not try it out?</p>
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
title: "StreamFX 0.11: What's (going to be) in it?"
|
||||
category: "News"
|
||||
tags: [ "StreamFX", "WordPress Archive" ]
|
||||
published: false
|
||||
---
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p></p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:heading -->
|
||||
<h2>Video Super Resolution</h2>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:paragraph -->
|
||||
<p></p>
|
||||
<!-- /wp:paragraph -->
|
||||
|
||||
<!-- wp:heading -->
|
||||
<h2>Video Denoising</h2>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:heading -->
|
||||
<h2>Custom Kernel Blur</h2>
|
||||
<!-- /wp:heading -->
|
||||
|
||||
<!-- wp:heading -->
|
||||
<h2>Reference-accurate Gaussian Blur</h2>
|
||||
<!-- /wp:heading -->
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
title: "Testing NVIDIA Maxine's Super Resolution Beta"
|
||||
category: Blog
|
||||
tags: ["NVIDIA", "Maxine", "Super Resolution", "Upscaling", "WordPress Archive"]
|
||||
---
|
||||
|
||||
<p class="block">With the release of NVIDIA Maxine, a number of exciting new Video, Audio and AR effects were shown. I wanted to try these out much earlier, but an unexpected problem prevented me from doing so. However thanks to NVIDIAs help, I've managed to get some of the examples running <em>most of the time</em>. Please note that the effect I'm testing is still considered Beta by NVIDIA, so final quality will likely differ from what I show here.</p><!--more-->
|
||||
|
||||
<p class="block">Since I only had gaming footage at hand for testing, and the Super Resolution effect seems to have been designed for real world content, I went to my balcony to record a quick video. The video was downscaled to 720p as well as 540p, then fed into the effect to upscale to two different resolution targets. Below is the result of this (may need an external player):</p>
|
||||
|
||||
<figure class="block">
|
||||
<video controls muted preload="none">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.web_.mp4" title="1080p Original">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.1440-strong.web_.mp4" title="1080p -> 1440p (1.333x)">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.720p.1080-strong.web_.mp4" title="720p -> 1080p (1.333x)">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.720p.1440-strong.web_.mp4" title="720p -> 1440p (2.0x)">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.540p.1080-strong.web_.mp4" title="540p -> 1080p (2.0x)">
|
||||
<source src="https://cdn.xaymar.com/blog/2021/06/nvvfx-test-2.540p.1620-strong.web_.mp4" title="540p -> 1620p (3.0x)">
|
||||
</video>
|
||||
</figure>
|
||||
|
||||
<p class="block">The video was taken with a Logitech Brio 4K on a long USB extension, set up for 1920x1080x60 capture as close to real world as possible. The results are quite clear, and show both the strengths and weaknesses of the effect clearly. At the current level, it seems to be missing any and all temporal capabilities, relying completely on spatial upscaling. This works well as long as you don't exceed an upscale of ~1.5x, above that you end up with significant artifacts.</p>
|
||||
|
||||
<p class="block">I'm excited to see what the future will bring for this.</p>
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: "A look into VoiceFX v0.3.0"
|
||||
category: "News"
|
||||
tags: [ "VoiceFX", "WordPress Archive" ]
|
||||
---
|
||||
|
||||
<p class="block">This release was plagued with odd bugs, reappearing problems, and delays from real life events happening to me. But now, after several months of nothing, VoiceFX v0.3.0 is available! Why not delve into what exactly was changed, with some visual examples?</p><!--more-->
|
||||
|
||||
<h2 class="block">Loading the effect should now be much faster!</h2>
|
||||
|
||||
<p class="block">Earlier VoiceFX versions had a bug that would cause VoiceFX to re-initialize everything every time the VST host sent <em>any</em> command to it, until the moment it was supposed to start processing. On some VST hosts, this was so bad that users reported VoiceFX freezing their system for minutes as the VST host was spamming a lot of commands. This is now a thing of the past, and initialization is only done once, resulting in a load time reduction of up to 75%!</p>
|
||||
|
||||
<h2 class="block">Significant reduction in Reset times!</h2>
|
||||
|
||||
<p class="block">If you've used VoiceFX, you may already have experienced in how slow VoiceFX can be when restarting playback, or seeking within a file. In the worst cases, you could end up spending more than a minute for multi-channel audio, or much longer with multiple instances of VoiceFX. In rare cases, it might have even led to a frozen system that needed to be restarted.</p>
|
||||
|
||||
<p class="block">This is now a thing of the past! Thanks to help from NVIDIA, resetting VoiceFX by seeking or restarting playback is now super fast. So fast that it was difficult to measure how much faster it was on a NVIDIA RTX 3090 unless I put some stressful load on the GPU. And the best of it all? This has no impact on quality!</p>
|
||||
|
||||
<columns class="block" count="2">
|
||||
<column>
|
||||
<figure class="block" lazyload>
|
||||
<video controls>
|
||||
<source src="https://cdn.xaymar.com/blog/2021/09/j2kUvHVIEd.mp4">
|
||||
</video>
|
||||
<figcaption>v0.3.0b3 and earlier</figcaption>
|
||||
</figure>
|
||||
</column>
|
||||
<column>
|
||||
<figure class="block" lazyload>
|
||||
<video controls>
|
||||
<source src="https://cdn.xaymar.com/blog/2021/09/nNgazrnM7h.mp4">
|
||||
</video>
|
||||
<figcaption>v0.3.0 and beyond</figcaption>
|
||||
</figure>
|
||||
</column>
|
||||
</columns>
|
||||
|
||||
<h2 class="block">Split VST3.x and 2.x Installers</h2>
|
||||
|
||||
<p class="block">Since users often had a mixed bag of VST hosts, some 3.x and 2.x, I originally created a binary that had both APIs supported. This turned out to be extremely user unfriendly, as users had to choose which VST host could see it, and which couldn't. With the new separate installers for either VST 3.x or 2.x version, users can now choose which VST host should be able to see it, or if it should just be available for both!</p>
|
||||
|
||||
<h2 class="block">Other Changes</h2>
|
||||
|
||||
<figure class="block float-right align-right" style="max-width: 256px;" lazyload>
|
||||
<img src="https://cdn.xaymar.com/blog/2021/09/explorer_lGEUp9cFB7.png">
|
||||
<figcaption>Log files!</figcaption>
|
||||
</figure>
|
||||
|
||||
<p class="block">There is of course some smaller changes, which don't really need a whole section, but still need to be mentioned:</p>
|
||||
|
||||
<ul class="block">
|
||||
<li>Non-specification-compliant VST 3.x hosts should now have less issues with the effect, but are still not ideal for using the effect.</li>
|
||||
<li>VoiceFX now generates log files at <code>%LOCALAPPDATA%\VoiceFX\logs</code>, which will be used to help users with issues. At the moment these log files are not automatically deleted, so it may be necessary to occasionally clear that directory, but this will eventually be automated.</li>
|
||||
<li>The identifier of the VSTs were changed to match the actual plugin name and author, so existing setups may break.</li>
|
||||
</ul>
|
||||
|
||||
<h2 class="block">Some final Words</h2>
|
||||
|
||||
<p class="block">I've released <a rel="noreferrer noopener" href="https://xaymar.itch.io/voicefx" target="_blank">VoiceFX on itch.io</a> where you can now buy the paid version, without going through Patreon or GitHub. While I can't promise an actual release cycle - I'm just a single person working on many things <em>plus</em> my actual job - I can promise that you won't miss out on actual updates. Some testing version may however not be uploaded to itch.io when they are immediately nuked due to a critical bug.</p>
|
||||
|
||||
<p class="block">Anyway, that was all I had to say. Go grab the lastest versions from <a rel="noreferrer noopener" href="https://blog.xaymar.com/projects/voicefx/" target="_blank">my Website</a>, <a rel="noreferrer noopener" href="https://xaymar.itch.io/voicefx" target="_blank">itch.io</a> or <a rel="noreferrer noopener" href="https://discord.com/channels/144178479322234881/821129721907118142/886456588267835433" target="_blank">from my Discord</a>!</p>
|
||||
Reference in New Issue
Block a user