HydrogenAudio

Hydrogenaudio Forum => General Audio => Topic started by: marcoxD95 on 2021-01-27 10:37:32

Title: Program or plugins that offer different kinds of upsampling algorithms?
Post by: marcoxD95 on 2021-01-27 10:37:32
To specify: I recently stumbled over the topic upsampling. I was working with "low quality" 24khz music audio that I sourced from videogames.

While I was working I noticed that different programs seem to use different kind of upsampling. With Winamp I was surprised since it actually added spectrums above 24khz/12khz per channel all the way up to 48khz, since that was what I was sampling to. After digging a bit this is supposedly called extrapolation and also aliasing or mirroring effect, where basically the spectrums look like they are mirrored after 24khz. I sadly dont know what kind exactly Winamp uses, no idea whatsoever but I guess it is different from other standard upsampling processes such as Foobar offers, since Foobar does not add/fill up information above the 24khz point and seems to keep the sound constant so that it does not sound different.

I am not too deep into this topic so please forgive if some of my expressions arent entirely correct. Anyway, it seems like most people are divided whether such procedures do improve audio and it seems to also depend on taste and listening tests. I personally felt like the low quality 24khz audio seemed brighter and clearer after the 48khz upsampling with the specified "Winamp" method.

What I would like to do, is experimenting with upsampling and try out different kind of algorithms and filters. I just wonder to what extend the audio quality of such low quality audio samples can be improved. The fact that I dont know what procedure Winamp used doesnt exactly help but I read of different kinds such as cubic, linear, etc.

Is there a program or plugin for Foobar or Audacity that offers these different kinds of upsampling and lets me experiment? I really would like to dig a bit deeper into the topic and maybe find even better sounding algorithms.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-27 10:42:05
MultiResampler by kode54
https://www.foobar2000.org/components/view/foo_dsp_multiresampler
Related discussion:
https://hydrogenaud.io/index.php?topic=112883.0

By the way, did you try this?
https://hydrogenaud.io/index.php?topic=114816.msg992983#msg992983
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: marcoxD95 on 2021-01-27 12:53:17
That MultiResampler is great and exactly what I was looking for! Thanks.

Judging from the spectrum and sound Winamp or VGMStream used Zero Hold Order as upsampling method/quality.

I compared my testing sample. I upsampled my original 24khz file to 48khz with all methods/qualities that the MultiResampler provides and started comparing by listening test.

I made a order from worst to "best" sounding, at least with my personal listening opinion, probably differs from person to person.

Sinc Interpolation sounds very close to the original audio source but perhaps a tad worse. Audio spectrum also seems to cut off after 24khz/12khz per channel. Definitely the worst method to me. After this comes the original 24khz source. After this it starts to get difficult because all other methods objectively sound almost the same to me and the differences are so small, that its almost unnoticeable. So yeah, to me Zero Hold Order, Bleep Synthesis, Blam Synthesis, Linear Interpolation and Cubic Interpolation sound almost the same. Perhaps Zero Hold Order sounds a bit more hard or metallic while Linear Interpolation sounds at least to me the most natural?

What is other peoples opinion on this? What is usually considered "best". I bet the opinions are widely divided and some even think upsampling is a waste for these kind of things I am trying to achieve - better sounding audio (at least to the general human hearing).

If I would ignore the listening test (Which as you read, in 60% of the cases seem to make little to no difference to me), I would probably consider Linear Interpolation the best, based on the spectrum viewer alone since these spectrums look the most "natural" to me but even the sound seems most "natural". But maybe I am completely off with my subjective opinion.

The related discussion is also interesting. So generally some people consider the sound worse and distorted? I cant really agree in my case. Perhaps it also depends on the source material and what kind of music it is. Low quality ADX 24khz audio upsampled does seem to sound definitely better to me.



Offtopic:
Regarding "Did you try this?": No, didnt notice there is a new tool for this yet. I might check it out later, I guess it doesnt come as a installable plugin?

Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-27 13:12:36
To me it is more like a EQ/synthesizer/sampler/low sample rate game music specific thing. In this case, the Nyquist frequency falls into the audible range. For typical content (44kHz or even 32kHz when considering age), it makes little or even no difference to me. This topic is controversial because some very expensive audiophile products don't implement a proper bandlimited filter, and result in obvious distortion in the audible band, like these:
https://www.stereophile.com/content/aqua-acoustic-quality-aqua-formula-xhd-da-processor-measurements
https://www.stereophile.com/content/borderpatrol-digital-analogue-converter-se-measurements
The reality is if the effect is audible and desired, there is no need to pay for them, one can simply do this with a free DSP plugin.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: Mark7 on 2021-01-27 14:40:36
I've noticed the same when upsampling 22khz to 44.1khz. Some resamplers (like SSRC) keep more of the "high" frequencies than others. To me that sounds better than resamplers which roll of high frequencies.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: DVDdoug on 2021-01-27 15:01:41
There is something called an Exciter Effect (https://en.wikipedia.org/wiki/Exciter_(effect)) (or "harmonic exciter" or "harmonic enhancer") that adds high frequency harmonics.    

For Audacity it's an optional plug-in (https://wiki.audacityteam.org/wiki/Nyquist_Effect_Plug-ins#Harmonic_Enhancer).   and in Audacity, you'd increase the "project rate" (project sample rate), then apply the effect, then export (which will export/render at the project rate).

...The "exciter" name is just a marketing name for the original analog effect.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: j7n on 2021-01-27 15:40:31
I prefer Cubic interpolation with module sounds as a balance between a steep cutoff and aliasing, and to avoid the cutoff frequency obviously shifting with the played note. Percussive, noisy sounds are nice with low quality resampling. In sampled music you could select the method depending on the voice. 12 kHz is not low quality. I remember being quite pleased with 11.025 sampling music in PC games when the word length was also finally 16-bits. And back then my hearing was better.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: marcoxD95 on 2021-01-27 16:23:15
There is something called an Exciter Effect (https://en.wikipedia.org/wiki/Exciter_(effect)) (or "harmonic exciter" or "harmonic enhancer") that adds high frequency harmonics.    

For Audacity it's an optional plug-in (https://wiki.audacityteam.org/wiki/Nyquist_Effect_Plug-ins#Harmonic_Enhancer).   and in Audacity, you'd increase the "project rate" (project sample rate), then apply the effect, then export (which will export/render at the project rate).

...The "exciter" name is just a marketing name for the original analog effect.

Interesting. I actually played around with the plugin and it indeed seems to slightly improve the original 24khz audio. However, I think Linear Interpolation sounds better and has more impact on the perceived quality on my test sample. I also went ahead and put that Exciter effect on top of the Linear Interpolation but I didnt really notice any difference anymore in terms of audio quality or sound. Maybe they are doing similar things so using them both makes no more difference.

Here are btw some test samples from which I am working and comparing with.
https://www.mediafire.com/file/rekotbn98nqbi95/Audio_WAV_Testsamples.7z/file

I still think I prefer Linear Interpolation the most out of all. Then again, I cant really say any difference to Cubic. They both sound the same to me. On spectrum I can see that cubic supposedly tries to make the higher frequencies seem more natural and removes "artifacts" above 20khz. I hardly doubt they are hearable but oh well. I mean from a pure technical standpoint cubic should be better right? I wonder what other people think or prefer.

I prefer Cubic interpolation with module sounds as a balance between a steep cutoff and aliasing, and to avoid the cutoff frequency obviously shifting with the played note. Percussive, noisy sounds are nice with low quality resampling. In sampled music you could select the method depending on the voice. 12 kHz is not low quality. I remember being quite pleased with 11.025 sampling music in PC games when the word length was also finally 16-bits. And back then my hearing was better.

So I guess Cubic should in theory be best, right? I mean I cant hear a difference between Linear and Cubic but if Cubic should be better in theory, I guess I just use that for my videogame rip masters.

Yeah well. I guess from your perspective 12khz isnt that low. For me personally I clearly notice a difference to "normal" 44.1/22.05khz music or 48/24khz audio. I primarily used 48khz since that is clean 2x from 24khz sources but no idea if it would actually matter which of these I use.

Shouldnt 12khz in audioterms be around MP3 80kbits? I mean... That sounds awful if you ask me. I dont even find CBR 128kbits bearable for "enjoyable" listening.




So I read a bit more into the topic on various websites and it seems like most would recommend cubic interpolation. It has less aliasing in the higher frequencies above 20khz, which my audio test samples also proof on spectrum viewer. Not like I think any normal human would be able to hear them but on paper its probably better without it. However, it also has no mentionable audio information above 22khz. Shouldnt I sample to 44.1khz with cubic interpolation then since 48khz seems a waste in this case? I often thought its safer/cleaner to sample cleanly like 24 - 48 - 96, etc but not sure if this is actually true when it comes to this topic. My original audio source is in 24khz.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: DVDdoug on 2021-01-27 17:15:42
Quote
So I guess Cubic should in theory be best, right?
I don't know what algorithms are the "best" but proper up-sampling isn't supposed to change the sound at all, and usually it doesn't.  It's not supposed to be an "effect" and there shouldn't be any side-effects.

Quote
For me personally I clearly notice a difference to "normal" 44.1khz music or 48khz audio.
Again, there's not supposed to be any difference.   Up-sampling or down-sampling between 44.1 and 48kHz (or any resampling above 44.1kHz) should also be transparent.   

Quote
Shouldnt 12khz in audioterms be around MP3 80kbits? I mean... That sounds awful if you ask me. I dont even find CBR 128kbits bearable for "enjoyable" listening.
With MP3 there are other artifacts besides the loss of high frequencies.    I don't really listen to low-bitrate MP3 but when I hear a "bad" MP3 the loss of highs is not the 1tst thing I notice.


P.S.
I just tried a quick experiment - I made a 15kHz sine wave in Audacity and exported as 80kbps MP3.   When I opened the MP3 it looked the same.   (In the context of music, it's more likely-to throw-away some higher frequencies).

Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: marcoxD95 on 2021-01-27 17:54:22
Quote
Shouldnt 12khz in audioterms be around MP3 80kbits? I mean... That sounds awful if you ask me. I dont even find CBR 128kbits bearable for "enjoyable" listening.
With MP3 there are other artifacts besides the loss of high frequencies.    I don't really listen to low-bitrate MP3 but when I hear a "bad" MP3 the loss of highs is not the 1tst thing I notice.


P.S.
I just tried a quick experiment - I made a 15kHz sine wave in Audacity and exported as 80kbps MP3.   When I opened the MP3 it looked the same.   (In the context of music, it's more likely-to throw-away some higher frequencies)

True. But I was refering to where the frequencies cut off. Of course MP3 transcoding introduces other problems as well.

I actually did the same test now and converted one of my CD 44.1 music files to MP3 80kbps just for fun and it cut off around 13khz for me. On top of that it also changed the sampling rate to 32khz. Kinda interesting MP3 does the sampling rate on the fly but it only seems to go below 44.1 when we are talking 100kbps and less, which no normal user would ever use. Btw, I used Foobar for conversion and latest LAME 3.99 encoder.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: [JAZ] on 2021-01-27 19:25:43
First, what's first:

The best kind of resampling is the one that does not distort the sound in the existing frequencies, and does not add sounds on nonexistent frequencies.

As such, Sinc interpolation and higher order interpolation are better than Zero Hold or linear.




Now... To your question:

What happens when the sound that you resample was actually meant to be distorted?
(Mostly because that was what the hardware/software produced when it was created)

Here, I show you two examples of two tracker modules created in the 90's:

This one was created with low quality samples, and with zero or linear interpolation: In comparison, one would prefer the "con aliasing" (aliased) version rather than the "sin aliasing" (not aliased) version, because the non-aliased sounds too muffled:

Tripilandia con aliasing: https://www.youtube.com/watch?v=-p18GzUXhXI&t=1s
Tripilandia sin aliasing: https://www.youtube.com/watch?v=wmYCY_rorxw

On the other hand, this other song was composed with a hardware soundcard that had something similar to cubic interpolation ( a Gravis Ultrasound).  In this case, the one without aliasing is cleaner, while the other one seems too bright:

Rainy Autumn sin aliasing : https://www.youtube.com/watch?v=kPkwIRhivBQ
Rainy Autumn con aliasing: https://www.youtube.com/watch?v=PDzlH__gp1w


What does this mean?
This means that generally, you cannot take a setting and be happy with this.
This is also the reason why the "Exciter" kind of DSP effects ( there was one Soundblaster card that included one), might help on some cases, but can cause ear fatigue or be unpleasant with other content.

Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: j7n on 2021-01-28 02:05:56
I still think I prefer Linear Interpolation the most out of all. Then again, I cant really say any difference to Cubic. They both sound the same to me. [..] Shouldnt 12khz in audioterms be around MP3 80kbits?
I believe that old soundcards generally used simple linear interpolation to upsample sounds to the output codec rate, at least mine did. Not sure about consoles, emulators seem to default to cubic. Cubic should preserve a slightly more of sharpness below the cutoff because it is steeper, and safeguard better against aliasing. But I only hear a clear difference around 7-10 kHz and can only perceive overall amount of brightness above that, not any aliasing. Sounds that naturally exist there are usually not tonal and would not alias obviously. To me the sample with linear seemed marginally duller, but I failed to ABX them.

Low bitrate MP3 has inaccuracies at all frequencies and time smearing as it tries to distribute the bit depth among the most important frequencies. PCM has full bit depth available at all times.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: Chibisteven on 2021-01-28 05:09:38
I still think I prefer Linear Interpolation the most out of all. Then again, I cant really say any difference to Cubic. They both sound the same to me. [..] Shouldnt 12khz in audioterms be around MP3 80kbits?
I believe that old soundcards generally used simple linear interpolation to upsample sounds to the output codec rate, at least mine did. Not sure about consoles, emulators seem to default to cubic. Cubic should preserve a slightly more of sharpness below the cutoff because it is steeper, and safeguard better against aliasing. But I only hear a clear difference around 7-10 kHz and can only perceive overall amount of brightness above that, not any aliasing. Sounds that naturally exist there are usually not tonal and would not alias obviously. To me the sample with linear seemed marginally duller, but I failed to ABX them.

Low bitrate MP3 has inaccuracies at all frequencies and time smearing as it tries to distribute the bit depth among the most important frequencies. PCM has full bit depth available at all times.

Consoles at least 6th Gen ones that I looked at seem to do simple linear re-sampling in their internal DSPs and usually only have one output rate.  Sometimes they support more than one output rate but this depends on which console you're using.
5th Gen maybe similar to 6th Gens but I have to look as those don't seem fixed in their sample rate output.
4th Gen ones seems all over the place but that era in general required more complicated sound hardware as cartridges were still the norm.  For the two most popular ones it was Zero Order Hold for one of them and Gaussian for the other one that was used.
3rd Gen and earlier is probably just as crude and simple as their internal synthesizers in general.  Zero order hold most likely for any samples played if the hardware supported it.

On the PC side, DirectX still uses simple linear re-sampling.  As for recent consoles, I don't really know what their doing as I mostly lost interest in them after the 6th Gen as they've become a bit like PCs.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-28 05:35:13
I think people should differentiate playback of low sample rate content and playback of sequencer formats. OP mentioned ADX and I suppose it is not being used as a sequencer sample format (perhaps @kode54 knows more about it). That means they are simply being played as-is, perhaps with pan, fade, mix, but not per-voice pitch-shifted, with LFO, envelope and automatable filters.

Emulators of sequencer formats should at least offer a mode which is faithful to the original hardware, which the music composer originally used to create that piece of music.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: knutinh on 2021-01-28 13:53:41
To specify: I recently stumbled over the topic upsampling. I was working with "low quality" 24khz music audio that I sourced from videogames.

While I was working I noticed that different programs seem to use different kind of upsampling. With Winamp I was surprised since it actually added spectrums above 24khz/12khz per channel all the way up to 48khz, since that was what I was sampling to. After digging a bit this is supposedly called extrapolation and also aliasing or mirroring effect, where basically the spectrums look like they are mirrored after 24khz. I sadly dont know what kind exactly Winamp uses, no idea whatsoever but I guess it is different from other standard upsampling processes such as Foobar offers, since Foobar does not add/fill up information above the 24khz point and seems to keep the sound constant so that it does not sound different.

I am not too deep into this topic so please forgive if some of my expressions arent entirely correct. Anyway, it seems like most people are divided whether such procedures do improve audio and it seems to also depend on taste and listening tests. I personally felt like the low quality 24khz audio seemed brighter and clearer after the 48khz upsampling with the specified "Winamp" method.

What I would like to do, is experimenting with upsampling and try out different kind of algorithms and filters. I just wonder to what extend the audio quality of such low quality audio samples can be improved. The fact that I dont know what procedure Winamp used doesnt exactly help but I read of different kinds such as cubic, linear, etc.

Is there a program or plugin for Foobar or Audacity that offers these different kinds of upsampling and lets me experiment? I really would like to dig a bit deeper into the topic and maybe find even better sounding algorithms.
From a dsp point of view, the most generally "correct" way to do upsampling is by brickwall filtering, usually some approximation to a sinc function.

I don't see why you would want to use cubic or linear interpolation, techniques popular in image processing for their low cost.

Sure, you can try to do better than Shannon and Nyquist by restricting to a particular input, intent and user preference. Spectral band replication in codecs is fairly successful at that. If your sound consists of separate "bleeps" and squeaks generated by simple mathematical constructs (square waves, sawtooth waves,...) perhaps you can outsmart generic dsp by guessing what the analytic formulae for the waveform was, regenerate that at a higher samplerate and get "plausible" high frequencies?

-k
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: ajp9 on 2021-01-28 18:43:52
From a compression point of view, some degree of fold-over from a lower bandwidth to a higher bandwidth can be used to make sound brighter or seemingly "transparent" even though it's an effect not a lossless replication. Aliasing in decimation should always be avoided, but in interpolation keeping some fold-over can often sound better than brick wall filtering. It's not DSP correct in terms of reproduction the way the samples are encoded, yes, but brick walls can not only make music sound muffled or dull but in other cases it can sound more harsh. Sometimes, basic interpolation works well for musicians as they see fit.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: knutinh on 2021-01-29 10:18:11
From a compression point of view, some degree of fold-over from a lower bandwidth to a higher bandwidth can be used to make sound brighter or seemingly "transparent" even though it's an effect not a lossless replication. Aliasing in decimation should always be avoided, but in interpolation keeping some fold-over can often sound better than brick wall filtering. It's not DSP correct in terms of reproduction the way the samples are encoded, yes, but brick walls can not only make music sound muffled or dull but in other cases it can sound more harsh. Sometimes, basic interpolation works well for musicians as they see fit.
If you want to speculatively fill in higher frequencies, are there not established techniques for (blind) bandwidth extension that does a better job than does very short resampling kernels?
https://en.wikipedia.org/wiki/Bandwidth_extension
https://ieeexplore.ieee.org/document/8462691
-k
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-29 10:42:00
The problem of ZOH, linear, and other forms of inadequate resampling is that apart from integer upsampling ratio, there will always be aliasing. For ADX it could be fine for 24kHz to 48kHz, especially if the original hardware uses the same mode, so that the musician/sound engineer heard the same thing.

For samplers and other applications where the sound engine is not tied to a particular hardware, even the simplest sequencer format supports 12TET pitch shifting, if not continuously like pitch bend, so not an apple to apple comparison.

By the way, SNES uses Gaussian, PS2 uses Catmull-Rom, according to PCSX2. Creative uses the E-MU 8-point interpolation algorithm, X-Fi's SoundFont synth uses a pretty high order IIR (or minimum phase FIR?) filter.

Earlier versions of Kontakt only use linear interpolation, due to the fact people mostly use them with GigaSampler libraries, and perhaps AKAI, where the interpolation quality is poor anyway. When you import SF2 to Kontakt, it sounded like shit, envelope parameters are wrong, interpolation is poorer than Creative/EMU's engine.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-29 10:46:55
From a compression point of view, some degree of fold-over from a lower bandwidth to a higher bandwidth can be used to make sound brighter or seemingly "transparent" even though it's an effect not a lossless replication. Aliasing in decimation should always be avoided, but in interpolation keeping some fold-over can often sound better than brick wall filtering. It's not DSP correct in terms of reproduction the way the samples are encoded, yes, but brick walls can not only make music sound muffled or dull but in other cases it can sound more harsh. Sometimes, basic interpolation works well for musicians as they see fit.
If you want to speculatively fill in higher frequencies, are there not established techniques for (blind) bandwidth extension that does a better job than does very short resampling kernels?
https://en.wikipedia.org/wiki/Bandwidth_extension
https://ieeexplore.ieee.org/document/8462691
-k
Modern sampler/synth often support more than 100 voices, so advanced methods like these are not practical for latency and processing power concerns. For games, even more processing power is reserved for other purposes.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: marcoxD95 on 2021-01-30 11:04:02
I am playing with PCSX2 quite often and I find the original PS2 Catmul-Rom sound bad. It sounds really rough and crackly. Weirdly enough I cannot remember this from back in my time but lets be honest, I still played PS2 on a old CRT TV so I probably didnt even notice such nuances.

I usually use Linear sound in the PCSX2 sound options because to me it sounds clearer and less muddled. I would honestly even use Cubic or Hermite mode but for some reason this really kills the performance in several PS2 games making them lag almost constantly when sound is played. I hardly doubt my hardware is at fault, since I have a beefy system and I can even crack other settings such as playing at max 8x resolution and full blending accuracy but that sound mixing setting seriously kills performance in most games so it cannot really be used practicable.

Btw, coincidence but I was actually wondering about that Hermite Resampling. Is it uncommon? According to PCSX2 it should be the best sound quality option. That MultiResampler doesnt offer it though. Was just wondering if I could try this one out too on my soundtrack rips. Doubt I would hear a difference from Cubic but it would be a interesting try never the less. If any Foobar plugin offered this that is. Could barely find anything about Hermite Interpolation. It is sometimes also refered to as Cubic Hermite Resampling but generally seems to be really uncommon and not much information on. Guess it is not used for traditional audio resampling?
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: [JAZ] on 2021-01-30 11:28:11
ZOH resampling should usually be lowpassed, maybe with a gentle rolloff. That could give some better experience.

We have to think that older equipment would not play the higher end (even the lower end) like we can nowadays.
That CRT TV is a good example, since they used to be filtered in order to not hear the pilot signal of the video. ( https://en.wikipedia.org/wiki/Pilot_signal )

I can't really comment on the Hermite resampling, but I seems it requires more points than cubic interpolation, which means more calculations and so more CPU usage.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-01-30 12:20:44
I am playing with PCSX2 quite often and I find the original PS2 Catmul-Rom sound bad. It sounds really rough and crackly. Weirdly enough I cannot remember this from back in my time but lets be honest, I still played PS2 on a old CRT TV so I probably didnt even notice such nuances.

I usually use Linear sound in the PCSX2 sound options because to me it sounds clearer and less muddled. I would honestly even use Cubic or Hermite mode but for some reason this really kills the performance in several PS2 games making them lag almost constantly when sound is played. I hardly doubt my hardware is at fault, since I have a beefy system and I can even crack other settings such as playing at max 8x resolution and full blending accuracy but that sound mixing setting seriously kills performance in most games so it cannot really be used practicable.

Btw, coincidence but I was actually wondering about that Hermite Resampling. Is it uncommon? According to PCSX2 it should be the best sound quality option. That MultiResampler doesnt offer it though. Was just wondering if I could try this one out too on my soundtrack rips. Doubt I would hear a difference from Cubic but it would be a interesting try never the less. If any Foobar plugin offered this that is. Could barely find anything about Hermite Interpolation. It is sometimes also refered to as Cubic Hermite Resampling but generally seems to be really uncommon and not much information on. Guess it is not used for traditional audio resampling?
I don't think the crackling and lag you experienced is inherent to any of the interpolation methods. The emulator tried to sync video and audio with time stretching and caused all kinds of nasty sounds.

Again, you must specify the audio format is a simple one (like ADX) or a sequencer format (e.g. PSF). For PSF you can't really simply use a DSP plugin to solve everything because the interpolation happens at per-voice (musical note or sound effect component) level, not after mixdown to stereo.

PSF plugin, in case it is (not) the format you are talking about:
https://www.foobar2000.org/components/view/foo_psf
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: knutinh on 2021-02-06 06:33:21
From a compression point of view, some degree of fold-over from a lower bandwidth to a higher bandwidth can be used to make sound brighter or seemingly "transparent" even though it's an effect not a lossless replication. Aliasing in decimation should always be avoided, but in interpolation keeping some fold-over can often sound better than brick wall filtering. It's not DSP correct in terms of reproduction the way the samples are encoded, yes, but brick walls can not only make music sound muffled or dull but in other cases it can sound more harsh. Sometimes, basic interpolation works well for musicians as they see fit.
If you want to speculatively fill in higher frequencies, are there not established techniques for (blind) bandwidth extension that does a better job than does very short resampling kernels?
https://en.wikipedia.org/wiki/Bandwidth_extension
https://ieeexplore.ieee.org/document/8462691
-k
Modern sampler/synth often support more than 100 voices, so advanced methods like these are not practical for latency and processing power concerns. For games, even more processing power is reserved for other purposes.
I dont understand this. If your application can work at a high intermediate rate (88, 96,...) then one can often apply «good» methods offline. The waveform is then going to be smooth, and one can use simpler methods for low cost/low delay realtime resampling.

For a vintage video game that includes sound snips at 22.05kHz or even lower rates (in order to save cartridge/cd rom space) surely we can afford to upsample those files offline using todays storage?

I am sure that there are some intricacies with samplers where you want to continously modulate resampling using the pitch bend wheel, but I am argueing that the issues of 1) maximizing (apparent) quality of limited recordings, and 2) changing samplerate in a realtime application are usually solved best separately.

-k
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-02-06 07:31:22
I guess you are a DSP-only guy without hand-on experiences in composition/sequencing and such.

Some software samplers running in DAWs have adjustable quality settings so that one can always use the high quality settings during offline rendering, and uses the lower quality ones during sequencing, but it also comes with the problem that the rendered output could sound different from the real time output so that the composer need to go back and forth comparing things. Some software samplers don't have these settings. On the other hand, if we are talking about something like keyboard workstations with built-in sampler/synth, they can only operate in real time, the same applies to live performance as well.

For the game music part, I am referring to the OP's issue that game emulators (that you can play games with them, not only music) and games are still limited by real time performance. For a game music player alone of course people can apply whatever quality settings they like, provided that the player plugins have such settings, but if the algorithm is nowhere similar to the original hardware, then it is not "emulation" anymore.

The bottom line is, you can do whatever you like if you write your own sampler and game music player with your own codes, but most often the one who write the application is not the one who use the application, and not the same one who composed the music, so there will always be conflicts, and compromise.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-02-06 12:55:34
Just another example of these things. Game music OST released on a CD, so by definition "offline".
https://youtu.be/TLhv802D_8o
First 15 seconds attached. Composers are not necessarily "DSP-aware", they just use whatever sound good, suitable and convenient for their compositions. Products with the "ideal" sound patches for composers are not necessarily distributed with the sample player using "ideal" synthesis technology in the eyes (if not ears) of DSP guys.

I am also not happy with the sound quality of the Dreamcast synthesizer, but I only have one favorite game (Ikaruga) with poor quality music in this console. I ended up with sequencing the whole OST myself, so everything follows my own taste.
https://www.nicovideo.jp/watch/sm16395588

Perhaps people who are not happy with kode54's plugins can also learn how to write their own interpolation algorithms, should it be AI, Hermite or other kinds of fancy algorithms, or sequence the music note by note like what I did. I even spent a year to compile my own SoundFont collection so that GM/GS/XG MIDI files play in the way I wanted.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: saratoga on 2021-02-06 16:59:58
Is there a program or plugin for Foobar or Audacity that offers these different kinds of upsampling and lets me experiment? I really would like to dig a bit deeper into the topic and maybe find even better sounding algorithms.

see "Polynomial Interpolators for High-Quality Resampling of Oversampled Audio" by Olli Niemitalo:
http://yehar.com/blog/wp-content/uploads/2009/08/deip.pdf

It is a great reference for different interpolators, their performance and their alias rejection.

Btw, coincidence but I was actually wondering about that Hermite Resampling. Is it uncommon? According to PCSX2 it should be the best sound quality option. That MultiResampler doesnt offer it though. Was just wondering if I could try this one out too on my soundtrack rips. Doubt I would hear a difference from Cubic but it would be a interesting try never the less. If any Foobar plugin offered this that is. Could barely find anything about Hermite Interpolation. It is sometimes also refered to as Cubic Hermite Resampling but generally seems to be really uncommon and not much information on. Guess it is not used for traditional audio resampling?

Cubic hermite interpolation is pretty common in audio.  Usually it is just called "cubic interpolation".  We use it in Rockbox. 

From a compression point of view, some degree of fold-over from a lower bandwidth to a higher bandwidth can be used to make sound brighter or seemingly "transparent" even though it's an effect not a lossless replication. Aliasing in decimation should always be avoided, but in interpolation keeping some fold-over can often sound better than brick wall filtering. It's not DSP correct in terms of reproduction the way the samples are encoded, yes, but brick walls can not only make music sound muffled or dull but in other cases it can sound more harsh. Sometimes, basic interpolation works well for musicians as they see fit.
If you want to speculatively fill in higher frequencies, are there not established techniques for (blind) bandwidth extension that does a better job than does very short resampling kernels?
https://en.wikipedia.org/wiki/Bandwidth_extension
https://ieeexplore.ieee.org/document/8462691
-k
Modern sampler/synth often support more than 100 voices, so advanced methods like these are not practical for latency and processing power concerns. For games, even more processing power is reserved for other purposes.
I dont understand this. If your application can work at a high intermediate rate (88, 96,...) then one can often apply «good» methods offline. The waveform is then going to be smooth, and one can use simpler methods for low cost/low delay realtime resampling.

For a vintage video game that includes sound snips at 22.05kHz or even lower rates (in order to save cartridge/cd rom space) surely we can afford to upsample those files offline using todays storage?

I am sure that there are some intricacies with samplers where you want to continously modulate resampling using the pitch bend wheel, but I am argueing that the issues of 1) maximizing (apparent) quality of limited recordings, and 2) changing samplerate in a realtime application are usually solved best separately.

I benchmarked Hermite in rockbox, I think it was 3 or 4 MHz on an ARM9 core per 44.1->48k stereo channel pair, so yes, on modern hardware polynomial resampling is basically free.  I'm also not sure that the number of voices matters either, since most emulators are probably mixing at the native rate and then resampling the output.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: bennetng on 2021-02-06 18:23:39
https://youtu.be/DOJquuBpenw
SNES SPC sample extraction.
The video shows the format uses very short, looped notes with only one pitch (or slightly more for some games) to cover the whole keyboard range. Thankfully it is a rather old console so graphics and other requirements are low.

foobar2000 SPC plugin with several kinds of interpolation methods to play with.
https://www.foobar2000.org/components/view/foo_snesapu
The real console uses Gaussian. I suppose if none of the available algorithms sound good then one can extract the samples, reconstruct the sound patches and sequence, then render the audio output with external samplers. Could be laborious.

I am also not happy with the sound quality of the Dreamcast synthesizer, but I only have one favorite game (Ikaruga) with poor quality music in this console. I ended up with sequencing the whole OST myself, so everything follows my own taste.
https://www.nicovideo.jp/watch/sm16395588
I did it with SFZ and a SoundFont I made, the highest quality setting should look like the attached screenshot. Quality can be considered extremely high for a sampler. No facny AI reconstruction though. The whole thing took me two weeks to sequence and there is (was?) no tool to extract anything (sequence, sample, play individual tracks separately) from the Dreamcast sequence format.
Title: Re: Program or plugins that offer different kinds of upsampling algorithms?
Post by: For Serious on 2021-02-20 04:55:52
Though I don't know the specifics of how the program does it, I have used Stereo Tool to do the same thing with old game music.
The original music tracks of StarCraft are in 22kHz wav. I always thought they sounded awesome but dull compared to other recordings. Then I got a hold of one of the tracks from a rare Korean CD (Starcraft Game Music Vol.1) and I had to do something about the missing frequencies.
In Stereo Tool I used the Absolute Highs setting combined with AutoEQ. The AutoEQ was hard to get tuned right, but the end results sound like they came from the same source as the CD now.