Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Good requantizers? (Read 9232 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Good requantizers?

Reply #25 –
Quote from: rutra80 on
Very interesting : ) Your sample has quite compressed dynamics so it favors low bit depths, but I tried it with some more challenging samples and it performs quite well.
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
Your dithering is quiet but quite spread across frequencies, some people find it more pleasant than very loud noise at very high frequencies...

In case of low rate samples when dithering with noise shaping is less useful - are there algorithms which instead of noise introduce some other kind of artifacts? Which requantizer uses simply rounding?
There are some audio editors which allow you to choose between simple rounding and dithering with or without noise shaping, e.g., Adobe Audition and, IIRC, Audacity.

To clarify, I did not implement any noise shaping in my DPCM codec above, to keep the code simple. Yes, there are some well performing dithering requantizers with noise shaping out there, e.g. the one in foobar2000 when converting to lower bit-depths. However, these are often so strong that, at low bit-depths, they increase the amplitude of the waveforms quite a bit, which increases the risk of audible clipping. Attached a more gentle noise shaper which you can use to convert to 8 bit or lower depths, along with a demo Wave file which makes comparative listening a bit easier than with my WDPK.wav above (BSBTrio.wav). You can also pass the resulting '...wav8bit.wav' into the DPCM codec I posted above, see also the files in the attached zip archive.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Good requantizers?

Reply #26 –
Quote from: rutra80 on
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
:o How loud are you (or those people) listening?
For me, at the volume that classical music gets quite loud, I need to add another 15 dB to start hearing noise with gesemann shaping. And that's when playing the noise alone, so it's not masked by music.

Re: Good requantizers?

Reply #27 –
Quote from: rutra80 on
I thought that conversion from DPCM to PCM at the same bit depth is lossless though.
Quote from: C.R.Helmrich on
The decoding part indeed does what rutra80 is looking for: it decodes the B-bit DPCM Wave file to a B-bit PCM Wave file, losslessly.
Out of curiosity, is there any practical advantage of using DPCM instead of LPCM in that case?

Re: Good requantizers?

Reply #28 –
DPCM, depending on the samples, may compress better with general purpose compression tools than linear PCM.

Re: Good requantizers?

Reply #29 –
Indeed, and you can actually notice that in the zip file which I uploaded in my reply #22 - the DPCM file is quite a bit smaller than the decoded LPCM file. It's generally beneficial to remove as much redundancy in the data as possible before packing it losslessly. That's how (7)zip, FLAC, and other lossless coders work so well.

Edit: Attached a visualization of the frequency response of the noise shaping filter code I posted above, in case anyone is interested. The blue line is the IIR filter response, the black line a special version of the absolute hearing threshold (with 50% free-field to diffuse-field correction) which the filter was fitted to at 44.1 kHz sampling rate.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Good requantizers?

Reply #30 –
Thanks again @C.R.Helmrich
Quote from: danadam on
Quote from: rutra80 on
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
:o How loud are you (or those people) listening?
For me, at the volume that classical music gets quite loud, I need to add another 15 dB to start hearing noise with gesemann shaping. And that's when playing the noise alone, so it's not masked by music.
At 8 bits?
One of them is a musician and he was listening on studio monitors.
The rest might had issues like crappy resampling or even another layer of requantizing...
It was tested on a quietly starting lowpassed piece getting full loud spectrum with time.

Re: Good requantizers?

Reply #32 –
Quote from: bennetng on
OK I made another non-DPCM companding format for fun, let's call it UA-law. It compresses to 8 bits and expands to 16 bits.
[attach type=image]20360[/attach]

To encode, drag one or more uncompressed 16-bit .wav files into UA-law.exe, new files will be saved in the same location as the original files, with "-ComP.waV" added at the end of the file names. The encoded files have same format and size as the original, but only the lowest +/-127 sample values are used, therefore volume level is very low, and can achieve high compression ratio with 7z and FLAC.

The program uses file name to switch between encode/decode modes, if the files end with "-ComP.waV" (case sensitive), decode mode will be engaged. To decode, drag the "-ComP.waV" files into UA-law.exe, the decoded files will end with "-ComPand.wav" and can be directly played.

If for whatever reason you can't or unwilling to run the program, I attached some sample files, using the infamous eig.wv, compared to u-law and a-law.


New version:
-Value -128 and -32768 are used in encoding and decoding.
-Added clipping warning caused by the encoding process.
-Added noise shaping with individually tuned coefficients for 44.1k, 48k and >=88.2k.
-Dither is not used due to the nonlinear quantization nature of UA-law.
-Unlike LPCM, the noise added by noise shaping is proportional to signal level instead of being constant.
-Sound quality no longer similar to A/u-law in any possible way. Much better.  eig.wv is fully transparent to me, and therefore I attached a new audio demo. Pay attention to the first few seconds, they are not silence.

Re: Good requantizers?

Reply #33 –
New version:
-Further improved dynamic range (noise floor) and noise shaping.
-The encoder will automatically retry at a lower level to avoid clipping like this (yes it supports multichannel files):
Code: [Select]
-------------------------------------------------------------------------------
H:\Antonio Vivaldi - The Four Seasons- Summer.wav
00:10:33.8666666 = 55907040 samples / 2-ch @ 44100Hz
Encoded at 253.3487 x 2.5019532 secs
-------------------------------------------------------------------------------
H:\Bedrich Smetana - Mein Vaterland- Die Moldau.wav
00:12:04.7600000 = 63923832 samples / 2-ch @ 44100Hz
Encoded at 255.3869 x 2.8378906 secs
-------------------------------------------------------------------------------
H:\Evangelion_3.33.0_You_Can_(Not)_Redo.wav
WAVE_FORMAT_EXTENSIBLE
00:01:47.9160000 = 31079808 samples / 6-ch @ 48000Hz
Encoded at 74.81786 x 1.4423829 secs
-------------------------------------------------------------------------------
H:\Konami Kukeiha Club - UNIVERSE -STAGE 1-.wav
00:02:55.9066666 = 15514968 samples / 2-ch @ 44100Hz
Clipped 0.2162019 dB. Retry...
Encoded at 104.5435 x 1.6826172 secs
-------------------------------------------------------------------------------
H:\Luis Fonsi - Despacito (feat. Daddy Yankee).wav
00:03:49.2400000 = 20218968 samples / 2-ch @ 44100Hz
Clipped 0.3593756 dB. Retry...
Encoded at 104.8891 x 2.1855468 secs
-------------------------------------------------------------------------------
H:\M - A Recordings - Shadow Bright by Bruce Stark M0011A.wav
00:03:00.0400000 = 15879528 samples / 2-ch @ 44100Hz
Encoded at 254.9944 x 0.7060547 secs
-------------------------------------------------------------------------------
H:\Phil Collins - Another Day in Paradise.wav
00:05:22.9600000 = 28485072 samples / 2-ch @ 44100Hz
Clipped 0.2162019 dB. Retry...
Encoded at 106.2014 x 3.0410156 secs
-------------------------------------------------------------------------------
It is recommended to use RAM drive for benchmarking, especially when decoding, and the input files should be long enough so that the timer has enough ticks (it runs at 10000000 ticks/sec) to get meaningful stats instead than showing something like "Decoded at +∞ x 0 secs". Also, when using something like foobar2000 to convert files, it launches multiple encoder instances (FLAC, mp3, Wavpack...) in multiple threads. The UA-law demo, being a demo, needs to show info in a console and does not support foobar2000's encoder pipeline at this moment.

Check the attached demo compared to IMA/MS ADPCM. The "-ComP.waV" file is smaller than them when encoded with FLAC, and the decoded file has lower distortion and noise than the ADPCM files.

It is possible to convert existing CDDA files to 16-bit but with higher sample rates, UA-law's noise shaper will further reduce the noise in audible frequencies, and the encoded files will still be smaller than the 44k ones with FLAC.

X

DSD files must and DXD files may contain lot of ultrasonic noise and will harm UA-law's performance, these files should be lowpassed or resampled to a lower rate (e.g. 96k) before encoding. After RG and/or resampling, simple 1-bit TPDF dither like Case's Smart Dither (make sure "high-pass" is checked because it is faster) should be used instead of heavily shaped dither like the one from foobar2000, and should give enough headroom to avoid intersample over or clipping the UA-law encoder so that there is no need to retry.

Re: Good requantizers?

Reply #34 –
Version 1.0.0.5

-Wrote my own filter class for better speed.

-24-bit support
24-bit files will be encoded to 12-bit and decoded to 23-bit padded to 24-bit. Users can manually convert 16-bit files to 24-bit to use the 12-bit version of UA-law. As mentioned previously, 24-bit input to UA-law should also be dithered but not noise shaped if they involve resampling and/or RG.

-flac integration
Put flac.exe in the same location as UA-law.exe.
Files end with ".wav" and ".flac" will be encoded to "-ComP.flaC".
Files end with "-ComPand.wav" are ignored.
Files end with "-ComP.waV" and "-ComP.flaC" will be decoded to "-ComPand.wav".
All of the above are case sensitive. I set the flac compression setting to the typical -8 without other special treatments, so use -8 as baseline when comparing file sizes with unencoded files, with the same version/build of the flac.exe you prefer to use.

When converting multiple flac files, UA-law may launch several flac processes to improve overall speed, benchmark results will not be consistent as they are affected by the background processes and order of input files.

-WAVE_FORMAT_EXTENSIBLE
UA-law shows this when the input files contain this tag, and preserves it during processing. RMAA for example cannot open these kinds of files. flac.exe may add this tag when the input files are 24-bit, foobar2000 only add this tag when the files contain more than two channels. There are guidelines from Microsoft specifying how this tag should be used, but different software behave differently.

The whole "package" should be idiot proof: just drag and drop files and/or folders (subfolders are ignored) into UA-law.exe, no command-line knowledge required. Copy the original audio files to a separate folder or RAM drive to evaluate UA-law as the temp files may clutter up your original folder, and refresh Windows Explorer (press F5) to update file name and size before and after conversion when necessary.

All used algorithms have nothing to do with typical lossy formats, just exponentially spaced amplitude values with noise shaping, the same way I don't think floating point audio formats and DSD are lossy by themselves.

Re: Good requantizers?

Reply #35 –
Version 1.0.0.6

-Improved handling of background flac processes to save energy without sacrificing speed.
-Reduced temp file disk space requirement in batch conversion.

Re: Good requantizers?

Reply #36 –
version 1.0.0.7

-Added a readme file.
-Fixed idle tone issues with some types of input signals.

This should be the final version, as an exercise of self-learning and proof of concept.