HydrogenAudio

Hydrogenaudio Forum => General Audio => Topic started by: rutra80 on 2021-06-17 21:16:19

Title: Good requantizers?
Post by: rutra80 on 2021-06-17 21:16:19
What requantizers do you know for changing PCM bit depth? For example from 16 to 8-bits.
I know about simply discarding bits, rounding, dithering with noise shaping.
This one sounds quite impressive: https://www.bitsnbites.eu/hiqh-quality-dpcm-attempts/
I also remember 2Bdecided saying that lossyWAV would be quite an efficient requantizer, but I see no options for forcing bit limits to use no more than 8-bits for example?
What else?
Title: Re: Good requantizers?
Post by: saratoga on 2021-06-18 03:35:05
Do you want PCM requantization or compression?  If the former, Sox is a good choice, but there are many other options too.  If you want compression, FLAC or MP3 have better compression than DPCM. 
Title: Re: Good requantizers?
Post by: nu774 on 2021-06-18 04:04:49
Comment from https://github.com/gcp/opus-tools/blob/master/src/opusdec.c#L148:

Quote
This implements a 16 bit quantization with full triangular dither and IIR noise shaping. The noise shaping filters were designed by Sebastian Gesemann based on the LAME ATH curves with flattening to limit their peak gain to 20 dB. (Everyone elses' noise shaping filters are mildly crazy)
The 48kHz version of this filter is just a warped version of the 44.1kHz filter and probably could be improved by shifting the HF shelf up in frequency a little bit since 48k has a bit more room and being more conservative against bat-ears is probably more important than more noise suppression.
This process can increase the peak level of the signal (in theory by the peak error of 1.5 +20 dB though this much is unobservable rare) so to avoid clipping the signal is attenuated by a couple thousandths of a dB. Initially the approach taken here was to only attenuate by the 99.9th percentile, making clipping rare but not impossible (like SoX) but the limited gain of the filter means that the worst case was only two thousandths of a dB more, so this just uses the worst case.
The attenuation is probably also helpful to prevent clipping in the DAC reconstruction filters or downstream resampling in any case.

And this is it:
https://wiki.hydrogenaud.io/index.php?title=Noise_shaping
(Sampling rate of 44100 Hz, derived from LAME's ATH formula)

Also I remember S. Gesemann published a re-quiantizer app based on this, built with Java,  here in HA... but I don't remember the exact thread.
Title: Re: Good requantizers?
Post by: rutra80 on 2021-06-18 08:53:25
Do you want PCM requantization or compression?  If the former, Sox is a good choice, but there are many other options too.  If you want compression, FLAC or MP3 have better compression than DPCM. 
I want requantization, not compression. Especially reduction to 8-14 bits.

I know about sox but it mostly does dithering (with noise shaping), I'm interested in other methods with as little noise as possible (or at least having adaptive requantization and noise shaping like lossyWAV, but I'm unable to output 8 bit PCM with it).
DPCM sounds very impressive but it decodes to 16bit PCM and after decoding I see more bits in use than selected.
Title: Re: Good requantizers?
Post by: saratoga on 2021-06-18 16:20:38
Do you want PCM requantization or compression?  If the former, Sox is a good choice, but there are many other options too.  If you want compression, FLAC or MP3 have better compression than DPCM. 
I want requantization, not compression. Especially reduction to 8-14 bits.

I know about sox but it mostly does dithering (with noise shaping), I'm interested in other methods with as little noise as possible (or at least having adaptive requantization and noise shaping like lossyWAV, but I'm unable to output 8 bit PCM with it).
DPCM sounds very impressive but it decodes to 16bit PCM and after decoding I see more bits in use than selected.

The reason encoding to DPCM sounds good is that it is a form of lossy compression.  If you don't want compression, then your options are much more limited since there is a fixed amount of noise that must be present in an uncompressed bitstream of a given size.  You can use things like noise shaping to move that noise around (usually to higher frequencies where it is less noticeable), but you can't get rid of it without either introducing compression or increasing the bitrate. 
Title: Re: Good requantizers?
Post by: rutra80 on 2021-06-20 14:14:52
So when on https://www.bitsnbites.eu/hiqh-quality-dpcm-attempts/ they show 8 bit file, there actually is 16 bits of data but compressed to a file of an 8 bit equivalent?
I thought that they actually requantize to 8 bits and the file is simply half of the 16 bit size no matter if in DPCM or PCM form.
Title: Re: Good requantizers?
Post by: kode54 on 2021-06-20 20:23:13
DPCM, and also ADPCM, compress the file to delta values, usually of lower precision than the actual sample size. The decoded output would actually be 16 bits, but the compressed file is either 8 or even 4 bits per sample. Information is sometimes lost in the compression process, due to precision limits or other errors, but it usually only differs on the level of mostly inaudible noise.
Title: Re: Good requantizers?
Post by: rutra80 on 2021-06-20 21:43:54
If we create 8 bit DPCM file, can we decode it losslessly to 8 bit PCM?
Title: Re: Good requantizers?
Post by: kode54 on 2021-06-20 22:11:32
Not really. And there's no point to encoding 8 bit PCM to 8 bit DPCM, except to possibly make certain waveforms more compressible with conventional compression algorithms on top of the DPCM transformation.

The site you mentioned above encodes 16 bit PCM to 8 bit DPCM, but the deltas still decode to 16 bit samples.
Title: Re: Good requantizers?
Post by: rutra80 on 2021-06-20 22:25:22
Yes I do mean about encoding 16bit PCM to 8bit DPCM, but I thought that it would decode losslessly to 8 bit PCM (that delta is only used for convenience, like L/R vs M/S coding). Now I understand that deltas are 8 bit and we need more than 8 PCM bits to decode it losslessly.

So, to summarize - are there any other interesting ways to requantize 16 PCM bits to 8 PCM bits except simply discarding bits, rounding, or dithering with noise shaping?
There's so many resampling algorithms...
Title: Re: Good requantizers?
Post by: saratoga on 2021-06-21 05:43:38
So, to summarize - are there any other interesting ways to requantize 16 PCM bits to 8 PCM bits except simply discarding bits, rounding, or dithering with noise shaping?

No, or at least none that are better.  An 8 bit PCM signal has an SNR of < ~48dB.  You can use noise shaping to move noise to low frequencies or to high frequencies, but the SNR is still limited to 48 dB.  If you want to increase SNR,  you need more bits or to compress those bits.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-06-21 09:59:54
There is also A-law and µ-law (https://en.wikipedia.org/wiki/A-law_algorithm) as another form of lossy compression with fixed bit-per-sample count, but as saratoga indicated, those don't sound better than DPCM since they are missing the prediction part (and are, therefore, less efficient).

Note that A-law and µ-law are similar in concept to a floating-point number representation. I just rediscovered this Float16 discussion (https://hydrogenaud.io/index.php?topic=90770.0) which I remembered, and A/µ-law are a kind of "Float8" representation. But all of this represents lossy requantization, I don't think any requantization to a fixed lower bit-depth can, in general, be lossless.

Chris
Title: Re: Good requantizers?
Post by: bennetng on 2021-06-21 11:22:26
These kinds of discussion really blurred the line between lossy and lossless. For instance, int32 and float32 only have 25 bits of overlapping lossless values (+/-16777216) so both formats are "lossy" if you are converting back and forth and it is unfair to call either format "lossless".

Let's say, if 16-bit a-law/mu-law were being chosen as the CDDA data format in the 1980s, would we still call 16-bit LPCM "lossless"?

BTW, if I have something to complain about typical floating point formats, I would say they have too many wasted NaNs, and there is no flexibility to choose the amount of exponent vs fraction bits. If you define your own floating point types, then they are not accelerated by generic processors.

Integers are wasteful in another way as well, of course.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-06-21 12:36:24
These kinds of discussion really blurred the line between lossy and lossless. For instance, int32 and float32 only have 25 bits of overlapping lossless values (+/-16777216) so both formats are "lossy" if you are converting back and forth and it is unfair to call either format "lossless".

Let's say, if 16-bit a-law/mu-law were being chosen as the CDDA data format in the 1980s, would we still call 16-bit LPCM "lossless"?
This thread is about requantization of existing digital waveform data to a lower bit-depth. So if your "master" audio format (e.g. Float16 taken from a hypothetical companding CDDA format, as you say) would not be requantized, there wouldn't be any reason to call it lossy IMHO.

I think the major source of confusion here is that people tend to mix up the concepts of "lossy" and "perceptual coding". Converting e.g. 16-bit PCM from CD to 8-bit PCM (or DPCM, or A/µ-law), or Float32 to Float16, is lossy since you lose information in that process. The only principal difference between that kind of bit-rate reduction and more sophisticated MP3/AACish compression is that the latter use an adaptive frequency representation along with a perceptual model to "hide" the distortion introduced by the requantizer (which, in case of MP3, AAC, and later MPEG standards is quite similar to A/µ-law, by the way).

Chris
Title: Re: Good requantizers?
Post by: bennetng on 2021-06-21 12:49:21
Yes. It makes more sense to use the term "perceptual coding".
Title: Re: Good requantizers?
Post by: rutra80 on 2021-06-21 16:14:47
By lossy/lossless I meant:

16bit PCM --lossy--> 8bit DPCM --lossless--> 16bit PCM

I was hoping it looks like that:
16bit PCM --lossy--> 8bit DPCM --lossless--> 8bit PCM

It's obvious that requantizing to lower bit depth is lossy. I thought that conversion from DPCM to PCM at the same bit depth is lossless though.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-06-21 18:30:37
Hmm, I guess that depends on the 'D', i.e. way the predictor in your particular DPCM format works. Generally speaking, when decoding each DPCM value (by adding the predicted value which is also requantized to 8 bit), you may in some cases end up with a larger value range than what would fit into 8 bit. But with proper format constraints in place, such situations could be avoided, and then indeed the decoding might be lossless. But I don't know of any such simple DPCM format (without adaptive requantization).

Chris
Title: Re: Good requantizers?
Post by: bennetng on 2021-06-25 19:05:13
OK I made another non-DPCM companding format for fun, let's call it UA-law. It compresses to 8 bits and expands to 16 bits.
X

To encode, drag one or more uncompressed 16-bit .wav files into UA-law.exe, new files will be saved in the same location as the original files, with "-ComP.waV" added at the end of the file names. The encoded files have same format and size as the original, but only the lowest +/-127 sample values are used, therefore volume level is very low, and can achieve high compression ratio with 7z and FLAC.

The program uses file name to switch between encode/decode modes, if the files end with "-ComP.waV" (case sensitive), decode mode will be engaged. To decode, drag the "-ComP.waV" files into UA-law.exe, the decoded files will end with "-ComPand.wav" and can be directly played.

If for whatever reason you can't or unwilling to run the program, I attached some sample files, using the infamous eig.wv (https://hydrogenaud.io/index.php/topic,49601.0.html), compared to u-law and a-law.

Title: Re: Good requantizers?
Post by: DVDdoug on 2021-06-25 20:37:30
Quote
16bit PCM --lossy--> 8bit DPCM --lossless--> 8bit PCM

It's obvious that requantizing to lower bit depth is lossy. I thought that conversion from DPCM to PCM at the same bit depth is lossless though.
No.    Any change (any irreversible change) is a loss of information/data.     And in the end you have a regular 8-bit PCM file with the usual quantization noise and resolution limits.

...In the video world they call it "LPCM" (linear PCM).

Off topic, but I'm old enough to remember the analog days and the early digital days.    I was surprised that normal-everyday digitization (CDs and in recording, etc.) was linear.    (Except I believe the telephone system never used linear PCM.)    
Title: Re: Good requantizers?
Post by: kode54 on 2021-06-25 23:13:49
FYI, both u-law and a-law use 8 bit samples to store the data, using something vaguely resembling floating point. The 8 bits contain a sign bit, an exponent (shift count), and a quantized almost mantissa of the data. Both have different ways of dealing with the exponent value. A-law, for instance, has the two lowest exponent values shortcut to adding two specific smaller fractions to the quantized value.

So they're sort of logarithmic precision samples, but not much else, since there's no delta math involved. They still have minor 8 bit quantization noise, but apparently A-law handles this better than u-law.

VGMStream contains decoders for both in its pcm_decoder.c source file, since there are a number of random games which use these formats.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-06-26 10:51:54
OK I made another non-DPCM companding format for fun, let's call it UA-law. It compresses to 8 bits and expands to 16 bits.
Can you give some more details on your algorithm? Which encoded 8-bit sample values map to which decoded 16-bit values?

Chris
Title: Re: Good requantizers?
Post by: bennetng on 2021-06-26 12:11:50
It's a .net program, so the source can be viewed with ILSpy.
For encoding, second column is the input value threshold, can handle -32768. Zeros are unchanged.
For decoding, column 1 is translated to column 3.
Code: [Select]
1       2       1
2       5       3
3       10      7
4       17      13
5       26      21
6       38      32
7       52      45
8       68      60
9       86      77
10      106     96
11      129     117
12      154     141
13      182     168
14      212     197
15      244     228
16      279     261
17      317     298
18      357     337
19      399     378
20      445     422
21      493     469
22      543     518
23      597     570
24      653     625
25      712     682
26      774     743
27      839     806
28      907     873
29      978     942
30      1052    1015
31      1128    1090
32      1209    1168
33      1292    1250
34      1378    1335
35      1468    1423
36      1561    1514
37      1658    1609
38      1758    1708
39      1861    1809
40      1968    1914
41      2078    2023
42      2192    2135
43      2310    2251
44      2432    2371
45      2557    2494
46      2686    2621
47      2819    2752
48      2956    2887
49      3097    3026
50      3242    3169
51      3391    3316
52      3544    3467
53      3701    3622
54      3863    3782
55      4029    3946
56      4200    4114
57      4375    4287
58      4554    4464
59      4738    4646
60      4927    4832
61      5121    5024
62      5319    5220
63      5522    5420
64      5730    5626
65      5944    5837
66      6162    6053
67      6385    6273
68      6614    6499
69      6848    6731
70      7087    6967
71      7332    7209
72      7583    7457
73      7839    7711
74      8101    7970
75      8368    8234
76      8642    8505
77      8921    8781
78      9207    9064
79      9498    9352
80      9796    9647
81      10100   9948
82      10411   10255
83      10728   10569
84      11051   10889
85      11381   11216
86      11718   11549
87      12062   11890
88      12413   12237
89      12771   12592
90      13136   12953
91      13509   13322
92      13888   13698
93      14276   14082
94      14671   14473
95      15073   14872
96      15484   15278
97      15902   15693
98      16328   16115
99      16763   16545
100     17206   16984
101     17657   17431
102     18116   17886
103     18585   18350
104     19062   18823
105     19548   19305
106     20043   19795
107     20547   20295
108     21060   20803
109     21583   21321
110     22116   21849
111     22658   22387
112     23210   22934
113     23771   23490
114     24343   24057
115     24926   24634
116     25518   25222
117     26121   25819
118     26735   26428
119     27360   27047
120     27995   27677
121     28642   28318
122     29300   28971
123     29970   29635
124     30651   30310
125     31345   30998
126     32050   31697
127     32767   32408
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-06-28 15:14:10
I was hoping it looks like that:
16bit PCM --lossy--> 8bit DPCM --lossless--> 8bit PCM

It's obvious that requantizing to lower bit depth is lossy. I thought that conversion from DPCM to PCM at the same bit depth is lossless though.
Hmm, I guess that depends on the 'D', i.e. way the predictor in your particular DPCM format works. ... with proper format constraints in place, such situations could be avoided, and then indeed the decoding might be lossless.
No.    Any change (any irreversible change) is a loss of information/data.     And in the end you have a regular 8-bit PCM file with the usual quantization noise and resolution limits.
Since your 'no' partially contradicts my answer, I sat down this morning and implemented a basic DPCM codec in Octave/Matlab to check. Attached the .m script, where you can change the fourth line to specify a different input/output file name (the encoded file gets a 'wavEnc.wav' extension, the decoded file a 'wavDec.wav' extension). Sorry, since it's C-style scripting, it's really slow, so I also attach demo Wave files.

The decoding part indeed does what rutra80 is looking for: it decodes the B-bit DPCM Wave file to a B-bit PCM Wave file, losslessly. You can verify that by tandem coding the decoded file, by specifying the ...wavDec.wav as input when running the script a second time. That should result in exactly the same encoded and decoded audio as in the first run. I scaled the dither such that, when the input audio is already at the target bit-depth, no noise will be added.

The crucial part is value clipping to B bit at the right places and the absence of companding and other adaptive types of quantization (as done by e.g. bennetng; that would increase the value range and, thereby, the bit-depth). Conceptually, this simple codec is actually very similar to how FLAC, and possibly some other lossless codecs with a "wasted bits" feature, work.

That was fun, reminded me of my first experiments into audio coding roughly 20 years ago :)

Chris
Title: Re: Good requantizers?
Post by: rutra80 on 2021-07-01 11:06:58
Very interesting : ) Your sample has quite compressed dynamics so it favors low bit depths, but I tried it with some more challenging samples and it performs quite well.
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
Your dithering is quiet but quite spread across frequencies, some people find it more pleasant than very loud noise at very high frequencies...

In case of low rate samples when dithering with noise shaping is less useful - are there algorithms which instead of noise introduce some other kind of artifacts? Which requantizer uses simply rounding?
Title: Re: Good requantizers?
Post by: bennetng on 2021-07-02 17:36:38
OK I made another non-DPCM companding format for fun, let's call it UA-law. It compresses to 8 bits and expands to 16 bits.
[attach type=image]20360[/attach]

To encode, drag one or more uncompressed 16-bit .wav files into UA-law.exe, new files will be saved in the same location as the original files, with "-ComP.waV" added at the end of the file names. The encoded files have same format and size as the original, but only the lowest +/-127 sample values are used, therefore volume level is very low, and can achieve high compression ratio with 7z and FLAC.

The program uses file name to switch between encode/decode modes, if the files end with "-ComP.waV" (case sensitive), decode mode will be engaged. To decode, drag the "-ComP.waV" files into UA-law.exe, the decoded files will end with "-ComPand.wav" and can be directly played.

If for whatever reason you can't or unwilling to run the program, I attached some sample files, using the infamous eig.wv (https://hydrogenaud.io/index.php/topic,49601.0.html), compared to u-law and a-law.


New version:
-Lower distortion, check the re-encoded eig.wv.
-Much faster en/decoding speed. About 2 seconds for an uncompressed ~700MB CDDA image on a 6 years old Haswell i3 using RAM disk.
-Source and .exe file included.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-07-03 14:15:32
Very interesting : ) Your sample has quite compressed dynamics so it favors low bit depths, but I tried it with some more challenging samples and it performs quite well.
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
Your dithering is quiet but quite spread across frequencies, some people find it more pleasant than very loud noise at very high frequencies...

In case of low rate samples when dithering with noise shaping is less useful - are there algorithms which instead of noise introduce some other kind of artifacts? Which requantizer uses simply rounding?
There are some audio editors which allow you to choose between simple rounding and dithering with or without noise shaping, e.g., Adobe Audition and, IIRC, Audacity.

To clarify, I did not implement any noise shaping in my DPCM codec above, to keep the code simple. Yes, there are some well performing dithering requantizers with noise shaping out there, e.g. the one in foobar2000 when converting to lower bit-depths. However, these are often so strong that, at low bit-depths, they increase the amplitude of the waveforms quite a bit, which increases the risk of audible clipping. Attached a more gentle noise shaper which you can use to convert to 8 bit or lower depths, along with a demo Wave file which makes comparative listening a bit easier than with my WDPK.wav above (BSBTrio (https://www.amazon.de/Vision-Bsb-Trio/dp/B000TLA6UY/).wav). You can also pass the resulting '...wav8bit.wav' into the DPCM codec I posted above, see also the files in the attached zip archive.

Chris
Title: Re: Good requantizers?
Post by: danadam on 2021-07-06 20:38:29
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
:o How loud are you (or those people) listening?
For me, at the volume that classical music gets quite loud, I need to add another 15 dB to start hearing noise with gesemann shaping. And that's when playing the noise alone, so it's not masked by music.
Title: Re: Good requantizers?
Post by: danadam on 2021-07-06 20:55:27
I thought that conversion from DPCM to PCM at the same bit depth is lossless though.
The decoding part indeed does what rutra80 is looking for: it decodes the B-bit DPCM Wave file to a B-bit PCM Wave file, losslessly.
Out of curiosity, is there any practical advantage of using DPCM instead of LPCM in that case?
Title: Re: Good requantizers?
Post by: kode54 on 2021-07-06 23:45:25
DPCM, depending on the samples, may compress better with general purpose compression tools than linear PCM.
Title: Re: Good requantizers?
Post by: C.R.Helmrich on 2021-07-07 00:23:26
Indeed, and you can actually notice that in the zip file which I uploaded in my reply #22 - the DPCM file is quite a bit smaller than the decoded LPCM file. It's generally beneficial to remove as much redundancy in the data as possible before packing it losslessly. That's how (7)zip, FLAC, and other lossless coders work so well.

Edit: Attached a visualization of the frequency response of the noise shaping filter code I posted above, in case anyone is interested. The blue line is the IIR filter response, the black line a special version of the absolute hearing threshold (with 50% free-field to diffuse-field correction) which the filter was fitted to at 44.1 kHz sampling rate.

Chris
Title: Re: Good requantizers?
Post by: rutra80 on 2021-07-08 10:58:56
Thanks again @C.R.Helmrich
I personally seem to like sox requantizing with noise shaped dither most, for example I tried gesemann noise shaping and I barely can hear any noise at all, but some people find it hissing and already begin to hear the very quiet dithering at 5kHz.
:o How loud are you (or those people) listening?
For me, at the volume that classical music gets quite loud, I need to add another 15 dB to start hearing noise with gesemann shaping. And that's when playing the noise alone, so it's not masked by music.
At 8 bits?
One of them is a musician and he was listening on studio monitors.
The rest might had issues like crappy resampling or even another layer of requantizing...
It was tested on a quietly starting lowpassed piece getting full loud spectrum with time.
Title: Re: Good requantizers?
Post by: danadam on 2021-07-08 17:20:33
For me, at the volume that classical music gets quite loud, I need to add another 15 dB to start hearing noise with gesemann shaping. And that's when playing the noise alone, so it's not masked by music.
At 8 bits?
Ah, ok. I should have catch that. Sorry.
Title: Re: Good requantizers?
Post by: bennetng on 2021-07-11 22:09:39
OK I made another non-DPCM companding format for fun, let's call it UA-law. It compresses to 8 bits and expands to 16 bits.
[attach type=image]20360[/attach]

To encode, drag one or more uncompressed 16-bit .wav files into UA-law.exe, new files will be saved in the same location as the original files, with "-ComP.waV" added at the end of the file names. The encoded files have same format and size as the original, but only the lowest +/-127 sample values are used, therefore volume level is very low, and can achieve high compression ratio with 7z and FLAC.

The program uses file name to switch between encode/decode modes, if the files end with "-ComP.waV" (case sensitive), decode mode will be engaged. To decode, drag the "-ComP.waV" files into UA-law.exe, the decoded files will end with "-ComPand.wav" and can be directly played.

If for whatever reason you can't or unwilling to run the program, I attached some sample files, using the infamous eig.wv (https://hydrogenaud.io/index.php/topic,49601.0.html), compared to u-law and a-law.


New version:
-Value -128 and -32768 are used in encoding and decoding.
-Added clipping warning caused by the encoding process.
-Added noise shaping with individually tuned coefficients for 44.1k, 48k and >=88.2k.
-Dither is not used due to the nonlinear quantization nature of UA-law.
-Unlike LPCM, the noise added by noise shaping is proportional to signal level instead of being constant.
-Sound quality no longer similar to A/u-law in any possible way. Much better.  eig.wv is fully transparent to me, and therefore I attached a new audio demo. Pay attention to the first few seconds, they are not silence.
Title: Re: Good requantizers?
Post by: bennetng on 2021-07-20 19:00:08
New version:
-Further improved dynamic range (noise floor) and noise shaping.
-The encoder will automatically retry at a lower level to avoid clipping like this (yes it supports multichannel files):
Code: [Select]
-------------------------------------------------------------------------------
H:\Antonio Vivaldi - The Four Seasons- Summer.wav
00:10:33.8666666 = 55907040 samples / 2-ch @ 44100Hz
Encoded at 253.3487 x 2.5019532 secs
-------------------------------------------------------------------------------
H:\Bedrich Smetana - Mein Vaterland- Die Moldau.wav
00:12:04.7600000 = 63923832 samples / 2-ch @ 44100Hz
Encoded at 255.3869 x 2.8378906 secs
-------------------------------------------------------------------------------
H:\Evangelion_3.33.0_You_Can_(Not)_Redo.wav
WAVE_FORMAT_EXTENSIBLE
00:01:47.9160000 = 31079808 samples / 6-ch @ 48000Hz
Encoded at 74.81786 x 1.4423829 secs
-------------------------------------------------------------------------------
H:\Konami Kukeiha Club - UNIVERSE -STAGE 1-.wav
00:02:55.9066666 = 15514968 samples / 2-ch @ 44100Hz
Clipped 0.2162019 dB. Retry...
Encoded at 104.5435 x 1.6826172 secs
-------------------------------------------------------------------------------
H:\Luis Fonsi - Despacito (feat. Daddy Yankee).wav
00:03:49.2400000 = 20218968 samples / 2-ch @ 44100Hz
Clipped 0.3593756 dB. Retry...
Encoded at 104.8891 x 2.1855468 secs
-------------------------------------------------------------------------------
H:\M - A Recordings - Shadow Bright by Bruce Stark M0011A.wav
00:03:00.0400000 = 15879528 samples / 2-ch @ 44100Hz
Encoded at 254.9944 x 0.7060547 secs
-------------------------------------------------------------------------------
H:\Phil Collins - Another Day in Paradise.wav
00:05:22.9600000 = 28485072 samples / 2-ch @ 44100Hz
Clipped 0.2162019 dB. Retry...
Encoded at 106.2014 x 3.0410156 secs
-------------------------------------------------------------------------------
It is recommended to use RAM drive for benchmarking, especially when decoding, and the input files should be long enough so that the timer has enough ticks (it runs at 10000000 ticks/sec) to get meaningful stats instead than showing something like "Decoded at +∞ x 0 secs". Also, when using something like foobar2000 to convert files, it launches multiple encoder instances (FLAC, mp3, Wavpack...) in multiple threads. The UA-law demo, being a demo, needs to show info in a console and does not support foobar2000's encoder pipeline at this moment.

Check the attached demo compared to IMA/MS ADPCM. The "-ComP.waV" file is smaller than them when encoded with FLAC, and the decoded file has lower distortion and noise than the ADPCM files.

It is possible to convert existing CDDA files to 16-bit but with higher sample rates, UA-law's noise shaper will further reduce the noise in audible frequencies, and the encoded files will still be smaller than the 44k ones with FLAC.

X

DSD files must and DXD files may contain lot of ultrasonic noise and will harm UA-law's performance, these files should be lowpassed or resampled to a lower rate (e.g. 96k) before encoding. After RG and/or resampling, simple 1-bit TPDF dither like Case's Smart Dither (https://www.foobar2000.org/components/view/foo_dsp_dither) (make sure "high-pass" is checked because it is faster) should be used instead of heavily shaped dither like the one from foobar2000, and should give enough headroom to avoid intersample over or clipping the UA-law encoder so that there is no need to retry.
Title: Re: Good requantizers?
Post by: bennetng on 2021-08-07 18:27:37
Version 1.0.0.5

-Wrote my own filter class for better speed.

-24-bit support
24-bit files will be encoded to 12-bit and decoded to 23-bit padded to 24-bit. Users can manually convert 16-bit files to 24-bit to use the 12-bit version of UA-law. As mentioned previously, 24-bit input to UA-law should also be dithered but not noise shaped if they involve resampling and/or RG.

-flac integration
Put flac.exe in the same location as UA-law.exe.
Files end with ".wav" and ".flac" will be encoded to "-ComP.flaC".
Files end with "-ComPand.wav" are ignored.
Files end with "-ComP.waV" and "-ComP.flaC" will be decoded to "-ComPand.wav".
All of the above are case sensitive. I set the flac compression setting to the typical -8 without other special treatments, so use -8 as baseline when comparing file sizes with unencoded files, with the same version/build of the flac.exe you prefer to use.

When converting multiple flac files, UA-law may launch several flac processes to improve overall speed, benchmark results will not be consistent as they are affected by the background processes and order of input files.

-WAVE_FORMAT_EXTENSIBLE
UA-law shows this when the input files contain this tag, and preserves it during processing. RMAA for example cannot open these kinds of files. flac.exe may add this tag when the input files are 24-bit, foobar2000 only add this tag when the files contain more than two channels. There are guidelines from Microsoft specifying how this tag should be used, but different software behave differently.

The whole "package" should be idiot proof: just drag and drop files and/or folders (subfolders are ignored) into UA-law.exe, no command-line knowledge required. Copy the original audio files to a separate folder or RAM drive to evaluate UA-law as the temp files may clutter up your original folder, and refresh Windows Explorer (press F5) to update file name and size before and after conversion when necessary.

All used algorithms have nothing to do with typical lossy formats, just exponentially spaced amplitude values with noise shaping, the same way I don't think floating point audio formats and DSD are lossy by themselves.
Title: Re: Good requantizers?
Post by: bennetng on 2021-08-10 06:58:32
Version 1.0.0.6

-Improved handling of background flac processes to save energy without sacrificing speed.
-Reduced temp file disk space requirement in batch conversion.
Title: Re: Good requantizers?
Post by: bennetng on 2021-11-20 17:58:53
version 1.0.0.7

-Added a readme file.
-Fixed idle tone issues with some types of input signals.

This should be the final version, as an exercise of self-learning and proof of concept.