Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC v1.4.x Performance Tests (Read 58596 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: FLAC v1.4.x Performance Tests

Reply #400 –
Quote from: Porcus on
Quote from: bennetng on
Quote from: Porcus on
Sure it is undithered?
The files were generated using Audition, and if you trust there is no bug, then it is undithered.
So, sample value created by way of "multiplying sample number by 2pi to certain precision" and getting different round-off every now and then?
Audition 1.5 was a 2004 product which is no longer supported, and I am not good enough to reverse engineer their code.

Within freewares I know, apart from foobar, Audacity and SoX also support tone generation, so you can try to generate some files and compare them.

Re: FLAC v1.4.x Performance Tests

Reply #401 –
sox -D -n -r 48000 -c 1 -b 16 4567sox.wav synth 10 sine 4567

-D for no dithering. Differs a little from yours, so there is a "not exact scine'nce" pun waking here.
At least each one-second chunk is identical for this one (edit: not sure if that is a good thing for testing ...), but still it xz's to bigger than yours.



Re: FLAC v1.4.x Performance Tests

Reply #402 –
Quote from: bennetng on
So I tried it. The total duration of the two attached playlists is 41h 39m 06s. The playlists contain single tracks and images, so not every single track name is shown.

-8
38.1 GB (41002782313 bytes)

-8b8192 -A "subdivide_tukey(3);blackman"
37.7 GB (40507157807 bytes)

5 files out of 306 still clipped with -2dB gain. Yes, if one keeps throwing Merzbow to the chain that requires like -10dB it would harm the stats, so don't do that. Clipping should be controlled to as low as possible, or not at all (e.g. by using RG max non-clip gain).

For simplicity and speed, I did all the conversions within foobar and foo_dsp_resampler.
[attach type=image]27188[/attach]
[attach type=image]27190[/attach]
Same corpus with same settings transcoded from pre-upsampled flac files (16/44 -> 24/192 with -2dB preamp), using foobar multi-file multi-thread, flac 1.4.3. Decoding times are single-threaded using foobar x64 benchmark, all tested on NVMe SSD, i3-12100.

-8p
Total encoding time: 1:49:49.297, 22.75x realtime
40794570053 bytes

-8b8192 -A "subdivide_tukey(3);gauss(22e-2)"
Total encoding time: 19:30.984, 128.05x realtime
40502851907 bytes

-8b8192 -A "subdivide_tukey(3);blackman;gauss(22e-2)"
Total encoding time: 21:16.360, 117.47x realtime
40419431443 bytes

-8b9216 -A "subdivide_tukey(3);blackman;gauss(22e-2)"
Total encoding time: 21:18.485, 117.28x realtime
40406946503 bytes

-8b16384 -A "subdivide_tukey(3);blackman;gauss(22e-2)"
Total encoding time: 21:33.359, 115.93x realtime
40404848546 bytes
Decoding: 240.030x realtime

-8e
Total encoding time: 1:07:58.406, 36.76x realtime
40330004928 bytes
Decoding: 241.866x realtime

-8b8192 -A "subdivide_tukey(4);blackman;gauss(22e-2)"
Total encoding time: 30:46.781, 81.19x realtime
40053310660 bytes
Decoding: 240.803x realtime

From the data above I think one can deduce how slow -8pe on this CPU would be, so I am not going to test it.

Quote from: bennetng on
gausswork
Gauss, like other windows, works best when coupled with an optimal blocksize. Also, the gauss parameter needed to be calculated for best performance. For example, the target 192kHz sample rate has a Nyquist of 96kHz, if the resampler is lowpassed at about 23kHz like the red plot below, then the value should be 23/96, so it would be gauss(24e-2), the green and yellow plots would be 23e-2 and 25e-2.
X

Blackman on the other hand describes a faster than expected overall decay trend at upper spectrum without doing anything very specific, so can catch more different filter shapes.

If the hardware does not support higher blocksizes, slightly increasing -l and -r would help a lot, like -l14 to -l16 and -r7. The increased decoding complexity should not be a big deal especially for mains-powered devices.

-b16384 can still cause inflation at 192kHz depending on materials, use with caution.

For those who did not follow the previous discussions, I am talking about optimizations for >=4x upsampled data, so the parameters listed above are not suitable for encoding "real" hi-res files.

Re: FLAC v1.4.x Performance Tests

Reply #403 –
8192 beats 16384 size-weighted in my tests on upsampling those 38 CDs, but it varies quite a lot. Medians could very well tell a different story. I messed up something and it is still running, but I can report on those two block sizes at least.

Using -A "subdivide_tukey(3);blackman" on everything, then
** In overall size:
At 96 kHz, -b8192 beats -b16384, and -eb8192 beats -eb16384.
At 192 kHz, same happens.
At 384 kHz, -b8192 beats -b16384 by around 0.12 percent, but -eb16384 beats -eb8192 by around 0.18 percent.

192 kHz, let's look into that further: No -e here.
* Classical music benefits from larger blocksize -b16384, 12 albums to 2; all except harpsichord and (near-zero) Cage's percussion works. Total impact 0.32 percent (not percentage points!), varying from -0.15 (harpsichord) to 0.63 percent (Bruckner, vocals)
Median impact = 0.37 = median absolute value impact.
But then the rest:
* The heavier music: -b8192 wins by 7 albums against 3, switching sign on impact to signify that:
Total impact -0.14 percent, varying from -0.71 (Laibach, biggest benefit for -b8192) to 0.24 percent (Gojira, that benefits from 16384).
Median impact = -0.24. Remove the sign for median absolute value impact.
* The others. -b8192 wins by 9 albums against 5
Total impact -0.28 percent, max benefit from -b8192 is -1.31 percent (Wovenhand, in this release that is singer/songwriter) and then -0.99 (Sopor Aeternus, that is something completely different: darkwave) - and on the other end, benefiting most from larger blocksizes are the jazz albums: 0.41 percent for both Davis and Johansson. Those were near-mono before dithering I think.
Median impact = -0.32 percent. Median absolute impact: 0.38.


Quote from: bennetng on
For those who did not follow the previous discussions, I am talking about optimizations for >=4x upsampled data, so the parameters listed above are not suitable for encoding "real" hi-res files.

... but who knows how many hi-res files are "real".

Re: FLAC v1.4.x Performance Tests

Reply #404 –
With unknown hi-res content I usually stuck with the welch;hann;flattop combo, different shapes and easy to memorize window names.

Here is an outlier that still likes -b4096 at 192k. subdivide_tukey(6/1) was used. At 44k -b1152 seems optimal.
X

Re: FLAC v1.4.x Performance Tests

Reply #405 –
After some investigations the ones that still like -b4096 at 192kHz include ethnic plucking instruments (e.g. Shamisen, Pipa) and high pitched percussive sounds in either acoustic or synthesized manner.

Re: FLAC v1.4.x Performance Tests

Reply #406 –
Quote from: bennetng on
With unknown hi-res content I usually stuck with the welch;hann;flattop combo, different shapes and easy to memorize window names.
As hann is tukey(1), which is "the extreme end" (rectangle being the other), then I would be mildly surprised if it is any good compared to the tried and tested midways. 
Only "mildly" because who knows what works until it is tried ...

Quote from: bennetng on
the ones that still like -b4096 at 192kHz
If you take those and compare
-b 4096 -r <R>
to
-b 8192 -r <R+1>
what do you get? Upping the partition order by 1 would double the partition number and maintain the partition size in samples - it would be more like the partial effect of -b alone. (The one you posted in #404 does not benefit from doubling block size and increasing the partitioning though.)

Re: FLAC v1.4.x Performance Tests

Reply #407 –
Quote from: Porcus on
As hann is tukey(1), which is "the extreme end" (rectangle being the other), then I would be mildly surprised if it is any good compared to the tried and tested midways. 
Only "mildly" because who knows what works until it is tried ...
Both hann and blackman are of the cosine sum family but blackman is one step narrower than hann, so it would catch a decay trend which is not as severe as blackman. flattop by itself doesn't make too much sense due to the shape, so I don't use it alone, I use it with other windows so that the unique shape can catch something other windows can't. I only use this combo with >= 88.2k content, with subdivide tukey no more than 5.

Quote from: Porcus on
If you take those and compare
-b 4096 -r <R>
to
-b 8192 -r <R+1>
what do you get? Upping the partition order by 1 would double the partition number and maintain the partition size in samples - it would be more like the partial effect of -b alone. (The one you posted in #404 does not benefit from doubling block size and increasing the partitioning though.)
When checking this I found something interesting. In my previous post I was comparing -8p with -8b8192 -A "subdivide_tukey(3);blackman;gauss(22e-2)" and assumed if -8p produced a smaller file then that file must be better with -b4096. There are 11 files, 51 minutes fall into this category. However if I only compare -8 with -8b8192 alone, there is only one file (Yu Miyake - WANDA WANDA) works better with -b4096, and only one more file (Cool Zone - The Blessed Place) works better with -b4096 when using -8 -A "subdivide_tukey(3);blackman;gauss(22e-2)", and the total duration of files work better with -b4096 is reduced to 8m 21s, out of  1d 17h 39m 06s. That said, it doesn't mean these 11 files are better with -8p than -8e.

-8e
969930824 bytes

-8p
996927661 bytes

Re: FLAC v1.4.x Performance Tests

Reply #408 –
So I tried to transcode some mp3 files I collected to flac, turns out I found nothing special. Total 1158 files, 2d 16h 47m 31s, bitrates from 124kbps VBR to 320kbps CBR. All RG scanned and converted with foobar's "prevent clipping according peak". Used Case Smart Dither plugin with highpass filter unchecked. All files are transcoded to 16/44.

-8
22712634863 dithered
22547632564 truncated

-8e
22706853763 dithered
22540297898 truncated

-8p
22693665912 dithered
22527026089 truncated

Among the dithered files, 54 files, 2h 33m 19s are smaller with -8e.
Among the truncated files, 78 files, 4h 11m 36s are smaller with -8e.
Among the dithered files, 49 of them are same as the truncated ones.
Among the remaining 5 dithered files, they don't share any similarity that I can think of.

Among all files showed smaller file sizes with -8e, this piece of music has the biggest file size differences when compared to -8p. Source mp3 file is in 128kbps CBR.

3621067 -8p dithered
3555347 -8e dithered
3172894 -8p truncated
3150222 -8e truncated

4 - 悲情城市 Variation 1
https://www.discogs.com/release/3913204-SENS-%E3%82%BB%E3%83%B3%E3%82%B9-%E6%82%B2%E6%83%85%E5%9F%8E%E5%B8%82-A-City-Of-Sadness
https://youtu.be/O6ciikFXUQw

Regardless of how the Youtube Opus version was encoded, the mp3 file I have looks like this:
X

Re: FLAC v1.4.x Performance Tests

Reply #409 –
Quote from: bennetng on
Quote from: Porcus on
As hann is tukey(1), which is "the extreme end" (rectangle being the other), then I would be mildly surprised if it is any good compared to the tried and tested midways. 
Only "mildly" because who knows what works until it is tried ...
Both hann and blackman are of the cosine sum family but blackman is one step narrower than hann, so it would catch a decay trend which is not as severe as blackman. flattop by itself doesn't make too much sense due to the shape, so I don't use it alone, I use it with other windows so that the unique shape can catch something other windows can't. I only use this combo with >= 88.2k content, with subdivide tukey no more than 5.
When filter length remains the same, the same window can work quite differently in different sample rates, especially with windows that don't take user-specified parameters. Changing -b can somehow change the effective filter length. While Tukey can be set to match Rectangular and Hann, Tukey should be used to do things that only Tukey can do.
X
X
X

The -1dB@25kHz DSD filter I mentioned in another thread is also apodized with a cosine sum family window using a different configuration.
X

Now, invert the graph and compared it to some DSD modulators:
https://pcmdsd.com/Software/PCM-DSD_Converter.html
X

Re: FLAC v1.4.x Performance Tests

Reply #410 –
Quote from: bennetng on
I collected some public domain 44.1/48k samples, made some SoundFonts and arranged in a DAW. One octave has 12 semitones, but it was/is pretty hard to find high quality public domain sample sets (especially, ethnic instruments) that sampled every semitone. For example, it may just has the white keys (7 per octave) or even fewer like 2-3 per octave.

I don't know if this is any interesting test to be honest - but there are a lot of twelve-tone works composed over the last century.
Surprised though, that didn't find more freely downloadable on the 'net. Yes there are quite a few on the Internet Archive, but most seem to be vinyl rips in mp3.

Re: FLAC v1.4.x Performance Tests

Reply #411 –
Codecs (at least not the "AI" ones) don't care about high level stuff that involve music theory. Codecs care more about production procedures like miking, post production etc. For example, a performance of a drum set recorded at the back of the hall can be quiet different from the same performance recorded at a close distance. The former would have most of the transients smoothed out and more reverberant which is more compatible with -b4096/4608 at 44/48k. The latter could be something like -b2304. But then if one uses heavy hand dynamic range compression/limiting on a closely recorded drum set then the transients would be once again disrupted again.

Classical music recordings generally don't use close miking, there could be a dedicated mic for the soloist of a Concerto but never one mic for each performer in a strings/brass/wind section. Even for the soloist (e.g. violin) the treatment can still be quite different from a solo violin used in a pop music arrangement.

Re: FLAC v1.4.x Performance Tests

Reply #412 –
A surprising result: a case of
-e not making a single bit difference on an entire thirty-eight minutes album. Bit-identical .flac files created, not only same size.

But it is after sox resampling it to 64000 Hz, 24 bits (with dither) - it does not occur in the unaltered CD image file.  So something is going on in that resampling - but the encoder guessing spot-on for an entire album?  In fact, boosting block size I got it to happen to more albums too.

So what I was about to do - and redo, due to human error - was to upsample material to check how block sizes and -e react.  (Cf. this suggestion from @bennetng - background below, to those who aren't reading this thread regularly.)
So in addition to trying a high sampling rate like 192k, I tried a small factor up, to see if merely half an octave empty spectrum on top would be enough.
Much to my surprise, the benefit of -e sometimes collapsed all the way to zero instead of increasing.

For one of the albums, the result was quite robust. This album: https://psycroptic.bandcamp.com/album/psycroptic-2  Like, put on track 4, it isn't like all the (sub)frames would be equal.
But it happens for more albums too. Including Miles Davis' Birth of the Cool (in the 'Complete Birth of the Cool' edition) - which doesn't sound at all like this to a human ear (encoders may beg to differ). But for Miles to match -e, I had to try -b32768, which reduces it to "only" 18604 chances to guess less-than-perfect at least once throughout the > 79 minutes.

So, too curious for my own good, I tried to investigate more about "when" the model selection algorithm would hit spot-on the same as a -e would. Psycroptic at 64 kHz again: 
-8 -l<something> with or without -e, tested -l 6 to -l 20:
-l 6: -e makes it differs in some eight subframes. 21 bytes.
-l 7 to -l 13: -e makes zero difference. -l 14, the difference is 84 bytes
-l 18: now the impact of -e has reached 0.1 percent
-l 20: here is where the estimation fails so much that it is bigger than -l 19. Indeed, bigger than -l 17.  Two percent bigger than with -e.

Next (mild) surprise coming up: Give it a few extra apodization functions to choose betweeen. Like, more opportunities --> even a good guessing algorithm would more often fail to hit spot-on, eh?
No!  With -A "subdivide_tukey(5);blackman", it is closer to -e, if anything:
Still the range where it is spot on to -e is -l 7 to -l 13.  But the neighbouring -l6 and -l14 are closer (10 and 4 bytes size difference), so is -l18
Still -l 20 is the point where the guesstimation algorithm fails to make it smaller than one order lower, but at least it beats -l 17.

Going the other way, down to -7: confirming it.

With higher sampling rate, we surely get a difference - like, for this particular album upsampled to 192 kHz, then
-8p improves a percent (!) over -8, but
-8e improves more than four percent over -8. Percent, not points.
(Bitrates are like 2824, compared to 2054 for 1/3 of the sampling rate - uncompressed size ratio is 3, for a perspective.)



Background info:
It is "known" that the -e switch can make a difference when the top frequency range is empty - and that is why it often happens on high resolution files, as they are sometimes sourced from normal resolution - and I was curious how much "empty spectrum" it would take for -e to kick in.
Those of you who are not familiar with what -e does: the reference encoder can choose between a number of ways to calculate the predictor - and would want to choose the one that leads to smaller file. Without -e, it makes an estimate of size per alternative, chooses the one that comes out best, and only that is compressed fully; with -e, it brute-forces through them all.  On CDDA material, the (gu)es(s)timate is so good that -e is very rarely worth it - if you have the patience to wait for -e, you could instead spend that time on more efficient ways to squeeze out bits, unless the material is quite peculiar.  On high resolution signals, there are more such "peculiar" material where the (gu)es(s)timation isn't that good.  Identifying such cases could improve upon the algorithm ... if one bothers.

Re: FLAC v1.4.x Performance Tests

Reply #413 –
In my previous test (Reply #408) I also found that MP3 files which occupied about 70% of the spectrum (15-16kHz cutoff) don't have clear advantage when using -e, though zero improvement is quite amusing.

Re: FLAC v1.4.x Performance Tests

Reply #414 –
Quote from: Porcus on
8192 beats 16384 size-weighted in my tests on upsampling those 38 CDs, but it varies quite a lot.
[...]
192 kHz, let's look into that further: No -e here.
* Classical music benefits from larger blocksize -b16384, 12 albums to 2

But, the classical corpus benefits from higher block sizes even when compressing the CDDA (no upsampling no DSPing, just --lax).
-7b8192 --lax beats -8 by 10 albums to 4, and also in total size. And the difference is about twice the size as between -7 and -8
Indeed, -7b8192 wins by "same 10 albums to 4" aganinst -8b4608 and against plain -7.

Even -7b4608 beats -8, though narrowly - sure everything here is narrow, difference btween -7b8192 (best among these) and plain -7 (worst) is only 0.07 percent. It is also much less than how on upsampled to 192 kHz, blocksize 16384 edged out 0.32 over 8192.

But the point is, this is material where bigger blocks than default already improves with CDDA - only we don't test that as much, it being out of subset. No surprise that this is precisely where bigger blocks start improving first.


(Oh, and: Even if -8b16384 did beat plain -7, you can forget about that block size for CDDA.)

Re: FLAC v1.4.x Performance Tests

Reply #415 –
Quote from: Porcus on
A surprising result: a case of
-e not making a single bit difference on an entire thirty-eight minutes album. Bit-identical .flac files created, not only same size.

But it is after sox resampling it to 64000 Hz, 24 bits (with dither) - it does not occur in the unaltered CD image file.  So something is going on in that resampling - but the encoder guessing spot-on for an entire album?

Oh this is weird. Impact of -e atop -8:

2.8 megabytes: savings by -8e instead of -8 on the CDDA corpus. That is .012 percentage points of the 23 GB WAVE uncompressed size. Not much!
1.7 megabytes: After having resampled to 44.1/24, this is what -e saves. FLAC size does however increase by 80 percent, so compression ratio worsens from 51 percent to 64 percent. Not really strange.
1.25 megabytes: Resampling instead to 64/24, this is what -e saves. Going 64 instead of 44.1 increases uncompressed size by 45 percent, but FLAC size increases by only 22 percent over 44.1/24: that is slightly better than the overall compression ratio of 53 percent on the 64/24.
Going 64/24 makes the encoder compress better in percentage terms, and the impact of -e much smaller than for CDDA: only .0025 percentage points of WAVE.

Now, pay attention to the magnitude, there is no decimal point here. Resampling to 96/24, the impact of -e is instead:
187 megabytes. (Lower when I add the blackman though, but > 160 anyway.)

OK, it is known that the FLAC encoder starts picking systematically wrong models when the signal looks like this. And the 187 grows overproportionally from then: Doubling sampling rate to 192 increases the -e impact to 691, blah blah blah.
But 64 is pretty much midway between 44.1 and 96. (Geometric mean is 65.066 kHz.)
And going to 64/24 instead of 44.1/24 makes it do better. Somewhere in that extra half octave from 64 to 96, something weird happens ...

(... is it the resampling algorithm? Should I try 87 and 89?)

Re: FLAC v1.4.x Performance Tests

Reply #416 –
Don't compare 16/44 -> 24/44 and 16/44 -> 24/64 because the former does not involve any resampling. Try something like 16/44 -> 24/48 vs 16/44 -> 24/64, and apply the same amount of negative gain for clipping prevention.

44.1k -> 48k has a small enough ratio, yet still retains the characteristics of clean resampling like steep lowpass and intersample peaks. Try omit the blackman as well, as subdivide_tukey should be much more effective for such a small resampling ratio.

Re: FLAC v1.4.x Performance Tests

Reply #417 –
I think sox resamples if you specifically ask it to do so in a DSP chain with volume change and everything?

Quote from: bennetng on
Try omit the blackman as well
There is no blackman in that post except "(Lower when I add the blackman though, but > 160 anyway.)"

Re: FLAC v1.4.x Performance Tests

Reply #418 –
Quote from: Porcus on
I think sox resamples if you specifically ask it to do so in a DSP chain with volume change and everything?
No. You can use SoX -V followed by your commands, SoX will list the used effects. For example, SoX may add dither automatically depending on the input commands, with -V you can confirm if it is the case or not.

Resample (by using rate) means changing the sample rate without changing playback speed and pitch. So if the sample rate remains unchanged, there is no resampling.

Re: FLAC v1.4.x Performance Tests

Reply #419 –
sox -V in.wav out-rate-48000.wav gain -4 rate 48000
Code: [Select]
sox INFO sox: effects chain: input        44100Hz  2 channels
sox INFO sox: effects chain: gain         44100Hz  2 channels
sox INFO sox: effects chain: rate       48000Hz  2 channels
sox INFO sox: effects chain: dither     48000Hz  2 channels
sox INFO sox: effects chain: output     48000Hz  2 channels
sox -V in.wav out-rate-44100.wav gain -4 rate 44100
Code: [Select]
sox INFO rate: has no effect in this configuration
sox INFO sox: effects chain: input        44100Hz  2 channels
sox INFO sox: effects chain: gain         44100Hz  2 channels
sox INFO sox: effects chain: dither       44100Hz  2 channels
sox INFO sox: effects chain: output       44100Hz  2 channels
You can see that when there is no resampling, there is no lowpass and intersample peaks as well.
X

Re: FLAC v1.4.x Performance Tests

Reply #420 –
Thanks for the correction. I ran it again on 48 kHz/24 resampling; just to pick a number between 64 and 96, I also resampled to 79 kHz/24 resampling. And 192/24 figures are in.
Megabytes saved going -8 to -8e, "new" numbers boldfaced:

0.326 for 48 kHz. Ridiculously low! So the resampling does something?
1.25 for 64 kHz. Low too. For comparison, -8p saves 10.3 megabytes (still only 0.04 percent!) over -8.
11.9 for 79 kHz.
187 for 96 kHz. That is around 0.6 percent (not percentage points), and overtakes big time the 53 megabytes impact of -p.
691 for 192 kHz. That is nearly 2 percent. For comparison, -8p saves half a percent.


Even if 48 kHz made for very low -e impact, I did not find any more bit-identical files. I did not scan them all at higher block sizes though, which was necessary to get equal files for the 64 kHz Sodom, Springsteen and Davis files.

So I went to the one that revealed the curious equality and even for -8, which produced bit-exactly the same file as -8e. The Psycroptic. Tested more sample rates.  -8 gave bit-identical files to -8e for
61 kHz, 64 and 67 kHz. And the former even equal when I lowered the block size to 2048, giving it twice as many chances to guess one wrong.
By doubling blocksize, thereby reducing the chances of guessing wrong, I also got -8 to agree with -8e on the 55 kHz and 58 kHz files. I managed the same for 70 and 73 and 76, but only when maxing out to block size -b65535.

So guessing spot-on what -e would find is still a bit peculiar - even if I at 64 kHz managed to get it for 4 of 38 albums ... ?


Re: FLAC v1.4.x Performance Tests

Reply #421 –
Consider the most used sample rates, the smallest upsampling factor next to 44.1 -> 48 is 48 -> 88.2 (1.8375x) and -e is going to work. 44.1 -> 79 is only about 1.79x. Technically, even 44100 -> 44101 is a valid upsampling, but it is not something people would do for any practical purpose.

Which also matched my findings on 44.1k MP3 transcoding as the overall cutoff is rarely lower than 15kHz, and I specifically capped the bitrate to not lower than 128kbps CBR and 124kbps VBR. I did not encode these MP3 files myself, I collected them, so they are likely encoded by many different encoders.

Re: FLAC v1.4.x Performance Tests

Reply #422 –
Quote from: bennetng on
So I tried it. The total duration of the two attached playlists is 41h 39m 06s. The playlists contain single tracks and images, so not every single track name is shown.
16/48 -> 24/88.2 using these settings, without dither:
X
X

-8p
Initial conversion include transcoding from APE, flac, WavPack, upsample and -2dB gain therefore not timed, but previous tests showed -8p spent 1.6x time as -8e.
32320498436 bytes

Below are transcoded from pre-upsampled flac files:

-8
Total encoding time: 9:23.046, 244.67x realtime
32347544572 bytes

-8e
Total encoding time: 33:32.563, 68.45x realtime
32324646092 bytes

-8b8192
Total encoding time: 9:11.843, 249.64x realtime
32303228485 bytes

-8b8192 -A "subdivide_tukey(3);welch;hann;flattop"[*1]
Total encoding time: 11:57.859, 191.90x realtime
32240773767 bytes

88 files, 9971799931 bytes are smaller with -8e[*2], same files in -8p used 9976078278 bytes.
218 files, 22344420158 bytes are smaller with -8p[*3], same files in -8e used 22352846161 bytes.

45 files, 2264363917 bytes in [*1] are bigger than the smallest combinations of [*2] and [*3].

The idea is when resampling ratio is approaching 2x, other windows are going to work as well, so the importance of -e is more about the ability to use a lower blocksize for hardware compatibility. Also, -e requires a clean and low ultrasonic noise floor, so "analog upsampling" like playback of 44.1k content at DAC output and recapture by an ADC using 96k is not going to work, for example, download the 24/96 AMPT files below:
https://archimago.blogspot.com/search?q=ampt

Re: FLAC v1.4.x Performance Tests

Reply #423 –
@ktf , you may know whether the following is expected, or maybe that it might be a symptom of something - you see it didn't happen in 1.3.4 on the sine sizes listed below. But, official flac-1.4.3\Win32 and flac-1.4.3\Win64 builds produce different files (yeah I know the FAQ has long tried to explain users that it is normal at least for builds compiled on different architectures and run on different architectures, but still: it appears new with 1.4.)

Sines - like in the numbers below - are by themselves no big deal yes we know, but the question is if this gives a trace leading to something overlooked or otherwise possible to improve easily.
And since it might be related to model selection algorithm, maybe it could explain that the retune you tested and proposed here did not work as well when HA users tested it on Windows?

Because 1.3.4 and 1.4.x uses different apodization functions for -8, I included runs with -8 -A "tukey(5e-1)" , with and without -e and -p too.  You see that it doesn't fare very well on 1.4.3.  But at the bottom of the table, it looks like 1.3.4 does select the tukey (rather than partial/punchout) when brute-forced, while 1.4 does not.
Also it seems that 1.4 benefits more from -e; is that likely to be the double precision?  And if so: should one try double precision also elsewhere?

Code: [Select]
960044	4567Hz-undither.wav
824535	(for -0 and -0e, no matter what version)
374679	4567Hz-undither-5_143-Win64.flac
374539	4567Hz-undither-8_but_tukeyonly_143-Win64.flac
373646	4567Hz-undither-5_143-Win32.flac
373506	4567Hz-undither-8_but_tukeyonly_143-Win32.flac

326994	4567Hz-undither-5_134-Win64.flac
BIT-ID	4567Hz-undither-5_134-Win32.flac

275581	4567Hz-undither-5e_134-Win32.flac
BIT-ID	4567Hz-undither-5e_134-Win64.flac

273276	4567Hz-undither-8_143-Win64.flac
BIT-ID	4567Hz-undither-8_143-Win32.flac

262519	4567Hz-undither-5e_143-Win64.flac
261820	4567Hz-undither-5e_143-Win32.flac

261015	4567Hz-undither-8_134-Win32.flac
BIT-ID	4567Hz-undither-8_134-Win64.flac
BIT-ID	4567Hz-undither-8_but_tukeyonly_134-Win32.flac
BIT-ID	4567Hz-undither-8_but_tukeyonly_134-Win64.flac

254241	4567Hz-undither-8e_but_tukeyonly_143-Win64.flac
253416	4567Hz-undither-8e_but_tukeyonly_143-Win32.flac
220693	4567Hz-undither-8e_but_tukeyonly_134-Win32.flac
BIT-ID	4567Hz-undither-8e_but_tukeyonly_134-Win64.flac
BIT-ID	4567Hz-undither-8e_134-Win32.flac
BIT-ID	4567Hz-undither-8e_134-Win64.flac

179285	4567Hz-undither-8p_143-Win32.flac
BIT-ID	4567Hz-undither-8p_143-Win64.flac

174746	4567Hz-undither-8p_134-Win32.flac
BIT-ID	4567Hz-undither-8p_134-Win64.flac

170643	4567Hz-undither-8pe_but_tukeyonly_143-Win64.flac
170118	4567Hz-undither-8pe_but_tukeyonly_143-Win32.flac

150973	4567Hz-undither-8pe_but_tukeyonly_134-Win64.flac
BIT-ID	4567Hz-undither-8pe_but_tukeyonly_134-Win32.flac
BIT-ID	4567Hz-undither-8pe_134-Win64.flac
BIT-ID	4567Hz-undither-8pe_134-Win32.flac

147183	4567Hz-undither-8e_143-Win64.flac
145394	4567Hz-undither-8e_143-Win32.flac
124466	4567Hz-undither-8pe_143-Win64.flac
124455	4567Hz-undither-8pe_143-Win32.flac

Again, sines by themselves are not the most interesting, so overnight I ran a couple of versions of the 38 albums in my signature. 1.4.3 only! 

* CDDA with -5, -5e, -8, -8e:  Win32 vs Win64 size differences are virtually nothing - averaging to less than a part per million.  (Yes the "High Tech Choruses" with test signals, including a sine, got some more bytes difference.)
* Same files upsampled to 96/24, -8 and -8e: Win32 vs Win64 size differences averaging to 103 (-8) vs 56 (-8e) parts per million, 32bit biggest files on most, still not much.  Worst were some metal albums at -8: 657 (Gojira), 456 (Emperor), 440 (Psycroptic).  (Impact of -e exceeded 2 percent with Judas Priest.)
* 96/24 upsamples, testing -8l16 and -8el16: Smaller differences. But -8l6 averages to a percent bigger files than -8 (max: +9.8% - and: pretty much unaffected by Win32/Win64 build) so ... avoid for now.

Takeaway from the latter is that as long as a lot of "high resolution" material is fake or for other reasons have very little content in the inaudible octave(s), the encoder isn't yet ready to increase "-l" in presets. Even if say a Linux build on a different architecture doesn't show such results - indeed that would be a sign there is something to tweak for stability.

Re: FLAC v1.4.x Performance Tests

Reply #424 –
Meanwhile, cf. Reply #420: The -8 vs -8e impact - is it the resampling or the 24 bit or something else?

Apparently, -8 hits closer to -8e for 24 bits files, than for 16 bits files. That is, size savings in -8e over -8, are larger for 16 bits even if the files are smaller.

I did two tests. One with this 38 CD corpus, and for a sanity check that it isn't an artefact of whatever sox command I have used, I afterwards tested a few 44.1/24 downloads, with similar result.


Test 1: The 38.
2.8 megabytes -e impact on the original CDDA. Out of approx 12 GB.
2.4 megabytes -e impact after resampling to 44.0/16 and no dither.
24 bits: 1.7 megabytes -e impact after resampling CDDA to 44.0/24 and no dither.  File sizes 23 GB, bitrate 1362.
24 bits: 1.7 megabytes -e impact when resampling the 96/24 upsample back to 44.1, keeping the 24 bits.  Note, this keeps a volume adjustment -v0.666 that was in the 96/24.
Then:
3.8 megabytes -e impact when ffmpeg'ing it to 16 bits (no dither).  Keeping the volume at -v0.666.  File sizes 11 GB, bitrate 678.

Running it through the sox resampler itself isn't enough to get the close results I saw for CDDA -> 48/24 upsample or CDDA -> 64/24 upsample. There is something about how much below Nyquist the low-pass sits.


Test 2: 44.1/24 lossless purchases (well some free), twenty-two kinda-arbitrarily-selected (no classical music though), one track from each, 2hrs34min in total - not much, but still enough to get a sanity check.

File sizes, -8 and then -8e savings, with and without dither:
24 bit orig.:
1 835 828 150, -8e saves 122 759
no dither:
1 025 018 227, -8e saves 199 641
16 bit dither rectangle
1 025 689 601, -8e saves 206 382
16 bit dither triangle:
1 026 407 484, -8e saves 179 793
16 bit dither "improved_e_weighted" (that's noise shaping)
1 049 624 168, -8e saves 310 590. Still only 0.03 percent impact.