Lossless Compression Test (multiple genres + high resolution + multichannel)

Topic: Lossless Compression Test (multiple genres + high resolution + multichannel) (Read 2357 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Follow-ups
- Assigned and unassigned speaker channels

Lossless Compression Test (multiple genres + high resolution + multichannel)

2024-06-28 21:23:46

For the past month, I’ve been running lossless encoders overnight on my computer. This effort resulted in another lossless bitrate table, which I hope provides a slightly different perspective compared to similar work.

The primary goal of this table is to compare compression ratios across various audio genres: pop-rock, classical, jazz, metal, electronic, and audiobooks. As a classical music enthusiast, I perfectly know how lossless encoders behave with this type of music (aiming for bitrates below 600 kbps) and how efficient they are compared to other kind of music. Adding different genres and keeping them separated adds an intriguing dimension to the analysis. In order to accomplish this, I attempted to allocate a suitable number of albums to each tested group of musical genre.

In addition to the standard CD audio material, I included post-CD content in my testing. This involved high-resolution audio at 96 KHz and 192 KHz, as well as multichannel audio (specifically 5.1 channels). While Martijn van Beurden had already conducted a thorough comparison in this domain, my contribution included additional audio material for a more extensive evaluation.

In summary, the following table is built from the following components:

Red Book CD (16/44.1) Albums:
A total of 253 complete audio albums, spanning 334 hours of music.
These albums are categorized into 7 musical genres.
Multichannel Albums:
There are 37 complete albums with a multichannel format (24/48/5.1).
These albums represent various genres.
High-Resolution Classical Music:
I included 35 albums exclusively featuring classical music at very high resolution (24/192 KHz).
Additionally, these albums were also tested at 24/96 KHz.
High-Resolution Various Genres:
A set of 40 albums covering different genres, all at high resolution (24/96 KHz).

In total, this table is based upon 400 albums (including two singles) and 428 hours of music, distributed across 11 distinct categories.

What’s not included: Speed performance testing is not part of this study. Factors such as CPU architecture, compiler variations, disk access, heat, and threaded tasks significantly impact encoding and decoding speed. I'm not very skilled for testing it accurately. For speed-related data, I recommend referring to Martijn van Beurden’s tests or browsing the forum and seek for Porcus's numerous tests.

The presentation is intentionally straightforward, available in a multi-tab sheet online. Feel free to utilize this data as needed! 😊

Now some details about the tested encoders:
FLAC:
I thoroughly tested all 9 presets—ranging from 0 to 8—using the reference FLAC encoder (version 1.43).
Additionally, I included one additional switch (-8p), but no further modifications (much slower, infinitesimal gain).
For other FLAC implementations (such as FFMPEG 7 and CueTools/Flake), I focused solely on the highest settings.
WAVPACK:
Although it wasn’t initially planned, I managed to test all 28 presets (from -f to -hhx6) using the reference WAVPACK encoder (version 5.70).
Testing extreme presets proved time-consuming, especially with high-resolution or multichannel samples. I probably won’t update the full table with minor releases of WavPack.
I also incorporated FFMPEG 7’s implementation, covering presets 0 to 5 (presets 6 to 8 were excluded due to their slow performance—slower than the slowest reference encoder preset—resulting in infinitesimal gains).
TTA:
Reference encoder and FFMPEG’s implementation were tested.
ALAC:
Three encoders were evaluated:
— REFALAC (reference encoder)
— FFMPEG
— CueTools (with default settings only)
HALAC (Hybrid Lossless Audio Codec):
I tested version 0.29 of this relatively new encoder.
OptimFROG:
The test was limited to the 253 Red Book files. I tested all 12 presets (ranging from 0 to 10, plus the MAX setting).
Multichannel support is not available, and I encountered issues with all files larger than 4GB, preventing completion of testing for the 96 KHz and 192 KHz classical music files.
However, data for various music at 96 KHz is fully available (all source files are below the 4GB limit).
Not Tested:
WMA Lossless, MP4 ALS, LA, RKAU, and Shorten were not included in the evaluation due to being deprecated, and having likely a very confidential user base.
Feel free to ask if you need further information or clarification! 😊

Results are >>here<<

(The sheet is also available here and can be downloaded if needed).

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #1 – 2024-06-28 21:25:19

CONCLUSIONS

The impact of musical genre on bitrate values and compression ratios is quite remarkable. For instance, audiobooks typically have a bitrate of around 25% (350 kbps), while metal music can reach up to 70% (1000 kbps). Interestingly, the performance of all encoders remains stable and is not significantly affected by the audio material.
The most significant variance is observed with Monkey’s Audio: while it consistently ranks as the second-best compressor with its highest settings, its performance is slightly worse when dealing with audiobooks, high-resolution audio (except classical music) and multichannel audio.
When it comes to classical music, the size difference between a strong compressor and an ultra-fast one is limited. However, in other genres, extreme settings and formats lead to more pronounced differences.

_____________________

Not surprisingly, OptimFrog remains the most powerful audio compressor across all types of audio material. However, it’s important to note that OptimFrog 5.100 lacks support for multichannel audio and encounters issues with files larger than 4GB. The last update was 8 years ago, and it’s uncertain whether there will be further updates. Interestingly, despite concerns about its encoding speed, modern CPUs handle the encoder quite efficiently. For instance, the extremely robust -preset 9 setting encodes at approximately ×110 realtime on my 3-year-old laptop and the resulting ratio really rocks! But decoding speed should be less comfortable (some operations like replaygain scanning, reencoding, integrity checker… will be strongly impacted).
TAK is remarkably close to perfection. It boasts impressive speed and consistently delivers outstanding encoding performance across all music genres. It also excels when handling high-resolution audio and multichannel content.

Monkey’s Audio performances are generally commendable, but it’s worth noting some discrepancies when using the highest setting (-c5000, also known as ‘insane’). These discrepancies are particularly noticeable with audiobooks, multichannel audio, and high-resolution content. While decoding speed hasn’t been specifically tested in this context, overall speed performance remains a weak point for the format, especially when it comes to decoding.
WavPack is arguably the most comprehensive compressor in terms of features, supporting DSD, lossy, hybrid, and floating-point formats. Notably, it excels in decoding speed and offers various asymmetric switches that enhance compression ratios without compromising decoding performance. Across all musical genres, WavPack’s slowest settings always achieve slightly more aggressive compression than TAK’s and Monkey’s Audio’s fastest settings. But at the price of much slower encoding speed.
WavPack benefits from a second implementation developed by the FFMPEG team. Although it provides less granularity, it features a straightforward preset system. The performance of both implementations is remarkably similar. However, it’s worth noting that FFMPEG’s version beats the reference encoder by a wider margin when handling multichannel files.
FLAC is undeniably popular, and it’s no surprise that there are multiple encoding tools available for it. These include the reference encoder, the FFMPEG implementation, CueTools, FLAKE, and likely others. However, FLAC’s compression performance isn’t its strongest feature.
Interestingly, FFMPEG’s FLAC outperforms several lossless formats when compressing 192 KHz classical music—even surpassing some of TAK’s and Monkey’s Audio’s fastest settings! Like with WavPack, FFMPEG also offers the strongest FLAC’s implementation for multichannel audio. So, which FLAC encoder is the best? It depends on the musical genres you’re working with, but the consistently top-ranking choice is the reference tool with the -8p setting.
TTA: the reference encoder doesn’t support high resolution nor multichannel files while the FFMPEG codec has no issues with them. For RedBook material, both encoders offer identical performance. FLAC always beats TTA in compression ratio with the exception of one musical group: billboard 2010-2020. TTA performances are poor with multichannel audio.
ALAC (Apple Lossless Audio Codec) has historically faced performance challenges, but the CueTools encoder has helped mitigate these weaknesses, as Porcus already suggested it in his own tests. In fact, CueTools’s ALAC performs nearly as well as FLAC with its default settings. Additionally, it’s possible that additional presets could further optimize the compression ratio, although I haven’t personally tested them. Unfortunately, CueTools’ ALAC doesn’t handle high-resolution and multichannel files. Overall performance remains low, except for audiobooks, which perform quite well with Cuetools—even better than Monkey’s fast.
HALAC: A new addition to the compressor family, HALAC is designed to be exceptionally fast, although this comes at the cost of compression performance. While its compression results are not impressive when compared to other options on the bitrate table, they are still acceptable. The encoding speed was fast during my test, although not stellar, which could be attributed to a lack of support for pipe encoding and potential bottlenecks from the storage system. Additionally, HALAC does not yet support high-resolution or multichannel audio. Nevertheless, considering its novelty, HALAC shows promise and may improve over time.
FOUND ISSUE:
FLAC -3 ratio is 46% on audiobooks while flac -2 and flac -4 are below 30%.

Side note: a similar test based on one genre was made by A_Man_Eating_Duck and is worth looking at:
https://hydrogenaud.io/index.php/topic,97310.0.html

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #2 – 2024-06-28 22:55:48

Can you confirm that ffmpeg's 192 KHz encodes are actually lossless, and the tool didn't do any format conversions?

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #3 – 2024-06-29 00:45:25

Thanks for sharing this, guruboolez!

I was doing a similar test suite of lossless encoders (I even tested that obscure Bonk encoder), but my house got taken away.

Maybe I don't have to finish that one after all.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #4 – 2024-06-29 08:51:22

Health to your labor for this detailed test, @guruboolez . The results are quite consistent.

HALAC(High Availability Lossless Audio Compression) is designed for reasonable compression ratio and ultra-fast processing. Encode and decode speeds and multithread performance are not mentioned in the tests. Frankly, I expected all this to happen. If only the compression ratio is important for codecs, then I've wasted my time so far. In my studies, I cannot activate a large number of operations at the expense of compression ratio. There are losses from the compression ratio even just for the multithread feature. I don't want to increase the processing density and put out a slower codec.

From the many user comments and requests I received(both professional and normal), they preferred to process much faster instead of compressing 1-2% more. Compressing data is not just reducing data to smaller sizes. It is to be able to do this faster using as little energy as possible. Unfortunately, achieving both of them is close to impossible. Accordingly, it is necessary to maintain the balance well here (Pareto front). I think someone who does such good tests can also do excellent tests in terms of speed. However, I recommend the next HALAC 0.3.x version for this.

Basically, there is not much difference between a 16-bit operation and a 24(16+8) bit operation. HALAC is not interested in SampleRate. I will be adding the 24 bit 2 channel option soon. If I can find the sample data I need to examine for multichannel, I can clarify the issue by using the necessary correlations. Maybe it can be a little better than coding each channel independently. What is missing here from my point of view is the lack of sample data.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #5 – 2024-06-29 10:56:12

Nice test, nice presentation. Off the top of my head, some remarks per codec:

* OptimFROG: Concerning decoding speed affecting (negatively) certain tasks, it should be mentioned that OptimFROG (and Monkey's and WavPack) can offer integrity checking without decoding. You will not get a confirmation that the audio matches the (optional) MD5, but the block checksums will be tested - and it is faster than decoding FLAC.

* Monkey's Audio: This codec cannot handle wasted bits, that is, "fake high resolution" when what is "fake" is the bits per sample - zero-padding up to look more impressive, or to fit a particular format. It is not necessarily "high-resolution audio"; say, what if an audiobook was recorded at 12 bits per sample? To put it on a CD, one will have to pad up to 16 bits, the format doesn't allow for less.
But since ALAC handles the audiobooks well, it is unlikely to be (a big part of) the explanation. ALAC doesn't do wasted bits either.
For most music CDs, there aren't any such wasted bits, and then Monkey's performs as good as its reputation. But for high resolution (and possibly, audiobooks), the prevalence of zeroed-out lower bits isn't known - so who knows how that impacts Monkey's performance. Big "YMMV" here, collection to collection.

* WavPack: Surprising on multichannel ffmpeg vs reference; did the source files have a WAVEFORMATEXTENSIBLE channel mask; i.e. the WAVE was recognized as 5.1 and not six unordered channels? Reference WavPack has a --pair-unassigned-chans that, for multichannel, will stereo-decorrelate pairs even if they are not assigned as this nor that. Maybe FFmpeg tries that anyway?
@bryant , any input? Difference seems too big to be explained by different block sizes?

* FLAC: You are getting into an apples-to-only-"near-apples" comparison, as reference FLAC at -0 to -8 only invoke parameters that produce "Subset" files for all signals: it is more restrictive by design. The higher FFmpeg presets happily produce files out of subset. Not necessarily a bad thing: if you want "something that compresses slightly better and is at worst only slightly less compatible", then the closest competition to FLAC-within-subset is not another format, but FLAC-outside-subset.
You mention that FLAC -3 does bad on audiobooks. Not at all surprising: -0 and -3 force dual mono encoding. Why is that "0 and 3 and nothing in between"? Because 0, 1, 2 restrict to a fixed set of predictors, and even if you have some chip that cannot do multiplication, you can still decode those files. -3 is the fastest variable-predictor setting.

* TTA: reference TTA and FFmpeg-TTA should produce the same bitstream. If you removed the tags from the latter, files should be bit-identical.
If you encoded from WAVE, the reference TTA encoder is picky about WAVE version. It means it rejects quite a few WAVE files that carry signals it could encode if piped. To be frank, the reference implementation sucks.

* ALAC: CUETools ported some of reference FLAC's tried and tested windowing functions into ALAC. Given how FLAC has been honed while Apple never cared about performance, it is not at all surprising that ALAC could be improved upon, if anyone bothered.

* HALAC: Even though it was never designed to compete when speed is not an issue, it is still worth a confirmation that it is behaves ... quite reasonable.

* WMAL: No WMAL here - let's all agree that nobody should use it?
(To the uninitiated: There have been Windows versions that cannot handle WMAL losslessly. It is abandonware.)

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #6 – 2024-06-29 12:12:19

Quote from: 2012 on 2024-06-28 22:55:48

Can you confirm that ffmpeg's 192 KHz encodes are actually lossless, and the tool didn't do any format conversions?

To answer your question, I had to recode a second time to FFMPEG-FLAC using this setting: -i pipe:0 -y -c:a flac -compression_level 12 %d. I verified that the bitrate is identical to my first encoding.
Now foobar2000 bit-to-bit comparison:

Code: [Select]

All tracks decoded fine, no differences found.

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] 2L • 2012 • Lasse Thoresen. Himmelkvad. Nordic Voices.wav"
"F:\§\t[192] 2L • 2012 • Lasse Thoresen. Himmelkvad. Nordic Voices.flac"
Compared 838925290 samples.
No differences in decoded data found.
Channel peaks: 0.977573 (-0.20 dBFS) 0.977547 (-0.20 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ACOUSENCE • 2010 • Debussy & Stravinsky. La Mer & The Rite of Springs. Jonathan Darlington & Duisburger Philharmoniker.wav"
"F:\§\t[192] ACOUSENCE • 2010 • Debussy & Stravinsky. La Mer & The Rite of Springs. Jonathan Darlington & Duisburger Philharmoniker.flac"
Compared 710397440 samples.
No differences in decoded data found.
Channel peaks: 0.977237 (-0.20 dBFS) 0.977237 (-0.20 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ALPHA • 2023 • Joseph Haydn. Symphonies 31, 59, 48. Il Giardino Armonico.wav"
"F:\§\t[192] ALPHA • 2023 • Joseph Haydn. Symphonies 31, 59, 48. Il Giardino Armonico.flac"
Compared 1002544640 samples.
No differences in decoded data found.
Channel peaks: 0.923012 (-0.70 dBFS) 0.922857 (-0.70 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ARCANA • 2023 • VA. Un Air d’Italie 'The Mandolin in Paris in the 18th Century'. Anna Schivazappa.wav"
"F:\§\t[192] ARCANA • 2023 • VA. Un Air d’Italie 'The Mandolin in Paris in the 18th Century'. Anna Schivazappa.flac"
Compared 795563520 samples.
No differences in decoded data found.
Channel peaks: 0.976384 (-0.21 dBFS) 0.887816 (-1.03 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ART INFINI • 2017 • Saint-Saëns, Schubert & Others. Arrangements for Cello & Harp. Hitomi Niikura.wav"
"F:\§\t[192] ART INFINI • 2017 • Saint-Saëns, Schubert & Others. Arrangements for Cello & Harp. Hitomi Niikura.flac"
Compared 832476115 samples.
No differences in decoded data found.
Channel peaks: 0.998451 (-0.01 dBFS) 0.933471 (-0.60 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] BERLINER PHILHARMONIKER • 2018 • Mahler. Symphony 6. Sir Simon Rattle.wav"
"F:\§\t[192] BERLINER PHILHARMONIKER • 2018 • Mahler. Symphony 6. Sir Simon Rattle.flac"
Compared 945719506 samples.
No differences in decoded data found.
Channel peaks: 0.890226 (-1.01 dBFS) 0.890226 (-1.01 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] CARPE DIEM RECORDS • 2017 • Various. The Wind Rose. Hirundo Maris, Petter Udland Johansen, Arianna Savall.wav"
"F:\§\t[192] CARPE DIEM RECORDS • 2017 • Various. The Wind Rose. Hirundo Maris, Petter Udland Johansen, Arianna Savall.flac"
Compared 858822213 samples.
No differences in decoded data found.
Channel peaks: 0.891224 (-1.00 dBFS) 0.891224 (-1.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ÇEDILLE • 2018 • Olagón. A Cantata In Doublespeak. Eighth Blackbird.wav"
"F:\§\t[192] ÇEDILLE • 2018 • Olagón. A Cantata In Doublespeak. Eighth Blackbird.flac"
Compared 1051576320 samples.
No differences in decoded data found.
Channel peaks: 0.953866 (-0.41 dBFS) 1.000000 (-0.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] CHANNEL CLASSICS • 2023 • Garth Knox. Open Spaces. Ragazze Quartet.wav"
"F:\§\t[192] CHANNEL CLASSICS • 2023 • Garth Knox. Open Spaces. Ragazze Quartet.flac"
Compared 671024641 samples.
No differences in decoded data found.
Channel peaks: 0.977452 (-0.20 dBFS) 1.000000 (0.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] DACAPO • 2022 • Ruud Langaard. Symphony 1. Sakari Oramo.wav"
"F:\§\t[192] DACAPO • 2022 • Ruud Langaard. Symphony 1. Sakari Oramo.flac"
Compared 639720960 samples.
No differences in decoded data found.
Channel peaks: 0.929258 (-0.64 dBFS) 0.929360 (-0.64 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] DECCA • 1960remast2022 • Wagner. The Golden Ring, Great Scenes. Wiener Philharmoniker & Georg Solti.wav"
"F:\§\t[192] DECCA • 1960remast2022 • Wagner. The Golden Ring, Great Scenes. Wiener Philharmoniker & Georg Solti.flac"
Compared 885050880 samples.
No differences in decoded data found.
Channel peaks: 0.977229 (-0.20 dBFS) 0.977230 (-0.20 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] DG • 2020 • Leonardo Vinci. Veni, Vidi, Vinci. Franco Fagioli, Il Pomo d'Oro & Zefira Valova.wav"
"F:\§\t[192] DG • 2020 • Leonardo Vinci. Veni, Vidi, Vinci. Franco Fagioli, Il Pomo d'Oro & Zefira Valova.flac"
Compared 819837440 samples.
No differences in decoded data found.
Channel peaks: 0.901855 (-0.90 dBFS) 0.901855 (-0.90 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] ERATO • 2023 • Various. Idylle. Léa Desandre & Thomas Dunford.wav"
"F:\§\t[192] ERATO • 2023 • Various. Idylle. Léa Desandre & Thomas Dunford.flac"
Compared 744192000 samples.
No differences in decoded data found.
Channel peaks: 0.928910 (-0.64 dBFS) 0.986442 (-0.12 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] EUDORA • 2018 • Bach. The Cello Suites (for guitar). Petrit Çeku.wav"
"F:\§\t[192] EUDORA • 2018 • Bach. The Cello Suites (for guitar). Petrit Çeku.flac"
Compared 1536988044 samples.
No differences in decoded data found.
Channel peaks: 0.824620 (-1.67 dBFS) 0.814213 (-1.79 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] GIMELL • 2015 • Arvo Pärt. Tintinnabuli. The Tallis Scholars & Peter Phillips.wav"
"F:\§\t[192] GIMELL • 2015 • Arvo Pärt. Tintinnabuli. The Tallis Scholars & Peter Phillips.flac"
Compared 772411392 samples.
No differences in decoded data found.
Channel peaks: 0.995926 (-0.04 dBFS) 0.760736 (-2.38 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] HARMONIA MUNDI • 2024 • Olivier Messiaen. Turangalîla Symphonie. Gustavo Gimeno.wav"
"F:\§\t[192] HARMONIA MUNDI • 2024 • Olivier Messiaen. Turangalîla Symphonie. Gustavo Gimeno.flac"
Compared 848988160 samples.
No differences in decoded data found.
Channel peaks: 0.988562 (-0.10 dBFS) 0.988555 (-0.10 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] HUNNIA • 2022 • Beethoven. Piano Sonata 29 Hammerklavier. Gergely Bogányi.wav"
"F:\§\t[192] HUNNIA • 2022 • Beethoven. Piano Sonata 29 Hammerklavier. Gergely Bogányi.flac"
Compared 575173117 samples.
No differences in decoded data found.
Channel peaks: 0.934515 (-0.59 dBFS) 0.986997 (-0.11 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] HYPERION • 2023 • Fauré. Nocturnes & Barcarolles. Marc-André Hamelin.wav"
"F:\§\t[192] HYPERION • 2023 • Fauré. Nocturnes & Barcarolles. Marc-André Hamelin.flac"
Compared 1885662680 samples.
No differences in decoded data found.
Channel peaks: 0.962155 (-0.34 dBFS) 0.970545 (-0.26 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] L'ENCELADE • 2018 • Reincken. Toccatas, Partitas & Suites for Harpsichord. Clément Geoffroy.wav"
"F:\§\t[192] L'ENCELADE • 2018 • Reincken. Toccatas, Partitas & Suites for Harpsichord. Clément Geoffroy.flac"
Compared 843005440 samples.
No differences in decoded data found.
Channel peaks: 0.861102 (-1.30 dBFS) 0.891184 (-1.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] LINN • 2012 • Bach. Cello Suites. Richard Tunnicliffe.wav"
"F:\§\t[192] LINN • 2012 • Bach. Cello Suites. Richard Tunnicliffe.flac"
Compared 1587693425 samples.
No differences in decoded data found.
Channel peaks: 0.588126 (-4.61 dBFS) 0.687851 (-3.25 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] LSO • 2023 • Richard Strauss. Also Sprach Zarathustra. François-Xavier Roth.wav"
"F:\§\t[192] LSO • 2023 • Richard Strauss. Also Sprach Zarathustra. François-Xavier Roth.flac"
Compared 571136000 samples.
No differences in decoded data found.
Channel peaks: 0.995426 (-0.04 dBFS) 0.997685 (-0.02 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] MYRIOS • 2023 • Bruckner. Symphony 4. François-Xavier Roth.wav"
"F:\§\t[192] MYRIOS • 2023 • Bruckner. Symphony 4. François-Xavier Roth.flac"
Compared 801876480 samples.
No differences in decoded data found.
Channel peaks: 0.889273 (-1.02 dBFS) 0.888990 (-1.02 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] NAÏVE • 2022 • Bach. Goldberg Variations (Piano). Tianqi Du.wav"
"F:\§\t[192] NAÏVE • 2022 • Bach. Goldberg Variations (Piano). Tianqi Du.flac"
Compared 1010403841 samples.
No differences in decoded data found.
Channel peaks: 0.954408 (-0.41 dBFS) 0.954453 (-0.40 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] NAXOS • 2023 • Lera Auerbach. 24 Preludes for Violin & Piano. Christine Bernsted, Ramez Mhaanna.wav"
"F:\§\t[192] NAXOS • 2023 • Lera Auerbach. 24 Preludes for Violin & Piano. Christine Bernsted, Ramez Mhaanna.flac"
Compared 619686400 samples.
No differences in decoded data found.
Channel peaks: 0.944057 (-0.50 dBFS) 0.944084 (-0.50 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] OMF • 2024 • Rameau. Pièces de clavecin. Marie Nishiyama.wav"
"F:\§\t[192] OMF • 2024 • Rameau. Pièces de clavecin. Marie Nishiyama.flac"
Compared 1050990080 samples.
No differences in decoded data found.
Channel peaks: 0.717725 (-2.88 dBFS) 0.565310 (-4.95 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] OUR RECORDINGS • 2023 • Corelli, Telemann, Händel. Corellimania. Petri, Perl, Esfahani.wav"
"F:\§\t[192] OUR RECORDINGS • 2023 • Corelli, Telemann, Händel. Corellimania. Petri, Perl, Esfahani.flac"
Compared 872399360 samples.
No differences in decoded data found.
Channel peaks: 0.710189 (-2.97 dBFS) 0.681710 (-3.33 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] PASSACAILLE • 2021 • Vivaldi. Concerti Particolari. Academia Montis Regalis & Enrico Onofri.wav"
"F:\§\t[192] PASSACAILLE • 2021 • Vivaldi. Concerti Particolari. Academia Montis Regalis & Enrico Onofri.flac"
Compared 699023360 samples.
No differences in decoded data found.
Channel peaks: 0.921511 (-0.71 dBFS) 0.921511 (-0.71 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] PENTATONE • 2024 • Pergolesi, Vivaldi. Stabat Mater & Nisi Dominus. Maarten Engeltjes.wav"
"F:\§\t[192] PENTATONE • 2024 • Pergolesi, Vivaldi. Stabat Mater & Nisi Dominus. Maarten Engeltjes.flac"
Compared 660561920 samples.
No differences in decoded data found.
Channel peaks: 0.941153 (-0.53 dBFS) 0.931884 (-0.61 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] REFERENCE RECORDINGS • 2015 • Beethoven. Symphonies 5 & 7. Manfred Honeck & Pittsburgh SO.wav"
"F:\§\t[192] REFERENCE RECORDINGS • 2015 • Beethoven. Symphonies 5 & 7. Manfred Honeck & Pittsburgh SO.flac"
Compared 822828943 samples.
No differences in decoded data found.
Channel peaks: 0.923884 (-0.69 dBFS) 0.924726 (-0.68 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] RESONUS • 2024 • Howard Skempton. 50 Preludes & Fugues for Organ. Matthew Owens.wav"
"F:\§\t[192] RESONUS • 2024 • Howard Skempton. 50 Preludes & Fugues for Organ. Matthew Owens.flac"
Compared 1409664000 samples.
No differences in decoded data found.
Channel peaks: 0.699645 (-3.10 dBFS) 0.631088 (-4.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] RICERCAR • 2023 • Lully, Philidor…. Fastes de la Grande Écurie. Syntagma Amici.wav"
"F:\§\t[192] RICERCAR • 2023 • Lully, Philidor…. Fastes de la Grande Écurie. Syntagma Amici.flac"
Compared 798356480 samples.
No differences in decoded data found.
Channel peaks: 0.921511 (-0.71 dBFS) 0.921511 (-0.71 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] SIGNUM • 2019 • Mozart. Apollo et Hyacinthus. The Mozartists & Ian Page.wav"
"F:\§\t[192] SIGNUM • 2019 • Mozart. Apollo et Hyacinthus. The Mozartists & Ian Page.flac"
Compared 882549760 samples.
No differences in decoded data found.
Channel peaks: 0.894637 (-0.97 dBFS) 0.988523 (-0.10 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] SONO LUMINUS • 2016 • Philip Glass. Dreaming Awake. Bruce Levingston, Ethan Hawke.wav"
"F:\§\t[192] SONO LUMINUS • 2016 • Philip Glass. Dreaming Awake. Bruce Levingston, Ethan Hawke.flac"
Compared 1263272844 samples.
No differences in decoded data found.
Channel peaks: 0.966094 (-0.30 dBFS) 0.966078 (-0.30 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] STEINWAY & SONS • 2016 • Scriabine. Piano Works. Klara Min.wav"
"F:\§\t[192] STEINWAY & SONS • 2016 • Scriabine. Piano Works. Klara Min.flac"
Compared 721715200 samples.
No differences in decoded data found.
Channel peaks: 0.966051 (-0.30 dBFS) 0.966051 (-0.30 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\t[192] WARNER • 2021 • Bach. Sonatas & Partitas (Violin). Augustin Hadelich.wav"
"F:\§\t[192] WARNER • 2021 • Bach. Sonatas & Partitas (Violin). Augustin Hadelich.flac"
Compared 1512832909 samples.
No differences in decoded data found.
Channel peaks: 0.875777 (-1.15 dBFS) 0.845521 (-1.46 dBFS)



Total duration processed: 1d 23:04:55.160
Time elapsed: 59:35.754
47.40x realtime

Everything is fine with FLAC/FFMPEG. Is there any reason you ask for it? Do you feel I have to check with FFMPEG WavPack as well?

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #7 – 2024-06-29 12:18:02

@Hakan Abbas :

Hi Hakan, I understand your concern. However, as I mentioned in my initial post, I am not skilled at accurately testing encoding speed (decoding should be easier for me, but with one potential bias: I will only test speed through the foobar2000 decoding speed component, which is excellent but potentially not consistent with the reference decoder).

The current purpose of this test is to create a kind of bitrate cartography—nothing more. Selecting albums for testing is already a challenging task because it is far from obvious. Additionally, compressing everything with various settings, especially slower ones, consumes a significant amount of time and electricity.

On the other hand, an accurate representation of encoding ratio holds more value than raw speed. Because it applies universally. Whether you have an old Pentium or a powerful Ryzen processor, the ratio remains unchanged, and the bitrate table still holds true. However, speed varies significantly from user to user and from period to period. When speed improves, it impacts our decisions. For instance, I recently switched to Wavpack and am using the hx6 setting—a setting I couldn't afford with my previous computer. While browsing old Wavpack debates in this forum, I found that the x6 switch was entirely unusable 15 years ago, but today, it should appear as acceptable in terms of speed for some people depending on their computer. Tomorrow, it may even be perceived as fast and maybe people will ask David Bryant to create further -xn settings.

Furthermore, in real-world scenarios, HALAC speed did not perform exceptionally well. If I were to encode a full hard drive to HALAC using tools like foobar2000, EZ CD Audio Converter, dBPoweramp, or BatchEncoder, it would not be as fast as expected due to USB limitations, drive constraints, and a CPU potentially working with up to 32 or even 64 simultaneous files. That’s a second limitation to measuring speed: what you get in a test could be much worse if real life practices.

Users can also very quickly form an accurate opinion about encoding speed—it only takes a few seconds to determine if a setting is usable or uncomfortable. In contrast, assessing what a setting offers in terms of ratio compared to another setting requires much more effort. Hence this test which is restricted to bitrate or ratio only.

Regardless of the scenario, it seems very obvious that a vast majority of people don’t base their decision solely on bitrate performance. Otherwise, FLAC would have become an abandoned lossless format long ago. Don’t be afraid by a single bitrate table.
Anyway, I’ll be glad to check how future version of HALAC will react with my corpus 😊

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #8 – 2024-06-29 12:52:51

@Destroid : I’m not entirely sure what happened to your house, but I hope you’re safe. If you have your own test, I think comparing our results would be highly interesting for some people (including me)!

@Porcus :

Thanks for all these comments and precisions.

Regarding Wavpack, I must admit that I wasn’t aware of the WAVEFORMATEXTENSIBLE channel mask. When I examine foobar2000’s info box, I see the following:
• 6 channels: FL FR FC LFE BL BR (81.8%)
• 6 channels: FL FR FC LFE SL SR (18.2%)
Is it OK? Could you guide me?

As for FLAC, I’ve long been aware that high FLAC settings are non-subset, but it’s essential to emphasize this point. I’m uncertain if it’s still an issue for Android or iOS playback. It might be worth investigating. However, personally, I’ve never trusted these borderline settings, nor would I recommend them to others. The difference is minimal anyway: 332.77 GB with FLAC 1.43 -8p versus 332.59 GB with FFMPEG FLAC -12 (remember that Flake -r264 doesn’t support high-resolution files). The variation is only 0.05% nowadays!

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #9 – 2024-06-29 13:20:54

@guruboolez I really respect what you do.

It will not be right to use Foobar2000 for the speed test. Because the way I don't understand HALAC works 3-4 times slower. This is definitely not normal. That's why I'm going to prepare it at GUI HALAC Converter. Of course, it will support other codecs. All codecs have a CLI. In real life, the best and professional is to test this way. Or not to attempt the speed test.

Obviously, the codecs (HALIC and HALAC) I have developed have been brutally tested by many experts so far and is still kept. Some of these tests are not public. However, for the first time, I come across a test where the transaction speed is disabled. In such cases, it is stated in which system (Processor, SSD, RAM ...) is performed in such cases and the results are calculated.

It is necessary to clarify the speed. The speed relationship of the codec is almost proportional compared to time and developing systems. In other words, if the HALAC can process a certain set of data in x seconds in a certain setting, the other can work in 10x. This ratio maybe 9x maybe 11x. If a slow codec in more modern systems is accelerated, the faster codec is more accelerated.

The lossless compression rate in sound data is limited. 2-3% compression increase may require non-practical processing time. In this case, the speed becomes more important, and as you say, FLAC is a good example of this. Just like JPEG, MP4, RAR.

The video below may give an idea to show which one is more important by certain authorities in terms of speed or compression ratio of a codec.
https://youtu.be/LQuamIL7S-s?si=Qe6oakRfuGI1I8T-&t=36

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #10 – 2024-06-29 14:29:46

Quote from: guruboolez on 2024-06-29 12:12:19

Everything is fine with FLAC/FFMPEG. Is there any reason you ask for it?

Just to confirm it's not a case of too good to be true since FFMPEG encoders performed well with this format.

Quote from: guruboolez on 2024-06-29 12:12:19

Do you feel I have to check with FFMPEG WavPack as well?

Since it's not a question of the encoders themselves being broken, retesting one 192KHz sample would be enough just to confirm ffmpeg doesn't do any wrong transformations in this use-case.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #11 – 2024-06-29 15:25:54

Quote from: Hakan Abbas on 2024-06-29 13:20:54

It will not be right to use Foobar2000 for the speed test. Because the way I don't understand HALAC works 3-4 times slower.

I have concerns about this, not only for HALAC. Many years ago, people observed significant discrepancies between two different systems—for example, Rockbox versus Win32 foobar2000—when handling certain audio formats. While trends are easily identifiable (such as Monkey’s Insane being consistently slower than FLAC), results for similar formats could still be unfair or biased.

Quote

This is definitely not normal. That's why I'm going to prepare it at GUI HALAC Converter. Of course, it will support other codecs. All codecs have a CLI. In real life, the best and professional is to test this way. Or not to attempt the speed test.

I completely understand your perspective! But as user, I'm not sure I'll be enthusiastic by the speed of any format if I cannot achieve it with my usual tools and for my encoding scenario. I mean: what’s the purpose of a speed test if can’t achieve this speed outside the test? I think there are no best solution for measuring speed but only compromises.
Making a GUI for HALAC is very nice but it has to be complete (support for many input formats, accurate tag transfert) to become useful.

Quote

However, for the first time, I come across a test where the transaction speed is disabled.

I can understand your feeling. My test can not satisfy you in this state, it’s obvious 😊But everyone is free to use the data I produced in my table and to complete it with other measures like speed.

Quote

It is necessary to clarify the speed. The speed relationship of the codec is almost proportional compared to time and developing systems. In other words, if the HALAC can process a certain set of data in x seconds in a certain setting, the other can work in 10x. This ratio maybe 9x maybe 11x. If a slow codec in more modern systems is accelerated, the faster codec is more accelerated.

Indeed, simplifying the speed values could be helpful, such as expressing FLAC as a range like “x500-x800” and TAK as “x400-x500.” I had this in mind when I built the table. But I'm still not entirely sure it solves the problem. What's exactly the point of measuring precisely the bitrate of ratio on one side if on the other side the test stays very approximative on speed? And won’t this erase some important granularity between two similar formats?

Quote

The lossless compression rate in sound data is limited. 2-3% compression increase may require non-practical processing time. In this case, the speed becomes more important, and as you say, FLAC is a good example of this. Just like JPEG, MP4, RAR.

You’re absolutely right! But the cultural context around audio is not the same than with still pictures or archives. For audio users bitrate is a very essential value that every software player shows. But photographers don't check the bit-per-pixel values of their pictures and picture viewers don't show this value. So even if the differences seem small from a rational standpoint, having a detailed bitrate table remains valuable for audio enthusiasts alike.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #12 – 2024-06-29 15:52:52

Quote from: 2012 on 2024-06-29 14:29:46

Since it's not a question of the encoders themselves being broken, retesting one 192KHz sample would be enough just to confirm ffmpeg doesn't do any wrong transformations in this use-case.

OK, I understand.
It reminds me one difference I noticed between reference Wavpack and FFMPEG's implementation: 24 bit audio are stored as 32 bit with FFMPEG (it's documented I think and it stays lossless so I won't call it an issue even if it's strange).

Source: 8:30:33.343 (2 940 800 917 samples) 96000 Hz 2 channels 24 Bits per sample
WV/FFMPEG: 8:30:33.343 (2 940 800 917 samples) 96000 Hz 2 channels 32 (fixed-point) Bits per sample

Code: [Select]

ll tracks decoded fine, no differences found.

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Aleksi Perälä - The-Colundi-Sequence-Level-17-2 (2019).wav"
"F:\§\se[96] Aleksi Perälä - The-Colundi-Sequence-Level-17-2 (2019).wv"
Compared 295249221 samples.
No differences in decoded data found.
Channel peaks: 0.943603 (-0.50 dBFS) 0.946794 (-0.47 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] ContaineR - YACKER (2024).wav"
"F:\§\se[96] ContaineR - YACKER (2024).wv"
Compared 173146509 samples.
No differences in decoded data found.
Channel peaks: 0.931830 (-0.61 dBFS) 0.933648 (-0.60 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Dark Sky Alliance - Interdwell (2024).wav"
"F:\§\se[96] Dark Sky Alliance - Interdwell (2024).wv"
Compared 415426560 samples.
No differences in decoded data found.
Channel peaks: 0.890971 (-1.00 dBFS) 0.891006 (-1.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] James Blake - Playing Robots Into Heaven (2023).wav"
"F:\§\se[96] James Blake - Playing Robots Into Heaven (2023).wv"
Compared 246157481 samples.
No differences in decoded data found.
Channel peaks: 0.977235 (-0.20 dBFS) 0.977237 (-0.20 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Jemapur - Xentral Processing Unit (Dance) (2023).wav"
"F:\§\se[96] Jemapur - Xentral Processing Unit (Dance) (2023).wv"
Compared 392294414 samples.
No differences in decoded data found.
Channel peaks: 0.967713 (-0.29 dBFS) 0.973262 (-0.24 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Justice - Hyperdrama (2024).wav"
"F:\§\se[96] Justice - Hyperdrama (2024).wv"
Compared 283902720 samples.
No differences in decoded data found.
Channel peaks: 0.988439 (-0.10 dBFS) 0.988439 (-0.10 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] ʞık - Basement Classics (2024).wav"
"F:\§\se[96] ʞık - Basement Classics (2024).wv"
Compared 380600062 samples.
No differences in decoded data found.
Channel peaks: 1.000000 (0.00 dBFS) 1.000000 (0.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Mehmet Aslan - The Sun is Parallel (2022).wav"
"F:\§\se[96] Mehmet Aslan - The Sun is Parallel (2022).wv"
Compared 246681870 samples.
No differences in decoded data found.
Channel peaks: 0.988553 (-0.10 dBFS) 0.988553 (-0.10 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] R.N.A.O Meets P.O.P.O - Organism (1980r2024).wav"
"F:\§\se[96] R.N.A.O Meets P.O.P.O - Organism (1980r2024).wv"
Compared 209952000 samples.
No differences in decoded data found.
Channel peaks: 1.000000 (0.00 dBFS) 1.000000 (0.00 dBFS)

Comparing:
"D:\AUDIOTESTS\BITRATE TABLE\400 FILES TEST\se[96] Underworld - Barbara Barbara, We Face A Shining Future (2016).wav"
"F:\§\se[96] Underworld - Barbara Barbara, We Face A Shining Future (2016).wv"
Compared 297390080 samples.
No differences in decoded data found.
Channel peaks: 0.989710 (-0.09 dBFS) 0.989710 (-0.09 dBFS)



Total duration processed: 8:30:33.343
Time elapsed: 7:02.571
72.49x realtime

So audiostream is proven to be lossless.

But what if I want convert it back to FLAC? Can I get back 24 bit audio with no loss? I tried to convert the 32bit WV file to 24 bit FLAC with foobar2000 (I just set the highest bps supported to 24 bit):

Code: [Select]

All tracks decoded fine, no differences found.

Comparing:
"F:\§\se[96] Aleksi Perälä - The-Colundi-Sequence-Level-17-2 (2019).wv"
"F:\§\se[96] Aleksi Perälä - The-Colundi-Sequence-Level-17-2 (2019).flac"
Compared 295249221 samples.
No differences in decoded data found.
Channel peaks: 0.943603 (-0.50 dBFS) 0.946794 (-0.47 dBFS)

Comparing:
"F:\§\se[96] ContaineR - YACKER (2024).wv"
"F:\§\se[96] ContaineR - YACKER (2024).flac"
Compared 173146509 samples.
No differences in decoded data found.
Channel peaks: 0.931830 (-0.61 dBFS) 0.933648 (-0.60 dBFS)

Comparing:
"F:\§\se[96] Dark Sky Alliance - Interdwell (2024).wv"
"F:\§\se[96] Dark Sky Alliance - Interdwell (2024).flac"
Compared 415426560 samples.
No differences in decoded data found.
Channel peaks: 0.890971 (-1.00 dBFS) 0.891006 (-1.00 dBFS)

Comparing:
"F:\§\se[96] James Blake - Playing Robots Into Heaven (2023).wv"
"F:\§\se[96] James Blake - Playing Robots Into Heaven (2023).flac"
Compared 246157481 samples.
No differences in decoded data found.
Channel peaks: 0.977235 (-0.20 dBFS) 0.977237 (-0.20 dBFS)

Comparing:
"F:\§\se[96] Jemapur - Xentral Processing Unit (Dance) (2023).wv"
"F:\§\se[96] Jemapur - Xentral Processing Unit (Dance) (2023).flac"
Compared 392294414 samples.
No differences in decoded data found.
Channel peaks: 0.967713 (-0.29 dBFS) 0.973262 (-0.24 dBFS)

Comparing:
"F:\§\se[96] Justice - Hyperdrama (2024).wv"
"F:\§\se[96] Justice - Hyperdrama (2024).flac"
Compared 283902720 samples.
No differences in decoded data found.
Channel peaks: 0.988439 (-0.10 dBFS) 0.988439 (-0.10 dBFS)

Comparing:
"F:\§\se[96] ʞık - Basement Classics (2024).wv"
"F:\§\se[96] ʞık - Basement Classics (2024).flac"
Compared 380600062 samples.
No differences in decoded data found.
Channel peaks: 1.000000 (0.00 dBFS) 1.000000 (0.00 dBFS)

Comparing:
"F:\§\se[96] Mehmet Aslan - The Sun is Parallel (2022).wv"
"F:\§\se[96] Mehmet Aslan - The Sun is Parallel (2022).flac"
Compared 246681870 samples.
No differences in decoded data found.
Channel peaks: 0.988553 (-0.10 dBFS) 0.988553 (-0.10 dBFS)

Comparing:
"F:\§\se[96] R.N.A.O Meets P.O.P.O - Organism (1980r2024).wv"
"F:\§\se[96] R.N.A.O Meets P.O.P.O - Organism (1980r2024).flac"
Compared 209952000 samples.
No differences in decoded data found.
Channel peaks: 1.000000 (0.00 dBFS) 1.000000 (0.00 dBFS)

Comparing:
"F:\§\se[96] Underworld - Barbara Barbara, We Face A Shining Future (2016).wv"
"F:\§\se[96] Underworld - Barbara Barbara, We Face A Shining Future (2016).flac"
Compared 297390080 samples.
No differences in decoded data found.
Channel peaks: 0.989710 (-0.09 dBFS) 0.989710 (-0.09 dBFS)



Total duration processed: 8:30:33.343
Time elapsed: 7:16.162
70.23x realtime

Everything is fine

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #13 – 2024-06-29 17:02:44

@guruboolez
A player/converter, such as Foobar2000, is really interesting that it can operate an external file at much lower speed than normal speed. A binary file with certain parameters will be called and run.
Yes, I think it is necessary to prepare a beautiful converter with HALIC Converter style when it relieves my work.
I already said in the first reference that the compression results of your tests are quite consistent. So there is no problem for me. What was important to me was at what speeds these results were achieved. Still thanks...

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #14 – 2024-06-29 20:07:20

HALAC is slow in this test because HALAC doesn't support encoding from stdin/pipe unlike other encoders tested. And fb2k can't directly feed source file to encoder. So time is wasted to write temporary wav file.
Just add pipe support and no separate GUI is needed.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #15 – 2024-06-29 21:41:01

What multi-channel files encoded with libavcodec wavpack encoder are compressed better?

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #16 – 2024-06-30 02:27:45

Quote from: guruboolez on 2024-06-29 12:52:51

@Destroid : I’m not entirely sure what happened to your house, but I hope you’re safe. If you have your own test, I think comparing our results would be highly interesting for some people (including me)!

Thanks guruboolez. It was a flooding issue, and let's just say the land owner and management acted inappropriately. Maybe for the best that I left.

Off the top of my head, codecs like Bonk, WMALL didn't seem too impressive. I remember the MPEG-lossless reference encoder at the highest setting (something like -7 switch?) did very well at compressing but at the cost of being extremely slow. There might have been one or two more other oddball lossless audio compressors that got tested.

The testing you conducted covers a lot more area than my testing (which was purely Red Book CD audio), so I didn't explore the various single/multi channel and high-resolution areas. Plus, not much variety in genres in my tests, and your results are quite interesting in that regard. And FLAC settings are practically non-existent- just the pure numeric switches.

The real reason for the testing was I wrote a script that had it's own internal timing "routine" that activated stopwatch-like when the audio compressor began and ended and excluded any duration copying WAV to RAMDisk and clearing the RAMDisk since the size isn't very vast. And of course it automated various compressors with different switches (lossy as well as lossless).

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #17 – 2024-06-30 07:45:03

Quote from: Bogozo on 2024-06-29 20:07:20

HALAC is slow in this test because HALAC doesn't support encoding from stdin/pipe unlike other encoders tested. And fb2k can't directly feed source file to encoder. So time is wasted to write temporary wav file.
Just add pipe support and no separate GUI is needed.

Thank you for the information. I will take care of this issue.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #18 – 2024-06-30 13:15:46

First, looking more into your .ods file, there is something missing about OptimFROG: It has zeroes for 96 kHz classical music stereo.
It also has zeroes where it should say "not working".

Quote from: guruboolez on 2024-06-29 12:52:51

Regarding Wavpack, I must admit that I wasn’t aware of the WAVEFORMATEXTENSIBLE channel mask. When I examine foobar2000’s info box, I see the following:
• 6 channels: FL FR FC LFE BL BR (81.8%)
• 6 channels: FL FR FC LFE SL SR (18.2%)
Is it OK? Could you guide me?

I don't know! I only know there is a thing ... somewhere ... maybe or maybe not there.
@bryant , is there any reason why any of these channel configurations should do worse in reference WavPack than in FFmpeg?

Quote from: guruboolez on 2024-06-29 12:52:51

As for FLAC [...] The difference is minimal anyway: 332.77 GB with FLAC 1.43 -8p versus 332.59 GB with FFMPEG FLAC -12

Yeah, that is almost spot-on equal to the "-p" difference. But it is the non-CDDA material where FFmpeg wins. On CDDA, FFmpeg -compression_level 12 gets beaten by reference FLAC -7.
On 192 kHz classical music, FFmpeg -compression_level 12 beats flac.exe -8p by 0.3 percentage points, which is quite a lot.
(It is imprecise to say this is about "Subset or not", because reference FLAC limits its presets to settings that would keep any signal within subset, and so for resolutions where Subset does allow -b 8192, the presets doesn't use them. -8b8192 -l16 I suppose would improve over -8, as FFmpeg goes up in block size when sampling rate increases. No "big" improvement, but enough to prove a point ... if one is interested.)

On a different note, there are individual differences. Take the electronic music, where the reference beats FFmpeg by half a percent point, that is quite a lot. Take the "lowest" and "highest" of classical CDDA: FFmpeg beats -8p by nearly half a percentage point on those two albums, yet still loses on the average. But this is I guess, to be expected for FLAC where the strategy for better compression is not "pack further", but "try something else and pick the best".

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #19 – 2024-07-01 03:06:40

Hi folks,

Something got me hooked up in this thread.

Quote

FOUND ISSUE:
FLAC -3 ratio is 46% on audiobooks while flac -2 and flac -4 are below 30%.

Quote

You mention that FLAC -3 does bad on audiobooks. Not at all surprising: -0 and -3 force dual mono encoding. Why is that "0 and 3 and nothing in between"? Because 0, 1, 2 restrict to a fixed set of predictors, and even if you have some chip that cannot do multiplication, you can still decode those files. -3 is the fastest variable-predictor setting.

So, we do have an issue or is that a feature (the restriction)? If it's not an issue, it should be explained in the main post.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #20 – 2024-07-01 04:56:49

@guruboolez Thanks so much for putting together this amazing test and presentation! I think a lot of times people will choose a lossless compressor based on features and compatibility (and possibly license) because lacking something there can be a deal-breaker. However, beyond that, the next obvious consideration is compression, because basically that is the whole point!

Speed can obviously be a consideration too, but I agree that it is less important in most situations unless it becomes absurdly slow. And encoding speed is less important that decoding speed because, as you say, it’s only ever done once. Decoding complexity can affect battery life for portable playback and “snappiness” in DAW programs, and so is worth considering in those niche scenarios.

The first surprise for me here was the results for “audiobooks” because any day that WavPack is competitive with Monkey’s Audio is a good day!

And it’s definitely odd that the FFmpeg implementation does so poorly compared to the reference encoder, but my guess is that it’s because FFmpeg doesn’t default to “optimize mono”, which many audiobooks must benefit from. Unfortunately the only audiobook I have (Malcolm Gladwell’s Outliers) doesn’t trigger that mode and compresses slightly better with FFmpeg.

What wasn’t obvious at first was the superior performance of FFmpeg’s WavPack encoder compared to the reference on multichannel, so I examined that in detail and ran some experiments. A couple of differences showed up immediately when encoding 24/48/5.1 material. The first is that the reference encoder puts 12,000 samples in each frame whereas FFmpeg puts 24,000 samples (which is more efficient because it reduces frame overhead). The second is that the reference encoder encodes the center and LFE channels into independent frames whereas FFmpeg encodes them into a stereo frame (just like the two front and two surround channels are). Again, this reduces frame overhead, but can also hurt compression because it’s interleaving completely unrelated information.

To figure out whether these were contributing factors, and to what degree, I used the reference encoder to simulate what FFmpeg was doing. That was easy with the frame size because the –blocksize option allows specifying this directly. Unfortunately there was no easy way to do the channel paring because WavPack has no override for the default behavior which is to pair the naturally paired channels (e.g., left and right) and put everything else in mono frames. The only options for pairing (--pair-unassigned-chans) only applies to unassigned channels and there’s no way to override the specified channel mask to make the channels unassigned except in raw PCM mode (which would have worked but doesn’t copy tags). I ended up hacking the command-line program to do this.

Here are the results for my 887,328,104 byte test file (Dogs from Pink Floyd's Animals, sorry @Porcus

), comparing the specified configuration to the unmodified reference encoder (using -hhx6 for all reference configurations):

configuration	improvement (bytes)	improvement (ratio)
reference --hhx6	0	0.000%
pairing FC+LFE	269510	0.030%
FFmpeg-3	294772	0.033%
blocksize=24000	835170	0.094%
FFmpeg-4	840858	0.095%
FFmpeg-5	922546	0.104%
pairing & blocksize	973190	0.110%
FFmpeg-6	990022	0.112%
FFmpeg-7	1038122	0.117%

As you can see, when matching the channel pairing and frame size, the reference WavPack encoder falls in between FFmpeg compression levels 5 and 6, which is pretty typical, so I think that mystery is pretty much solved (pairing may help more in other samples).

Just to clarify questions from @Porcus , there is no encoding difference between the “back” channels and the “side” channels (we just remember which it was). The FFmpeg behavior seems fixed to encoding multichannel streams as paired stereo frames, regardless of the actual channel configurations. I think this is a little weird in some cases (e.g., center and LFE) but it turns out to provide almost identical compression (see above). It’s definitely more weird in lossy / hybrid mode, but of course FFmpeg doesn’t do that.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #21 – 2024-07-01 07:50:03

So, channel decorrelation. WavPack:

Quote from: bryant on 2024-07-01 04:56:49

As you can see, when matching the channel pairing and frame size, the reference WavPack encoder falls in between FFmpeg compression levels 5 and 6, which is pretty typical, so I think that mystery is pretty much solved (pairing may help more in other samples).

Of course (assuming file format 5 supports it) one can try both to pair or not to pair, at least for x4 up where the user has actively chosen to spend time and effort. Both unassigned and "assigned but not paired in 5.70".
But pairing did only improve 0.03%, sooo ... of course there is a mileage-may-vary and it could be bigger here and there, but

Block size matters more, but I guess there are good reasons to limit that to a quarter of a second (which also is TAK's max, by the way!) over half a second. Hunch, if someone sets up a streamserver over a dubious line or something?
If so, it makes sense for WavPack to be careful (compared to the heavier formats that cannot be streamed this way anyway), and a quarter of a second is also much more data for multichannel.

FLAC:

Quote from: krafty on 2024-07-01 03:06:40

So, we do have an issue or is that a feature (the restriction)? If it's not an issue, it should be explained in the main post.

Strictly speaking a feature, even if it might not be perceived as one > 20 years later. For all that you know, someone has embedded a dual-mono-only FLAC decoder into a canned player and encodes everything with -3 ... and also, who would use -3 if not for that? Oh yeah, those of us who do testing

But OTOH, caveat emptor. Users would indeed think that the presets are ordered by speed/compression trade-off, but -1&-2 vs -3 are not.

So, ... maybe not "Issue", but "You might want to know".

BTW as speed goes, it seems -0 could be made faster by boosting block size.

Re: Lossless Compression Test (multiple genres + high resolution + multichannel)

Reply #22 – 2024-07-01 14:53:04

Quote from: Porcus on 2024-07-01 07:50:03

But OTOH, caveat emptor. Users would indeed think that the presets are ordered by speed/compression trade-off, but -1&-2 vs -3 are not.

They were, 20 years ago. But CPUs changed, they became "more superscalar" and things like out-of-order execution and branch prediction became way more potent. Also, FLAC changed, all code got faster but some parts more than others. So, the presets aren't ordered by speed/compression anymore. They never really were that strictly, just in general.

However, there is little reason to change the presets are people are accustomed to certain presets means certain things.

Notice