Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Lossless codec comparison - part 3: CDDA (May '22) (Read 5595 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #25 –
Also, given that an iPod Video (dual-core ARMv4 at 80 MHz) running Rockbox as it existed circa 2010 can decode TTA, ALAC,  and some Monkey's Audio files in realtime (as well as FLAC at about 6x realtime) a lot of this is basically a rounding error on anything approaching a modern CPU.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #26 –
Oh, a lot of this isn't noticeable in practice - or, dwarfed by other phenomena, like drive write speed and how much time the software spends doing tag transfers afterwards, for that matter.
But if you use Monkey's Insane (well, Matt Ashland called it "insane", you were warned!), you might have to wait a while for transcodes, and for scanning for ReplayGain or dynamic range or maybe acoustic ID for tagging, or for bitcomparisons ... if you do many at once.

 (I have asked before if battery life is much affected anymore ... maybe it isn't? Back in the Rockbox days, you would enjoy more battery life using FLAC than using MP3.)

Filesize isn't as much of a concern anymore either - unless your SSD or memory card is running full. If you don't have any more lossless files than can fit on your work laptop (or your phone!) you have an entire extra backup that way.

But the sport is still fun to watch.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #27 –
So how much of the news is the change in corpus? Of course it moves all sizes downwards, but it also made TAK overtake the ape.

Some of the codecs have improved their compression (like FLAC), some do not - a Monkey's file at -cN000 is the same no matter what version. WavPack seems pretty much the same (slightly bigger due to the frame-checksums that make for fast verification, countering the very small mono-as-stereo part of the corpus) - and looking back at the TAK 2.3.1 and 2.3.2 release notes and tests, the size differences are very small for CDDA, we are talking a hundredth of a percentage point or so.

So - by rough eyeballing, not precise calculations - this is a bit on how the corpus matters:
 * ktf's Revision 5 corpus compress to slightly below 50 percent size for -p4m and Insane. "Revision 4" compressed to around 54. So this corpus is around 8 percent (not points!) "more compressible".
 * Revision 4: TAK -p4m produced files about 1 percent larger than Monkey's Insane. In Revision 5, it narrowly beats every Monkey. So the change in corpus has made for
 -> 9 percent smaller files on TAK -p4m
 -> 8 percent smaller files on Monkey's Insane.
Question arises, is any of those figures off compared to the rest? WavPack, ALAC and LA (all the default settings) make for slightly above 8 as well - again by eyeballing. And the codecs don't happened to have changed much. Also, presuming that revision 4 has a WavPack -h in there (it quotes four WavPack settings but has tested six!), we can also make sense out of what happens between WavPack default and -h:
 -> Like Monkey's, TTA makes the same files no matter what version (including using ffmpeg!); it was around TAK  -p0m, slightly better than any FLAC, between WavPack default and -h which in turn was around Monkey's fast; now overtaken by TAK -p0whatever and FLAC -678 (FLAC has improved!).

Which does indicate that well, the revision 5 corpus is "more TAK friendly". But
 * That does not mean this is a "biased towards TAK corpus"; it could be that it is less biased than the the narrower revision 4 corpus.
 * The "only" thing that makes for suspicion towards this corpus is that it is broad and unweighted and brings file size down to the 50 percent mark, and that looks a bit low - but that is viewed through the biased eyes of a metalhead.
 * Is it by much? ~ 8 percent smaller files for everything, with ~8 percent-of-those-8-percent better for TAK? Let's stop here and think for a little: how much faster is running a 100m flat on 9.92 vs 10.00? Quite a lot if you are chasing olympic medals, pretty much nothing for practical purposes. The reason it is striking is that Monkey's was the to-go high-compression codec of 2005, and "whoah, you beat Monkey's Insane" with a fast asymmetric codec is still raising eyebrows.


All this on CDDA. The success of WavPack -hx4 on hi-res shows it is a slightly different animal. For example.


And, concerning this "noticeable in practice": I see from my TAK 2.3.1 testing that it would decode like 20 percent faster than 2.3.0 even when reading from spinning HDD. Whether that is "noticeable" ... up to opinion.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #28 –
Quote from: Porcus on
So how much of the news is the change in corpus?
I guess that's hard to tell in general. I could have chosen to take the same corpus as last time to be able to compare, but that would propagate any advantages certain codecs have on certain material. I instead chose to make a 'fresh' start.

One could argue the inclusion of the single instrument material and the number of orchestral sources provide an advantage to codecs that do well with tonal content, and the addition of exotic material (like the chiptune and microsound sources) provide an unfair advantage to certain codecs that just happen to perform well there. I think I made a balanced corpus including a wide variety of material one would want to losslessly encode, not just music. If we would continue to compare codecs based on sources that were likely similar to the ones used in tuning the codec, we won't learn anything new.

Also, it seems reasonable to assume asymmetric codecs are better suited to deal with material that is different than the usual. See for example Ryodi Ikeda - Dataplex, which I think is very different from what would most people would consider music. Wavpack -x4, FLAC, ALAC, TAK, Shorten and ALS, all asymmetric codecs, do much better compared to symmetric codecs like WavPack, Monkey's Audio, TTA and LA. OptimFROG, being a bit of both, also very well here. That's why I think it is important to include material that is off the beaten path. I didn't do that (as much) in the previous revisions.
Music: sounds arranged such that they construct feelings.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #29 –
Quote from: ktf on
Quote from: Porcus on
So how much of the news is the change in corpus?
I guess that's hard to tell in general. I could have chosen to take the same corpus as last time to be able to compare, but that would propagate any advantages certain codecs have on certain material. I instead chose to make a 'fresh' start.

* Hm, since I am too lazy to check: Is the Revision 4 corpus a subset of the Revision 5, or did you also remove signals?

* Also, since I cannot check: the Pokémon album you have both as hi-res and as CDDA. Are they ... what we in music would speak of loosely as "the same mastering" (in that the difference would be very low volume if the 192 were carefully resampled to 44.1)? The results are a bit different between 192 and 44.1

* What was the recording chain of your diffuse sound fields recording? Is there any suspicion about that phenomenon "2" that TBeck points out in reply #12?

* I looked at the CDDA results the other day with fresh eyes, and there are some "unexpected" (well maybe not after Revision 4): In line with what you point out about where asymmetric codecs shine, it seems that the symmetric benefit at "denser" music. (Also that is how LA benefits vs OptimFROG?). But then at the most noisy, this Merzbow track fools the ape - and so also with the Merzbow album you included, although it isn't the least compressible in your corpus.
... it wouldn't be hard to visualize, if one is bothered to do the work: sort the albums by compressibility (say averaging some codecs at their max setting to order the signals) and present how TAK -p4m vs Monkey's Extra/Insane do against each other, say per decile/quartile. And what frog level it takes to beat them (and LA!).

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #30 –
Quote from: Porcus on
* Hm, since I am too lazy to check: Is the Revision 4 corpus a subset of the Revision 5, or did you also remove signals?
I started mostly from scratch, only adding sources from revision 4 when I couldn't find better alternatives. Most revision 4 sources did not return in revision 5.

Quote
* Also, since I cannot check: the Pokémon album you have both as hi-res and as CDDA. Are they ... what we in music would speak of loosely as "the same mastering" (in that the difference would be very low volume if the 192 were carefully resampled to 44.1)? The results are a bit different between 192 and 44.1
They were generated from the same programs with the same emulator but with different settings.

Quote
* What was the recording chain of your diffuse sound fields recording? Is there any suspicion about that phenomenon "2" that TBeck points out in reply #12?
Sounds were recorded with a Zoom H4n and normalized in Audacity. So it it highly likely that OptimFROG benefits from the mentioned holes, yes.

Quote
it seems that the symmetric benefit at "denser" music.
Could you give an example, I don't see that in the data.

If you take The Ambient Visitor, Bobby McFerrin and Bach, which compress very well, I see a difference between FLAC -8 and OptimFROG max of about 4%-point, or about 10%. Looking at Jeroen van Veen that is about 6% point or > 20%.  However Skrillex and Merzbow see about 3% point difference, but because these compress much less, this translates into a difference of only about 4%. So, I think I see the opposite trend?
Music: sounds arranged such that they construct feelings.

Re: Lossless codec comparison - part 3: CDDA (May '22)

Reply #31 –
Actually it could be a point to distinguish "CDDA" between on one hand "CD rips" as in "music from commercial CDs or 44.1/16 stereo downloads" and "other sources converted to 44.1/16".

As for this:

Quote from: ktf on
Quote
it seems that the symmetric benefit at "denser" music.
Could you give an example, I don't see that in the data.

If you take The Ambient Visitor, Bobby McFerrin and Bach, which compress very well, I see a difference between FLAC -8 and OptimFROG max of about 4%-point, or about 10%. Looking at Jeroen van Veen that is about 6% point or > 20%.  However Skrillex and Merzbow see about 3% point difference, but because these compress much less, this translates into a difference of only about 4%. So, I think I see the opposite trend?

Well if you were to take white noise, all would evaluate to 100 percent, and you could then say it benefits the asymmetric because they are generally not so efficient and here they are equal. So that is a point.

What made me suspicious about this - and might have caught me in major confirmation bias! - is the observation that
* revision 5 has smaller files (better compression)
and
* TAK overtakes the insane ape and FLAC overtakes TTA.
So what I based my impression was how a couple of pairs that generally measure about the same do vary: high-setting TAK/Monkey's score compare to each other. Also TAK vs OptimFROG default and high-setting FLAC compared to TTA, but I have to admit I had an eye constantly on TAK/ape.

Anyway, from alphabet down, disregarding those which end up "close to 50", and the oddball signals. "*" seems to confirm my perception, "-" against it, "." well uh.
* Animals as Leaders: TTA and Monkey slightly better than FLAC and TAK
- Alea Diane: counter, against my perceived observation yes
. Krauss: would be a "*" but is maybe too close to 50 and default frog doesn't shine
- Vivaldi:
* Apocalyptica: TTA and Monkey slightly better than FLAC and TAK
. Berlage: counter for TTA, but TAK beats ape.
. Bert Kaempfert: TTA and Monkey slightly better than FLAC and TAK, but this is close to average
* McFerrin: TAK beats ape and default frog
. Cavallro: close to 50
* Coldplay. Although default frog does not shine, Monkey's soundly beats TAK and TTA beats FLAC
* Confido Domino Minsk. Well TAK doesn't win by much, but this isn't TTA's fave
*? Daft Punk. Maybe the ape blinded me.
skipping some odds and near-50
* Dvorak: frog and TTA not happy
. Epica: I had this as "*" because of the ape, but ... ah maybe not the others
* Equilibrium: look at ape and TTA
* Fanfare Ciocarlia: ditto but to a lesser extent
*? Fatboy Slim: maybe unfair to judge this based solely upon how well the frog fares
- Flanders recorder.
*? Fors/Bjelland. TTA does not agree, it likes this piece.
. FreeSound ambient: odd signal, I scrolled past
* Verdi: TAK beats default frog, FLAC narrowly beats TTA
- Horacio Vaggione disagrees with me
. and so would Gotovsky do except the frog isn't overly happy
. Stravinsky ditto
*? In Flames. I had this as an example due to the ape, but comparing all three I was maybe ...
* Powerslave. Of course I paid attention to what happens to that album ...
.? Jean Guillou. Would be a - hadn't it been for the frog.

That was page 32 of 64, looks like a place to stop.