Skip to main content
Topic: 24-bit codec comparisons? (Read 16036 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

24-bit codec comparisons?

Has anyone done a comparison of lossless codecs' performance on 24-bit audio?

I'm used to all lossless codecs performing about the same, but for 24-bit, this clearly isn't so; WavPack seems to work a lot better than FLAC, even though they're basically tied on CD audio.

24-bit codec comparisons?

Reply #1
Has anyone done a comparison of lossless codecs' performance on 24-bit audio?

No, please go ahead 
In theory, there is no difference between theory and practice. In practice there is.

24-bit codec comparisons?

Reply #2
I suspect that TBeck has some figures, following his TAK development and testing.

I'm not sure how large the corpus is.
I'm on a horse.

24-bit codec comparisons?

Reply #3
Any ideas for test material? I produce music, and nearly all of my 24-bit files are mixes-in-progress and stuff... very little "end product" in 24-bit. I could download some lossless DVD-A rips and 24/96 vinyl recordings, I suppose.

24-bit codec comparisons?

Reply #4
If the source was recorded, then it passed through an ADC, and ADCs generally record noise in many/most of the last 8 bits anyway, so I don't think the choice of source is all that important.  Though purely electronic music may be a bit different, since you wouldn't have thermal noise mixed in.

24-bit codec comparisons?

Reply #5
I may be missing something, but if the lower 8 bits are noise (i.e., incompressible), won't every codec perform the same on that part, thus making the comparison dependent on what's in the top 16 bits?

24-bit codec comparisons?

Reply #6
I may be missing something, but if the lower 8 bits are noise (i.e., incompressible), won't every codec perform the same on that part, thus making the comparison dependent on what's in the top 16 bits?

That's how it should be, but it turns out that it's not always the case.

With FLAC there is a limitation in the size of the values that can be represented in the Rice coder, so when the residuals get too large the encoder starts storing values without any compression. I also seem to remember that Monkey's Audio also has some limitation with large magnitude samples because of its MMX usage, although this may have been fixed in newer versions.

So, when testing 24-bit capabilities it would be good to have some loud samples and some with lots of high frequencies to make sure the codecs are exercised fully.

24-bit codec comparisons?

Reply #7
I would actually like to test out some of the lossless codecs to post here as I have some 24-bit rips. Before I do, I wanna make sure I have everything I need...

I plan on testing: WavPack, FLAC, Monkey's Audio, and TAK. Time permitting, OptimFROG and LA.
As far as I understand, I just need to use the tool timethis.exe to get encoding time? Do I need anything else? If not, I'm going to start working on this very soon.

24-bit codec comparisons?

Reply #8
I plan on testing: WavPack, FLAC, Monkey's Audio, and TAK. Time permitting, OptimFROG and LA.

LA doesn't support 24-bits, so you can take that off your list. You might consider replacing it with TTA because it handles 24-bits, and it also handles 32-bit floats in case you considered testing that also.

I look forward to seeing your results! 

24-bit codec comparisons?

Reply #9
As far as I understand, I just need to use the tool timethis.exe to get encoding time? Do I need anything else? If not, I'm going to start working on this very soon.
I would recommend TIMER.EXE, rather than TIMETHIS.EXE.  TIMER.EXE allows you to report CPU-only times, rather than CPU+IO times, which TIMETHIS.EXE will only report. TIMER.EXE reports both ("Global Time" = CPU+IO; "Process Time" = CPU-only).

This thread may provide some further considerations.
I'm on a horse.

24-bit codec comparisons?

Reply #10
Any ideas for test material? I produce music, and nearly all of my 24-bit files are mixes-in-progress and stuff... very little "end product" in 24-bit. I could download some lossless DVD-A rips and 24/96 vinyl recordings, I suppose.

Well, there is a lot of 24 bit material available from live recording crowd. Especially on archive.org in the live music section. Most of it audience records made on 722's. But a few bands allow soundboard 24 bit plugins. If you goto archive.org check you can find 24bit soundboards from Younder Mountain String Band, Dark Star Orchestra or Ryan Adams(plus a few others here and there). You can find some good 24 bit audience recordings from Bela Fleck, String Cheese Incident plus quite a few others.


24-bit codec comparisons?

Reply #12
This thread may provide some further considerations.
Thank you, I really appreciate that Synthetic Soul! You're guide will help me out a lot
No problem.  It's just one way of doing things, but I hope there will be some information that may help.

If you have any further questions reagarding my processes I would suggest posting to that thread (I am tracking it) so that everyone can benefit from my infinite wisdom.  Feel free to PM concerning subjects that may not be relevant to that thread though.
I'm on a horse.

24-bit codec comparisons?

Reply #13
I have also noticed this myself. 

One day last week I was comparing the decode speeds (and file sizes) of FLAC (-0), TAK (turbo), and WavPack (fast) in Foobar.  When I used a 16/44.1 file, obviously FLAC had the fastest decode speed (but much larger file), followed by TAK, and finally came WavPack (which had a slightly smaller file size than TAK, but was fairly slower on the decode speed).  I started to consider converting all my whole audio collection from WavPack to TAK even though I knew this would take a few hours of encoding time.

But then I started to test some of my 24/96 files.  To my great surprise, WavPack not only had the smallest file size, but it also had the fastest decoding speed (by a pretty decent margin).  FLAC (which I thought would come in first) was runner up (again with the largest file size), with TAK coming in dead last (although much better file size compared to FLAC, though still bigger than WavPack).

Even though I would like to see better decode speeds at 16/44.1, I decided to stick with WavPack as my codec of choice (besides, David is a stand up guy  ).  I was very impressed with how well WavPack scales, and conversely disappointed with how bad FLAC/TAK scales.  Is this bad scaling just an inherit flaw in the FLAC/TAK algorithm or does this just happen to be one of WavPack's strengths (or maybe a mix both)? 

Now if David would just suck it up for the team and sleep with some of the iAudio engineers, we could possibly get official WavPack support on the upcoming Cowon A3.  Then life would be great!

24-bit codec comparisons?

Reply #14
But then I started to test some of my 24/96 files.  To my great surprise, WavPack not only had the smallest file size, but it also had the fastest decoding speed (by a pretty decent margin).  FLAC (which I thought would come in first) was runner up (again with the largest file size), with TAK coming in dead last (although much better file size compared to FLAC, though still bigger than WavPack).

Even though I would like to see better decode speeds at 16/44.1, I decided to stick with WavPack as my codec of choice (besides, David is a stand up guy  ).  I was very impressed with how well WavPack scales, and conversely disappointed with how bad FLAC/TAK scales.  Is this bad scaling just an inherit flaw in the FLAC/TAK algorithm or does this just happen to be one of WavPack's strengths (or maybe a mix both)?

It's always a bit dangerous to draw conclusions regarding compression performance from limited file sets. These are the results of my 24 Bit /96 Khz test corpus:

Code: [Select]
WavPack   -f      58,08 %
TAK       Turbo   54,52 %


Here TAK is performing 3,5 % better than WavPack. Possibly your files are something special. I would be interested to analyze a snippet (10 to 30 seconds) of one of your files to check what is going on. Could you please send me one via email?

For the speed: Those fast modes are usually limited by the speed of the io system (unless you have a really ultra fast hard disk array). I doubt, that TAK's codec is encoding or decoding slower than WavPack on those settings, but it's possible that TAK's io system is responsible for worse results. With 24 Bit /96 Khz TAK 1.0.1 is using quite large io buffers which can considerably decrease the speed. Funny, but i am just working on a new io sub system for TAK 1.0.2 which will use considerably smaller buffer and at least on my system achieves significantly higher speeds than V1.0.1 on high resolution audio.

  Thomas

24-bit codec comparisons?

Reply #15
Here TAK is performing 3,5 % better than WavPack. Possibly your files are something special. I would be interested to analyze a snippet (10 to 30 seconds) of one of your files to check what is going on. Could you please send me one via email?

There is one case where i would expect Wavpack to perform better than TAK: Possibly the files are not really 24 bit. If audio data with less bits (possibly because the ADC is only generating for instance 20 bit) is converted to 24 bit, the lower bits will contain constant patterns. If those patterns are beeing detected by the encoder, about 6 percent per bit can be saved. Currently TAK detects only the most common case where the low bits are all zero, while WavPack can also take advantage of other patterns. This can also result in a speed advantage.

I thought such files ("pseudo 24-Bit") are too rare to put much effort into their handling. But i may add support for other constant bit patterns to one of the next versions. It's very easy to do and very fast.

24-bit codec comparisons?

Reply #16
Ah I'm sorry I should have mentioned that since I don't care about encoding time (I always encode while I'm at work), I use the -fx6 switch for WavPack since it has no impact on decoding speed.  Perhaps that's why we are seeing different results (it was ever so slight, I believe the bit-rate difference was like 3000k vs 3005k).  So I'm guessing the default -f switch probably would produce a larger file.  Sorry for the confusion. 

To be honest though Tom the different file sizes between codecs doesn't bother me as much as the different decoding speeds between codecs (as long as the file sizes are close).  I can stomach the slight increased size of the TAK file (vs the WavPack file), but I couldn't stomach a ~15% drop in decoding speed (example: 75x real-time vs 64x real-time for Bohemian Rhapsody).  The new IO system in TAK 1.0.2 does sound very interesting.  I will make sure that when TAK 1.0.2 is released, I will retest and reevaluate my codec options.

Also I can also assure you it is indeed a 24/96 file (not a 20-bit file).  I ripped the 24/96 PCM Stereo files from my "Queen: A Night at the Opera" DVD-Audio disc.  I'm fairly sure they are true 24/96 files (though if I'm wrong please correct me).

I'm sorry Tom if I came off as being critical of the TAK codec (or FLAC for that matter), but I was so impressed by TAK's performance with 16/44.1 files, I was really expecting similiar results with 24/96 files too.  So you can imagine the disappointment I had when I ran my higher resolution tests for TAK.  I wouldn't worry about it though since I'm sure the overwhelming majority of files out there are 16/44.1, not 24/96.  It's really exciting to see the TAK codec develop; I wish you the best of luck.

24-bit codec comparisons?

Reply #17
Now if David would just suck it up for the team and sleep with some of the iAudio engineers, we could possibly get official WavPack support on the upcoming Cowon A3.  Then life would be great!


24-bit codec comparisons?

Reply #18
I have a suggestion for a file base. It's not perfect but it has some advantages. First, its a native 24/96 recording. Second, its readily available to everyone. Finally, it comes with the original md5 and ffp signatures so that we all know that we are dealing with the same files:

Original 24bit
http://www.archive.org/details/paf2007-06-...41v.m222.flac24
download site:
ftp://ia350640.us.archive.org/0/items/paf...41v.m222.flac24

16bit version of same:
http://www.archive.org/details/paf2007-06-...41v.m222.flac16
download site
ftp://ia350630.us.archive.org/0/items/paf...41v.m222.flac16

Here is a different recording, of the same show but in 24/48:
http://www.archive.org/details/phil2007-06...k21.nbox.flac24
download site:
ftp://ia301321.us.archive.org/2/items/phi...k21.nbox.flac24

Like wise there is a 16bit version of the above mentioned 24/48 recording:
http://www.archive.org/details/phil2007-06...k21.nbox.flac16
download:
ftp://ia350624.us.archive.org/2/items/phi...k21.nbox.flac16

Please note that these are all LEGAL FILES. Also, the ftp download locations are very fast, I average about 700k/s from my cable modem.

I'm doing an informal file size comparison of flac, wavpack, tak and ape at max compression. I'll post it in a few days.

24-bit codec comparisons?

Reply #19
I can stomach the slight increased size of the TAK file (vs the WavPack file), but I couldn't stomach a ~15% drop in decoding speed (example: 75x real-time vs 64x real-time for Bohemian Rhapsody).


Just out of curiosity, what is the practical difference between a 75x and 64x real-time decoding speed that you find so difficult to stomach?

24-bit codec comparisons?

Reply #20
Has anyone done a comparison of lossless codecs' performance on 24-bit audio?

I'm used to all lossless codecs performing about the same, but for 24-bit, this clearly isn't so; WavPack seems to work a lot better than FLAC, even though they're basically tied on CD audio.


I can whole heartedly agree with that.  I recently digitally ripped all my DVD-Audio disks and found that wavpack was much better than FLAC, sometimes I was looking at 10% extra compression.  Also Wavpack could do 88200Hz and 19200hz files, FLAC just fell over.

I use wavpack -hhx6 for everything now, it may be incredibly slow, but it gives great compression ratios.

24-bit codec comparisons?

Reply #21
Ah I'm sorry I should have mentioned that since I don't care about encoding time (I always encode while I'm at work), I use the -fx6 switch for WavPack since it has no impact on decoding speed.  Perhaps that's why we are seeing different results (it was ever so slight, I believe the bit-rate difference was like 3000k vs 3005k).  So I'm guessing the default -f switch probably would produce a larger file.  Sorry for the confusion. 

Thanks for the info! I will try this on my file sets. I have to admit, that i am not very familar with the effect of WavPack's options.

BTW: There is a similar switch for TAK: -p0e or p0m will select extra or maximum evaluation and shouldn't affect the decoding speed.

To be honest though Tom the different file sizes between codecs doesn't bother me as much as the different decoding speeds between codecs (as long as the file sizes are close).  I can stomach the slight increased size of the TAK file (vs the WavPack file), but I couldn't stomach a ~15% drop in decoding speed (example: 75x real-time vs 64x real-time for Bohemian Rhapsody).  The new IO system in TAK 1.0.2 does sound very interesting.  I will make sure that when TAK 1.0.2 is released, I will retest and reevaluate my codec options.

I hope it will help a bit. I forgot, that the speed also depends on the cpu type. TAK is for instance performing less well on a P4.

Also I can also assure you it is indeed a 24/96 file (not a 20-bit file).  I ripped the 24/96 PCM Stereo files from my "Queen: A Night at the Opera" DVD-Audio disc.  I'm fairly sure they are true 24/96 files (though if I'm wrong please correct me).

This helps to rule out the constant bits hypothesis...

I'm sorry Tom if I came off as being critical of the TAK codec (or FLAC for that matter), but I was so impressed by TAK's performance with 16/44.1 files, I was really expecting similiar results with 24/96 files too.  So you can imagine the disappointment I had when I ran my higher resolution tests for TAK.  I wouldn't worry about it though since I'm sure the overwhelming majority of files out there are 16/44.1, not 24/96.  It's really exciting to see the TAK codec develop; I wish you the best of luck.

Please don't take me wrong. I am always very excited if someone posts such results contrary to my expectations because it shows me, that i may have missed something and may have to improve TAK. I already have a nice collection of extraordinary behaving 16-bit files, but my high resolution corpus is still quite small. It's always possible that i have missed somthing.

Could you please try one of those files with -p0e, -p0m and p1m? This could help me to find the reason for the unexpected results.

  Thomas

24-bit codec comparisons?

Reply #22
Quote
Just out of curiosity, what is the practical difference between a 75x and 64x real-time decoding speed that you find so difficult to stomach?
Well my 24/96 files already reach up to a max of five cpu usage in foobar; I really don't want to go any higher than that.  But when I said "couldn't stomach" I was really talking about transcoding (remember all my files are already WavPack).  Since the files are about the same in size, why would I spend a few hours transcoding my high resolution files just to lose 15% in decoding speed?  That's what I can't stomach right now.

Quote
Could you please try one of those files with -p0e, -p0m and p1m? This could help me to find the reason for the unexpected results.
Yes I could.  I'll post back later today with results.

[EDIT]Ah sorry guys went out and saw Die Hard 4 tonight, I'll delay the testing till tomorrow.[/EDIT]

24-bit codec comparisons?

Reply #23
I use wavpack -hhx6 for everything now, it may be incredibly slow, but it gives great compression ratios.


Hmm, are you sure it's better than -hx6?

I abandoned the idea of ever using -hh a while back when I compared -hhx6 and -hx6 on some files, and found that -hhx6 was never much better, and was often a little bit worse. Those were 16-bit files, however.

24-bit codec comparisons?

Reply #24
Also Wavpack could do 88200Hz and 19200hz files, FLAC just fell over.
FLAC handles any sample rate up to 655350Hz.  probably the problem was that 88.2kHz and 19.2kHz are not in the subset so you have to use the --lax switch when encoding with flac.exe.

FLAC performance on 24bit is not tuned but will improve.  some things are already done but not released yet.

 
SimplePortal 1.0.0 RC1 © 2008-2019