HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: Wombat on 2014-06-27 16:55:48

Title: 24-bit vs. 16-bit Audio listening test
Post by: Wombat on 2014-06-27 16:55:48
I didn't see it mentioned here on hydrogen.
Archimago did a nice job preparing a simple test with offering files, collecting data and some conclusion!
INTERNET TEST: 24-bit vs. 16-bit Audio - Can you hear the difference? (http://archimago.blogspot.de/2014/04/internet-test-24-bit-vs-16-bit-audio.html)
Procedure (http://archimago.blogspot.de/2014/06/24-bit-vs-16-bit-audio-test-part-i.html)
Results and more (http://archimago.blogspot.de/2014/06/24-bit-vs-16-bit-audio-test-part-ii.html)

It is a very ambitious and fun to follow project by a hobby music lover that deserves some compliment no matter what some fault-finders will cite.
Title: 24-bit vs. 16-bit Audio listening test
Post by: ferday on 2014-06-27 20:09:31
Hey thanks for posting, I completed the test right around first announcement, and totally forgot to check in to see the results!

For completeness I did a lot of trials with varying equipment (even though i knew that i would personally not be able to tell them apart ha ha) and it was a lot of fun.

the results are exactly what most here would've guessed they should be...

Props to archimago for his efforts, his blog is getting better all the time and is always a fun read
Title: 24-bit vs. 16-bit Audio listening test
Post by: lithopsian on 2014-06-27 21:58:54
Cool.  I was impressed by the cost of the systems being used!
Title: 24-bit vs. 16-bit Audio listening test
Post by: TomasPin on 2014-06-27 22:04:33
Wow, this is really neat. Not surprised by the 50/50 results (what you'd call guessing), of course.

Thanks Archimago for carrying this out, will check out his blog. And thanks for letting us know, Wombat.
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-06-27 22:08:29
High confidence responses: 50%, 49%, 44% right.


Title: 24-bit vs. 16-bit Audio listening test
Post by: TomasPin on 2014-06-27 23:01:07
High confidence responses: 50%, 49%, 44% right.

Oh, right. That's what I get for skimming through it.
Title: 24-bit vs. 16-bit Audio listening test
Post by: Audible! on 2014-06-28 00:43:18
This just goes to show that Barry Diament's claims about 24/96 audio being insufficiently high resolution are unequivocally true
Title: 24-bit vs. 16-bit Audio listening test
Post by: Hotsoup on 2014-06-28 01:48:52
These participants should've had their wives or girlfriends take the test for them from in the kitchen or whatever other nearby room. I hear that's where the sweet spot is since they can always tell the difference.
Title: 24-bit vs. 16-bit Audio listening test
Post by: IgorC on 2014-06-28 02:04:33
Imo the test would be more useful if low and middle anchor would be included. 10 bits, 14 bits or similar. Both dithered and noise shaped.
Title: 24-bit vs. 16-bit Audio listening test
Post by: Hotsoup on 2014-07-01 20:11:31
Discussion thread about this test already closed by mods on SHTV, probably emotions running too high. Some of it is amusing.

Can You Hear the Difference @ SHTV (http://forums.stevehoffman.tv/threads/internet-test-can-you-hear-the-difference-between-24-bit-and-16-bit-audio.352185/)

Title: 24-bit vs. 16-bit Audio listening test
Post by: Kohlrabi on 2014-07-01 20:37:47
Imo the test would be more useful if low and middle anchor would be included. 10 bits, 14 bits or similar. Both dithered and noise shaped.
Why? the question was whether people can spot a difference between 16/44.1 and 24/96 audio, and the answer seems to quite clearly support the null hypothesis that there is no audible difference. A "low" anchor at 14 bit might have only embarrassed audiophiles even more. 
Title: 24-bit vs. 16-bit Audio listening test
Post by: Hotsoup on 2014-07-01 20:58:12
..the question was whether people can spot a difference between 16/44.1 and 24/96 audio..
I think he kept the sample rate at 96, just changed the bit depth. At least that's how I read it.
Quote
The dithering process was basic. Using an older version of Adobe Audition (version 3.0.1), a flat triangular dither of 0.5 bits was utilized with settings as shown:

(http://4.bp.blogspot.com/-EcO9SxYS9Rg/U6PJIsGNi_I/AAAAAAAADLc/WAQzb6sxxD8/s1600/Dither.png)
Quote
The sample rate was kept at 96kHz. These are very conservative settings and no advanced settings like noise shaping was utilized as featured in some of the "better" dithering algorithms like iZotope's MBIT+ or Weiss' POWr, etc. Adobe Audition again was used to convert the dithered 16-bit data back to a 24-bits container.

The 24-bit and (effective) 16-bit versions were randomly assigned as Sample A or B and files were enumerated 1 to 6 in the final package downloaded by the respondents.

Title: 24-bit vs. 16-bit Audio listening test
Post by: yourlord on 2014-07-01 21:33:38
Discussion thread about this test already closed by mods on SHTV, probably emotions running too high. Some of it is amusing.

Can You Hear the Difference @ SHTV (http://forums.stevehoffman.tv/threads/internet-test-can-you-hear-the-difference-between-24-bit-and-16-bit-audio.352185/)


Quote
Grant

    daglesj said: ?
        So erm how many of you here have speakers etc. that have a frequency range higher than 22Khz?


    Being able to hear those frequencies isn't the point. What happens in that area of the spectrum directly affects the fundamentals that we do hear.


That made my brain hurt..If the source frequencies above 20KHz affected the fundamentals in the audible range, then a 44.1kHz sample rate will capture and reproduce that affect because it exists in the audible range below 22kHz.. Ow...

Is it really that hard to understand that you don't have to reproduce a 30kHz tone to reproduce it's effects on the signal within the audible range? You captured those audible effects when you sampled the source.. OW!! Make it stop!!
Title: 24-bit vs. 16-bit Audio listening test
Post by: krabapple on 2014-07-01 22:49:22
Discussion thread about this test already closed by mods on SHTV, probably emotions running too high. Some of it is amusing.

Can You Hear the Difference @ SHTV (http://forums.stevehoffman.tv/threads/internet-test-can-you-hear-the-difference-between-24-bit-and-16-bit-audio.352185/)


Quote
Grant

    daglesj said: ?
        So erm how many of you here have speakers etc. that have a frequency range higher than 22Khz?


    Being able to hear those frequencies isn't the point. What happens in that area of the spectrum directly affects the fundamentals that we do hear.


That made my brain hurt..If the source frequencies above 20KHz affected the fundamentals in the audible range, then a 44.1kHz sample rate will capture and reproduce that affect because it exists in the audible range below 22kHz.. Ow...

Is it really that hard to understand that you don't have to reproduce a 30kHz tone to reproduce it's effects on the signal within the audible range? You captured those audible effects when you sampled the source.. OW!! Make it stop!!



No, don't you see?  The ultrasonic content has to be there in real time, otherwise the brain magic doesn't happen.  ;>

Did they bring up Oohashi et al at all? 




Title: 24-bit vs. 16-bit Audio listening test
Post by: IgorC on 2014-07-02 00:30:30
Imo the test would be more useful if low and middle anchor would be included. 10 bits, 14 bits or similar. Both dithered and noise shaped.
Why? the question was whether people can spot a difference between 16/44.1 and 24/96 audio, and the answer seems to quite clearly support the null hypothesis that there is no audible difference. A "low" anchor at 14 bit might have only embarrassed audiophiles even more.

Hehehe. Yes, if 14 bits with excelent quality dithering and noise shaping would be very hard (or even impossible for almost all people ) to spot so they would realize how it's hopeless trying to hear any tiny difference between 16 and 24 bits.
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-07-02 01:34:18
To have any effect during playback you need lots of energy up there... which just is not there unless you do lots of nonlinear processing (compression etc.).
Title: 24-bit vs. 16-bit Audio listening test
Post by: 2Bdecided on 2014-07-02 10:13:23
I think he kept the sample rate at 96, just changed the bit depth. At least that's how I read it.
Quote
The dithering process was basic. Using an older version of Adobe Audition (version 3.0.1), a flat triangular dither of 0.5 bits was utilized with settings as shown:

(http://4.bp.blogspot.com/-EcO9SxYS9Rg/U6PJIsGNi_I/AAAAAAAADLc/WAQzb6sxxD8/s1600/Dither.png)
Wow. If AA3 still works like CEP in this respect, that's not even proper dither. You need a dither amplitude of 1 for it to work properly.

Can anyone with AA3 check?
New file: 96kHz stereo 32-bit.
Generate tones: 1Hz, no modulation, only the fundamental, -80dB both channels, sine wave, duration 10 seconds.
Convert sample type: 96kHz, stereo, 16-bit, enable dithering, dither depth = 0.5 bits, triangular, no noise shaping.
Amplitude: amplify: +70dB

Listen to the result. If the dither was working, you'd hear continuous white noise. When I do the above, the white noise pulses, meaning the dither is broken.

If you do the above but with dither depth = 1.0 bits, you get continuous white noise at the end.


I'm not claiming the difference is ever audible, or even detectable with any normal audio source - but it's interesting that the test gave the best possible chance of detecting the 16-bit version by using no noise shaping and broken "dither".

Cheers,
David.
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-07-02 11:20:52
It's not broken, it is just that the amplitude of the dither is so low that you get a mix of dither noise and quantization error.
Title: 24-bit vs. 16-bit Audio listening test
Post by: Wombat on 2014-07-02 14:55:47
I think he kept the sample rate at 96, just changed the bit depth. At least that's how I read it.
Quote
The dithering process was basic. Using an older version of Adobe Audition (version 3.0.1), a flat triangular dither of 0.5 bits was utilized with settings as shown:

See also If stupid dither is already enough... (http://forums.slimdevices.com/showthread.php?101742-INTERNET-TEST-24-bit-vs-16-bit-Audio-Closed-and-Results-Being-Posted-Soon&p=784194&viewfull=1#post784194)
I tried several times to play with low dither and on normal music i can't tell the diffeence, sometimes even truncated. When you amplify silent parts you normaly hear noise and adding 0.5 does sound clean on these samples imho. Also choosing the lower amplitude may made it harder for cheaters on this test. So i suggested to use a low amount to exaggerate
btw. anyone can convince the SoX team to implement Dither Depth?
Title: 24-bit vs. 16-bit Audio listening test
Post by: eahm on 2014-07-02 15:13:07
That made my brain hurt..

On an Italian forum they are talking about frequencies that you don't hear but have benefits on your body anyway so better to keep high quality lossy files or lossless ...and that's why ABX test are useless.
Title: 24-bit vs. 16-bit Audio listening test
Post by: castleofargh on 2014-07-02 15:46:27
That made my brain hurt..

On an Italian forum they are talking about frequencies that you don't hear but have benefits on your body anyway so better to keep high quality lossy files or lossless ...and that's why ABX test are useless.

somewhere in a book they are talking about UV frequencies that you don't hear but help for vitamin D that have benefits on your body anyway so better to keep UV in our music... and that's why ABX tests are useless.

you were so close to making an actual point. if only one part of the argument had like ... any relation with the rest, then it might have been actual reasoning. reason is one of those nifty logic tricks you can use to convince people. better luck next time.
Title: 24-bit vs. 16-bit Audio listening test
Post by: 2Bdecided on 2014-07-02 16:53:22
It's not broken, it is just that the amplitude of the dither is so low that you get a mix of dither noise and quantization error.
No, I know it's not computationally/algorithmically broken, but in the context of Lipschitz and Vanderkooy mathematically correct dither it's broken!

I didn't want to ever say so but this was my idea, Archimago wanted to use Noise Shaped dither at first  See also If stupid dither is already enough... (http://forums.slimdevices.com/showthread.php?101742-INTERNET-TEST-24-bit-vs-16-bit-Audio-Closed-and-Results-Being-Posted-Soon&p=784194&viewfull=1#post784194)
I tried several times to play with low dither and on normal music i can't tell the diffeence, sometimes even truncated.
Wombat, I invented LossyWAV; you don't need to convince me that truncation can be inaudible

I love it when people excuse their inability to ABX the unABXable on a lifetime attending rock concerts.

My hearing is fine for my age, but poor for (say) an 18 year old. Even so, in many ways I find myself far more critical than I used to be. This is with audible differences below 15kHz though, not inaudible ones.

Cheers,
David.
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-07-02 19:30:11
you were so close to making an actual point


I think that he was just relaying that ridiculous point, because it fits to "made my brain hurt".
Title: 24-bit vs. 16-bit Audio listening test
Post by: drfisheye on 2014-07-02 20:46:11
I wonder if classical music is really the best for these tests. Classic recordings are often recorded with the mics standing far from the instruments, giving a rather undetailed sound IMO. I much prefer up close recordings. Like you find in jazz and pop music (excluding the overly loud recordings).

Not that I would expect a different outcome in this test. I just wouldn't pick classical music as example for the best recorded music.
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-07-02 22:31:59
For these kind of tests you need high dynamic range. Classical is possibly one of the genres with highest DR.
Title: 24-bit vs. 16-bit Audio listening test
Post by: drfisheye on 2014-07-03 00:37:33
For these kind of tests you need high dynamic range. Classical is possibly one of the genres with highest DR.

Numbers 2 and 3 of the test aren't very dynamic. The Goldberg variation is just a piano playing on the same level recorded from a distance with loads of reverberation, blurring the detail.

Let's put it this way, if you want to see a lot of detail of something you take a close look, you don't take a distance.

(BTW, can't we just measure the DR of an actual concert so we know what maximum DR is needed? I'm pretty sure the DR in a concert hall doesn't exceed 96db above the noise floor?)
Title: 24-bit vs. 16-bit Audio listening test
Post by: Wombat on 2014-07-03 01:43:23
I wonder if classical music is really the best for these tests. Classic recordings are often recorded with the mics standing far from the instruments, giving a rather undetailed sound IMO. I much prefer up close recordings. Like you find in jazz and pop music (excluding the overly loud recordings).

Not that I would expect a different outcome in this test. I just wouldn't pick classical music as example for the best recorded music.

Depends who you ask. If asking where high bitrate helps it is very often argued with classic music needing it. The label samples of this test are from sell these in HiBits exactly for that reason and are well regarded afaik. The piano sample at least it is a Bösendorfer. If that doesn't need its timbre preserved...
Title: 24-bit vs. 16-bit Audio listening test
Post by: xnor on 2014-07-03 01:44:08
Let's put it this way, if you want to see a lot of detail of something you take a close look, you don't take a distance.

Not really. Low bit depth causes lower dynamic range so you need dynamic material. If the bit depth is too low you should hear distortion+noise (due to the 0.5 bits of dither used).

The files are highly dynamic, up to about 70 dB (difference between min and max RMS with 50ms window) except for Vivaldi, which is more like 45 dB. That is excluding the fade in/out silence.


Quote
(BTW, can't we just measure the DR of an actual concert so we know what maximum DR is needed? I'm pretty sure the DR in a concert hall doesn't exceed 96db above the noise floor?)

More like 70 - 80 dB max.
Title: 24-bit vs. 16-bit Audio listening test
Post by: ajinfla on 2014-07-03 20:01:50
This test is invalid because you can't hear both tracks at the same time to compare. Aural memory is way shorter than you think.
Plus I find it extremely stressful to sit in the recliner listening to classical music, without seeing and knowing whether its a CD or SACD. Not surprising no difference heard under such duress. I prefer Pink Floyd anyway. The electric guitars sound more natural and organic in 24bits.
Title: 24-bit vs. 16-bit Audio listening test
Post by: lvqcl on 2014-07-03 20:05:48
So, "without [...] knowing whether its a CD or SACD" = "duress". 

The electric guitars sound more natural and organic in 24bits.

Re-read TOS#8.
Title: 24-bit vs. 16-bit Audio listening test
Post by: yourlord on 2014-07-03 20:25:09
Your sarcasm detector is broken..
Title: 24-bit vs. 16-bit Audio listening test
Post by: lvqcl on 2014-07-03 20:39:19
Really? Well, English isn't my native language, so I don't always recognize it with the absence of [sarcasm] tag...
Title: 24-bit vs. 16-bit Audio listening test
Post by: pdq on 2014-07-03 20:50:29
ajinfla seems to make a habit of posting in a way that it is difficult to tell what is his real point of view.
Title: 24-bit vs. 16-bit Audio listening test
Post by: Arnold B. Krueger on 2014-07-15 15:02:29
This test is invalid because you can't hear both tracks at the same time to compare.


That has been proposed and tried as well.  It is very hard to obtain signficant isolation between the alternatives, so its not clear to the listener what the listener is listening to.

Quote
Aural memory is way shorter than you think.


I think the Science of that is pretty well understood, with good recolection of sonic details extending for at least a few seconds.

Quote
Plus I find it extremely stressful to sit in the recliner listening to classical music, without seeing and knowing whether its a CD or SACD.


I dunno. Would a hovering salesman help your relaxation? I hear that people are able to reliably determine all kinds of differences in audio stores...

Quote
Not surprising no difference heard under such duress. I prefer Pink Floyd anyway. The electric guitars sound more natural and organic in 24bits.


What said only classical could be used?

One of the best publicized hi-rez related DBTs around by Meyer and Moran involved a mixture of classical and popular works.

I still think the best DBT was the one that the music industry unintentionally organized by mastering DVD-As and SACDs from a hodge podge of new and legacy recordings that themselves were limited by all sorts of different formats including analog tape. Eventually some objectivists fired up their FFTs saw the obvious band limiting and blew the whistle.  I can find no example of any subjectist reviewer pointing this out in a published review, so we must presume that none of them heard any differences.  That DBT must have extended over 10 years or more.



SimplePortal 1.0.0 RC1 © 2008-2021