Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 96 vs. 48 or 44.1 kHz sampling --> scientific test (Read 71463 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

96 vs. 48 or 44.1 kHz sampling --> scientific test

http://www.hfm-detmold.de/texts/de/hfm/eti...ten/seite1.html


http://www.hfm-detmold.de/texts/de/hfm/eti...ten/seite9.html

etc.

(seite = site / page)



it is a summary of German diploma work, written 1998, so it should be independent from companies like Zoony etc.

It was quite a good reading, I hope, somebody of the germans could translate important things, sorry, I have no time.

So, the result was:




44.1 kHz, 16 bit:

they used Tascam-DAT-Recorder DA-30 MKII as reference for CD standard, 44.1/16.

They tell, that this was picked out very clearly from the majority.

(btw, they show a graph, that says, that 16 bit has in mid frequency range a noise level, which is higher than our hearing abilities. very interesting !)

so, 44.1/16 is not sufficient.






48 kHz, 24 bit

one DAC was preferred over analogue original, but why could not be figured out. So there was difference but which ?

another DAC was able to reproduce the analogue original as best DAC of all tested, better than all the 96 kHz DACs !!!!!

in blind test it was 50/50 %, so this DAC could not be distinguihsed from original source.



96 kHz:

Dacs had worse performance than the 48 khZ ones.
but some were "beta-versions...".



Look at the graphs, especially site 9.

Sorry for this short post, but I think, it may interest you...

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #1
An dieser Stelle muss aber sicherlich berücksichtigt werden, dass eine unvoreingenommene Bewertung der Klangkriterien kaum möglich war, da sich jeder Testteilnehmer leicht ausrechnen konnte, dass hinter dem verrauschten Signal ein Wandler stecken muss. Die Bewertung, wie sie das Diagramm zeigt, ist daher sicherlich etwas überzeichnet und auf unbewusste, voreingenomme Meinungen zurückzuführen.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #2
Quote
so, 44.1/16 is not sufficient.


HA...  HA...  HA...    (w00t)  :x 
Very nice, indeed... I should get a Magnum and shoot brain out of my head when I read such "arguments"...

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #3
Well, if I understand the German right, they did illustrate that listeners could distuinguish a CD quality recording from an 'original' (how did they play back _that_ ?).

The thing is that the results seem to depend too much on the used DAC rather than on the format itself, e.g. the 48khz DAC being chosen over the 96khz ones.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #4
just a moment, it doesn't help, if you cite single parts of texts.

btw, those results are quite agreeing to main opinions of this board, there is no reason to bash that work.


@garfs posted lines:

they refer to the first test part, where i am not so clear about, if they did it blind.

well, they tell, the noise of that 44.1/16 source (DAT) was easy to listen, so they ranked 16/44.1 as worst.

btw, in the latter test, which is blind, 16/44.1 is rated worst, too, very clearly.




Well, somewhere they write clearly, that this test showed more the single characteristics of each DAC.

Compare the 24/48 DACs and the 24/96 DACs.

(at that time, 1998 !!!!, the 96/24 DACs were not perfect, not as good as the 48/24 DACs. This is one result of their test.

They write, that obviously the sound is more infleunced by the analogue in and outs circuits of the DACs, then by the digital circuits....

Well, and as they took only one example for 16/44.1, a Tascam DAT (which is used in a lot studios, the reason, why they took it, so, it cannot be so bad ?) it can be, that this DAT has perhaps a worser analogue output than other 44.1/16 DACs, who knows ?!


So, they have not only the result, that their Tascam DAT produced some noise at 44.1/16, they write as result, that very probably 96 kHz sampling is overkill.
The good blind-tested 48 kHz DACs proved, that transparency is already available at 48 kHz sampling (which does not exclude, that 44.1 kHz sampling , 20 kHz sound, would be transparent, too.
But these 48 DACs had 24 bit.

So, they tell, that the 24 bit is an improvement over 16 bits.
Not the 96 kHz are an listenable (ABXable) improvement, 96 kHz are waste of space, which we will agree mainly, without great headaches.


well, this 24 bit better than 16 bit thing is unfortuanetly not really tested by them. Their test-setup was mainly for 96 kHz against other sampling frequencies.
And therefor they have a nice result, 48 kHz perfect.


Their graphs are unfortunately not the best ones.
you can get some valuable information out of them.
but in this summary they give only very little numbers, statistics.
I assume, that there is more statistical proof in the real diploma work.

But the whole test-setup, and how it is written, shows, that they wanted to get real blind-tested results, like we want here at HA.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #5
Garf, I have not read this article yet but it seems to me, that there might be an explanation for the 24-bit difference that we haven't thought of. A few weeks ago this problem just popped up in my mind.

Here it is:  the Nyquist theorem is true if - and only if - each sample is stored with infinite precision.

In other words: if we truncate each sample of the original signal to 16 bits, the added quantization noise will - after the signal passes through the DAC - affect not only the noise floor of the resulting signal, but also various issues such as phase, separation between frequencies, etc.

I've been thinking about verifying that in Matlab, but had no time for that so far. Does it make any sense to you ?

[Edit]
Now about the test:  It seems pretty stupid to me. I mean, why not use the SAME 24-bit/96kHz DAC each time, and downsample/requantize the signal for the lower sampling rate tests ?  Sounds like the only way to avoid comparing the DAC's rather than the sound quality.
[/Edit]

 

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #6
Quote
Well, if I understand the German right, they did illustrate that listeners could distuinguish a CD quality recording from an 'original' (how did they play back _that_ ?).

The thing is that the results seem to depend too much on the used DAC rather than on the format itself, e.g. the 48khz DAC being chosen over the 96khz ones.

yep, they write it, that every DAC will have its own influence due to analogue in and out circuits of each DAC.


actually they did NOT take a good CD-recording.

They played music in another room and used best available studio techniques to (record and) transmit it to the several to be tested DACs.



They had one very important test-setup for testing the main question, they asked: 96 vs. 48 kHz.
as this would be dependent on DAcs, too, they took a 96 kHz recording, and used some downsampling to 48 kHz, and then playback through 96 kHz, again, to test this 48 kHz-signals gainst the 96 kHz-originals , both playable through same 96-DAC.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #7
Quote
They had one very important test-setup for testing the main question, they asked: 96 vs. 48 kHz.
as this would be dependent on DAcs, too, they took a 96 kHz recording, and used some downsampling to 48 kHz, and then playback through 96 kHz, again, to test this 48 kHz-signals gainst the 96 kHz-originals , both playable through same 96-DAC.

Which, unsurprisingly, resulted/results in virtually no perceivable difference for the listeners.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #8
I think it's pretty clear from this "study" that the participants were evaluating the anti-aliasing filters and not the signal converters. This issue comes up a lot in these tests, and this is where most of them fall flat.

Not that there's anything wrong with evaluating analog domain front ends, but it shows how well the "studiers" understand the nature of the beast, and the need for a proper setup. I seem to recall David R working with what appeared to be a proper setup for this experiment way back (something about double blind tests at a tradeshow comes to mind)...

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #9
Here is how i would make such a test :

1. Record a jazz combo live with a simple, classic 2 microphone setup ( to preserve runtime delay best possible ), using a very high quality 24 Bit/ 96 Khz ADC

2. Downsample the recorded digital signal to 44.1 and 48 Khz afterwards on a PC, with a normal FFIR filter ( or whatever ), but using a very high internal precision to avoid rounding errors ( like 32 bit FP ), and dither the signal

3. Play back those downsampled signals using a state-of-the-art 24 Bit / 96 KHz DAC and an excellent stereo monitoring system

4. Let the user in an ABX decide what sounds 'best' or most natural to them

If you use different DACs you will hear the differences between them, and not the sampling frequency.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #10
Hey, hey, people... stop, please!!! Sometimes I just wonder like Alice in the Wonderland  what is being discussed here... Music stays completely aside, the talk si about something what almost nobody really needs. Just remember the old days back in 1955 to 1965 and excellent jazz recordings especially from Rudy van Gelder era done just with two mics in his living room on 2- (or 3-) track machines... The songs being recorded in one take, maybe some takes were made for each song and the whole album was made within a days (very often in one or two...) The recordings (be it on LP or later a CD re-issue) have excellent quality, maybe with higher noise but who really concerns? This is what really matters, real music in the real world... not these academical extracts... 

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #11
You're certainly entitled to your opinion (so why do you hang out at hydrogen audio, then?). Back to the issue...


96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #12
This "study" is completely flawed.
Untill someone does a test like NumLOCK and ChristianHJW suggested, i have nothing to learn from that "study".

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #13
Sorry SK1,

please tell me, what is flawed in this study ?

Can you read german ?
Who translated for you ?


Tell facts, what the flaws are !


Hey, this study needs at least 9 sites. And these 9 or 10 sites are only an extract out of perhaps 60 sites, dunno.



cite:

" Here is how i would make such a test :

1. Record a jazz combo live with a simple, classic 2 microphone setup ( to preserve runtime delay best possible ), using a very high quality 24 Bit/ 96 Khz ADC

remark user: afaik I have read, this has been done nearly that way. They had some musicians playing, perhaps 1, 2 or 3, classical instruments, and this was recorded at the best way, I remember that classic 2-microphone setup.

Well, you should all know, that this "High-School (a kind of special university, specialzed on studies for becoming sound-ingeneer)", where the test took place, is the only kind besides another in Berlin, in Germany. So, they have the knowledge, all (if not more), like we have here at HA.
They have around 40 - 50 students, all becoming sound-ingeneers at the end.
btw, these guys were the test-listeners.
So they would have had enough to perform a solid statistic, I assume (well, the information about carried out statistics is very small in this article, but there is some. I want to add, that 2 main lessons of the students are maths and physics, specialized on sound....) The statistical stuff will fill the a lot of sites of the official diploma work.............





2. Downsample the recorded digital signal to 44.1 and 48 Khz afterwards on a PC, with a normal FFIR filter ( or whatever ), but using a very high internal precision to avoid rounding errors ( like 32 bit FP ), and dither the signal


reamrk user: in one test-setup, this was carried out similarily to 48 kHz, I had mentioned it above. They were aware of the fact, that every DAC may sound a different way...




3. Play back those downsampled signals using a state-of-the-art 24 Bit / 96 KHz DAC and an excellent stereo monitoring system


remark user : done, written above.




4. Let the user in an ABX decide what sounds 'best' or most natural to them

If you use different DACs you will hear the differences between them, and not the sampling frequency. "

remark user:
that was carried out, otherwise, I wouldn't have wasted my time, to inform you about this study.

Well, I haven't found a flaw in this study. They were aware of all the mentioned things until now.
Why are some of you critizising this study ?
If you find a flaw, describe it. But you should not tell here possible flaws, which *might* have occured, like if they were beginners.


hmm, I know, it is difficult for most of you, to read those german sites.
(I assume, that this study wasn't spreaded so far, as it tells quite clearly, that 96 kHz is not necessary, 48 kHz are enough. The big music-industry had surely no interest to THIS result, lol !!!)
About that 16 bit vs. 24 bit thing, the study tells nothing new, perhaps I have written/interpreted too much in my first post.)
I have read, what guys these 40-50 students,( = participants of the blind -tests ) are.
They are selected out of a quite big number of volunteers, who want all to become sound-engineer, master for recording professional in studios or live etc.
So, they select the best ones, theoretical and practical skills.
Then they let them study at thier school.
So, the guys are quite young (20-25 max.) (haven't lost thier hearing abilities so far...)
and they are trained for musicality listening. They must play instruments , before they start their studies there...
and knowledge about electronics, acoustics, electrical, physics are reqwuired too, ot they are teached at that school.

In fact, they would be ideal HA-members, if they would start dealing with lossy codecs, too, and some training to listen to artifacts of lossy encoders, but this could be learned quite fast...

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #14
Quote
"in one test-setup, this was carried out similarily to 48 kHz, I had mentioned it above. They were aware of the fact, that every DAC may sound a different way..."

Can you be more specific as to how this was tested? What do you mean by "this was carried out similarily to 48 kHz" ? Your first post didn't suggest this test approach was used...

Isn't the fact that this study was conducted 5 years ago somewhat important? Multi-bit sigma-delta modulators weren't around then.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #15
Quote
[...]
what is flawed in this study ?
[...]

A different DAC was used each time, which is not the safest thing to do when you're desperately trying to notice differences beyond 90dB SNR !

Remember, what you basically want, is to distinguish between the following two binary 24-bit samples, just by listening:

YYYYYYYYYYYYYYYY00000000 (24 bits truncated to 16 bits)
YYYYYYYYYYYYYYYYXXXXXXXX (24 bits not truncated)

If any of both converters has a linearity problem below 2^-15, it's already too much for the test to be valid.
The main reason for this, is that I doubt seriously that a 16-bit converter would produce perfectly bit-accurate output down to the LSB.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #16
Numlock,
now I am even asking myself, if you have read the study ?

they have never tested for 24 bit to 24->16 truncated, or 16 bit.

hey, they tested 96 vs. 48 kHz @ 24 bit,
those single 44.1/16 test showed the bottom line...



hmm, no, they have not tested every time another DAC.
That was on purpose.

If they would have done the opposite:
only one DAC from one company, then we would cry (and we were right then): silly, test based only on one DAC, product.

They compared each DAC vs. the original.
and the testers should tell, which sounds better, DAC or original.
and that leaded to intersting varying results.

It showed, that DACs behaved quite different !

But it showed for the one 48 kHz DAC, that this was a perfect DAC, its output was indistinguishable from source/orignal. And the other 48 kHz DACs showed excellent performance, too.

For the new 96 kHz DACs the results were different.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #17
Quote
...I doubt seriously that a 16-bit converter would produce perfectly bit-accurate output down to the LSB.

What's the LSB?  Im working on 3wk and were looking at a pair of these...  http://www.apogeedigital.com/products/prod_minime.html
If I am able to get them at my house before hand to configure Ill do some tests, and record the results to wavs and flac them so you can abx yourself...  If 3wk does that.  They might decide to stay with an all sdpif setup, instead of integrating this one "goodie" we wanna get.   

anywho, whats LSB?

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #18
I took the time and read the complete article.

user is correct, those people made a good job ( if only they didnt use the crappy Manger speakers  ), the test was very well structured and the results seem clear.

The only thing i missed is what i was stating above, that is to compare the 96 KHz sampled signal with the very same signal, but limited in badnwidth. They have not done so, but always compared the sampled signal with the analog reference ( live band ).

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #19
Least Significant Bit in this case.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #20
thanks, ill read up about its important, if something had a high lsb, that would be worse right?

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #21
Slightly OT

Sound is so subjective, in that I think most test are always flawed to some extent, but if the general consensus is for a certain encoder/amplifier/decoder/DAC to be better than another then the test can be said to have verifiable results.

What is your take on testing sound quality? Is it completely subjective or objective? I would like to hear other thoughts on this topic.

Cheers
-=MusePack... Living Audio Compression=-

Honda - The Power of Dreams

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #22
Quote
Numlock,
now I am even asking myself, if you have read the study ?

I tried, yes.

Quote
they have never tested for 24 bit to 24->16 truncated, or 16 bit.

hey, they tested 96 vs. 48 kHz @ 24 bit,
those single 44.1/16 test showed the bottom line...

Sorry, I really thought that besides the 96kHz-DAC vs 48kHz-DAC test (where 48 was better), they tried to use the dCS 952 (Seite 7) in 16, 20 and 24-bit modes and compare. Well, it turns out my German definitely sucks. Indeed they were just mentioning the DACs' specs. My apologies.

Quote
hmm, no, they have not tested every time another DAC.
That was on purpose.

If they would have done the opposite:
only one DAC from one company, then we would cry (and we were right then): silly, test based only on one DAC, product.

They compared each DAC vs. the original.
and the testers should tell, which sounds better, DAC or original.
and that leaded to intersting varying results.

Ok.

Quote
It showed, that DACs behaved quite different !

But it showed for the one 48 kHz DAC, that this was a perfect DAC, its output was indistinguishable from source/orignal. And the other 48 kHz DACs showed excellent performance, too.

For the new 96 kHz DACs the results were different.

But, why didn't they take three 96-kHz DAC's, and compare what they produce when the input changes ? From what you said, using different DAC for different frequencies really is biased by the component's properties, isn't it ?

Well, to my mind, here is how I understood the tests :
- is a 48kHz converter ALWAYS perceptually as good as a 96kHz one ?  => this test concludes "yes".
- is a 24kHz lowpassed signal perceptually as good as the analog one ? => also, result is "yes".

Missing stuff would be :
- is 20-bit and more, better than 16 bits ? if it is, then at what sampling rate does the difference vanish ? => would be a nice test to do.
- does the reference analog source REALLY contain frequencies above 24kHz ?

Again, sorry about by misunderstanding of that test.
Btw, I definitely agree with your remark about the music industry.

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #23
Quote
...it tells quite clearly, that 96 kHz is not necessary, 48 kHz are enough...


NumLOCK:

Do you remember what I wrote you in the PM last week? 20bit/48kHz is sufficient enough for any high(est) end audio... 

96 vs. 48 or 44.1 kHz sampling --> scientific test

Reply #24
Quote
NumLOCK:

Do you remember what I wrote you in the PM last week? 20bit/48kHz is sufficient enough for any high(est) end audio... 

Well, yes I know, I know.. 

I won't argue with the 48kHz, since I agree with it 

To be sincere I'm still wondering what implications a truncation to 20bit really has. Doesn't the Shannon only apply to discrete, ANALOG domain ? Does truncating to N-bits only rise the noise floor ?

Cheers