HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: pawelq on 2011-06-18 23:14:59

Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-18 23:14:59
I would like to share results of a blind listening test that was conducted on June 16th, 2011 in Warsaw, Poland.

The full description of the test is available at http://www.audio.e-snp.net/ (http://www.audio.e-snp.net/), but it’s in Polish, so I’ll provide a synopsis below.

The test question was "can we hear any effect of A/D/A conversion at 24/96 performed with a studio recorder". Frankly, both the test organizer and me (let's say, the test statistician), were pretty much convinced that no difference would be detectable.

The test music was played using a gramophone (Bergman Audio Sindre with Air Tight PC-1 cartridge), gramophone preamp (Air Tight ATE-2005), * , amplifier (Soulution 720 / 710), and loudspeakers (Hansen Audio Prince V2). In “analog” trials, the path was as described above. In “digital” trials, a studio recorder (Tascam DV-RA1000) set into “monitor” mode (i.e., A/D then D/A, at 24 bit and 96 kHz) was inserted in place of the asterisk.

The participants listened to about 1 minute of “Falling Alice” from Chick Corea’s “Mad Hatter” LP from 1976, mint condition. The test was conducted in a 5.8 m x 7.8 m acoustically adapted room.

There were 10 listeners (neither the test organizer nor me participated; he was switching the connections, I was several thousand kilometers away). All listeners listened together, being in the same room. They left the room for connection switching. During listening, the test organizer remained in the back of the room, invisible to the listeners. The listeners are (and me and the organizers) are members of a small Polish “sensible audiophile” internet forum.

There were 13 test trials, in 7 seven of them there was A/D/A conversion (D), in 6 purely analog path was used (A). The order of D and A trials was random.

Prior to the test, the listeners familiarized themselves with the supposed difference in A vs. D sound and with the recording, which was played a few times in A and D configuration. The listeners received answer cards on which they marked the trials as A or D. They were asked to answer in each trial, even if they were unsure. Prior to the test, they also provided answers to three questions: “Do you think that the effect of digitalization will be audible (yes/no)?”, “Do you consider yourself an experienced listener of vinyl records, using high-quality equipment (yes/no)?”, “How much of your listening time is spent listening to vinyl records (in %)”.

The results were analyzed in two main ways.
In the individual analysis, we checked if any of the participants identified A vs. D at a statistically significant level. With one-way binomial test with Šidák correction (due to multiple listeners. i.e., multiple tests) we determined that at most one error (12/13 correct) is allowed to pass this test (p=0.017; for 11/13 p=0.107).
In group analysis, we converted the results to proportion correct (e.g., 8/13=0.615) and used one-way Wilcoxon one-sample test to determine if the median of proportion correct was significantly higher than 0.5. (Additionally, we calculated one-way one-sample t-test, however, due to the small sample size, the normality assumption could not be reliably tested and we consider the results of t-test to be less trustworthy.)

The results table, a plot of proportion correct for each listener, and a more detailed description of the analysis are provided at http://www.audio.e-snp.net/wyniki.php (http://www.audio.e-snp.net/wyniki.php) (FYI, in the result table A stands for “analog” trials, C stands for “digital” trials, column 1 is listener number, columns 2-4 are answers to the three questions (NIE=no, TAK=yes), the bottom row shows what actually happened in the trials.)
No listener answered with 0 or 1 error, which was required for statistically significant outcome of the individual analysis. There was one 11/13 (0.846) result, and two 10/13 (0.769) results. No one scored below 6/13 (0.462).

The interesting thing is, that average proportion correct was 0.631, and it was significantly higher than chance (p=0.0322, Wilcoxon test; possibly unreliable t-test: p=0.0093). My interpretation is that the A/D/A process done with the Tascam recorder did audibly influence the signal.
No association of answers to any of the three questions with the proportion of correct answers was found.

Any comments?

Title: 24/96 digitalization - can it be audible
Post by: Notat on 2011-06-19 01:01:00
Did you carefully match the level of the system with and without the Tascam inserted i.e. are you sure that the Tascam had exact unity gain? Listeners can hear can minute level differences and usually perceive them as qualities other than different level.
Title: 24/96 digitalization - can it be audible
Post by: Juha on 2011-06-19 07:23:22
By the specs, Tascam DV-RA1000 does not have very good A/D convertor(s) so, it would be nice to see RMAA (http://www.rightmark.org/) results of it.

Juha
Title: 24/96 digitalization - can it be audible
Post by: itisljar on 2011-06-19 09:21:43
By the specs, Tascam DV-RA1000 does not have very good A/D convertor(s) so, it would be nice to see RMAA (http://www.rightmark.org/) results of it.

Juha


By what specs?
Title: 24/96 digitalization - can it be audible
Post by: Juha on 2011-06-19 10:29:31
By the specs, Tascam DV-RA1000 does not have very good A/D convertor(s) ... .

Juha


By what specs?



DV-RA1000 - http://tascam.com/product/dv-ra1000/downloads/ (http://tascam.com/product/dv-ra1000/downloads/)
DV-RA1000HD - http://tascam.com/product/dv-ra1000hd/downloads/ (http://tascam.com/product/dv-ra1000hd/downloads/)

IMO, digital sources for this type of analog source against its digitalized version comparisons should be prepared using HQ A/D equipment (min. Mytek ADC, Benchmark ADC1, Prism sound AD-2, etc.).

Juha
Title: 24/96 digitalization - can it be audible
Post by: DonP on 2011-06-19 11:50:45
Just throwing out some guesses here of things that might give clues:

Levels set so clipping might have happened (like when the stylus hit the record)?

Sometimes with equipment like that "monitor mode" is meant for monitoring, not an output used for production, and is not up to the full specs. 

What sort of time delay is there in the A/D/A process?  Could there be some crosstalk or bleed through  that would give a subtle  pre-echo of the original analog?
Title: 24/96 digitalization - can it be audible
Post by: WernerO on 2011-06-19 16:08:09
By the specs, Tascam DV-RA1000 does not have very good A/D convertor(s)


It is a textbook implementation of the PCM1804. It measures very well. Please point in detail at the specs that told you that it was not very good.

Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-19 16:17:53
Thanks everyone for you comments.  And sorry for my slow response, I have to discuss the details with the other test organizer, and being on different continents and in different time zones, we need time for this.

are you sure that the Tascam had exact unity gain?

The difference was <0.2 dB.


it would be nice to see RMAA (http://www.rightmark.org/) results of it.

Wouldn't it require using measurement equipment that has better specs than the Tascam ?


Levels set so clipping might have happened (like when the stylus hit the record)?

The preamplifier was muted before lowering the stylus and unmuted afterwards. It's a slow, ~2s mute/unmute. Placing the stylus and muting/unmuting was done by a person unaware whether the current trial was A or D.


Sometimes with equipment like that "monitor mode" is meant for monitoring, not an output used for production, and is not up to the full specs.

We used the regular Analog Out RCA outputs. As far as we know, the procedure is identical to normal "production" recording except that there is no recording.


What sort of time delay is there in the A/D/A process?  Could there be some crosstalk or bleed through  that would give a subtle  pre-echo of the original analog?

We'll look into the possibility of measuring this delay. On the other hand, Tascam specs say that crosstalk is <-97 dB


At this point we have to admit that we have found a potential confound, although it might be irrelevant. We ensured that the levels were below clipping by observing Tascam's clipping indicators during pre-test runs. Everything seemed fine. But we also recorded the test music using the same configuration and settings, and there was tiny amount of slight clipping in the recorded file, namely 21/98 samples in L/R channels, for a total of >17 million samples in each channel. We are not sure if this can be audible/significant, but it is a methodological flaw, and we'll try to find a way of avoiding it, should we re-do the test, or do similar tests.



Title: 24/96 digitalization - can it be audible
Post by: Juha on 2011-06-19 20:13:24
By the specs, Tascam DV-RA1000 does not have very good A/D convertor(s)


It is a textbook implementation of the PCM1804. It measures very well. Please point in detail at the specs that told you that it was not very good.


If it's BB PCM1804 chip as you suggest then it's specs are:

    Dynamic Range: 112 dB (Typical)
    SNR: 111 dB (Typical)
 
By the DV-RA1000HD documentation Dynamic Range for DAC is 120dB.

Specs from Owners Manual  (Analog to Analog)

    Dynamic Range: >103 dB (DVD +RW) (>96 CD-R/RW)
    SNR: >103 dB

Bench document results  (Analog to Analog)

    Dynamic Range: 107 dB
    SNR: ~107 dB


As comparison, E-MU 0404 USB - http://www.emu.com/products/product.asp?ca...lSpecifications (http://www.emu.com/products/product.asp?category=610&subcategory=611&product=15185&nav=technicalSpecifications)
RMAA - http://ixbtlabs.com/articles2/proaudio/emu-0404-usb-p2.html (http://ixbtlabs.com/articles2/proaudio/emu-0404-usb-p2.html)

So, it's not the worst implementation found in Tascam but, isn't it just as using a 18 -bit A/D stage that supports 24-bit resolution (though, would that extra 2-3 bit range you could get by using ADCs I mentioned have much mean in this type of test?).


Juha
Title: 24/96 digitalization - can it be audible
Post by: Notat on 2011-06-20 00:14:16
are you sure that the Tascam had exact unity gain?

The difference was <0.2 dB.

My recollection is that best practice for a sensitive test like this is <0.1 dB.
Title: 24/96 digitalization - can it be audible
Post by: saratoga on 2011-06-20 00:24:18
So, it's not the worst implementation found in Tascam but, isn't it just as using a 18 -bit A/D stage that supports 24-bit resolution (though, would that extra 2-3 bit range you could get by using ADCs I mentioned have much mean in this type of test?).


I don't know what you're trying to say with "18 -bit A/D stage that supports 24-bit resolution", but according to Google its a 24 bit converter.

Title: 24/96 digitalization - can it be audible
Post by: Northpack on 2011-06-20 00:48:53
So, it's not the worst implementation found in Tascam but, isn't it just as using a 18 -bit A/D stage that supports 24-bit resolution (though, would that extra 2-3 bit range you could get by using ADCs I mentioned have much mean in this type of test?).


I don't know what you're trying to say with "18 -bit A/D stage that supports 24-bit resolution", but according to Google its a 24 bit converter.

I think this is nit-picking anyway. Remember that the analogue source for this test is a record from 1975. It would be spectacular enough if if would suggest that a properly done 16bit A/D/A conversion were audible.
Title: 24/96 digitalization - can it be audible
Post by: krabapple on 2011-06-20 03:55:32
Just to be clear, this is all about an average score?  All the individual scores failed to reach the statistical significance threshold?


Title: 24/96 digitalization - can it be audible
Post by: 2Bdecided on 2011-06-20 10:11:23
82/130.

Assuming you didn't cherry pick the data, I reckon it's fair to take that as one block. I'm sure that results in a very small p value, but I can't find a large enough p-value table to check.


I'd be very worried about a 0.2dB level difference, and quite worried about clipping (depending on the content).

If X should not be audible, but apparently might have been audible, the fact you also have Y and Z which are known to be (sometimes) audible is rather significant.

Cheers,
David.
Title: 24/96 digitalization - can it be audible
Post by: WernerO on 2011-06-20 11:18:09
isn't it just as using a 18 -bit A/D stage that supports 24-bit resolution (though, would that extra 2-3 bit range you could get by using ADCs I mentioned have much mean in this type of test?).


Yes, the noise performance of the ADC side of the DV-RA1000 is about 18 bit equivalent. I measured input SNR at 106dB  or so, unweighted, 22kHz, IIRC. I only ever measured one ADC better under the exact same conditions; that one was 19 bit equivalent.

The products you show are not 2-3 bits better.






Title: 24/96 digitalization - can it be audible
Post by: usernaim on 2011-06-20 16:21:27
Interesting that no one did worse than 6/13.  Just by intuition I find that to be potentially informative.

But the multiple listeners issue is bothersome.  Is there any possibility that the listeners influenced each other? 

Also, as far as applicability, we don't tend to listen to our hi-fi that way, with all that acoustic interference.  I don't know if that makes the test more or less difficult or doesn't matter, but it might matter.
Title: 24/96 digitalization - can it be audible
Post by: [JAZ] on 2011-06-20 18:41:04
Interesting that no one did worse than 6/13.  Just by intuition I find that to be potentially informative.


Doing 6/13 is not an indicative of being on the good track of things. 6/13 (and 7/13) is an indicative of chance. If you flip a coin, there's as much possibilities to get any of each sides. If you flip it twice, the randomness would suggest you get different sides each time, so achieveing a 1/2.

That's why here at hydrogenaudio we try to make people aware of ABX, and the way to understand the values ( asking to reach 95% or in some cases even 99% of success).

In fact, there is as much possibilities to reach 16/16 than to reach 0/16. 0/16, if not done by chance, would suggest that a difference was clearly noticed, but the user misinterpreted which was which when answering.


About the test itself, I am not knowledgeable enough to find what would make it incorrect, but having 3 out of 10 listeners with a result of 10/13 or better is something to try to understand.  There is no proof (they didn't pass the test) that they could hear a difference, but the results imply that there might have been something.
Title: 24/96 digitalization - can it be audible
Post by: usernaim on 2011-06-20 19:46:08

Interesting that no one did worse than 6/13.  Just by intuition I find that to be potentially informative.


Doing 6/13 is not an indicative of being on the good track of things. 6/13 (and 7/13) is an indicative of chance. If you flip a coin, there's as much possibilities to get any of each sides. If you flip it twice, the randomness would suggest you get different sides each time, so achieveing a 1/2.

That's why here at hydrogenaudio we try to make people aware of ABX, and the way to understand the values ( asking to reach 95% or in some cases even 99% of success).

In fact, there is as much possibilities to reach 16/16 than to reach 0/16. 0/16, if not done by chance, would suggest that a difference was clearly noticed, but the user misinterpreted which was which when answering.


About the test itself, I am not knowledgeable enough to find what would make it incorrect, but having 3 out of 10 listeners with a result of 10/13 or better is something to try to understand.  There is no proof (they didn't pass the test) that they could hear a difference, but the results imply that there might have been something.

I disagree.  If the listeners are guessing, you would expect a normal distribution centered at 50/50.  That we have people who got 10/13 would be expected.  That no one got 3/13 or 4/13 or even 5/13 is what stands out about the distribution.  It is also, ultimately, why the overall mean was statistically significant.  If bad guessers balanced out good guessers the mean would be 50/50.  They didn't.
Title: 24/96 digitalization - can it be audible
Post by: Parelius on 2011-06-20 20:25:41
We ensured that the levels were below clipping by observing Tascam's clipping indicators during pre-test runs. Everything seemed fine. But we also recorded the test music using the same configuration and settings, and there was tiny amount of slight clipping in the recorded file, namely 21/98 samples in L/R channels, for a total of >17 million samples in each channel. We are not sure if this can be audible/significant, but it is a methodological flaw, and we'll try to find a way of avoiding it, should we re-do the test, or do similar tests.


I don't have any backing for this, so please just overlook if it is too far out.

I've been told that my MH ULN8 should operate at -6db (if I'm not wrong) at the input for the A/D converters to prove their best. Is that just a «fairytale»?; and if not, is this something that is common to A/D converters and could also apply to the Tascam unit? (Nothing said about the input in this test, as I can see.)

Apology if I'm breaking any rules here.
Title: 24/96 digitalization - can it be audible
Post by: Notat on 2011-06-20 23:09:23
We ensured that the levels were below clipping by observing Tascam's clipping indicators during pre-test runs. Everything seemed fine. But we also recorded the test music using the same configuration and settings, and there was tiny amount of slight clipping in the recorded file, namely 21/98 samples in L/R channels, for a total of >17 million samples in each channel. We are not sure if this can be audible/significant, but it is a methodological flaw, and we'll try to find a way of avoiding it, should we re-do the test, or do similar tests.


I don't have any backing for this, so please just overlook if it is too far out.

I've been told that my MH ULN8 should operate at -6db (if I'm not wrong) at the input for the A/D converters to prove their best. Is that just a «fairytale»?; and if not, is this something that is common to A/D converters and could also apply to the Tascam unit? (Nothing said about the input in this test, as I can see.)

Apology if I'm breaking any rules here.

Professional recording is usually done at -24 dBFS or so. As has apparently been demonstrated here, it does not pay to be stingy with headroom. Watch your meters. There is no standard for what a clipping indicator means.
Title: 24/96 digitalization - can it be audible
Post by: Parelius on 2011-06-20 23:10:20
^
Sleeping in class. «overlook» should be «ignore». Sorry for that. (Didn't find any edit button.)
Title: 24/96 digitalization - can it be audible
Post by: AndyH-ha on 2011-06-21 04:56:08
In case it isn't clear, leaving headroom when recording is to prevent unexpectedly high input levels from clipping. It has nothing to do with the quality of the A to D (barring some unusual ADC), or how the converter operates.  As long as there is no clipping. the input can be extremely near, or even at, 0dBfs.
Title: 24/96 digitalization - can it be audible
Post by: WernerO on 2011-06-21 07:03:11
I've been told that my MH ULN8 should operate at -6db (if I'm not wrong) at the input for the A/D converters to prove their best.


Many delta-sigma ADC chips have a slightly rising distortion in the top half of their input range.

Title: 24/96 digitalization - can it be audible
Post by: AndyH-ha on 2011-06-21 09:01:32
What is the definition of a half of the input range?

Does this problem ever show up in tests of soundcards?

If so, where can some revealing results be found?
Title: 24/96 digitalization - can it be audible
Post by: Kees de Visser on 2011-06-21 09:46:03
leaving headroom when recording is to prevent unexpectedly high input levels from clipping.
Under studio conditions it's very well possible to make an educated guess about the max spl and take some risk. And if clipping happens it's often no problem to record that part again. Live recording is different, so a larger headroom margin can be required. Last weekend I recorded airplanes on an airshow and even with plenty of headroom the large 3-engine airplane took me by surprise. 24-bit ADC is no luxury under these conditions because the quieter  parts will require at least 20dB gain during post production.
Title: 24/96 digitalization - can it be audible
Post by: WernerO on 2011-06-21 11:26:59
If so, where can some revealing results be found?


PCM1804 datasheet.

(http://img200.imageshack.us/img200/5373/thd.gif)

Not what I would call a significant problem.

Title: 24/96 digitalization - can it be audible
Post by: Northpack on 2011-06-21 15:53:52
I don't think that we have to speculate about possible minor shortcomings of the Tascam when the test is known to have two major problems:

- Level matching within 0.2dB is insufficient and could be well audible
- Clipping

It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.
Title: 24/96 digitalization - can it be audible
Post by: Martel on 2011-06-21 17:26:54
I'm no statistics expert but it seems like too little data. Have you tried flipping a coin 130 times instead and processing the results the same way? What kind of conclusion would you jump to if it gave you the same result?
Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-21 19:26:34
Just to be clear, this is all about an average score?  All the individual scores failed to reach the statistical significance threshold?
Yes - but keep in mind that due to low number of trials and due to correction for multiple participants, the threshold for individual analysis was very high - 12/13 =92.3%. Unfortunately we could not run more trials, with 50 trials and 10 participants just 74% (37/50) would suffice.

82/130.

Assuming you didn't cherry pick the data, I reckon it's fair to take that as one block.
I did not cherrypick. All data from all participants are used, exactly as reported in the asnwer cards. Actually, one of the top-scorer's errors was a correction from initially correct response, had he not corrected, he would pass the individual statistical threshold. But we take his (wrong) correction as final.

I refrained from taking the data as one block as you suggested, because they were 13 submeasurents (trials) in 10 measurements (participants), and assuming individual differences in perceptual abilities, the 13 submeasurements within each of 10 participants were not independent. Anyway, the 95% confidence interval of the 82/130 proportion is 0.5452 to 0.7089 (or 0.5413 to 0.7125, using another method), it does not include 0.5, so would I consider the proportion to be significantly different from 0.5.

I'd be very worried about a 0.2dB level difference, and quite worried about clipping (depending on the content).
Yeah, clipping is not good, and we should have avoided it. As i said though, there was very little of it, just a few cut of peaks over tens of seconds of recording.

As of the level difference, are there any published data showing sensitivity to level differences <0.2 dB in complex/musical signals? Anyway, if this is significant, we would need a different A/D/A device, one that it is closer to ideal unity gain

But the multiple listeners issue is bothersome.  Is there any possibility that the listeners influenced each other?
This cannot be excluded, but there was no way to get access to the room and equipment for 10 individual sessions. We have no reports of such influence from the participants, but subconscious influence is theoretically possible

Also, as far as applicability, we don't tend to listen to our hi-fi that way, with all that acoustic interference.
What kind of acoustic interference are you referring to?

- Level matching within 0.2dB is insufficient and could be well audible
[...]
It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.
Again, I'd appreciate references that established jnd for intensity in complex signals to be <0.2 dB

I'm no statistics expert but it seems like too little data. Have you tried flipping a coin 130 times instead and processing the results the same way? What kind of conclusion would you jump to if it gave you the same result?
I believe this part has been taken care of properly without physically flipping a coing. That's what we have math for.
Title: 24/96 digitalization - can it be audible
Post by: 2Bdecided on 2011-06-22 10:23:41
I'd be very worried about a 0.2dB level difference, and quite worried about clipping (depending on the content).
Yeah, clipping is not good, and we should have avoided it. As i said though, there was very little of it, just a few cut of peaks over tens of seconds of recording.

As of the level difference, are there any published data showing sensitivity to level differences <0.2 dB in complex/musical signals?

- Level matching within 0.2dB is insufficient and could be well audible
[...]
It should be repeated with levels matched within 0.1dB and lowered input level to give really significant results.
Again, I'd appreciate references that established jnd for intensity in complex signals to be <0.2 dB
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.

* - The closest is 0.25dB under quite different conditions: F.E. Toole and S. Olive, "The Modification of Timbre by Resonances: Perception and Measurements", JAES vol 36, # 3, March 1988, pp 122-142

Cheers,
David.
Title: 24/96 digitalization - can it be audible
Post by: Martel on 2011-06-22 13:43:43
I believe this part has been taken care of properly without physically flipping a coing. That's what we have math for.

Well, any statistical analysis is easily misinterpreted or skewed towards a certain point by formulating the tested hypothesis in a certain way.
Why not test a hypothesis that all of the participants were just guessing and their scores were drawn from a coin-flip-distribution random number "generator"?
That was my point, sorry for not making it more apparent.
Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-22 13:55:12
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.


Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".


Why not test a hypothesis that all of the participants were just guessing and their scores were drawn from a coin-flip-distribution random number "generator"?
That was my point, sorry for not making it more apparent.


I suggest that you re-read the thread, paying special attention to "binomial test", "Šidák correction", and "confidence interval of proportion".

Title: 24/96 digitalization - can it be audible
Post by: Northpack on 2011-06-22 14:39:30
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.

Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".

AFAIK the common standard for level-matching is <= 0.1dB. I don't know whether this value has been empirically deducted but it seems to be sufficient.
Title: 24/96 digitalization - can it be audible
Post by: Axon on 2011-06-22 16:24:11
FWIW, I recall once being able to successfully ABX a synthetic signal to at least a 0.05db difference in volume, with IEMs. But I did not think I could have been able to repeat that feat with speakers or with "real" music, and I've always worried that those Etys might have distortion levels varying significantly with sound level....
Title: 24/96 digitalization - can it be audible
Post by: knutinh on 2011-06-23 12:29:01
There are none*. But then, there are no published references showing that a 20kHz LPF and 16-bit quantisation is audible in musical signals either. So, great care must be taken.


Sure. I am trying to find out how close level-matching needs to be. In other words, what is the basis and the limit for the "better matching" requests. I mmagined that we would do a similar test with levels matched to 0.06 dB, and how do I know that a HA'er won't say "hey, it should have been 0.05 or better".

Good point.

I guess one answer woul be that "the more controversial your findings are, and the less able others are to repeat your test, the more painstaking you have to be in controlling these things in order to make a difference to 95% of the sane readers. The last 5% will dispute your findings no matter what you do".

0.1dB is a number that I have seen several times. It might be a number that is both practical to achieve and well enough ahead of what we believe that humans can distinguish.

I think that at this point, getting independent confirmation is most important.

-k
Title: 24/96 digitalization - can it be audible
Post by: Brand on 2011-06-23 13:38:11
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.
Title: 24/96 digitalization - can it be audible
Post by: Northpack on 2011-06-23 14:34:54
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.

An impressive result. The only drawback is if the original was 0.0 it could possibly have caused some minor clipping. It would have been better if you had normalized to, say, -1,0 and -1.1 instead, just to be really sure that there's no clipping.
Title: 24/96 digitalization - can it be audible
Post by: Kees de Visser on 2011-06-23 17:44:54
An impressive result.
You can try for yourself with this online hearing test:
http://www.audiocheck.net/blindtests_index.php (http://www.audiocheck.net/blindtests_index.php)
Since it's relatively easy to match levels in the OP setup, 0.1 dB seems reasonable, just to eliminate yet another variable.
OTOH moving one's head will result in larger level (and spectral) differences than 0.1dB, so perhaps it's a moot point.
Title: 24/96 digitalization - can it be audible
Post by: Northpack on 2011-06-23 18:54:26
An impressive result.
You can try for yourself with this online hearing test:
http://www.audiocheck.net/blindtests_index.php (http://www.audiocheck.net/blindtests_index.php)

It seems impossible for me to ABX 0.1dB, 0.2dB only if I try very hard.
Title: 24/96 digitalization - can it be audible
Post by: dhromed on 2011-06-23 19:56:23
An impressive result.
You can try for yourself with this online hearing test:
http://www.audiocheck.net/blindtests_index.php (http://www.audiocheck.net/blindtests_index.php)


Using headphones, and coming home from work after a ride in the bus and being very active in the kitchen, I dropped to 6/10 at 1dB. A second pass with more focus got me back to 10/10.
0.5dB was a guessing game. For completeness I tried 0.2 and 0.1, but I heard nothing. Pitch test got me to 10c, and then it was all the same.

The 440Hz tone made my ears bleed after a while, though.
Title: 24/96 digitalization - can it be audible
Post by: Brand on 2011-06-24 12:56:43
FWIW, I just ABXed a song with a 0.1 dB difference (normalized the original, which was at 0.0 to -0.1). I used headphones and focused on a very short sample repeatedly. So probably this is not the same methodology as you used, but it's still something I'd look into for such tests.

An impressive result. The only drawback is if the original was 0.0 it could possibly have caused some minor clipping. It would have been better if you had normalized to, say, -1,0 and -1.1 instead, just to be really sure that there's no clipping.

I was thinking the same, actually. And also since one file was processed and the other wasn't. So I did as you suggested, one at -1.0 and the other at -1.1 (separate processing, of course). I managed a 8/8, then perhaps got a bit impatient, stopped at 9/11. It's not the 10/10 I'm usually aiming at, but take it as you will. I might give it another shot on a calmer day. (It's stormy right now.)
Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-24 13:03:23
Just a question. Can you guys replicate this kind of accuracy at 0.1-0.2 dB difference if you take off your headphones and go to another room for a few minutes between trials? This is how our test was conducted (and we used loudspeakers and music, not headphones and 440 Hz tone).
Title: 24/96 digitalization - can it be audible
Post by: Brand on 2011-06-24 13:54:45
I haven't tried yet, so I won't make any claims. But if I were to guess I'd say no. Even with careful listening on the headphones it wasn't easy and as I've hinted before I had to constantly repeat the involved samples to spot the differences.

But since we're scrutinizing this, what's the reasoning for only using one song for the test and this song in particular ("Falling Alice")?
Title: 24/96 digitalization - can it be audible
Post by: pawelq on 2011-06-25 05:48:57
But since we're scrutinizing this, what's the reasoning for only using one song for the test and this song in particular ("Falling Alice")?


Only one song - mostly time constraints. We had 13 trials, add time needed for device switching, and you will realize that using multiple songs was not really feasible.

Why this song? Well, we had a pretty long discussion with very diverse suggestions, and this song was suggested by one participant mainly because of variety of timbres involved, including human voice, and relatively transparent texture. And everybody agreed to use it :-). An additional factor was that a fully analog (i.e., old enough) LP in good condition could be provided.

Most people agreed that the genre/style of music was not too unfamiliar for too many of them. Anecdotally, familiarity turned out to be a factor, as the top-scorer was the owner of the LP....
Title: 24/96 digitalization - can it be audible
Post by: Brand on 2011-06-27 22:33:27
I previously kinda implied that using headphones made the ABX easier.
I've now finally tried the 0.1 dB ABX with speakers and I thought it was actually easier than with headphones. Uploaded files here (http://www.hydrogenaudio.org/forums/index.php?showtopic=89376).

I think usually for lossless-lossy ABXing headphones are better, but I guess with volume differences speakers could have an advantage. (Feeling the sound with the whole body etc.)


I still didn't try ABXing the same way you did for the test, though. I was repeating the short pieces back and forth. But theoretically, if the difference was in fact 0.2 dB instead of 0.1 dB, I would imagine it could be possible even with just a single listen and a bigger pause in between.
Title: 24/96 digitalization - can it be audible
Post by: Axon on 2011-06-28 21:33:03
:F

If it's easier with speakers, then I'm inclined to at least suspect that the difference might be distortion-induced.
Title: 24/96 digitalization - can it be audible
Post by: DonP on 2011-06-28 23:44:49
:F

If it's easier with speakers, then I'm inclined to at least suspect that the difference might be distortion-induced.


That's the second post you've made in this thread suggesting that a small difference in volume (the other time, .05 dB) was detectable due to distortion.

By what mechanism would distortion differ significantly over a .05dB power change?  Or even 1 dB? 

I can certainly accept that speakers can have audible distortion.  The question is how much the distortion can change, beyond just scaling with the main signal, over a very small range of level.
Title: 24/96 digitalization - can it be audible
Post by: Dynamic on 2011-06-30 00:14:40
I have a couple of queries regarding methodology (and I've only read your brief description in English so I don't know the details)

First, if there's a difference in delay, it's possible that speaker-to-stylus feedback (e.g. vibrations) could influence the sound quality in different ways. I guess high speed reel-to-reel magnetic studio tape would be an analogue alternative source that is immune to this and could be recorded (once) from the original vinyl source or real instruments.

During listening, the test organizer remained in the back of the room, invisible to the listeners.

Invisibility is certainly important, as is truly random order, so that's good procedure. It might be important that there is no other possible communication. For example, use a lamp rather than speaking to indicate the system is ready for testing and maintain a constant time-delay between auditions regardless of whether the digital component is switched into or out of use or left as the previous test (and constant time the light is lit).

A further concern with communal listening is that it's possible for some of the listeners to be influenced by each other (even subconsciously), causing their answers to be correlated rather than completely independent which can mess up the statistics and cause significance to be incorrectly inferred on a flawed assumption of statistical independence.

It's amazing how subtle things can be picked up subconsciously without trying. For example I recently heard a story of a man inadvertently made his wife think she was psychic. He was doing a really awful card trick with a friend when his wife came in (all three were staunch rationalists). The husband said "OK, guess the colour of the next card" and the friend would say "Black". Then "Right, what colour is this one?" and friend replied "Red.". They got it wrong now and then to make it seem plausible and the wife asked to have a go. Bizarrely, and unexpectedly to all concerned, she started to guess perfectly and was freaked out and thought she could be psychic or getting extra-sensory perception. She was livid with her husband when he explained what had happened. It turns out she'd subconsciously picked up the secret code without consciously realising it, and in this case it was right for red, okay for black. One might presume that the brain is reinforcing neural pathways that link various word to the same outcome and while it feels like you're picking red or black out of thin air, the tenuous neural connection can become strong enough to tip the balance of near-randomness in favour of the right answer with remarkable consistency.

It's possible that the person who switched the signal path might tap their foot subconsciously behind the curtain in a different way despite trying desperately not to influence the listeners at all.

This sounds like a good experiment, and it might be a significant result, but it can be so difficult to eliminate all possible sources of statistical correlation and effects beyond those intended for measurement. If you do test again, best of luck. It might even be interesting to rewire the Tascam internally to disconnect the line out and wire directly from the line in (with a plain analogue cable) while leaving the line in attached to the ADC if the clipping or level lights can be observed to provide a null control trial unbeknown to all participants.
Title: 24/96 digitalization - can it be audible
Post by: Arnold B. Krueger on 2011-06-30 12:18:11
There were 10 listeners (neither the test organizer nor me participated; he was switching the connections, I was several thousand kilometers away). All listeners listened together, being in the same room. They left the room for connection switching. During listening, the test organizer remained in the back of the room, invisible to the listeners.


The presence of a person who knew the actual status of each trial in the room is to me the most obvious weak spot in the test.

It is unecessary for him to be there, and it should have never happened.

I can hypothesize a long list of tells that may have unconsciously communicated what he knew to the people in the room. Of course I wan't there and don't know what actually happened.
Title: 24/96 digitalization - can it be audible
Post by: RonaldDumsfeld on 2011-07-01 23:29:50
Is there a possibility that some convertor chips are designed from scratch to convert at multiples of 44.1 but have the ability to resample to multiples of 48 and vice versa? Are these things supposed to be equally good at conversion (as far as the human ear is capable) at all sample rates?
Title: 24/96 digitalization - can it be audible
Post by: Axon on 2011-07-13 01:57:47
:F

If it's easier with speakers, then I'm inclined to at least suspect that the difference might be distortion-induced.


That's the second post you've made in this thread suggesting that a small difference in volume (the other time, .05 dB) was detectable due to distortion.

By what mechanism would distortion differ significantly over a .05dB power change?  Or even 1 dB? 

I can certainly accept that speakers can have audible distortion.  The question is how much the distortion can change, beyond just scaling with the main signal, over a very small range of level.

I haven't forgotten about this, and I think you may be right. I can't really back up my original claim, on continued reflection.

I was sorta thinking that, in the context of harmonics increasing faster than increases in the magnitude of the fundamental, there ought to be some sort of continuum between "static nonlinear distortion" and "clipping". Ie, a tiny change in volume could cause a huge relative change in harmonic distortion at the onset of clipping, but if you define the distortion statically, ie y=x+0.001x^2, there's no relative change whatsoever -- the second harmonic is always some fixed level below the first harmonic. And my hypothesis only works if the relative change is so high as to cause some harmonic to no longer get masked.

But for that to be the case, I think the system would have to be well on its way to being limited/clipped to begin with. If a 0.05db volume difference causes say a 1db level change for some particular harmonic, making it audible, then if the system has another 10db of clean headroom, we ought to expect that this harmonic ought to b e able to increase by another 200db....
Title: 24/96 digitalization - can it be audible
Post by: 2Bdecided on 2011-07-13 12:58:22
If one of the speaker cones has a hard physical restriction that is reached when the volume is increased a little, then that would be a good example of what you describe.

Trivial to set up with test tones and a little speaker modification

Hard to imagine how this could happen with real music and a system with demonstrably useful amounts of headroom.

Though I'm often amazed how much hidden distortion there is in some systems, which is only revealed by narrow/low bandwidth sources.

Cheers,
David.
Title: 24/96 digitalization - can it be audible
Post by: Axon on 2011-07-13 16:37:20
The best idea I could come up with is if the distortion was somehow "multiplexed" into several harmonics, but irregularly. That is, for a given volume increase, harmonic A would bump up by a significant amount, but the other harmonics would stay the same. Then for some further volume increase, harmonic B would increase, but A and all the others would stay the same.

This would give the observed behavior I would need for my hypothesis. But it's ridiculously thin. Also, it's ridiculous.
Title: 24/96 digitalization - can it be audible
Post by: Arnold B. Krueger on 2011-07-13 20:04:38
Is there a possibility that some convertor chips are designed from scratch to convert at multiples of 44.1 but have the ability to resample to multiples of 48 and vice versa? Are these things supposed to be equally good at conversion (as far as the human ear is capable) at all sample rates?


As a rule converter chips rely on an external clock to control the speed at which they work. IOW, they just follow the clock and work the same no matter what it is.
Title: 24/96 digitalization - can it be audible
Post by: knutinh on 2011-07-13 21:48:45
So what is the conclusion on the topic at hand?

-k
Title: 24/96 digitalization - can it be audible
Post by: AndyH-ha on 2011-07-14 06:59:09
My apologies for not having the energy to read enough of yet another 24 bit argument, and maybe getting very off topic here, but while converter chips may function to the clock given them at the moment, the results are not always the same. Maybe mine is a special case, somehow defective: I have a Echo Mia that does rather poorly at 22050Hz, in spite of seeming to be very good at higher sample rates.

I've recorded many hundreds of hours of spoken audio that ended up at 22050Hz final product. It finally dawned on me that recording there to begin with would be more efficient. I normally use the Mia computer for those recordings.

Results were not so satisfactory, although not really unusable. There were definitely different than recording at 44.1kHz and downsampling. I verified that the Audiophile 2496 containing computer has no problem at 22050 but I did not find it convenient to switch for general use, so I just continue to record at 44.1 and downsample.