Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Compilation of blind test reports (Read 10046 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Compilation of blind test reports

I was wowed when  I came across this -- someone went to a lot of trouble to compile most of the print and online audio equipment DBTs since the 90's

first post here:
http://www.head-fi.org/t/486598/testing-au...laims-and-myths

btw pretty sure he's wrong about Meyer & Moran only using CD-sourced SACDs, but I'll correct him on that ;




Compilation of blind test reports

Reply #1
Moran and Meyer said they used "mostly" this (and each test subject got to pick whatever they felt most comfortable with, if I understand correctly):
Quote
Patricia Barber – Nightclub (Mobile Fidelity UDSACD 2004)
Chesky: Various -- An Introduction to SACD (SACD204)
Chesky: Various -- Super Audio Collection & Professional Test Disc (CHDVD 171)
Stephen Hartke: Tituli/Cathedral in the Thrashing Rain; Hilliard Ensemble/Crockett (ECM New Series 1861, cat. no. 476 1155, SACD)
Bach Concertos: Perahia et al; Sony SACD
Mozart Piano Concertos: Perahia, Sony SACD
Kimber Kable: Purity, an Inspirational Collection SACD T Minus 5 Vocal Band, no cat. #
Tony Overwater: Op SACD (Turtle Records TRSA 0008)
McCoy Tyner Illuminati SACD (Telarc 63599)
Pink Floyd, Dark Side of the Moon SACD (Capitol/EMI 82136)
Steely Dan, Gaucho, Geffen SACD
Alan Parsons, I, Robot DVD-A (Chesky CHDD 2003)
BSO, Saint-Saens, Organ Symphony SACD (RCA 82876-61387-2 RE1)
Carlos Heredia, Gypsy Flamenco SACD (Chesky SACD266)
Shakespeare in Song, Phoenix Bach Choir, Bruffy, SACD (Chandos CHSA 5031)
Livingston Taylor, Ink SACD (Chesky SACD253)
The Persuasions, The Persuasions Sing the Beatles, SACD (Chesky SACD244)
Steely Dan, Two Against Nature, DVD-A (24,96) Giant Records 9 24719-9
McCoy Tyner with Stanley Clark and Al Foster, Telarc SACD 3488


Source: Boston Audio Society

But with the exception of Steely Dan, Two Against Nature (which mentions "24,96"), how exactly would one go about finding this out? Even if one owned all these discs I doubt a single one reads on its liner notes: "Oh, by the way, this is from a CD resolution master." (even if that might be true in some instances, I have no idea).

Compilation of blind test reports

Reply #2
It is now known that many SACD releases were  actually transcribed from digital and analog sources that are sorely lacking in significant content > 22 or 24 KHz.  Some came from digital masters sampled at 44 or 48 Khz, and some came from analog tape recordings that had a natural steep cut offs in the 22-24 KHz range. Many were also made with microphones with steep cutoffs as low as 12 KHz.

Exactly which of the above fit into that category is unknown to me, but sources indicate that on the average up to half of them can reasonably be expected to suffer from these problems. 

Therefore the Meyer and Moran tests may not have been what some interpreted them to be. Their tests may have been contaminated with with large amounts of program material that did not have significant content > 22 KHz. Therefore, these recordings cannot be reasonably expected to sound different in their tests which in essence involved the audibility of removal of all program material > 22 KHz.  If the > 22 KHz program material was not there to remove, then removing it must have no effect!

Compilation of blind test reports

Reply #3
analog tape recordings that had a natural steep cut offs in the 22-24 KHz range.


A limitation in the recording (e.g. write heads etc.), or in the tape medium itself? (Like: could we be listening to digitally faithfully reproduced tape hiss caused by master tape deteroriation, which no-one cared about in old days when the read heads could not pick it up anyway?)

Compilation of blind test reports

Reply #4
Therefore the Meyer and Moran tests may not have been what some interpreted them to be. Their tests may have been contaminated with with large amounts of program material that did not have significant content > 22 KHz. Therefore, these recordings cannot be reasonably expected to sound different in their tests which in essence involved the audibility of removal of all program material > 22 KHz.  If the > 22 KHz program material was not there to remove, then removing it must have no effect!



Wanna bet that didn't stop reviewers in Stereophile, TAS, etc, from gushing over the improved 'almost vinyl like'  'hi rez' sound? ;>

(I suspect that was M&M's ultimate point, though their specific stimulus for doing the tests was Bob Stuart's poorly-founded touting of hi rez in JAES)


Remenber, audients were allowed to pick their own examples...presumably recordings they thought best showed the difference.

 

Compilation of blind test reports

Reply #5
Agreed, Krabapple, but if doing it again I'd say they should specifically limit their selections to one of "those" magazines "top ten best SACDs we've ever heard" sorts of lists. They have such lists, yes? [I don't read them so I wouldn't know.]
---

I can just see this will be the test doubters' next lame excuse: "Well none of those SACDs M&M chose in Test 2 have a crash cymbal or solo jingling car keys, so there's no appreciable acoustical power above 22.5kHz anyways..." [Not to mention there are only a handful of mics that pick up such frequencies, and they aren't typically used in studio work, I'm guessing. But I'm not a studio mic expert.]

...and then when they fix that in Test 3 we'll get: "But the speakers they used have poor polar response curves* (off axis dispersion) in the 30kHz range so other than on axis output, they were hardly energizing the room with key frequencies that make SACD sound so much better.."

See, none of these things are limitations when they tout how much better SACD is to CD; it's only when M&M run tests that don't support their beliefs that it's an issue! There's no end to it.

* I guess they use waterfall plots for this these days, but same thing.

Compilation of blind test reports

Reply #6
analog tape recordings that had a natural steep cut offs in the 22-24 KHz range.


A limitation in the recording (e.g. write heads etc.), or in the tape medium itself?


That depends, but it is more likely that it is the tape heads that are doing more to set the observed FR limits than anything else. There are steep cut offs during playback that commence when the wavelength of the signal on the tape is the same size or smaller than the gap of the tape head. Tape head gap sizes are usually constrained by the technology used to create them, and also the other problems that arise when they are very small.

Quote
(Like: could we be listening to digitally faithfully reproduced tape hiss caused by master tape deteroriation, which no-one cared about in old days when the read heads could not pick it up anyway?)


Back in the day we were pretty happy when our tape recorders were reasonably flat up to say 16 KHz, particularly at higher recorded levels. There are other problems, such as the fact that the dynamic range of audio tape and the recorder decreases with increasing frequency.

Most professionally-made recordings made in the late 1970s and early 80s were not afflicted with clearly audible tape hiss. The record/playback equipment and media were good enough to keep noise levels reasonably low. We also had various noise reduction strategies such as Dolby A that were very effective at reducing audible tape hiss so that it wasn't a problem.

We have better tape heads for playback today than we had back in the day. However, anybody doing transcriptions of old tapes might be conflicted by the polar ideas of playing the tape back as it was intended to be played back when it was recorded, versus playing the tape from the archives back with the best technology that is available today.

Compilation of blind test reports

Reply #7
Therefore the Meyer and Moran tests may not have been what some interpreted them to be. Their tests may have been contaminated with with large amounts of program material that did not have significant content > 22 KHz. Therefore, these recordings cannot be reasonably expected to sound different in their tests which in essence involved the audibility of removal of all program material > 22 KHz.  If the > 22 KHz program material was not there to remove, then removing it must have no effect!


Wanna bet that didn't stop reviewers in Stereophile, TAS, etc, from gushing over the improved 'almost vinyl like'  'hi rez' sound? ;>


I have made this point on public forums with the hope of obtaining a reply from John Atkinson starting several months back, but I know of no meaningful reply.

Quote
(I suspect that was M&M's ultimate point, though their specific stimulus for doing the tests was Bob Stuart's poorly-founded touting of hi rez in JAES)


David Greisinger had covered this issue on his web site a number of years ago. He found that many DVD-A and SACD recordings were based on analog recordings whose bandpass and resolution was itself quite limited.

Quote
Remember, audients were allowed to pick their own examples...presumably recordings they thought best showed the difference.


Unfortunately Catch-22 applies. Being charitable, I will allow that the listeners did this as effectively as they could, but they were probably basing their choices based on on listening tests, not FFT analysis. Since variations in response > 20 KHz are not audible, that could not have been a relevant criteria for their choices.

In fact they may have chosen recordings based on their perceptions of good sonic detail in the highest audible registers, which unfortunately had the untended consequence of providing better masking for any program material > 22 KHz that might have actually been there. Better masking would mean even poorer results on listening tests for discrimination based on brick wall filtering at 22 KHz.

Compilation of blind test reports

Reply #8
I can just see this will be the test doubters' next lame excuse: "Well none of those SACDs M&M chose in Test 2 have a crash cymbal or solo jingling car keys, so there's no appreciable acoustical power above 22.5kHz anyways..." [Not to mention there are only a handful of mics that pick up such frequencies, and they aren't typically used in studio work, I'm guessing. But I'm not a studio mic expert.]

...and then when they fix that in Test 3 we'll get: "But the speakers they used have poor polar response curves* (off axis dispersion) in the 30kHz range so other than on axis output, they were hardly energizing the room with key frequencies that make SACD sound so much better.."

See, none of these things are limitations when they tout how much better SACD is to CD; it's only when M&M run tests that don't support their beliefs that it's an issue! There's no end to it.


It might be good to anticipate as much of this as possible in the next round of proper listening evaluationss.

To clarify, your typical crash cymbal has its peaks in its power response well below 20 KHz, and usually even below 10 KHz.

You are precisely right about the general run of studio mics lacking response > 20 Khz, and they are even worse if not perfectly aligned and close to the sound sources. Ditto for speakers.

Compilation of blind test reports

Reply #9
To clarify, your typical crash cymbal has its peaks in its power response well below 20 KHz, and usually even below 10 KHz.


I understand your point regarding masking, in the earlier post, which I agree with, however it is my understanding that spectrographs of the crash cymbal show it has the strongest power output (40%) in the > 20KHz range (not peaks, but rather everything) , at least compared to any other acoustic, conventional musical instrument I've ever seen measured, even more so than the musical triangle. [Which I had previously thought would have had a lot of content up there, but I was wrong. It is only 1%.]. That's the only reason why I mentioned it.

The best case trumpet's >20 KHz power output is a measly 2%, by comparison, in the study I link to below, but that may be a candidate as well.

The only things even higher in power output (up there) aren't conventional acoustic instruments, such as jangling car keys (68%), which I mentioned, other similar sound effects/noises, and electrically generated wideband noise or other test tones, I suppose, specifically generated for one's test.

I don't think there is such a thing as a conventional instrument with strong > 23KHz content that doesn't also have the strong (masking)  content in the nearby, lower (actually audible to some young people) octave(s) which you are correct to warn about. However, I don't see M&M's detractors bringing that up as an argument though, since it then puts the ball in their court to name a qualifying example that would work, yet not have the same masking problem you speak of, and they can't. It simply doesn't exist on any conventional, music SACD, I suspect.

Quote
Instruments Without Harmonics....
...
Fig.  Instrument................... SPL............ 10 dB Above.........Percentage
.........................................(dB).............Bkgnd. to.............of Power
............................................................What Freq.?..........Above 20 kHz

10. Speech Sibilant..............72.    ............>40 kHz        ..............1.7
11. Claves.........................104.    ...........>102  "          .................3.8
12. Rimshot........................73.    ............>90  "          ...................6.
13. Crash Cymbal...............108.    ...........>102  "        ................40.
14. Triangle.........................96.    ...........>90  "          ...................1.
15. Keys jangling..................71.    ...........>60  "        .................68.
16. Piano...........................111.    ...........>70  "          ...................0.02
-

Source:
http://www.cco.caltech.edu/~boyk/spectra/spectra.htm

Compilation of blind test reports

Reply #10
Sorry to jump into this part of the forum.
But I am interested in this last post that suggests that cymbals produce much of their sound above 20k.
Do we know how much about 22k or 24k?
Also do we know if the ear or our bodies in general respond to this near-ultrasonic energy? Given your non-musical examples, I wonder where nails down the blackboard would sit?

Compilation of blind test reports

Reply #11
But I am interested in this last post that suggests that cymbals produce much of their sound above 20k.
Do we know how much about 22k or 24k?

Lots. The power is quite strong there, even stronger than in its 5-10kHz range it seems, however the output in the range you ask about would quite possibly be psychoacoustically masked by the equally strong output in the audible octave just below it, which I believe was Mr. Krueger's point:

[You can inspect any of the other musical instruments' spectrums, labeled "Figure #x" in the article I linked to in my previous post, by clicking their hypertext links there, if interested.]

Quote
Given your non-musical examples, I wonder where nails down the blackboard would sit?


The irritation of fingers scratched on a blackboard has nothing to do with ultrasonics, it is simply humans' high sensitivity to that audible range (strong around 2-4 kHz), the same range being important for speech intelligibility but also sensing an animal's cries when in distress, even at a great distance*, although the blackboard/nail screech is of course chaotic and annoying noise.

*From an evolutionary stand point, animals most alert to predator attacks, even at a great distance, would be most likely to flee in time or run back to camp to protect their young. They would be Darwin's "fittest" and would breed more of their kind, also equally sensitive to that same frequency range, while the kinds not as sensitive in this range would get eaten, not breed, and die off.

Compilation of blind test reports

Reply #12
thank you for your inforamative post mzil