Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: DBT Is Flawed Because Bob Stuart Says So (Read 55146 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

DBT Is Flawed Because Bob Stuart Says So

Reply #75
Interesting that you cite that article. What you find there is not an open honest debate of the topic, but rather a sham discussion where many critical comments were censured out of existence.


Well, this very sane incoming letter from one Lawrence S. Makow was included, at least— by Robert Harley himself, even.


It is not unusual for a few mild letters to be allowed to filter through to make it look like there was no censorship. There was.

BTW IME not all High End audio forums are censured this way. AFAIK the Stereophile forum may kick people off, but any posts that get deleted along the are pretty purple.

DBT Is Flawed Because Bob Stuart Says So

Reply #76
http://www.avguide.com/forums/blind-listen...lawed-editorial
Oh, that's hilarious, because it says...

Quote
But blind listening tests lead to the wrong conclusions even when the experimenters’ motives are pure. A good example is the listening tests conducted by Swedish Radio (analogous to the BBC) to decide whether one of the low-bit-rate codecs under consideration by the European Broadcast Union was good enough to replace FM broadcasting in Europe.

...

Swedish Radio said the codec was good enough to replace analog FM broadcasts in Europe. This decision was based on data gathered during the 20,000 “double-blind, triple-stimulus, hidden-reference” listening trials.

Well, maybe they did, maybe they didn't - but I can tell you one thing with absolute certainty - the real BBC/EBU tests a decade later found that no codecs were transparent (at the bitrates they tested, having sought out problem samples for each codec), and only one (AAC main profile) avoided introducing annoying artefacts on at least some audio.

Double-blind triple stimulus hidden reference testing of course - the same testing methodology that supposedly "caused" Sweedish radio to determine that a codec was fit to replace FM in 1991. Nothing to do with the criteria (EBU criteria reportedly quoted by Sweedish radio DOES NOT demand transparency).

Truth of the matter is, I don't know of any codecs that are universally transparent at the ("low") bitrates they're designed for. Many are transparent most of the time, but that's a different statement entirely. And most HA regulars if they use / when they used lossy codecs for "archiving" used much higher bitrates than those codecs were designed for.

Which is why it's extra funny to try to imply that Sweedish radio did 20000 tests in 1991 (what codecs were around in 1991?!?!!!!!) and couldn't hear anything wrong because they used blind testing. But I guess if you're against blind testing, you'll accept this reported "fact" without question.

audioclaudio will suggest I reject it because I like blind testing. No, I reject it because an audible difference was proven by blind testing...
http://www.mp3-tech.org/programmer/docs/w2006.zip

Cheers,
David.


DBT Is Flawed Because Bob Stuart Says So

Reply #78
The flaws, such as you have presented them, are "flaws" with human beings. They are present equally in sighted and blind tests.

Yes, they are. However, the fact one test is sighted and the other is blinded is what obviously causes differences in the impact these flaws have on human perception. The problem with this is one cannot always know if such a "flaw" is causing differences in perception instead of preventing them. This is because one cannot know everything there is to know about how the human hearing system works. Due to complexity constraints, if one decides to eliminate one or more effects caused by one or more "flaws" then it's not always technically possible to also prove exactly what impact this decision will have on one or more effects caused by one or more other "flaws".
There are other flaws that are only present in sighted tests, but removed by blind tests.

Yes, there are. However, this doesn't prove there are no flaws that are only present in blind tests, but removed in sighted tests.
There are flaws that are more present in blind tests if you choose to do them in a certain way. Fine. Do them in a different way.

Again, yes. However, doing them in a different way does not always make them reliale.
We can't rebut your argument further because, having taken these points into consideration, you have no argument left.

The argument I have left is you still haven't proven a thing. The simple fact I haven't proven a thing doesn't also mean that you have; on top of this, I am not the one being too afraid to admit that I haven't.
If you feel there is some other problem with double blind testing that it not present in sighted testing, and cannot be removed from blind testing, please explain it.

I think the interview with Bob Stuart already did explain it rather exceptionally well.

DBT Is Flawed Because Bob Stuart Says So

Reply #79
http://www.avguide.com/forums/blind-listen...lawed-editorial
Oh, that's hilarious, because it says...

Quote
But blind listening tests lead to the wrong conclusions even when the experimenters’ motives are pure. A good example is the listening tests conducted by Swedish Radio (analogous to the BBC) to decide whether one of the low-bit-rate codecs under consideration by the European Broadcast Union was good enough to replace FM broadcasting in Europe.



Another thing the cited article says that is really quite revealing is:

"Robert Harley -- Thu, 05/21/2009 - 09:37
I'm not aware of any formal DBT of standard-resoution digital audio with high-resolution digital audio. The difference between 44.1kHz/16-bit digital audio and 176.4kHz/24-bit is obvious, in my experience. It is, in fact, so obvious that no one (no one that is a disinterested experimenter, that is) has bothered to organize and conduct it."

He's obviously 100% oblivious to the following well-known rather highly relevant article:

Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback
Authors: Meyer, E. Brad; Moran, David R.
Affiliation: Boston Audio Society, Lincoln, MA, USA
JAES Volume 55 Issue 9 pp. 775-779; September 2007

Hey, it had only been out for a year and a half and plastered all over the web the whole time...  ;-)


DBT Is Flawed Because Bob Stuart Says So

Reply #80
I think the interview with Bob Stuart already did explain it rather exceptionally well.


What you've done for me and I thank you for it,  is to make me aware of the highly outdated model of hearing that both you and Stuart have staked your careers on.

You don't read very much, do you?

The way you've been sloughing posts around here also says something about your writing.

Ever hear of a "Logic tight box"?

DBT Is Flawed Because Bob Stuart Says So

Reply #81
I'll take the opportunity now to cite db1989's recent and inconvenient suggestion that audioclaudio investigate burden of proof.

DBT Is Flawed Because Bob Stuart Says So

Reply #82
@Arny:
You do know that censure and censor are different words with different meanings, right?


Yeah, yeah, yeah! ;-)

I'm still a little jet lagged after spending most of last week in Orange County and spending all day Saturday getting back.  Sundays are busy for me, so I didn't get much chance to rest.

I was talking with my spell checker and hastily clicked the wrong option.

Thanks!

DBT Is Flawed Because Bob Stuart Says So

Reply #83
Personally, I consider this topic done.  Anything more is feeding the troll.

He has no intention (i would say even interest) in listening to what we say, and what he writes is of no help to us.

Let him go to the forums that he's more familiar with.

DBT Is Flawed Because Bob Stuart Says So

Reply #84
He's obviously 100% oblivious to the following well-known rather highly relevant article:

Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback
Authors: Meyer, E. Brad; Moran, David R.
Affiliation: Boston Audio Society, Lincoln, MA, USA
JAES Volume 55 Issue 9 pp. 775-779; September 2007

Hey, it had only been out for a year and a half and plastered all over the web the whole time...  ;-)

It was later proven that the Meyer & Moran tests were not using truly HiRes material, but PCM converted to DSD. Bob Stuart was one of the test subjects BTW.

The audible difference between 16/44.1 and proper HiRes is actually quite enormous.

DBT Is Flawed Because Bob Stuart Says So

Reply #85
It was later proven that the Meyer & Moran tests were not using truly HiRes material, but PCM converted to DSD. Bob Stuart was one of the test subjects BTW.

Reference?

DBT Is Flawed Because Bob Stuart Says So

Reply #86
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.

Evidence for what is otherwise a violation TOS #8?

DBT Is Flawed Because Bob Stuart Says So

Reply #87
It was later proven that the Meyer & Moran tests were not using truly HiRes material

...

The audible difference between 16/44.1 and proper HiRes is actually quite enormous.
Oh, wake up.

The listeners were allowed to choose their own material that they believed best showed the benefits of hi-res.

That they came forward with a wide variety of content, some DSD, some 96k+, and some so old it probably had littlcontent above 20k - and then couldn't hear the effects of downconverting any of it to 44.1kHz, shows that the "enormous" benefit of HiRes wasn't audible to them before, during, or after the test!


But if you want to believe that there was "no true HiRes" material in that test, and that's why it failed, carry on.

DBT Is Flawed Because Bob Stuart Says So

Reply #88
Cite the law that says people taking ABX tests are not allowed to practice beforehand in any fashion they choose (e.g.: sighted, blind, standing on their head worshipping the flying spaghetti monster).

I was referring to the actual test itself, NOT the practicing beforehand.


The above  shows how thoroughly you misunderstand ABX: ABX allows practicing during the test because you have the option of comparing clearly identified A versus clearly identified B at any time during the test that you wish.

That's aside from the fact that your hearing model is refuted by modern science...

Two strikes!

I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.

As for my hearing model being refuted by modern science, like I said, this is not even my hearing model, but the hearing model described by Bob Stuart in the TAS interview I linked to earlier in this thread. I also said I don't know if the way I interpreted this interview was correct or not.

DBT Is Flawed Because Bob Stuart Says So

Reply #89
The argument I have left is you still haven't proven a thing. The simple fact I haven't proven a thing doesn't also mean that you have; on top of this, I am not the one being too afraid to admit that I haven't.
Blind testing solves a problem with sighted testing. The problem is proven. The solution is proven. There are acres of peer reviewed literature across tens of disciplines showing this.

Your alleged problems with blind testing are unproven, and you propose no solution.

This is why the ball is not in my court. Please try to understand this.

Cheers,
David.

P.S. this is not the same as me saying "there is no problem" either!

DBT Is Flawed Because Bob Stuart Says So

Reply #90
What you've done for me and I thank you for it,  is to make me aware of the highly outdated model of hearing that both you and Stuart have staked your careers on.
I think Bob applies a transmission-line model of the cochlear. As much as I can remember (and I've been out of psychoacoustics for a decade!) that's a fine model to use with the right parameters.

Finding those parameters is tricky - you usually determine them by comparing known psychoacoustic results (which come from double-blind tests!) with the predictions of the model, and tweaking the model to match reality.

The transduction is well understood, but some of the brain processing isn't - and the brain forms a massive feedback loop with the cochlear.

Cheers,
David.

DBT Is Flawed Because Bob Stuart Says So

Reply #91
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.

Evidence for what is otherwise a violation TOS #8?

Quote
Robert Harley -- Tue, 07/15/2008 - 10:22
I must disagree with Michkhol's reasoning on all three points. .

First, my assertion that audible differences exist between 44.1kHz/16-bit, 96kHz/24-bit, and DSD is based on my own listening experiences. This includes hearing a live microphone feed and comparing it to standard resolution PCM, high-resolution PCM, and DSD. Have you performed the same test and reached a different conclusion? Or are you content to base your belief that they all sound identical on the results of a published test? (A “test” devised and conducted, by the way, by two individuals with a long history of attempting to discredit audiophiles.) Are you suggesting that I should reject my own direct experience and conclude that I was simply deluded? Is everyone who hears a difference between standard- and high-resolution digital audio similarly deluded—people like Meridian’s Bob Stuart, Keith Johnson, dCS founder Mikey Storey, Peter McGrath, and other credentialed and respected leaders in the field who have decades of academic research and hands-on experience with the subject?

Second, Swedish Radio was, in fact, attempting to discover audible artifacts in the codecs. Read the paper (“Subjective Assessments on Low-Bit-Rate Audio Codecs” by C. Grewin and T. Ryden, published in the Proceedings of the 10th International AES Conference”). I’ve read the paper, and attended its presentation at an AES conference in London.

More to the point, you’re trying to dodge the essential facts of the Swedish Radio affair; 60 listeners over 20,000 double-blind trials failed to hear an artifact of the codec that was obvious to a single listener using non-blind techniques. No amount of parsing the language of Swedish Radio's mandate gets you around that fact.

Finally, your attempt to dismiss Peter McGrath’s experience merely by the fact that I don’t know the specific D/A converter he was listening through is grasping at straws.


http://www.avguide.com/forums/blind-listen...lawed-editorial

While this may perhaps not meet your definition of "objective evidence", it sure does look like objective evidence to a guy like me.

DBT Is Flawed Because Bob Stuart Says So

Reply #92
I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.
If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again. That's how people are able to detect which one of the three near identical signals contains the almost masked signal in psychoacoustic tests - if what Bob and you were saying scuppered such things, the thresholds measured in such tests would be totally wrong. Yet the threshold measured in such tests matches what's expected from the physiology of the ear. The real trick is to cut cats open, probe their nerves, and intercept the signals on the way to their brains. People used to do that (and worse). We know a lot about hearing from such experiments. We know our conscious limits match the raw signals that are reaching our brains, and we know a lot about the apparatus feeding those signals to our brains.

Cheers,
David.

DBT Is Flawed Because Bob Stuart Says So

Reply #93
While this may perhaps not meet your definition of "objective evidence", it sure does look like objective evidence to a guy like me.
"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"

Goodness, if you think that's "objective", I think we're wasting our time here until you invest in a dictionary!

DBT Is Flawed Because Bob Stuart Says So

Reply #94
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.
You've really shot yourself in the foot here. If this was true, an ABX test, even if it was the most hostile test ever, should be trivial to pass.

How does a "quite enormous" difference disappear completely when someone says the words "which one are you listening to then?"


DBT Is Flawed Because Bob Stuart Says So

Reply #95
I think Bob applies a transmission-line model of the cochlear.
That's cochlea, not cochlear. It's in the book titled "Auditory Neuroscience - Making Sense of Sound" by Jan Schnupp, Israel Nelken and Andrew King.

DBT Is Flawed Because Bob Stuart Says So

Reply #96
I think Bob applies a transmission-line model of the cochlear.
That's cochlea, not cochlear. It's in the book titled "Auditory Neuroscience - Making Sense of Sound" by Jan Schnupp, Israel Nelken and Andrew King.



Let's start:  "how do you even imagine that one can hear a difference between two systems, one with noise 98 dB down and the other 146dB down, when the level is set to peak at 96dB?"

How do you explain how "obviously difference" fails to show up in even the worst kind of ABX test?

I have yet to see a whit of evidence that "high-rez" matters for final presentation to a listener.  Have you any, bearing in mind that citing non-blind-testing proves nothing but the incompetence, the complete and total incompetence, of the person citing it as evidence.

Bear in mind the hard evidence for the persistance of loudness memory while you're at it.
-----
J. D. (jj) Johnston

DBT Is Flawed Because Bob Stuart Says So

Reply #97
If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again.
Once again, our opinions differ. I suggest you watch that Audio Myths Workshop vid I linked to earlier, more in particular the part where Poppy Crum shows how it does work.
Quote
Do you hear an 's'? I think I hear an 's'.

"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"
Add Barry Diament of Soundkeeper Recordings to that long list of 'clever' people. He publicly stated that, in his own opinion, the audible difference between properly recorded 2x (96 kHz or 88.2 kHz) and properly recorded 4x (192 kHz or 176.4 kHz) samplerate is even bigger than the audible difference between 1x (48 kHz or 44.1 kHz) and 2x. He also wrote on ComputerAudiophile forum that, if he could invite you to his recording studio, it would take him two seconds to prove the audible difference between 16/44.1 and 24/192 to you.
How does a "quite enormous" difference disappear completely when someone says the words "which one are you listening to then?"
Judging by my own listening experience, the difference doesn't disappear even one bit.

DBT Is Flawed Because Bob Stuart Says So

Reply #98
How do you explain

Quote
Robert Harley -- Mon, 07/14/2008 - 10:28
My point is that the better-sounding piece of equipment more easily allows you to forget the equipment and focus on the music. Electronic artifacts are a constant reminder that we're listening to a re-creation, rather than to music itself.

Greater musical enjoyment comes not from listening in analytical mode for specific sonic attributes, but through lower awareness of the electro-mechanical system between listener and performer. The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Maybe this is how.

DBT Is Flawed Because Bob Stuart Says So

Reply #99
Quote
The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Fine...  Where are the double-blind tests showing that the listener "feels different" or enjoys the music more when the otherwise inaudible artifacts are removed.

But in fact, Mr. Harley's quote doesn't exactly say that...  He seems to say people enjoy music more on better sounding equipment even though they are not thinking about sound quality or why they are enjoying it more.    That seems reasonable.  And if you asked the person about the sound quality later they might say, "That was a really nice-sounding sound system."

Yes, you may feel better when listening to more expensive equipment that has no audible difference, or to higher resolution audio...  As long as you know what you are listening to (non-blind).