HydrogenAudio

Hydrogenaudio Forum => General Audio => Topic started by: audioclaudio on 2012-08-08 20:17:50

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-08 20:17:50
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part. This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.
Often, if not practically always, it is too time consuming and/or too uneconomical to conduct an ABX test properly, i.e. in such way that expectation bias doesn't creep in through the back door etcetera. Experts in auditory neuroscience and psychoacoustics have gathered experimental evidence which appears to indicate the following. Humans who remember different things are perfectly capable of hearing the same sounds differently as a result of remembering different things. For example, Bob Stuart of Meridian, who has a Ph.D in neuroscience, believes that it is perfectly possible for a person to not hear a specific detail in a piece of music when it is played back on one particular system "A", then to discover this specific detail by listening to the same music again on a better, more resolving system "B" next, and then, finally, to turn back to the previous system "A" and always hear this detail on system "A" even though the detail could previously not be heard on system "A". Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-08 20:51:28
I'm sorry, but how is any of this "the trouble with abx testing"? Rather this is the trouble with determining what is right and wrong when differences are subtle, and abx testing is the tool that makes this possible.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-08 20:56:17
One trouble is the misapplication of ABX results.  The other is the silly argument that there is a problem with the method of testing because the results can be misapplied.

Regarding Bob Stuart, except for the fast switching part, sure, sounds reasonable.  So?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-08 22:20:31
I'm sorry, but how is any of this "the trouble with abx testing"? Rather this is the trouble with determining what is right and wrong when differences are subtle, and abx testing is the tool that makes this possible.

Yes, it is the tool that makes this possible, at least in theory. However, in practice, it is typically more often than not extremely difficult to find enough people who will participate in such testings.

One trouble is the misapplication of ABX results.  The other is the silly argument that there is a problem with the method of testing because the results can be misapplied.

Regarding Bob Stuart, except for the fast switching part, sure, sounds reasonable.  So?

I don't know why you disagree on the fast switching part, because there appears to be substantial evidence to support it. Experts claim a natural process normally occurs in the brain while we listen; it gradually builds a map of sound objects. When combined with auditory stimulae which the brain can manage to logically fit together with it, this map causes our perception to adapt accordingly. With rapid switching, this natural process does not occur.
My point is, if even world's best experts fail to come to an agreement on what's required to be able to design a reliable ABX test, which IMHO appears to be the case BTW, then who are we to judge those who are skeptical towards the objectivists?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-09 00:42:59
I don't know why you disagree on the fast switching part, because there appears to be substantial evidence to support it.

I guess I'd have to take your word for it on that one.

Quote
With rapid switching, this natural process does not occur.

You seem to misunderstand an extremely important part about ABX testing: ABX can only be used to objectively demonstrate differences.  A failed test does not prove two things sound the same to all people under all circumstances for all time.

Quote
My point is, if even world's best experts fail to come to an agreement on what's required to be able to design a reliable ABX test, which IMHO appears to be the case BTW, then who are we to judge those who are skeptical towards the objectivists?

I can come up with all sorts of fanciful things that you cannot disprove. Will you be giving me the same credibility?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-09 12:11:14
Many of these "problems" apply to any listening test where you must compare two sources and decide what you think about them - and that includes sighted listening tests! Therefore it's wrong to raise these "problems" as criticisms of ABX/DBT.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-09 13:44:07
I'm sorry, but how is any of this "the trouble with abx testing"? Rather this is the trouble with determining what is right and wrong when differences are subtle, and abx testing is the tool that makes this possible.

Yes, it is the tool that makes this possible, at least in theory. However, in practice, it is typically more often than not extremely difficult to find enough people who will participate in such testings.

It's actually quite simple. Person A claims to be able to hear something. If he then voluntarily takes an ABX test for, say, a dozen or so trials, he either has provided statistically significant results that support his claim, or else his claim is unsubstantiated (though not necessarily false). Unsubstantiated claims remain just that, unsubstantiated.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-09 19:12:16
I don't know why you disagree on the fast switching part, because there appears to be substantial evidence to support it.

I guess I'd have to take your word for it on that one.

Well, I said "there appears to be" because I am no expert on the matter. There was a TAS interview by Robert Harley with Bob Stuart about it, which was published some years ago.
Quote
With rapid switching, this natural process does not occur.

You seem to misunderstand an extremely important part about ABX testing: ABX can only be used to objectively demonstrate differences.  A failed test does not prove two things sound the same to all people under all circumstances for all time.

No, I understand this and I am not trying to argue against it. However, it implies that, if Bob Stuart is correct about the consequence of rapid switching, ABX might not be capable of demonstrating differences which could be easily noticed in listening sessions during which the natural process would be given the time and chance to occur, so, again, this would indicate ABX is generally very often too time consuming and/or too economically unfeasible.
Quote
My point is, if even world's best experts fail to come to an agreement on what's required to be able to design a reliable ABX test, which IMHO appears to be the case BTW, then who are we to judge those who are skeptical towards the objectivists?

I can come up with all sorts of fanciful things that you cannot disprove. Will you be giving me the same credibility?

This is exactly the problem I was actually trying to describe. If nobody had looked for planets outside the Solar System, nobody would have ever discovered any. That is what science is, or ought to be, all about in the first place. Without science, there can be no objectiveness. If I have to name one thing I have in common with Bob Stuart, it's the fact we both don't believe in Black Magic (and we both love music). 
Many of these "problems" apply to any listening test where you must compare two sources and decide what you think about them - and that includes sighted listening tests! Therefore it's wrong to raise these "problems" as criticisms of ABX/DBT.

Yes, I believe that is entirely correct. It partly explains why I usually prefer not to partake in any listening tests of any kind whatsoever. I am in the same camp as Robert Harley about them. Meaning, differences between sounds are magnified when I listen purely for pleasure rather than for making comparisons. I know, it may seem a little bit strange perhaps, but nevertheless, it's true.
I'm sorry, but how is any of this "the trouble with abx testing"? Rather this is the trouble with determining what is right and wrong when differences are subtle, and abx testing is the tool that makes this possible.

Yes, it is the tool that makes this possible, at least in theory. However, in practice, it is typically more often than not extremely difficult to find enough people who will participate in such testings.

It's actually quite simple. Person A claims to be able to hear something. If he then voluntarily takes an ABX test for, say, a dozen or so trials, he either has provided statistically significant results that support his claim, or else his claim is unsubstantiated (though not necessarily false). Unsubstantiated claims remain just that, unsubstantiated.

Obviously. Like I said though, I suspect that alot of these "unsubstantiated claims" remain unsubstantiated only because ABX itself is what can severely impair one's ability to "hear something", and that this could perhaps help to explain why alot of knowledgeable people, including a fair number of experts (there's that dirty word again...), are leaning more towards, or at least are not so very skeptical towards, the subjectiveness part of the picture than some others might find logical.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-09 19:48:20
If nobody had looked for planets outside the Solar System, nobody would have ever discovered any. That is what science is, or ought to be, all about in the first place.

How does requiring objective confirmation of claims prevent curiosity?

Quote
Meaning, differences between sounds are magnified when I listen purely for pleasure rather than for making comparisons. I know, it may seem a little bit strange perhaps, but nevertheless, it's true.

The needle on my placebo effect meter just pegged.

Funny how otherwise obvious differences vanish under scrutiny, isn't it?

Quote
ABX itself is what can severely impair one's ability to "hear something"


That this is it true is highly questionable.  Next time you present a "fact" I recommend supporting it with evidence rather than anecdotes.

While there may be a limit on one's patience and/or resources which may hinder proper ABX testing, ABX testing does not require arbitrary time limits.  This tired "time limit" excuse still doesn't change where the burden of proof falls when pitting theories that are corroborated with little or no evidence against theories that are corroborated with evidence or are falsifiable but have yet to be falsified.

Regarding "experts" I can find some who say global warming is not real.  I don't know of any in this camp who don't also have some kind of incentive to hold this position, however.  You might consider whether the same can't be said about some of these neuroscientists.  Of course some kind of vested interest doesn't necessarily mean that they are wrong either.  The point is that expert testimony can be cherry-picked.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-09 20:42:51
Obviously. Like I said though, I suspect that alot of these "unsubstantiated claims" remain unsubstantiated only because ABX itself is what can severely impair one's ability to "hear something", and that this could perhaps help to explain why alot of knowledgeable people, including a fair number of experts (there's that dirty word again...), are leaning more towards, or at least are not so very skeptical towards, the subjectiveness part of the picture than some others might find logical.

Let's say that someone claims to be able to hear a difference, but only under conditions A, B and C. Therefore ABX won't work.

Very sinple, do the ABX test under conditions A, B and C and let's see what you can do. If you say that one of the conditions is that it takes you an hour or two to hear the difference, fine. Take your time. Get back to me when you are done. I will still need to see ABX test results.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-09 22:07:35
My sincerest apologies for a mistake I made in my previous post.  For those who might not have seen it in original form, I had interpreted "differences between sounds are magnified when I listen purely for pleasure [...] it's true" as universally applying to all people.

FWIW I am currently performing MUSHRA tests under contract and can absolutely assure you with no uncertainty that there are many details that would have otherwise gone overlooked had I been listening just for pleasure.  Perhaps if you were ever to stake your reputation and put a decent wage on the line by attempting to demonstrate that you can hear subtle differences between two things you would likely find the same thing.  Or, perhaps you have comparable experience and have simply come to the opposite conclusion. If so I'd like to hear about it.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-10 00:56:17
Quote
If nobody had looked for planets outside the Solar System, nobody would have ever discovered any. That is what science is, or ought to be, all about in the first place.

How does requiring objective confirmation of claims prevent curiosity?

Those who did keep on looking for planets outside the Solar System were about to give up on it because nobody wanted to believe them and therefore nobody cared to invest in "that kind of research". As soon as the objective confirmation hit the news like a bombshell, new questions were raised, as people from all over the world all of a sudden did become very curious about these planets.
ABX itself is what can severely impair one's ability to "hear something"

There you go again.

While there may be a limit on one's patience and/or resources which may hinder proper ABX testing, ABX testing does not require arbitrary time limits.  This tired "time limit" excuse still doesn't change how the burden of proof falls when dealing with falsifiable claims vs. unfalsifiable claims.

Regarding "experts" I can find some who say global warming is not real.  I don't know of any in this camp who don't also have some kind of incentive to hold this position, however.  You might consider whether the same can't be said about some of these neuroscientists.  Of course some kind of vested interest doesn't necessarily mean that they are wrong either.  The point is that expert testimony can be cherry-picked.

I agree. I never said ABX testing is an invalid way of testing. If an ABX test fails, all that means is further testing is in order. But demanding further testing from a group of people is one thing. Getting that same group of people to become curious enough to participate in such further testing is still usually a whole lot less doable IMO. It depends on how big the reward is versus how much effort it takes, I guess. Personally, I, find rapid switching in an ABX test more stressful than almost anything.
Obviously. Like I said though, I suspect that alot of these "unsubstantiated claims" remain unsubstantiated only because ABX itself is what can severely impair one's ability to "hear something", and that this could perhaps help to explain why alot of knowledgeable people, including a fair number of experts (there's that dirty word again...), are leaning more towards, or at least are not so very skeptical towards, the subjectiveness part of the picture than some others might find logical.

Let's say that someone claims to be able to hear a difference, but only under conditions A, B and C. Therefore ABX won't work.

Very sinple, do the ABX test under conditions A, B and C and let's see what you can do. If you say that one of the conditions is that it takes you an hour or two to hear the difference, fine. Take your time. Get back to me when you are done. I will still need to see ABX test results.

What if condition A says I can only hear a difference when listening purely for pleasure, condition B says I can't listen purely for pleasure while I'm being forced to listen for differences instead, and condition C says ABX won't work because ABX inevitably forces me to listen for differences? That would be a deadlock situation. It's only a hypothesis, sure. But then, how exactly were you going to prove that the difference I am hearing is called a hallucination? Long story short, ABX and DBT are very valuable tools. They just ain't always perfect for the job.
My sincerest apologies for a mistake I made in my previous post.  For those who might not have seen it in original form, I had interpreted "differences between sounds are magnified when I listen purely for pleasure [...] it's true" as universally applying to all people.

FWIW I am currently performing MUSHRA tests under contract and can absolutely assure you with no uncertainty that there are many details that would have otherwise gone overlooked had I been listening just for pleasure.  Perhaps if you were ever to stake your reputation and put a decent wage on the line by attempting to demonstrate that you can hear subtle differences between two things you would likely find the same thing.  Or, perhaps you have comparable experience and have simply come to the opposite conclusion. If so I'd like to hear about it.

No worries, just because your avatar looks like an extremist doesn't also mean I think you are one.  In fact, my first record that I owned as a kid was the 7" 45 rpm single of the Smurf song.
The differences in sound that are magnified when I listen purely for pleasure are not really audible to me in the sense that I can easily describe them, but they do have a big enough impact on things like fatiguingness and involvingness for me to still notice them in the long run. It can take several weeks, and sometimes it even takes months.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-10 01:39:19
The differences in sound that are magnified when I listen purely for pleasure are not really audible to me in the sense that I can easily describe them, but they do have a big enough impact on things like fatiguingness and involvingness for me to still notice them in the long run. It can take several weeks, and sometimes it even takes months.

...and they can be shown through ABX testing since ABX testing has no inherent time limits.  Absent of objective data, I call bullshit.

Why should I believe this over placebo effect?  Aside from anecdotes arriving from sighted tests, can you name two independent studies that demonstrate either fatigue or feeling more or less involved is the difference between otherwise identical sounds?  Which do believe has a larger body of objective supporting evidence, placebo effect or fatigue/involvement?  The fatigue/involvement card has been played many times on this forum already and has never made it past "anecdotal."  Here's your opportunity to raise its status to "credible."  Will you rise to the occasion or will you do like everyone before you and simply make excuses for what is continuing to squarely solidify as an excuse?

Let's be clear: if you can qualify perceived differences in sound then you should be able to test for them through ABX.  If removing expectation bias eliminates the ability to perceive these differences then one really must question what is actually influencing the perception, the quality originally theorized or expectation bias.

BTW, fast switching is an tool that may or may not be available to the participant.  When it is available, its use should be optional.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-10 10:18:08
What if condition A says I can only hear a difference when listening purely for pleasure, condition B says I can't listen purely for pleasure while I'm being forced to listen for differences instead, and condition C says ABX won't work because ABX inevitably forces me to listen for differences? That would be a deadlock situation. It's only a hypothesis, sure. But then, how exactly were you going to prove that the difference I am hearing is called a hallucination? Long story short, ABX and DBT are very valuable tools. They just ain't always perfect for the job.

You've moved outside science and into "religion", because you have an untestable belief. Untestable, because we know that expectation bias/placebo exists, but you believe there's a real difference that only exists in situations where it's not possible to remove expectation bias / placebo.

Therefore there is no test that can differentiate between an imagined difference, and a "real" difference that only "exists" in situations where an imagined difference cannot be ruled out.

This is heading into the realms of philosophy, or trees falling down in forests that don't make a sound because no one is there to hear them. It's not testable by science. It's not testable at all.

Convenient that, isn't it?


Your most extreme position, that even thinking about whether you hear something (rather than just listening entirely for pleasure) will remove the difference, at first sound a little like the problem of observation changing the particles being observed in physics. Or Schrödinger's cat. (http://en.wikipedia.org/wiki/Schr%C3%B6dinger's_cat). Except, at best, it's about your state-of-mind, and the reality of the audio signal received by your ears exists independently of your state of mind.


I think the convincing thing about blind listening is this: even while listening "blind", many people are still convinced they hear a difference. It's not the act of blinding the sources or being made to compare them that makes the difference go away. It continues to exist in the mind of the observers - they report that they hear it. Their answers show that they do not.

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-10 10:57:59
Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.


I found the source for that (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=2). Haven't fully read it yet myself.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-10 14:14:38
Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.


I found the source for that (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=2). Haven't fully read it yet myself.

Yes, and the page before that. http://www.avguide.com/article/tas-194-mer...t-harley?page=1 (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=1)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: krabapple on 2012-08-11 01:23:04
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part. This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.
Often, if not practically always, it is too time consuming and/or too uneconomical to conduct an ABX test properly, i.e. in such way that expectation bias doesn't creep in through the back door etcetera. Experts in auditory neuroscience and psychoacoustics have gathered experimental evidence which appears to indicate the following. Humans who remember different things are perfectly capable of hearing the same sounds differently as a result of remembering different things. For example, Bob Stuart of Meridian, who has a Ph.D in neuroscience, believes that it is perfectly possible for a person to not hear a specific detail in a piece of music when it is played back on one particular system "A", then to discover this specific detail by listening to the same music again on a better, more resolving system "B" next, and then, finally, to turn back to the previous system "A" and always hear this detail on system "A" even though the detail could previously not be heard on system "A". Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.



So, what do you mean by 'fast switching'?  Do you mean, using short audio samples, or do you mean, making the actual switching moment as instantaneous as possible?

I suspect you mean the former.  If so, that's not 'fast switching'  And btw, how long ago did Bob Stuart get his PhD, and was it specifically in 'neuroscience'?  He's been running Meridian since 1977.  Has he kept up with the neuroscience literature since then?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: krabapple on 2012-08-11 01:26:59
Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.


I found the source for that (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=2). Haven't fully read it yet myself.

Yes, and the page before that. http://www.avguide.com/article/tas-194-mer...t-harley?page=1 (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=1)



"What I think is outrageous is to say we understand everything about how the human hearing system works, " 

Well, Bob, who is saying that, really? 
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-11 09:58:38
Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.

Yes, and the page before that. http://www.avguide.com/article/tas-194-mer...t-harley?page=1 (http://www.avguide.com/article/tas-194-meridian-audios-bob-stuart-talks-robert-harley?page=1)


"What I think is outrageous is to say we understand everything about how the human hearing system works, " 

Well, Bob, who is saying that, really?


Can we all say "Straw man argument" ;-)

The reference seems very confused or confusing;

Quote
"Robert: That brings to mind a conversation we had at CES about why blind listening tests may not be reliable. You said that when exposed to sound, our brain builds a model over time of what’s creating that sound. The rapid switching in blind testing doesn’t allow that natural process to occur, and we get confused.

Bob: That’s right. Perception happens on lots of different time scales.


Well, yes!

Quote
There’s something called the conscious present, which is a period of time over which some of this integration into an object would happen.


No!  Counscious(ness) is being aware of one's own existence, sensations, thoughts, surroundings, etc

(two big problems: First the correct form of the word was not used, and secondly consciousness acts over time where time is an independent parameter. Time is not consciousness itself.) These guys are obviously way over their head even entering this discussion!

If you were dropped into a concert hall, how long would it take you to really understand what it is you’re hearing? It can take several seconds, or even minutes, before you’re listening fully into the space.

Quote
Sometimes when you’re looking for a difference between A and B, you can hear it quickly. Other times the difference between A and B can come on a time scale of minutes or even longer where you find that you’ve changed something and you don’t notice a change but find that you have a very different connection to the music. But if you are doing quick switching that mechanism gets broken.


Major conflation!  Small details in sound only remain in the brain for a very short period of time - the time scale is seconds at most. You may remember that you heard a small detail that you converted into a memory, but the actual sensation related to the small detail comes and goes almost instantly.


Quote
The problem with A/B switching, or blind listening tests,


Another major error involving conflation - blind tests and A/B switching are completely independent. You can do either independent of the other. Equating them again shows that the people talking are way out of their depth


Quote
is that it doesn’t always eliminate things that we find to be important on a lot of time scales.


Hello! Aren't we talking about hearing small details and not eliminating the perception of them?

Quote
Obviously you can do blind listening on long time scales, and that’s good.


Actually, it has been found that listening on long time scales erases the reliable perception of small details. The most important use of long term listening is to search for short passages of music where the audible difference we are investigating is portrayed most clearly.h


All of the confusion I see above IME is greatly helped by true blind testing. Has either done more than diddle around with blind testing? I don't see it here!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-11 10:27:07
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part.


Wow! I've never seen so many critical mistakes in one sentence!

First off there is no intent that ABX tests be objective. The goals are that they be relevant, reliable and accurate. Tests involving listening by people are inherently subjective and that is a good thing. In fact the goal is to involve subjects (people) and in some sense be as subjective as possible.

The reason for collecting modest numbers of test results is to manage the inherent variations in subjective tests near the threshold of reliable detection. If a test necessarily involves a large number of trials, the effect being studied is probably so small that it is actually not significant to normal listening. When something is clearly audible statistical significance is easily obtained with a fairly small number of trials.

Quote
This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.


Wrong. The above is only true for very small differences. Trained listeners can be very accurate and reliable if they are listening to a difference that is even just a little bit above the threshold of audibility.

One of the more obvious things about listening tests that one obtains from practical experience is that when they involve things that can be actually heard, they are generally fast and easy.

Most of the anecdotes about zillions of trials and frustrated listeners come from people who in retrospect were trying to hear things that aren't audible.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 10:34:15
So, what do you mean by 'fast switching'?  Do you mean, using short audio samples, or do you mean, making the actual switching moment as instantaneous as possible?

I suspect you mean the former.  If so, that's not 'fast switching'

From what I can read in the article, it means using short audio samples.
And btw, how long ago did Bob Stuart get his PhD, and was it specifically in 'neuroscience'?  He's been running Meridian since 1977.  Has he kept up with the neuroscience literature since then?

He's still a very highly acclaimed expert in psychoacoustics, the "golden ears" type of person, and IIRC he is still one of the most experienced members of the AES (Audio Engineering Society) group. He is also famed for being the inventor of (IMO) breakthrough innovative technologies such as the apodizing filter, which BTW has recently been adopted by Dolby TrueHD, and for the work he has done in the development of the MLP (Meridian Lossless Packing) codec, which is commonly used in DVD-Audio, HD DVD and Blu Ray.
"What I think is outrageous is to say we understand everything about how the human hearing system works, " 

Well, Bob, who is saying that, really?

People who believe ABX is ALWAYS the perfect tool under ANY condition, and who demand objective evidence about EVERYTHING, but who, at the same time also, fail to provide objective evidence to backup their belief system?
From the same article:
"The problem with A/B switching, or blind listening tests, is that it doesn’t always eliminate things that we find to be important on a lot of time scales."
A bit further down the paragraph:
"What we’re looking for is not only that we can hear a difference but also that it is more musically satisfying."
You could always try and find objective evidence to show it's bullshit. However... :cough: I wish you the best of luck with that! :cough:
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-11 11:07:33
So, what do you mean by 'fast switching'?  Do you mean, using short audio samples, or do you mean, making the actual switching moment as instantaneous as possible?

I suspect you mean the former.  If so, that's not 'fast switching'

From what I can read in the article, it means using short audio samples.


Actually, you need both.


Quote
Quote from:  link=msg=804891 date=0
And btw, how long ago did Bob Stuart get his PhD, and was it specifically in 'neuroscience'?  He's been running Meridian since 1977.  Has he kept up with the neuroscience literature since then?

He's still a very highly acclaimed expert in psychoacoustics, the "golden ears" type of person, and IIRC he is still one of the most experienced members of the AES (Audio Engineering Society) group.


In your opinion, which based on your posts seems to be poorly-informed in this area.

Quote
He is also famed for being the inventor of (IMO) breakthrough innovative technologies such as the apodizing filter, which BTW has recently been adopted by Dolby TrueHD, and for the work he has done in the development of the MLP (Meridian Lossless Packing) codec, which is commonly used in DVD-Audio, HD DVD and Blu Ray.


None of which gives someone chops related to doing proper subjective tests.


Quote from: audioclaudio link=msg=0 date=
People who believe ABX is ALWAYS the perfect tool under ANY condition,


Straw man argument.

Quote from: audioclaudio link=msg=0 date=
and who demand objective evidence about EVERYTHING



Straw man argument compounded by abuse of the word "objective"


Quote from: audioclaudio link=msg=0 date=
but who, at the same time also, fail to provide objective evidence to backup their belief system?


I would summarize my belief system in this area as being dominated by the idea that sighted evaluations involving small audible differences are strongly influenced by false positives. What proof of that do you require?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 14:08:23
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part.


Wow! I've never seen so many critical mistakes in one sentence!

First off there is no intent that ABX tests be objective. The goals are that they be relevant, reliable and accurate. Tests involving listening by people are inherently subjective and that is a good thing. In fact the goal is to involve subjects (people) and in some sense be as subjective as possible.

The reason for collecting modest numbers of test results is to manage the inherent variations in subjective tests near the threshold of reliable detection. If a test necessarily involves a large number of trials, the effect being studied is probably so small that it is actually not significant to normal listening. When something is clearly audible statistical significance is easily obtained with a fairly small number of trials.

You basically just confirmed the whole thing I said. Because the problem with ABX testing is the fact it cannot always be used to produce truly objective results, it cannot always be considered relevant, accurate and reliable.
http://en.wikipedia.org/wiki/Objectivity_(science) (http://en.wikipedia.org/wiki/Objectivity_(science))
Quote
This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.


Wrong. The above is only true for very small differences. Trained listeners can be very accurate and reliable if they are listening to a difference that is even just a little bit above the threshold of audibility.

That is only your opinion. What you call "small differences" might very well be less small to me.
One of the more obvious things about listening tests that one obtains from practical experience is that when they involve things that can be actually heard, they are generally fast and easy.

Most of the anecdotes about zillions of trials and frustrated listeners come from people who in retrospect were trying to hear things that aren't audible.

...Or the bias that ABX fails to eliminate is exactly what's causing the fact things aren't audible.

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 14:54:09
Actually, you need both.

Bob Stuart says it breaks the natural process if short audio samples are used. By breaking the natural process, bias is introduced inevitably. My point is not whether or not Bob Stuart is correct about what he says. It's the simple fact that, although I haven't, ironically you still haven't proven anything at all, either. So please, just stop trying to pretend that you have.
Quote
Quote from:  link=msg=804891 date=0
And btw, how long ago did Bob Stuart get his PhD, and was it specifically in 'neuroscience'?  He's been running Meridian since 1977.  Has he kept up with the neuroscience literature since then?

He's still a very highly acclaimed expert in psychoacoustics, the "golden ears" type of person, and IIRC he is still one of the most experienced members of the AES (Audio Engineering Society) group.


In your opinion, which based on your posts seems to be poorly-informed in this area.

Quote
He is also famed for being the inventor of (IMO) breakthrough innovative technologies such as the apodizing filter, which BTW has recently been adopted by Dolby TrueHD, and for the work he has done in the development of the MLP (Meridian Lossless Packing) codec, which is commonly used in DVD-Audio, HD DVD and Blu Ray.


None of which gives someone chops related to doing proper subjective tests.

As per my above comment, none of your contributions to this thread give you chops about it so far, either...
Quote
People who believe ABX is ALWAYS the perfect tool under ANY condition,


Straw man argument.

Quote
and who demand objective evidence about EVERYTHING



Straw man argument compounded by abuse of the word "objective"

You still don't get the sarcasm in any of this part, do you?   
Quote from:  link=msg=804914 date=0
Quote from: audioclaudio link=msg=0 date=
but who, at the same time also, fail to provide objective evidence to backup their belief system?


I would summarize my belief system in this area as being dominated by the idea that sighted evaluations involving small audible differences are strongly influenced by false positives. What proof of that do you require?

I don't require proof of that at all, because as a matter of fact I never disagreed with it. The McGurk effect can never be denied, for example. My stance on sighted evaluations is the same as Bob Stuart's, meaning I am convinced there are at least SOME situations where they CAN be more relevant, accurate and reliable than double blinded evaluations, even if the differences in sound are said to be "small".
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: [JAZ] on 2012-08-11 16:20:52
audioclaudio and Arnold  : These two last pages of the thread are really difficult to follow, and don't really help at all to make a decision that answers the topic's title.


First of all, I would like to rewind and set a straight and concrete meaning of several words, since it's not really clear what you're against:

- Objective test/Objectivism ( http://en.wikipedia.org/wiki/Objectivism (http://en.wikipedia.org/wiki/Objectivism) . In this thread, really http://en.wikipedia.org/wiki/Objectivity_%28science%29 (http://en.wikipedia.org/wiki/Objectivity_%28science%29) ) : The main idea behind this is that the person doing the test does not matter, since the reality exists independently of the subject taking the test (1 meter is 1 meter no matter who checks it) . Objectivity does make possible to repeat tests and *verify* a result.

- Subjective test/Subjectivism ( http://en.wikipedia.org/wiki/Subjectivism (http://en.wikipedia.org/wiki/Subjectivism) ) : The main idea under subjectivism is that we know the world thanks to our sensory system, and as such, only what we can perceive is what exists (ok. This is just one interpretation. Subjectivism has many variants, from phylosophical, to theological).


- ABX Test: An Objective test that evaluates the response (subjective or not) of the participants to the items at test, and obtains a (statistically) representable result that can be considered reliable and, as such, valid for the specific group at which the study was directed.

- ABX Test, application: An ABX test can only say that a difference exists between the two items being evaluated by the determined subgroup at which the study is directed.  (Concretely, this means that it can never be used to say that a difference does not exist. At much, with many failed tests, it can be an indication that such difference is not probable).

In case of Hydrogenaudio, this means that it allows us to *verify* that an *audible difference* between two audio samples (being them two different codecs, two soundcards, two amplifiers....) does exist for the person or people involved in doing the ABX test. If enough tests validate this, then, and only then, can be considered that the difference is real and not due to some error in the test.


Hydrogenaudio is an Objectivism Forum. Let's put that straight and undeniably clear. The high emphasis on ABX tests, the relation with the science world, and the participation of people directly related to the subject of this forum (like the actual coders of audio codecs) is an indication of it.



Reasons why ABX tests are a requirement in order to get objective tests, reliable and representable for the intended audience:


A) Expectation bias ( http://en.wikipedia.org/wiki/Expectation_bias (http://en.wikipedia.org/wiki/Expectation_bias) ): A subject, intentionally or not, can be influenced by its subconsciusness: Someone listening to a song in a newly bought equipment does have expectation bias when evaluating its sound quality against an older, less good equipment. (while a difference might exists, the subject could be perceiving his feeling that a difference exists and not the real difference).

B) Representative: To pass an ABX test, it is required to correctly identify a percentage of the trials, and they cannot be cherry picked. ABX pretends to reduce the possibilities of: chance/luck, hidden elements at test (i.e. a difference caused by something that is not the item at test), unconscious influence from other subjects (that's the requirement for the test to be *double* blind).


Setting up an audio ABX test is not necesarily easy, Here are some examples of what can go wrong:

- Playing at different sample rates: soundcards, soundcard drivers, OS audio stack, or even the playback application can cause problems not related to the original audio, but to the different sampling rate being used.
- Volume mismatch: It is important that the samples at test are correctly leveled, so that the subject evaluates the audio, not the difference in audio amplitude. Differences of less than 1dB can be audible.
- Clipping/distortions caused by playback settings/equipment: It is not strange that audio clips digitally today. While the clip in itself could be heard and could determine a limitation or problem depending on the intended use, the problem could be gone just by setting the amplification properly.



When is an ABX necessary:

ABX test, as said above, help to determine that a difference exists. For Hydrogenaudio it is required when an *audible* difference is reported and:

- It is not commonly known that it exists
- It is against the general knowledge that says such difference should not be audible.
- It is not measurable by other means that could clearly determine otherwise (i.e. an audible difference in lossless codecs can be discarded if the codec works and so the problem, if any, exists somewhere else).



To finish this post, I'd like to add that ABX is not intended to avaluate if an item is better than another. In order to determine such thing, it is necessary that they are found to be different. Once that is verified, the difference can be avaluated between both items.

It's been said that ABX does not impose a limitation of how long/short a listening period runs for, neither puts it limits in how many times you switch back an forth to determine a difference/preference.
Also note that the existence of a difference is objective. The avaluation of the difference, while some rules apply, is more subjective.
That's why on the public listening tests, the statistic methods (like MUSHRA) that are applied on the final results of the group try to reduce the influence of subjective decisions.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-11 16:39:37
I am convinced there are at least SOME situations where they CAN be more relevant, accurate and reliable than double blinded evaluations, even if the differences in sound are said to be "small".


Convinced by what?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 18:56:53
Can we all say "Straw man argument" ;-)

Actually no, you can't...
The reference seems very confused or confusing;

To me, it all seems very perfectly clear.
No!  Counscious(ness) is being aware of one's own existence, sensations, thoughts, surroundings, etc

(two big problems: First the correct form of the word was not used, and secondly consciousness acts over time where time is an independent parameter. Time is not consciousness itself.) These guys are obviously way over their head even entering this discussion!

Eh?

http://books.google.be/books?id=s52lUTjYPH...%22&f=false (http://books.google.be/books?id=s52lUTjYPHoC&pg=PA150&lpg=PA150&dq=%22conscious+present%22&source=bl&ots=cU2JATD-j-&sig=fCz3hK619dZRi_9ZyIuOz_C5GDA&hl=nl&sa=X&ei=cXUmUJu9OcOChQeY8ICABw&ved=0CEAQ6AEwADgy#v=onepage&q=%22conscious%20present%22&f=false)

http://psycnet.apa.org/books/13261/011 (http://psycnet.apa.org/books/13261/011)
but the actual sensation related to the small detail comes and goes almost instantly.

Says who?
blind tests and A/B switching are completely independent. You can do either independent of the other. Equating them again shows that the people talking are way out of their depth

Where exactly do you see Bob Stuart equating them? 
Quote
is that it doesn’t always eliminate things that we find to be important on a lot of time scales.


Hello! Aren't we talking about hearing small details and not eliminating the perception of them?

We are talking about eliminating bias. I thought so much was obvious...
Actually, it has been found that listening on long time scales erases the reliable perception of small details. The most important use of long term listening is to search for short passages of music where the audible difference we are investigating is portrayed most clearly.

Yes, listening on long time scales CAN erase the reliable perception of small details. However, the same thing applies to listening on short time scales. Both short AND long time scales are important to me, because I normally listen to music for both short and long periods of time. Or were you trying to suggest that musicians ought to not create long pieces of music just because it partly erases my perception? 
All of the confusion I see above IME is greatly helped by true blind testing. Has either done more than diddle around with blind testing? I don't see it here!

No offense, but the main reason why you're so confused IMO is simply because you seem to know very little about the science of psychoacoustics. Well... neither do I, like I already said but, I think I can safely guarantee that Bob Stuart does know one heck of alot more about it than all of the people who replied to this thread combined.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 19:15:11

neither puts it limits in how many times you switch back an forth to determine a difference/preference.

Had you actually even read the interview article, perhaps you would have known that switching back and forth is exactly the kind of thing that inevitably introduces bias except maybe if the test subjects are people who have the memory of a goldfish? 
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-11 19:30:16
I am convinced there are at least SOME situations where they CAN be more relevant, accurate and reliable than double blinded evaluations, even if the differences in sound are said to be "small".


Convinced by what?

By believing in Black Magic. =P
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: [JAZ] on 2012-08-11 21:35:34

neither puts it limits in how many times you switch back an forth to determine a difference/preference.

Had you actually even read the interview article, perhaps you would have known that switching back and forth is exactly the kind of thing that inevitably introduces bias except maybe if the test subjects are people who have the memory of a goldfish? 


Why are you always running away? I've said that it imposes no limits. It does not mean that you *have to*. And if that introduces bias, then, take a rest and come back.
There is nothing innerently wrong in asking the subject if it can correctly identify an item. The only limitations imposed by the ABX test is in avoiding conscious or unconscious preference for one of the items being tested.


Anyway, If that's the only thing of my reply that you find wrong, let's leave it that way and please, specify in which terms (examples or situations) a sighted test can be more reliable than a blind test, as you said in your last sentence, and which you completely avoided to answer at dhromed's request.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: bennetng on 2012-08-12 05:51:11
Quote
What if condition A says I can only hear a difference when listening purely for pleasure, condition B says I can't listen purely for pleasure while I'm being forced to listen for differences instead, and condition C says ABX won't work because ABX inevitably forces me to listen for differences? That would be a deadlock situation. It's only a hypothesis, sure. But then, how exactly were you going to prove that the difference I am hearing is called a hallucination? Long story short, ABX and DBT are very valuable tools. They just ain't always perfect for the job.


What if condition A says a person can tell lies, condition B says I can't believe a person if I can't prove he is telling the truth or not, and condition C says non-blind tests won't work because such methods can't prevent a person from telling lies? Remember, what you are doing here is to make other people believe that you can hear the differences. In order to achieve this you need to provide a proof that you are not lying. If you can't, why should I believe you?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 11:47:28
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part.


Wow! I've never seen so many critical mistakes in one sentence!

First off there is no intent that ABX tests be objective. The goals are that they be relevant, reliable and accurate. Tests involving listening by people are inherently subjective and that is a good thing. In fact the goal is to involve subjects (people) and in some sense be as subjective as possible.

The reason for collecting modest numbers of test results is to manage the inherent variations in subjective tests near the threshold of reliable detection. If a test necessarily involves a large number of trials, the effect being studied is probably so small that it is actually not significant to normal listening. When something is clearly audible statistical significance is easily obtained with a fairly small number of trials.


You basically just confirmed the whole thing I said.


No, I denied it for reasons that you have not addressed, and now you have added further illogic to the discussion.

Quote
Because the problem with ABX testing is the fact it cannot always be used to produce truly objective results, it cannot always be considered relevant, accurate and reliable.


Excluded middle argument. Nothing in the real  world is perfect or ideal.

In the real world we have sighted evaluations as supported by many are just so horribly flawed in so many of the cases that they are exclusively relied upon! This is even though better alternatives exist and are well known! It is hard for me to believe that so much of what is said by well-meaning but totally mislead people is even said any more!

Since you have raised your criteria for acceptability to utter perfection, please explain how you even get out of bed in the morning? ;-)

Are you seriously telling me that by the criteria of relevant, accurate and reliable, sighted evaluations are indeed perfect as compared to any of the well-known blind testing paradigms as they apply? (note that you have made the common mistake of lumping all blind testing into ABX)


Quote
Quote from:  link=msg=804906 date=0
Quote
This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.


Wrong. The above is only true for very small differences. Trained listeners can be very accurate and reliable if they are listening to a difference that is even just a little bit above the threshold of audibility.



That is only your opinion.


Just mine, eh? ;-)

No its a fact that anybody can find out for themselves if they are willing to do a little home work.

You aren't very well read, are you?

Quote
What you call "small differences" might very well be less small to me.


I'm not going to go into the world of dueling perceptions with you. The above sentence is speculation without proof or even evidence. It merits no reply.


Quote
Quote from:  link=msg=804906 date=1344677227
One of the more obvious things about listening tests that one obtains from practical experience is that when they involve things that can be actually heard, they are generally fast and easy.

Most of the anecdotes about zillions of trials and frustrated listeners come from people who in retrospect were trying to hear things that aren't audible.


...Or the bias that ABX fails to eliminate is exactly what's causing the fact things aren't audible.


The above sentence doesn't make any sense to me. Please clarify.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 11:57:00
Actually, it has been found that listening on long time scales erases the reliable perception of small details. The most important use of long term listening is to search for short passages of music where the audible difference we are investigating is portrayed most clearly.

Yes, listening on long time scales CAN erase the reliable perception of small details.


Thanks for that!

Quote from: audioclaudio link=msg=0 date=
However, the same thing applies to listening on short time scales.


Proof?

I don't think you've noticed the obvious: If listening on both long and short scales erases the perception of small details, then small details can't ever be perceived. Well that is unless you can figure out how to avoid both situations.

You need to read "This is Your Brain on Music". It lays out the current science about the perception of music. When you've followed up on all of its footnotes, let me know what you think.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-12 11:58:26
Re. bennetng's post:

Calling someone who refuses an ABX test a liar isn't going to get the discussion very far, though in this case I don't know that it will much matter since it has (again) been steered away from science into the realm of religion. Besides being inflammatory, people suffering from placebo effect actually consciously perceive a difference, just that this perception was not formed from external stimuli that has reached the outer ear.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 12:03:10
No offense, but the main reason why you're so confused IMO is simply because you seem to know very little about the science of psychoacoustics.


You get to be wrong... very wrong!

One obvious fact is that nothing has actually been said about psychoacoustics so far, and yet you brought it up!

Quote
Well... neither do I, like I already said


That's clear, but you seem to be unable to act rationally on that knowledge.


Quote from:  link=msg=804956 date=0
but, I think I can safely guarantee that Bob Stuart does know one heck of alot more about it than all of the people who replied to this thread combined.


Face it, you're a fan!

If Bob Stuart knew very much at all about modern psychoacoustics and believed it, his company's product line would be vastly different and he would have probably made far more real advances in audio.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 12:13:02


First of all, I would like to rewind and set a straight and concrete meaning of several words, since it's not really clear what you're against:

- Objective test/Objectivism ( http://en.wikipedia.org/wiki/Objectivism (http://en.wikipedia.org/wiki/Objectivism) . In this thread, really http://en.wikipedia.org/wiki/Objectivity_%28science%29 (http://en.wikipedia.org/wiki/Objectivity_%28science%29) ) : The main idea behind this is that the person doing the test does not matter, since the reality exists independently of the subject taking the test (1 meter is 1 meter no matter who checks it) . Objectivity does make possible to repeat tests and *verify* a result.

- Subjective test/Subjectivism ( http://en.wikipedia.org/wiki/Subjectivism (http://en.wikipedia.org/wiki/Subjectivism) ) : The main idea under subjectivism is that we know the world thanks to our sensory system, and as such, only what we can perceive is what exists (ok. This is just one interpretation. Subjectivism has many variants, from phylosophical, to theological).


I snipped much of your post because it is IMO so good and so non-controversial.

The main point I'd like to make here is that as I understand objectivity and subjectivity which is almost identical to what you posted as far as it goes, they are compatible and actually complement and may even demand each other.

BTW. I see test equipment and test methodologies as "perception enhancers".


The golden ears fail right at this basic point. This was the first point I made in my 2005 debate with John Atkinson. It appears that none of that king's horses and none of that king's men could ever put him back together again. That's because he fell off the fence of Reason so long that his yolk and white had congealed. ;-)

They need for there to be a conflict between the two even though it need not exist. That's just part of the crumbling foundation of their grotesquely flawed dogma. The dogma needs to exist to support their commercial interests. It's all about money, with the irony that if they straightened up and flew right, they'd probably make a ton more money and push Audio miles ahead of where it is.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-12 12:16:13
Re. Knowledge and belief in modern psychoacoustics:
That's assuming this knowledge actually influenced product development instead of only being used as bonifides in order to market to the gullible more effectively.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 12:36:10
Re. Knowledge and belief in modern psychoacoustics:
That's assuming this knowledge actually influenced product development instead of only being used as bonifides in order to market to the gullible more effectively.



Right, and proof that the above is dominating in the man's life is the fact that he is still flogging the market with an "enhanced CD player".

Considerable evidence was presented in the late 1980s by Masters and Clark and never effectively rebutted, that the good CD players of that day were sonically transparent. The first generation players weren't sonically transparent but most if not all of the audible flaws were tiny and only noticeable under fairly artificial conditions.

People like Ethan and I and some other HA contributors and at least one published, refereed paper in the JAES have revisited this issue several times since then and it still seems to be true.

In a rational world, people would simply leave CD players to people who want to do things like that and move on. Instead many people keep making up and telling fairy tales so that they can continue to flog the market and milk the big bucks from the poorly-informed.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Wombat on 2012-08-12 18:36:53
Meanwhile in a different place our member Julf did a blog entry that relates very well to the things you discuss here. An interesting food for thought imho
http://www.computeraudiophile.com/blogs/ju...c-thinking-246/ (http://www.computeraudiophile.com/blogs/julf/double-blind-control-experiment-and-scientific-thinking-246/)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-12 20:30:05
I am convinced there are at least SOME situations where they CAN be more relevant, accurate and reliable than double blinded evaluations, even if the differences in sound are said to be "small".

Convinced by what?

By believing in Black Magic. =P

I asked a serious question. Things happened to you that convinced you. I would like to know what these things are. Maybe it was an article you read, maybe it was a thought experiment you did, maybe it was a weird experience with a set of speakers.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-12 22:35:34
My stance on sighted evaluations is the same as Bob Stuart's, meaning I am convinced there are at least SOME situations where they CAN be more relevant, accurate and reliable than double blinded evaluations, even if the differences in sound are said to be "small".


The evidence against that viewpoint is more than overwhelming. Sighted evaluations are  completely invalidated by a stupendously well-documented, and readily observable tendency towards false positives, except when the difference is outstandingly readily perceptiable. IOW, if you can ABX the difference perfectly with both ears plugged, then it is probably obvious enough that you may not need a blind test to confirm.

Of course the issue of sight in audiophile evaluations is only part of the long list of grotesque problems with almost all audiophile listening tests.

In general audiophile listening tests:

(1) Are not level-matched.
(2) Don't offer close comparisons.
(3) Are often not even done in the same room.
(4) Are often not even done on the same day.
(5) Don't involve using the identical same program material.

GMAB!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 10:00:41
Quote from:  link=msg=804956 date=0
but, I think I can safely guarantee that Bob Stuart does know one heck of alot more about it than all of the people who replied to this thread combined.


Face it, you're a fan!

If Bob Stuart knew very much at all about modern psychoacoustics and believed it, his company's product line would be vastly different and he would have probably made far more real advances in audio.
I'm a fan too - they've also done what even you would probably call "real advances in audio" - e.g. active speakers and DSP.

audioclaudio's implication that Bob Stuart is some sole acting genius, and the individual author of the technologies audioclaudio mentions, is ludicrous - he works with others (often Peter Craven, but you will find many other alumni of audio engineering, past and present, as co-authors on papers and patents). There are actual real world engineers working at Meridian developing and implementing some of this stuff.

I think audioclaudio underestimates the knowledge base here at HA. It would be rude to name drop, but it would be wise if audioclaudio wouldn't assume that one man "knows more" than everyone here.

It would be wise to ignore anything that the marketing department of any company puts out.


The arguments raised against double blind testing raised in this thread are old news, and have been rebutted many times. At the end of the day, it comes down to listening in the same way as you would listen in a sighted test, removing all other variables, not knowing what you're listening to, and checking how likely your answers are down to chance. It's not some mind altering madness inducing torture technique. Or at least, it doesn't have to be!

The unavoidable torture is maybe finding out that, when you don't know what you're listening to, the differences you thought you heard disappear. People may just have to agree to differ on why that happens.

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 14:23:45
Quote from:  link=msg=804956 date=0
but, I think I can safely guarantee that Bob Stuart does know one heck of alot more about it than all of the people who replied to this thread combined.


Face it, you're a fan!

If Bob Stuart knew very much at all about modern psychoacoustics and believed it, his company's product line would be vastly different and he would have probably made far more real advances in audio.
I'm a fan too - they've also done what even you would probably call "real advances in audio" - e.g. active speakers and DSP.


Sure they did it, but were they first and was it anything but an obvious application of a technology that was developed by someone else for a different purpose?

I breadboarded a digital music player on a hybrid computer in 1973...  So what?


Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 15:28:52
It does not mean that you *have to*.

I know that; in fact I never said the opposite.
And if that introduces bias, then, take a rest and come back.
Taking a rest doesn't magically erase my memory. Hence, taking a rest doesn't eliminate my cognitive bias. What happens if I listen to the same music on (1) system A, then (2) system B, then (3) system A again, is as follows. When I'm listening to system B, what I hear is something that's being cognitively biased as a result of what I can remember about how the music sounded on system A. First, let's assume system A is a less resolving system than B. While blinded, I can hear a detail with B that wan't noticeable at all with A. Still blinded, now let's assume I switch back to system A (according to step 3 in the experiment I described above). I can now still hear the detail. Why? Simple. It's because my memory is telling me that B sounded subjectively better, and that that's what the music should sound like. As a direct result from this, my brain now automatically fills in the missing detail where it previously couldn't. It previously could not, because the information required for that had previously not even been stored in my memory yet.

The only limitations imposed by the ABX test is in avoiding conscious or unconscious preference for one of the items being tested.

My above explanation, which is actually Bob Stuart's instead of mine (at least, if I understood the TAS article correctly...), shows that "avoiding conscious or unconscious preference" is not always technically possible if switching back and forth between audio samples is being allowed during these tests.

Anyway, If that's the only thing of my reply that you find wrong, let's leave it that way and please, specify in which terms (examples or situations) a sighted test can be more reliable than a blind test, as you said in your last sentence, and which you completely avoided to answer at dhromed's request.

"If you had the memory of a goldfish, maybe it would work." Seeing something can introduce bias because we can remember what we see, but that doesn't necessarily also mean hearing something can NEVER introduce bias. I can hear things while I'm blinded, things I can remember. I don't have to see what might be causing them to be able to remember that I heard them. So because I can remember them, I cannot always avoid being biased by them. If you don't believe that humans can be biased by things they can remember about a sound they heard, please watch the start of this vid.
http://www.youtube.com/watch?v=BYTlN6wjcvQ (http://www.youtube.com/watch?v=BYTlN6wjcvQ)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 15:43:08
audioclaudio's implication that Bob Stuart is some sole acting genius, and the individual author of the technologies audioclaudio mentions, is ludicrous

I never implied that he is some sole acting genious, nor that he is the individual author of the technologies. I may be less of a genius than he is, but I'm not THAT stupid. 
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 15:50:46
Meanwhile in a different place our member Julf did a blog entry that relates very well to the things you discuss here. An interesting food for thought imho
http://www.computeraudiophile.com/blogs/ju...c-thinking-246/ (http://www.computeraudiophile.com/blogs/julf/double-blind-control-experiment-and-scientific-thinking-246/)

As a matter of fact I'm a registered user on CA forum too, albeit under a different nick. I've had some public discussions with Julf in the past.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-13 16:11:30
So, audioclaudio, do you notice that you are the only one here arguing against the applicability of ABX testing in proving that a difference is audible? Did you wonder why that is?

Is it A: Everyone else here is so biased that we would discount any evidence presented to us that would counter that belief?

Or is it B: We have been waiting more than a decade for someone to present such evidence, and instead all we have gotten is folks like yourself with lots of theories and conjecture and passing along of everybody else's theories and conjectures?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 16:16:51
As a direct result from this, my brain now automatically fills in the missing detail where it previously couldn't. It previously could not, because the information required for that had previously not even been stored in my memory yet.
Cite the law that says people taking ABX tests are not allowed to practice beforehand in any fashion they choose (e.g.: sighted, blind, standing on their head worshipping the flying spaghetti monster).

Quote
I don't have to see what might be causing them to be able to remember that I heard them. So because I can remember them, I cannot always avoid being biased by them.
...and that's the point! If you can demonstrate your bias while blinded then congratulations, you passed.

I'm not THAT stupid.
Your continued (and lately childish) arguing from authority doesn't paint you as all that smart either.

I've had some public discussions with Julf in the past.
Do you want a medal? I doubt Wombat's post was intended solely for you. While you're the latest uneducated opponent of objective testing, this thread is not about you. In fact, it isn't even about the debate you are trying to have (albeit incompetently). (EDIT: Thanks to the thread split this thread can be about your religious argument and all it's fallacies. )
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 16:33:54
And if that introduces bias, then, take a rest and come back.
Taking a rest doesn't magically erase my memory.


True, there is no magic. Erasing small details or rather not holding them long, is how your brain works.

Obviously you have intentionally ignored my advice and have chosen to remain poorly-informed.

Whatever, dude!


Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 16:36:46
So, audioclaudio, do you notice that you are the only one here arguing against the applicability of ABX testing in proving that a difference is audible? Did you wonder why that is?

Is it A: Everyone else here is so biased that we would discount any evidence presented to us that would counter that belief?

Or is it B: We have been waiting more than a decade for someone to present such evidence, and instead all we have gotten is folks like yourself with lots of theories and conjecture and passing along of everybody else's theories and conjectures?

:cough: It's probably C: Everyone else NOT here has been waiting more than a decade for someone to present evidence to support YOUR theories and conjectures? :cough:
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 16:38:34
window <- comprehension of science
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-13 16:46:38
I can hear things while I'm blinded, things I can remember. I don't have to see what might be causing them to be able to remember that I heard them. So because I can remember them, I cannot always avoid being biased by them.


That doesn't make a sighted test more reliable than a blind test. It's not an answer to my question, either.

Quote
can now still hear the detail. Why? Simple. It's because my memory is telling me that B sounded subjectively better, and that that's what the music should sound like. As a direct result from this, my brain now automatically fills in the missing detail where it previously couldn't. It previously could not, because the information required for that had previously not even been stored in my memory yet.


So your point is that due to this phantom detail, the ABX will show that A and B are equal, which would be wrong, since hypothetically, B is better. The blind test has failed.

Let's take that to its logical extreme: I play A, and there's nothing but silence. Then B, which contains music. Then I switch back to A, and because I've already heard B, my brain subconsciously superimposes the memory of the music onto the silence. I then falsely judge that A and B sound the same.

Obviously, you're not arguing that. So it becomes a question of magnitude. How tiny would a detail have to be before it can be superimposed on music without your conscious consent; before it messes up blind tests? If a detail is clearly audible, we can detect its absence when knowing it should be there; if it is not, then we hear nothing in either case.

It is a kind of defeatist stance: believing that objective methods are unreliable because the brain makes your perceive things that are not there. But the same goes for subjective methods! I've mentioned it before: it seems to come down to a crazy notion that things cannot be known and that knowledge can not be had.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 17:13:05
Cite the law that says people taking ABX tests are not allowed to practice beforehand in any fashion they choose (e.g.: sighted, blind, standing on their head worshipping the flying spaghetti monster).

I was referring to the actual test itself, NOT the practicing beforehand.
...and that's the point! If you can demonstrate your bias while blinded then congratulations, you passed.

By saying this, you are agreeing with me on the fact that, due to this bias, double blinded tests are indeed flawed.
Your continued (and lately childish) arguing from authority, doesn't paint you as all that smart either.

So far, the "authority" (please note the parentheses) has failed to provide objective evidence to support the theory which claims double blinded tests are NOT flawed. Ironically, on top of this, many of the people here who believe in this theory are consistently and childishly demanding objective evidence to support the theory which claims double blinded tests ARE flawed.
Do you want a medal? I doubt Wombat's post was intended solely for you. While you're the latest uneducated opponent of objective testing, this thread is not about you. In fact, it isn't even about the debate you are trying to have (albeit incompetently).
You fail to understand that I am NOT an opponent of objective testing. I am only trying to point out that double blinded testing is flawed, and why.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-13 17:19:41
So, audioclaudio, do you notice that you are the only one here arguing against the applicability of ABX testing in proving that a difference is audible? Did you wonder why that is?

Is it A: Everyone else here is so biased that we would discount any evidence presented to us that would counter that belief?

Or is it B: We have been waiting more than a decade for someone to present such evidence, and instead all we have gotten is folks like yourself with lots of theories and conjecture and passing along of everybody else's theories and conjectures?

:cough: It's probably C: Everyone else NOT here has been waiting more than a decade for someone to present evidence to support YOUR theories and conjectures? :cough:

You know, for awhile I thought you might be just someone that needed to overcome his ignorance on a subject in which he badly needed educating. Now it has become quit evident that you are nothing but a troll, and not worth my attention. 
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: db1989 on 2012-08-13 17:25:06
Your continued (and lately childish) arguing from authority, doesn't paint you as all that smart either.

So far, the "authority" (please note the parentheses) has failed to provide objective evidence to support the theory which claims double blinded tests are NOT flawed. Ironically, on top of this, many of the people here who believe in this theory are consistently and childishly demanding objective evidence to support the theory which claims double blinded tests ARE flawed.

I . . . What?

Have you ever heard anyone talk about a ‘burden of proof’?

Also:


""
Those are quotation marks.

()
Those are parentheses.

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 17:31:53
Once again, audioclaudio, you are confusing religion with science.

Regarding what I'm agreeing to, I'm trying to take what you're saying literally. If you hear someing with one sample that you don't hear in the other and that correlates with the actual samples then congratulations. If you're talking about some real difference that has now ghosted itself onto both samples then the results are going to indicate that you can't demonstate a difference. Take a break, do the test over if you must.  I hope your grasping at straws doesn't now lead you to say the difference is now permanent; otherwise, how will you ever going to distinguish the difference again?!?

Have you ever had this ghosting happen to you?  Did it go away?  What do you think caused it to go away?  If it did go away and you think you know the reason, why can't an ABX test be used to confirm it?

How exactly is a sighted test going to be more reliable?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: drewfx on 2012-08-13 17:55:38
If past listening experiences bias our future listening, then it does so for sighted testing as well as for ABX testing.

IOW it's not a flaw in ABX, it's a flaw in our brain's ability to determine what's real. Which is the main reason for doing ABX in the first place.

I suggest that you might want to just drop this particular line of your argument.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 18:00:31
He has no argument.

He's simply regurgitating something said by someone else (in whom he has put his blind faith) who is trying to justify why people should buy his expensive technology without having to demonstrate tangible benefit in a scientifically acceptable fashion.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 18:28:40
window <- comprehension of science
Where's the HA "like" button?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 18:33:26
So far, the "authority" (please note the parentheses) has failed to provide objective evidence to support the theory which claims double blinded tests are NOT flawed. Ironically, on top of this, many of the people here who believe in this theory are consistently and childishly demanding objective evidence to support the theory which claims double blinded tests ARE flawed.
The flaws, such as you have presented them, are "flaws" with human beings. They are present equally in sighted and blind tests.

There are other flaws that are only present in sighted tests, but removed by blind tests.


There are flaws that are more present in blind tests if you choose to do them in a certain way. Fine. Do them in a different way.


We can't rebut your argument further because, having taken these points into consideration, you have no argument left.


If you feel there is some other problem with double blind testing that it not present in sighted testing, and cannot be removed from blind testing, please explain it.


We already know the way in which blind testing beats sighted testing: it removes placebo / expectation bias. There are objective studies showing this - let google deliver the objective evidence for this superiority to your screen

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: drewfx on 2012-08-13 18:36:36
He has no argument.


I consider it a purely hypothetical argument that we get here every few months - IF there are some kind of tiny differences that are real and audible but somehow still manage to evade detection under ABX testing, then it follows that ABX is flawed.

Of course no one ever manages to provide any actual evidence whatsoever that such stealth differences exist and are audible.

So we are left with the supposition that they conceivably somehow might exist and are therefore EXTREMELY IMPORTANT.

But personally I don't find purely hypothetical arguments particularly compelling.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 18:39:00
I can hear things while I'm blinded, things I can remember. I don't have to see what might be causing them to be able to remember that I heard them. So because I can remember them, I cannot always avoid being biased by them.


That doesn't make a sighted test more reliable than a blind test. It's not an answer to my question, either.

I agree. It only shows that both sighted and double blind tests can be, but not necessarily always are, unreliable. If there can be no reliable way to show that a test is indeed reliable, logically we must accept the possibility that it's not. The test being blinded or sighted does not change this fact. In answer to your question, here is the link to another article about this subject. As you can see, it has been debated over to great length.
http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)
So your point is that due to this phantom detail, the ABX will show that A and B are equal, which would be wrong, since hypothetically, B is better. The blind test has failed.

Let's take that to its logical extreme: I play A, and there's nothing but silence. Then B, which contains music. Then I switch back to A, and because I've already heard B, my brain subconsciously superimposes the memory of the music onto the silence. I then falsely judge that A and B sound the same.

Obviously, you're not arguing that. So it becomes a question of magnitude. How tiny would a detail have to be before it can be superimposed on music without your conscious consent; before it messes up blind tests? If a detail is clearly audible, we can detect its absence when knowing it should be there; if it is not, then we hear nothing in either case.

It is a kind of defeatist stance: believing that objective methods are unreliable because the brain makes your perceive things that are not there. But the same goes for subjective methods! I've mentioned it before: it seems to come down to a crazy notion that things cannot be known and that knowledge can not be had.

Well, yes. It IMO is not a crazy notion at all, though. Take neuroscience for example, it's not an exact science. In fact, if all knowledge could always be had then perhaps we wouldn't be needing science in the first place. After all, the goal of science is not only to obtain knowledge, but to at least TRY and obtain knowledge, so here's yet another lovely anecdote. One cannot prove the existence of the Higgs boson until one actually DOES find it. But then, how are people supposed to find the little bugger if they all just refuse to even keep looking?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Wombat on 2012-08-13 18:45:18
As a matter of fact I'm a registered user on CA forum too, albeit under a different nick. I've had some public discussions with Julf in the past.

This doesn´t make any of your statements more valid, maybe even more the other way around.
One thing i may add to this discussion is that here on Hydrogenaudio we have several people that stumbled across audible probs while doing no abx tests and surprise, surprise are able to reproduce these in proper abx sessions. Now guess what i think about the claims of people that are not able to stand an abx test?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Nick.C on 2012-08-13 18:46:04
One cannot prove the existence of the Higgs boson until one actually DOES find it. But then, how are people supposed to find the little bugger if they all just refuse to even keep looking?
The search for the Higg's Boson has continued because its existence would "fit" with current particle theory. On the other hand, proof of its existence would be subjected to a 5 sigma test - objectively, repeatably.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 18:47:01
So we are left with the supposition that they conceivably somehow might exist and are therefore EXTREMELY IMPORTANT.

Also, we somehow lose this "argument" because we, "cannot prove that they don't exist and that they aren't important."
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: drewfx on 2012-08-13 18:51:55
So we are left with the supposition that they conceivably somehow might exist and are therefore EXTREMELY IMPORTANT.

Also, we somehow lose this "argument" because we, "cannot prove that they don't exist and that they aren't important."

What if we can prove it hypothetically? Does that count? 
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 18:52:12
But then, how are people supposed to find the little bugger if they all just refuse to even keep looking?

Again, how does a requirement for objective evidence keep people from being curious?

Where has anyone in this discussion discouraged research into the unknown?

Let pioneers pioneer! Just don't expect the scientific community to accept their untested theories on faith, because they may be untestable; especially if they fly in the face of theories that are testable and are supported by the results of these tests.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-13 19:05:31
In answer to your question, here is the link to another article about this subject. As you can see, it has been debated over to great length.
http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)


Well, if nothing else, I am now convinced that you are convinced.

if they all just refuse to even keep looking?

Who is refusing?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 19:30:10
Cite the law that says people taking ABX tests are not allowed to practice beforehand in any fashion they choose (e.g.: sighted, blind, standing on their head worshipping the flying spaghetti monster).

I was referring to the actual test itself, NOT the practicing beforehand.


The above  shows how thoroughly you misunderstand ABX: ABX allows practicing during the test because you have the option of comparing clearly identified A versus clearly identified B at any time during the test that you wish.

That's aside from the fact that your hearing model is refuted by modern science...

Two strikes!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 19:35:01
In answer to your question, here is the link to another article about this subject. As you can see, it has been debated over to great length.
http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)


Interesting that you cite that article. What you find there is not an open honest debate of the topic, but rather a sham discussion where many critical comments were censured out of existence.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 19:45:40
You mean censored.

Speaking of the ability to audition samples during a test, some interfaces (like that in foobar2000) allow the participant to audition the known samples A and B in addition to the unknown samples, X and Y.  There are no limitations on how often these samples can be auditioned or how long one can take before committing to an answer.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 19:50:04
Speaking of the ability to audition samples during a test, some interfaces (like that in foobar2000) allow the participant to audition the known samples A and B in addition to the unknown samples, X and Y.  There are no limitations on how often these samples can be auditioned or how long one can take before committing to an answer.


Since when is there a Y sample in an ABX test? ;-)

Nevertheless, I agree with you about auditioning A and B at any time.

I have to admit that I smiled the first time I used that feature after I put it in. It made things soooo much easier!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 19:55:36
It kinda puts a monkey wrench into the ghosting excuse, no?

...of course this will be ignored with the rest of the inconvenient points/questions raised by myself and others.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 20:02:40
One cannot prove the existence of the Higgs boson until one actually DOES find it. But then, how are people supposed to find the little bugger if they all just refuse to even keep looking?


Straw man argument. This discussion is not about no longer looking. It is about the terms and conditions for verifying that it has actually been found.

Also, lets say that you somehow manage to find, as opposed to imagine, fatal flaws in ABX. What about all the other forms of DBTs that might apply but are in fact different tests? Since they are different tests, you would need to also find fatal flaws in them or accept their validity...

One of the confirmations of ABX is the fact that we have numerous ways of confirming or denying the audibility of as wide range of defects.  Some are related, others come at the question in different ways. In some cases the means of verification are as different as could be imagined. In general they all agree about the threshold of audibility for any defect that is studied that way, including ABX.

We have numerous audible defects whose intensity we can control at will. We can and have established the conditions under which they are heard by most people, a few people, and nobody. We have done this in some cases by various independent means. The independent means, including ABX, agree.

This contrasts with sighted evaluations which generally have positive results for just about everything that rides somebody's hobby horse.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: dhromed on 2012-08-13 20:02:45
Interesting that you cite that article. What you find there is not an open honest debate of the topic, but rather a sham discussion where many critical comments were censured out of existence.


Well, this very sane incoming letter from one Lawrence S. Makow (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial#comment-5264) was included, at least— by Robert Harley himself, even.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 20:05:49
Interesting that you cite that article. What you find there is not an open honest debate of the topic, but rather a sham discussion where many critical comments were censured out of existence.


Well, this very sane incoming letter from one Lawrence S. Makow (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial#comment-5264) was included, at least— by Robert Harley himself, even.


It is not unusual for a few mild letters to be allowed to filter through to make it look like there was no censorship. There was.

BTW IME not all High End audio forums are censured this way. AFAIK the Stereophile forum may kick people off, but any posts that get deleted along the are pretty purple.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 20:50:37
http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)
Oh, that's hilarious, because it says...

Quote
But blind listening tests lead to the wrong conclusions even when the experimenters’ motives are pure. A good example is the listening tests conducted by Swedish Radio (analogous to the BBC) to decide whether one of the low-bit-rate codecs under consideration by the European Broadcast Union was good enough to replace FM broadcasting in Europe.

...

Swedish Radio said the codec was good enough to replace analog FM broadcasts in Europe. This decision was based on data gathered during the 20,000 “double-blind, triple-stimulus, hidden-reference” listening trials.

Well, maybe they did, maybe they didn't - but I can tell you one thing with absolute certainty - the real BBC/EBU tests a decade later found that no codecs were transparent (at the bitrates they tested, having sought out problem samples for each codec), and only one (AAC main profile) avoided introducing annoying artefacts on at least some audio.

Double-blind triple stimulus hidden reference testing of course - the same testing methodology that supposedly "caused" Sweedish radio to determine that a codec was fit to replace FM in 1991. Nothing to do with the criteria (EBU criteria reportedly quoted by Sweedish radio DOES NOT demand transparency).

Truth of the matter is, I don't know of any codecs that are universally transparent at the ("low") bitrates they're designed for. Many are transparent most of the time, but that's a different statement entirely. And most HA regulars if they use / when they used lossy codecs for "archiving" used much higher bitrates than those codecs were designed for.

Which is why it's extra funny to try to imply that Sweedish radio did 20000 tests in 1991 (what codecs were around in 1991?!?!!!!!) and couldn't hear anything wrong because they used blind testing. But I guess if you're against blind testing, you'll accept this reported "fact" without question.

audioclaudio will suggest I reject it because I like blind testing. No, I reject it because an audible difference was proven by blind testing...
http://www.mp3-tech.org/programmer/docs/w2006.zip (http://www.mp3-tech.org/programmer/docs/w2006.zip)

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 20:52:05
@Arny:
You do know that censure and censor are different words with different meanings, right?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 21:00:21
The flaws, such as you have presented them, are "flaws" with human beings. They are present equally in sighted and blind tests.

Yes, they are. However, the fact one test is sighted and the other is blinded is what obviously causes differences in the impact these flaws have on human perception. The problem with this is one cannot always know if such a "flaw" is causing differences in perception instead of preventing them. This is because one cannot know everything there is to know about how the human hearing system works. Due to complexity constraints, if one decides to eliminate one or more effects caused by one or more "flaws" then it's not always technically possible to also prove exactly what impact this decision will have on one or more effects caused by one or more other "flaws".
There are other flaws that are only present in sighted tests, but removed by blind tests.

Yes, there are. However, this doesn't prove there are no flaws that are only present in blind tests, but removed in sighted tests.
There are flaws that are more present in blind tests if you choose to do them in a certain way. Fine. Do them in a different way.

Again, yes. However, doing them in a different way does not always make them reliale.
We can't rebut your argument further because, having taken these points into consideration, you have no argument left.

The argument I have left is you still haven't proven a thing. The simple fact I haven't proven a thing doesn't also mean that you have; on top of this, I am not the one being too afraid to admit that I haven't.
If you feel there is some other problem with double blind testing that it not present in sighted testing, and cannot be removed from blind testing, please explain it.

I think the interview with Bob Stuart already did explain it rather exceptionally well.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 21:05:39
http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)
Oh, that's hilarious, because it says...

Quote
But blind listening tests lead to the wrong conclusions even when the experimenters’ motives are pure. A good example is the listening tests conducted by Swedish Radio (analogous to the BBC) to decide whether one of the low-bit-rate codecs under consideration by the European Broadcast Union was good enough to replace FM broadcasting in Europe.



Another thing the cited article says that is really quite revealing is:

"Robert Harley -- Thu, 05/21/2009 - 09:37
I'm not aware of any formal DBT of standard-resoution digital audio with high-resolution digital audio. The difference between 44.1kHz/16-bit digital audio and 176.4kHz/24-bit is obvious, in my experience. It is, in fact, so obvious that no one (no one that is a disinterested experimenter, that is) has bothered to organize and conduct it."

He's obviously 100% oblivious to the following well-known rather highly relevant article:

Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback
Authors: Meyer, E. Brad; Moran, David R.
Affiliation: Boston Audio Society, Lincoln, MA, USA
JAES Volume 55 Issue 9 pp. 775-779; September 2007

Hey, it had only been out for a year and a half and plastered all over the web the whole time...  ;-)

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 21:09:35
I think the interview with Bob Stuart already did explain it rather exceptionally well.


What you've done for me and I thank you for it,  is to make me aware of the highly outdated model of hearing that both you and Stuart have staked your careers on.

You don't read very much, do you?

The way you've been sloughing posts around here also says something about your writing.

Ever hear of a "Logic tight box"?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 21:10:22
I'll take the opportunity now to cite db1989's recent and inconvenient suggestion that audioclaudio investigate burden of proof.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-13 21:12:55
@Arny:
You do know that censure and censor are different words with different meanings, right?


Yeah, yeah, yeah! ;-)

I'm still a little jet lagged after spending most of last week in Orange County and spending all day Saturday getting back.  Sundays are busy for me, so I didn't get much chance to rest.

I was talking with my spell checker and hastily clicked the wrong option.

Thanks!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: [JAZ] on 2012-08-13 21:36:08
Personally, I consider this topic done.  Anything more is feeding the troll.

He has no intention (i would say even interest) in listening to what we say, and what he writes is of no help to us.

Let him go to the forums that he's more familiar with.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 21:45:49
He's obviously 100% oblivious to the following well-known rather highly relevant article:

Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback
Authors: Meyer, E. Brad; Moran, David R.
Affiliation: Boston Audio Society, Lincoln, MA, USA
JAES Volume 55 Issue 9 pp. 775-779; September 2007

Hey, it had only been out for a year and a half and plastered all over the web the whole time...  ;-)

It was later proven that the Meyer & Moran tests were not using truly HiRes material, but PCM converted to DSD. Bob Stuart was one of the test subjects BTW.

The audible difference between 16/44.1 and proper HiRes is actually quite enormous.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: pdq on 2012-08-13 21:51:19
It was later proven that the Meyer & Moran tests were not using truly HiRes material, but PCM converted to DSD. Bob Stuart was one of the test subjects BTW.

Reference?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-13 22:41:36
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.

Evidence for what is otherwise a violation TOS #8?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 22:50:38
It was later proven that the Meyer & Moran tests were not using truly HiRes material

...

The audible difference between 16/44.1 and proper HiRes is actually quite enormous.
Oh, wake up.

The listeners were allowed to choose their own material that they believed best showed the benefits of hi-res.

That they came forward with a wide variety of content, some DSD, some 96k+, and some so old it probably had littlcontent above 20k - and then couldn't hear the effects of downconverting any of it to 44.1kHz, shows that the "enormous" benefit of HiRes wasn't audible to them before, during, or after the test!


But if you want to believe that there was "no true HiRes" material in that test, and that's why it failed, carry on.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 22:54:12
Cite the law that says people taking ABX tests are not allowed to practice beforehand in any fashion they choose (e.g.: sighted, blind, standing on their head worshipping the flying spaghetti monster).

I was referring to the actual test itself, NOT the practicing beforehand.


The above  shows how thoroughly you misunderstand ABX: ABX allows practicing during the test because you have the option of comparing clearly identified A versus clearly identified B at any time during the test that you wish.

That's aside from the fact that your hearing model is refuted by modern science...

Two strikes!

I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.

As for my hearing model being refuted by modern science, like I said, this is not even my hearing model, but the hearing model described by Bob Stuart in the TAS interview I linked to earlier in this thread. I also said I don't know if the way I interpreted this interview was correct or not.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 22:56:05
The argument I have left is you still haven't proven a thing. The simple fact I haven't proven a thing doesn't also mean that you have; on top of this, I am not the one being too afraid to admit that I haven't.
Blind testing solves a problem with sighted testing. The problem is proven. The solution is proven. There are acres of peer reviewed literature across tens of disciplines showing this.

Your alleged problems with blind testing are unproven, and you propose no solution.

This is why the ball is not in my court. Please try to understand this.

Cheers,
David.

P.S. this is not the same as me saying "there is no problem" either!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 23:08:03
What you've done for me and I thank you for it,  is to make me aware of the highly outdated model of hearing that both you and Stuart have staked your careers on.
I think Bob applies a transmission-line model of the cochlear. As much as I can remember (and I've been out of psychoacoustics for a decade!) that's a fine model to use with the right parameters.

Finding those parameters is tricky - you usually determine them by comparing known psychoacoustic results (which come from double-blind tests!) with the predictions of the model, and tweaking the model to match reality.

The transduction is well understood, but some of the brain processing isn't - and the brain forms a massive feedback loop with the cochlear.

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 23:18:08
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.

Evidence for what is otherwise a violation TOS #8?

Quote
Robert Harley -- Tue, 07/15/2008 - 10:22
I must disagree with Michkhol's reasoning on all three points. .

First, my assertion that audible differences exist between 44.1kHz/16-bit, 96kHz/24-bit, and DSD is based on my own listening experiences. This includes hearing a live microphone feed and comparing it to standard resolution PCM, high-resolution PCM, and DSD. Have you performed the same test and reached a different conclusion? Or are you content to base your belief that they all sound identical on the results of a published test? (A “test” devised and conducted, by the way, by two individuals with a long history of attempting to discredit audiophiles.) Are you suggesting that I should reject my own direct experience and conclude that I was simply deluded? Is everyone who hears a difference between standard- and high-resolution digital audio similarly deluded—people like Meridian’s Bob Stuart, Keith Johnson, dCS founder Mikey Storey, Peter McGrath, and other credentialed and respected leaders in the field who have decades of academic research and hands-on experience with the subject?

Second, Swedish Radio was, in fact, attempting to discover audible artifacts in the codecs. Read the paper (“Subjective Assessments on Low-Bit-Rate Audio Codecs” by C. Grewin and T. Ryden, published in the Proceedings of the 10th International AES Conference”). I’ve read the paper, and attended its presentation at an AES conference in London.

More to the point, you’re trying to dodge the essential facts of the Swedish Radio affair; 60 listeners over 20,000 double-blind trials failed to hear an artifact of the codec that was obvious to a single listener using non-blind techniques. No amount of parsing the language of Swedish Radio's mandate gets you around that fact.

Finally, your attempt to dismiss Peter McGrath’s experience merely by the fact that I don’t know the specific D/A converter he was listening through is grasping at straws.


http://www.avguide.com/forums/blind-listen...lawed-editorial (http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial)

While this may perhaps not meet your definition of "objective evidence", it sure does look like objective evidence to a guy like me.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 23:19:05
I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.
If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again. That's how people are able to detect which one of the three near identical signals contains the almost masked signal in psychoacoustic tests - if what Bob and you were saying scuppered such things, the thresholds measured in such tests would be totally wrong. Yet the threshold measured in such tests matches what's expected from the physiology of the ear. The real trick is to cut cats open, probe their nerves, and intercept the signals on the way to their brains. People used to do that (and worse). We know a lot about hearing from such experiments. We know our conscious limits match the raw signals that are reaching our brains, and we know a lot about the apparatus feeding those signals to our brains.

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 23:21:33
While this may perhaps not meet your definition of "objective evidence", it sure does look like objective evidence to a guy like me.
"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"

Goodness, if you think that's "objective", I think we're wasting our time here until you invest in a dictionary!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-13 23:24:25
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.
You've really shot yourself in the foot here. If this was true, an ABX test, even if it was the most hostile test ever, should be trivial to pass.

How does a "quite enormous" difference disappear completely when someone says the words "which one are you listening to then?"

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-13 23:33:02
I think Bob applies a transmission-line model of the cochlear.
That's cochlea, not cochlear. It's in the book titled "Auditory Neuroscience - Making Sense of Sound" by Jan Schnupp, Israel Nelken and Andrew King.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Woodinville on 2012-08-14 00:07:23
I think Bob applies a transmission-line model of the cochlear.
That's cochlea, not cochlear. It's in the book titled "Auditory Neuroscience - Making Sense of Sound" by Jan Schnupp, Israel Nelken and Andrew King.



Let's start:  "how do you even imagine that one can hear a difference between two systems, one with noise 98 dB down and the other 146dB down, when the level is set to peak at 96dB?"

How do you explain how "obviously difference" fails to show up in even the worst kind of ABX test?

I have yet to see a whit of evidence that "high-rez" matters for final presentation to a listener.  Have you any, bearing in mind that citing non-blind-testing proves nothing but the incompetence, the complete and total incompetence, of the person citing it as evidence.

Bear in mind the hard evidence for the persistance of loudness memory while you're at it.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-14 00:14:23
If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again.
Once again, our opinions differ. I suggest you watch that Audio Myths Workshop vid I linked to earlier, more in particular the part where Poppy Crum shows how it does work.
Quote
Do you hear an 's'? I think I hear an 's'.

"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"
Add Barry Diament of Soundkeeper Recordings to that long list of 'clever' people. He publicly stated that, in his own opinion, the audible difference between properly recorded 2x (96 kHz or 88.2 kHz) and properly recorded 4x (192 kHz or 176.4 kHz) samplerate is even bigger than the audible difference between 1x (48 kHz or 44.1 kHz) and 2x. He also wrote on ComputerAudiophile forum that, if he could invite you to his recording studio, it would take him two seconds to prove the audible difference between 16/44.1 and 24/192 to you.
How does a "quite enormous" difference disappear completely when someone says the words "which one are you listening to then?"
Judging by my own listening experience, the difference doesn't disappear even one bit.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: audioclaudio on 2012-08-14 00:33:53
How do you explain

Quote
Robert Harley -- Mon, 07/14/2008 - 10:28
My point is that the better-sounding piece of equipment more easily allows you to forget the equipment and focus on the music. Electronic artifacts are a constant reminder that we're listening to a re-creation, rather than to music itself.

Greater musical enjoyment comes not from listening in analytical mode for specific sonic attributes, but through lower awareness of the electro-mechanical system between listener and performer. The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Maybe this is how.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: DVDdoug on 2012-08-14 00:56:27
Quote
The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Fine...  Where are the double-blind tests showing that the listener "feels different" or enjoys the music more when the otherwise inaudible artifacts are removed.

But in fact, Mr. Harley's quote doesn't exactly say that...  He seems to say people enjoy music more on better sounding equipment even though they are not thinking about sound quality or why they are enjoying it more.    That seems reasonable.  And if you asked the person about the sound quality later they might say, "That was a really nice-sounding sound system."

Yes, you may feel better when listening to more expensive equipment that has no audible difference, or to higher resolution audio...  As long as you know what you are listening to (non-blind).
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Woodinville on 2012-08-14 01:18:24
How do you explain

Quote
Robert Harley -- Mon, 07/14/2008 - 10:28
My point is that the better-sounding piece of equipment more easily allows you to forget the equipment and focus on the music. Electronic artifacts are a constant reminder that we're listening to a re-creation, rather than to music itself.

Greater musical enjoyment comes not from listening in analytical mode for specific sonic attributes, but through lower awareness of the electro-mechanical system between listener and performer. The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Maybe this is how.


That's so far off I don't know where to start. You can listen any way you want in a blind test, you just don't know what you're listening to.

So sorry, that's baloney.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: knutinh on 2012-08-14 06:45:22
"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"
Add Barry Diament of Soundkeeper Recordings to that long list of 'clever' people. He publicly stated that, in his own opinion, the audible difference between properly recorded 2x (96 kHz or 88.2 kHz) and properly recorded 4x (192 kHz or 176.4 kHz) samplerate is even bigger than the audible difference between 1x (48 kHz or 44.1 kHz) and 2x. He also wrote on ComputerAudiophile forum that, if he could invite you to his recording studio, it would take him two seconds to prove the audible difference between 16/44.1 and 24/192 to you.

Why is it that you have no scientific sources? Would you not think that the people whose career is built upon investigating physics and perception and documenting their findings in such a way that it passes peer-review would be better suited to support your claims than people who are good at twisting equalizer knobs to make pleasing audio recordings?

What if we were discussing the health consequences of smoking, would you use Philip Morris as a source that it is actually healthy to smoke? (they got to know, they have been chruning out gazillions of sigarettes, right?).

-k
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: probedb on 2012-08-14 09:06:56
Add Barry Diament of Soundkeeper Recordings to that long list of 'clever' people. He publicly stated that, in his own opinion, the audible difference between properly recorded 2x (96 kHz or 88.2 kHz) and properly recorded 4x (192 kHz or 176.4 kHz) samplerate is even bigger than the audible difference between 1x (48 kHz or 44.1 kHz) and 2x. He also wrote on ComputerAudiophile forum that, if he could invite you to his recording studio, it would take him two seconds to prove the audible difference between 16/44.1 and 24/192 to you.


You did the the bit in bold didn't you? That doesn't make it true, it's one persons opinion.

I'm really no expert at all but all I see from you is blah, blah, blah, Bob Stuart, blah, blah, Bob Stuart.

I'm not entirely sure why people continue to argue, it's like trying to stop a priest believing in a god, not gonna happen unless you have Dougal from Father Ted on your side
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-14 12:17:58
Barry Diament? Really?!?

No...




...really?!?!?

This is the guy who first made an impression on this forum for stating how a CD-R copy sounds better than the original source disc?

Like Stuart, Diament has a vested interest in having people believe that up is down.

What a wonderful example to use in committing the logical fallacy of argument from authority.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 13:59:24
He's obviously 100% oblivious to the following well-known rather highly relevant article:

Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback
Authors: Meyer, E. Brad; Moran, David R.
Affiliation: Boston Audio Society, Lincoln, MA, USA
JAES Volume 55 Issue 9 pp. 775-779; September 2007

Hey, it had only been out for a year and a half and plastered all over the web the whole time...  ;-)

It was later proven that the Meyer & Moran tests were not using truly HiRes material, but PCM converted to DSD.


I believe that the conversion in question was well-documented in the article.

However, the idea that converting DSD to PCM necessarily causes a significant loss of resolution is a false claim. In fact, 24/192 has far more resolution than  SACD.

Your apparent ignorance of Shannon's relevant theorems is noted.


Quote
Bob Stuart was one of the test subjects BTW.


And???

Quote
The audible difference between 16/44.1 and proper HiRes is actually quite enormous.


True as long as you do sighted evaluations.

Your major problem at HA is that many of us have done bias-controlled evaluations of various kinds (not just ABX)  and know that at the very best, the differences are very subtle and in general they are vanishing.

Let's be clear - it is now known that the Meyers and Moran tests might be flawed because M&M  took at face value,  vendor claims that SACDs and DVD-As are high resolution recordings.

We now know that detailed technical analysis puts the actual percentage of all such recordings (not just the ones M&M used) closer to 50% high resolution. IOW the vendors didn't tell the truth, and it came around and bit them when M&M believed them.

You see there was a sort of shadow DBT that had already happened - all of these CD (or worse) resolution recordings were sold on DACDs and DVD-As as being high rez, but they weren't.  For quite a while no golden eared reviewers or audiobphiles blew the whistle. In the end, most if not all people who noticed the loss of resolution did so by means of technical tests, not listening tests!

But many of us have been doing similar tests with recordings known to be high resolution, for over a decade. No joy!

Again, you are revealing what you don't actually have any personal experience with which is lots!
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 14:06:05
I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1).


Of course! That's why ABX has allowed people to repeat step(1) and step(2) in a completely transparent and open way, whenever they want to including right in the midst of a trial.

Obviously your belaboring this particular point shows that you  have zero hands-on experience with ABX and have been listening to people who either lack any experience with ABX at all, or they completely missed many important points about ABX when they did try it.

What you need  to understand is that many well-known golden ears seem to get hysterical at even the mention of ABX.  I can name names based on personal experiences. They literally have to be carried away in some cases!  So, they may evaluate ABX while in a hysterical state, and what they report after is not complete or reliable.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 14:15:32
Quote
Robert Harley -- Tue, 07/15/2008 - 10:22

First, my assertion that audible differences exist between 44.1kHz/16-bit, 96kHz/24-bit, and DSD is based on my own listening experiences.


Since Harley has been pretty consistently negative about blind tests for at least 2 decades, its safe to assume that none of the below is based on blind tests.

Quote
This includes hearing a live microphone feed and comparing it to standard resolution PCM, high-resolution PCM, and DSD.


Been there, done that and with numerous other people.

Quote
Have you performed the same test and reached a different conclusion?


Obviously not the same test, but many like them.

Quote
Or are you content to base your belief that they all sound identical on the results of a published test?


Not at all.

Quote
(A “test” devised and conducted, by the way, by two individuals with a long history of attempting to discredit audiophiles.)


I believe that Harley is referring to Vanderkooy and Lipshitz. I know those guys personally and he appears to be libelling them. First off, they are actually audiophiles themselves so discrediting audiophiles would be like intellectual suicide. Secondly, their targets are mostly some of the more visible bozos and charlatans in high end audio, not high end audio in total.


Quote
Are you suggesting that I should reject my own direct experience and conclude that I was simply deluded?


This is yet another excluded-middle argument because the kind of error that Harley is talking about is more like being fooled by an illusion than some kind of mental defect.  Let's be clear here - everybody is affected by sighted and other bias. The mental defect is denying it.

Quote
Is everyone who hears a difference between standard- and high-resolution digital audio similarly deluded—people like Meridian’s Bob Stuart, Keith Johnson, dCS founder Mikey Storey, Peter McGrath, and other credentialed and respected leaders in the field who have decades of academic research and hands-on experience with the subject?


None of the hands-on experimenters mentioned are exactly what you'd call advocates of reliable listening tests. ;-)  Most if not all of them have their own dogs in the fight.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 14:18:31
How do you explain

Quote
Robert Harley -- Mon, 07/14/2008 - 10:28
My point is that the better-sounding piece of equipment more easily allows you to forget the equipment and focus on the music. Electronic artifacts are a constant reminder that we're listening to a re-creation, rather than to music itself.

Greater musical enjoyment comes not from listening in analytical mode for specific sonic attributes, but through lower awareness of the electro-mechanical system between listener and performer. The listener might not be aware of why he feels a closer connection to the music with better equipment. One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.
Maybe this is how.



The obvious fallacy is the idea that blind test advocates use only cheap inferior equipment.  Visible shades of class warfare.

It's just another piece of evidence that much of high end audio is strongly ego-driven, not nearly as sound quality driven as many would like to pretend.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-14 15:19:36
Let's be clear - it is now known that the Meyers and Moran tests might be flawed because M&M  took at face value,  vendor claims that SACDs and DVD-As are high resolution recordings.

We now know that detailed technical analysis puts the actual percentage of all such recordings (not just the ones M&M used) closer to 50% high resolution. IOW the vendors didn't tell the truth, and it came around and bit them when M&M believed them.

You see there was a sort of shadow DBT that had already happened - all of these CD (or worse) resolution recordings were sold on DACDs and DVD-As as being high rez, but they weren't.  For quite a while no golden eared reviewers or audiobphiles blew the whistle. In the end, most if not all people who noticed the loss of resolution did so by means of technical tests, not listening tests!

Now this would have made for good material in the parent discussion.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-14 17:52:42
If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again.
Once again, our opinions differ. I suggest you watch that Audio Myths Workshop vid I linked to earlier, more in particular the part where Poppy Crum shows how it does work.
Quote
Do you hear an 's'? I think I hear an 's'.

"lots of other 'clever' people who also happen to reject orthodox scientific testing say the same thing, so I must be right!"
Add Barry Diament of Soundkeeper Recordings to that long list of 'clever' people. He publicly stated that, in his own opinion, the audible difference between properly recorded 2x (96 kHz or 88.2 kHz) and properly recorded 4x (192 kHz or 176.4 kHz) samplerate is even bigger than the audible difference between 1x (48 kHz or 44.1 kHz) and 2x. He also wrote on ComputerAudiophile forum that, if he could invite you to his recording studio, it would take him two seconds to prove the audible difference between 16/44.1 and 24/192 to you.
How does a "quite enormous" difference disappear completely when someone says the words "which one are you listening to then?"
Judging by my own listening experience, the difference doesn't disappear even one bit.
So given this apparently enormous indestructible difference, how come we've had HiRes for 13 years, and no successful ABX tests against CD?

There are published industry and broadcaster DBTs of HD video, used to determine at which viewing distance the improvement becomes visible. How come the audio industry couldn't manage it for HiRes audio?

If a sound is masked (undetectable), it doesn't work this way. The memory doesn't bring it back once its masked again.
Once again, our opinions differ. I suggest you watch that Audio Myths Workshop vid I linked to earlier, more in particular the part where Poppy Crum shows how it does work.
Quote
Do you hear an 's'? I think I hear an 's'.
given that I understand multimodal perception and auditory scene analysis, I understand how these tricks work.

To suggest this disproves masking thresholds is damn close to suggesting that optical illusions show how picture coding will never work. Amazingly, optical illusions cope with JPEG encoding just fine

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Stone Free on 2012-08-14 18:03:32
I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.

I can think of a similar situation where details that could not be detected before a second step, that became obvious after it.

I would genuinely like to understand how come this is possible, the situation is as follows:

I'm watching a TV programme and a character says something, and I just can't understand what they've said, so I rewind my PVR and re-listen to them speak, nope, can't hear it, so I turn up the volume real loud play it several times, and no, I can't understand what they are saying.

However I rewind it once more and turn on the subtitles, so now I know what they actually said, and somehow from that point on if I rewind and replay it without subtitles, I can no longer re-create the situation of being unable to hear what was said.

What is the scientific reason behind this?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: db1989 on 2012-08-14 18:24:54
What is the scientific reason behind this?
Er, I don’t think there’s anything really scientific about it. You now know something that you did not know before. How could you ever recreate a time when you did not know it?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: lvqcl on 2012-08-14 18:43:51
(http://pmrb.net/blog/wp-content/uploads/2010/08/dalmatian.jpg)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Ron Jones on 2012-08-14 18:48:32
Quote
Robert Harley -- Mon, 07/14/2008 - 10:28
One doesn't have to be consciously aware of a reduction in artifacts to appreciate the deeper musical involvement the reduction in those artifacts engenders.

Appreciation denotes a level of consciousness. Unless he's referring to some sort of subconscious level of appreciation (which wouldn't be worthwhile to the listener in any way), this statement doesn't make sense.

This statement also suggests that a reduction in artifacts will lead to "deeper musical involvement", which isn't necessarily the case.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-14 19:05:47
With the previous examples about subtitles and the dalmation in mind, someone is going to come away thinking that listening to hi-res material will reveal details that didn't exist on the standard-res format and these details will forever be ghosted when listening to the standard-res format. Because of this phenomenon ABX can never work.

Of course ABX has been shown to work just not with the enormous differences of hi-res over standard-res.  Every passed ABX test must be the exception!

The key difference is that the actor really did say what the subtitles said and the dalmation really was in that picture.  IOW there never was any difference to begin with.  The equivalent is more like succesfully training yourself to hear a particular coding artifact.  Once you know what to listen for you can spot it.  Unortunately for those now trained, the original and the coded now sound different, rather than the same as certain people would like for you to believe.

Now how come I don't hear people say that there's an enormous difference between hi-res and standard-res but once you listen to the hi-res version you no longer hear this difference and the standard-res version sounds every bit as good?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 19:41:51
I'll try to be more clear. My point was if step (2) reveals a certain detail after step (1) did not reveal it, it's perfectly possible that we can switch back to system A and always hear this detail even though the same detail could not be heard during step (1). The explanation why this is perfectly possible is because step (2) did not only reveal this detail, but at the same time also it caused our brain to stored important information about this detail, into the memory of our brain. The fact this information is still accessible to our brain after we finished step (2) is because we do not have the power to erase it from our memory. Hence, we do not have the power to eliminate the bias that can result from this information, regardless of whether there was any practicing involved.

I can think of a similar situation where details that could not be detected before a second step, that became obvious after it.

I would genuinely like to understand how come this is possible, the situation is as follows:

I'm watching a TV programme and a character says something, and I just can't understand what they've said, so I rewind my PVR and re-listen to them speak, nope, can't hear it, so I turn up the volume real loud play it several times, and no, I can't understand what they are saying.

However I rewind it once more and turn on the subtitles, so now I know what they actually said, and somehow from that point on if I rewind and replay it without subtitles, I can no longer re-create the situation of being unable to hear what was said.

What is the scientific reason behind this?



There are at least two different effects.

(1) Once you learn that you are supposed to hear something, you will hear it even if it isn't there. I believe this is the essence of the McGurk Effect. McGurk Effect (http://en.wikipedia.org/wiki/McGurk_effect)

(2) Long term learning to decode audio does take place. For example since I've been watching BCC America quite a bit, I can more easily and accurately decode English with a British accent.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-14 19:45:39
Now how come I don't hear people say that there's an enormous difference between hi-res and standard-res but once you listen to the hi-res version you no longer hear this difference and the standard-res version sounds every bit as good?


I don't know how this relates but in general the enormous difference goes away when you start doing blind tests, any number of apparently successful sighted evaluations preceding or intermingled notwithstanding.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: mzil on 2012-08-14 21:15:28
I'm watching a TV programme and a character says something, and I just can't understand what they've said,...

However I rewind it once more and turn on the subtitles, so now I know what they actually said, and somehow from that point on if I rewind and replay it without subtitles, I can no longer re-create the situation of being unable to hear what was said.

What is the scientific reason behind this?


It's pareodolia (http://www.skepdic.com/pareidol.html). Here's a hilarious version of it that works the way our TV and closed captioning works, ie humans tendency towards suggestibility:
Oh Four Tuna. (http://www.youtube.com/watch?v=scCTty3KDLk)

It works both if the suggested words are true OR fictiional.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-14 22:23:52
I don't know how this relates but in general the enormous difference goes away when you start doing blind tests, any number of apparently successful sighted evaluations preceding or intermingled notwithstanding.
Apparently this difference may never come back thanks to being exposed to the hi-res version in which case double-blind or sighted tests will no longer reveal any differences. The upside to this is you only need to audition (insert gear or media of your choice here) and can go back to whatever (insert gear or media of your choice here) for it to provide the same stunningly wonderful experience!  Of course if this phenomenon can be reset then the excuse that double-blind tests can't ever indicate a successful outcome falls flat on its face.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-14 23:03:09
OT, but here's the best example I've seen of only being able to hear something when you're told what it is (posted recently to another thread)...
http://www.youtube.com/watch?v=muCPjK4nGY4 (http://www.youtube.com/watch?v=muCPjK4nGY4)

greynol makes a far more useful and serious point. These criticisms, if true, prevent listening comparisons full stop (period). And yet people compare, and report enormous differences. It's insane.

Arny too repeats the truth that those who have actually tried double blind testing have a far better insight into the limits of human perception than those who haven't.

Cheers,
David.

P.S. I love some of Barry Diament's recordings. They sound pretty great as -V2 mp3s.  I don't think the new albums are as good as the first one (and that's aside from the music!) - he managed a perfect acoustic mix on the first album (Lift) which is the way I'd like to hear most recordings. He hasn't managed to match that perfection on the others (IMO). Different musicians and different halls, I know, but it suggests to me that the perfect microphone placement on Lift may have been partly luck, or at least something that's not so easily recreated. And mp3 vs CD vs DSD is irrelevant in this case compared with placing the microphone 1 foot too far back.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-14 23:32:18
greynol makes a far more useful and serious point.

Thank goodness for logical conclusions and preceding posts by other members. I wonder if that's why earlier questions on the subject were being evaded. 

And mp3 vs CD vs DSD is irrelevant in this case compared with placing the microphone 1 foot too far back.

We forgot about the superior CD-R copy.

Seriously though, I've been happy with the remastering work he did in the '80s and perhaps other work, but I don't typically get too caught up with that type of stuff unless the engineer was grossly incompetent beyond annoying stylistic trends. What's unfortunate is that while the man obviously has gifted ears, it would seem he has limited his potential by not embracing unbiased critical listening exercises (to Arny's point).
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Hotsoup on 2012-08-14 23:45:21
Arny too repeats the truth that those who have actually tried double blind testing have a far better insight into the limits of human perception than those who haven't.

Not to continue veering off topic, but all I needed was about 45 minutes of calibrating my subwoofer until realizing it wasn't even turned on. Any golden ear I thought I had was gone after that. ABX's have done the same.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-15 00:18:36
I have a feeling audioclaudio decided to stop digging his hole, so I think letting the discussion progress elsewhere is fine except that anything worthwhile to future readers may go overlooked when the topic title comes up in a search.

Having this thread meander into the weeds does not concern me anywhere near as much as what these posts did to their parent topic that is linked in our TOS.

Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Stone Free on 2012-08-15 09:38:34
What is the scientific reason behind this?
Er, I don’t think there’s anything really scientific about it. You now know something that you did not know before. How could you ever recreate a time when you did not know it?
Its not that you now know what was said, but that you now hear what was said, even though you couldn't before.

The McGurk effect seems to be about vision affecting the perception of sound, whereas this is knowledge affecting perception.  I don't think it is Pareidolia either as that seems to be about the brains attempt to order/classify things and producing incorrect links.

Without experiencing it I would have thought that it should still sound indistinct as it did before.  I wonder if someone told me the following video would contain the word before I heard it, whether I would hear it correctly with that foreknowledge.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: 2Bdecided on 2012-08-15 10:13:59
I've been thinking more about the Bob Stuart statement in the original context: when listening to more "revealing" audio gear, you often hear things that you didn't before - but then you'll notice them on less "revealing" audio gear. This is suggested as a reason that ABX cannot work - because you can't "unhear" things.

Laying aside the question of whether it's the gear or your state of mind that causes the revelation ("this gear is really good - I'll listen very carefully to see what it reveals"  ), I have to say that, every time it's happened to me (usually with different headphones), I can still very clearly hear the differences between the headphones, even though the newly revealed feature is now audible through both.

Contrast this with the situation where something really is removed (IME a barely audible jangling of keys in a live recording, removed by a 16kHz low pass filter), and the keys were audibly missing from the LPF rendition, in a blind test, even though I knew they should be there, and they were the only audible clue to the LPF (there being little HF content apart from those keys). No amount of imagination could bring back something that was missing, and there was no other clue that something was missing.


Again, those who do ABX tests often find the opposite to what Bob's anecdote implies - ABX testing, while removing completely imagined differences, often lets you demonstrate that you could hear differences that were so subtle that you doubted them yourself. I have done several tests where I suspected I was guessing, but passed the test (without cherry picking).

Myself, I still wonder if HiRes vs CD will be successfully ABXed one day. I mean, I'm sure we can create a rigged test (bad converters, extreme content) where they can, but I wonder if something fairer will ever be passed. I don't discount the possibility, though given how hard it appears to be, it can't be that big a difference. It can't be "Life vs Death", as described by Stereophile.

Cheers,
David.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: knutinh on 2012-08-15 11:15:10
Myself, I still wonder if HiRes vs CD will be successfully ABXed one day. I mean, I'm sure we can create a rigged test (bad converters, extreme content) where they can, but I wonder if something fairer will ever be passed. I don't discount the possibility, though given how hard it appears to be, it can't be that big a difference. It can't be "Life vs Death", as described by Stereophile.

Contrary to popular belief among sensualist audiophiles, I think that most audio-interested scientists and sceptics would love for that to happen.

Me, I woud be happy if the audio industry moved focus (and R&D funding) away from snake-oil and towards the difficult stuff that everyone agree affects listening experience. There is a _slight_ chance that homeopathy works, that 192kHz is better than 44.1kHz and that I will become a professional football-player. Instead of wasting an infinite amount of resources searching for those improbabilities, I would rather spend limited resources on the stuff that likely do matter. Such as proper health-care. Proper microphone placement. Getting your daily exercise...

-k
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-15 15:38:21
Myself, I still wonder if HiRes vs CD will be successfully ABXed one day. I mean, I'm sure we can create a rigged test (bad converters, extreme content) where they can, but I wonder if something fairer will ever be passed. I don't discount the possibility, though given how hard it appears to be, it can't be that big a difference. It can't be "Life vs Death", as described by Stereophile.

Contrary to popular belief among sensualist audiophiles, I think that most audio-interested scientists and sceptics would love for that to happen.


Of course. Reality is that many have put a lot of effort into obtaining positive results for the audibility of whatever, but since only negative results were obtained, they quietly moved on.

Quote
Me, I woud be happy if the audio industry moved focus (and R&D funding) away from snake-oil and towards the difficult stuff that everyone agree affects listening experience. There is a _slight_ chance that homeopathy works, that 192kHz is better than 44.1kHz and that I will become a professional football-player. Instead of wasting an infinite amount of resources searching for those improbabilities, I would rather spend limited resources on the stuff that likely do matter. Such as proper health-care. Proper microphone placement. Getting your daily exercise...


That's the big picture of audio since probably the early 80s or so. Around then good electronic equipment generally became good enough that further significant improvements in SQ were improbable. There has been tremendous progress since then, but mostly in the realms of price/performance, and mostly in the form of applying technological advances first obtained in other areas.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: drewfx on 2012-08-15 16:57:26
There is a _slight_ chance that homeopathy works, that 192kHz is better than 44.1kHz and that I will become a professional football-player.

I don't think there's even the most heavily diluted of slight chances that homeopathy works.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-15 17:03:53
Would using deluded have diluted your pun?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Woodinville on 2012-08-15 20:26:58
Me, I woud be happy if the audio industry moved focus (and R&D funding) away from snake-oil and towards the difficult stuff that everyone agree affects listening experience.



I happen to know that will be part of the subject of the Heyser lecture this coming AES convention.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: AndyH-ha on 2012-08-15 22:34:42
Quote
No amount of imagination could bring back something that was missing, and there was no other clue that something was missing.


This is not something I attempt to keep up with, but my memory says that I first learned about it in a psychology of perception university class decades ago: subjects can be made to believe/expect that they are listening to a signal that contains X and they will be certain they hear X when it is not being presented. This kind of result gets reported every now and then by the popular press in studies of wine tasting and "consumer" responses to expensive vs inexpensive goods.

Rather recently, perhaps not more than a year ago, I read an abstract. or report, about some new DBT and deception experiments. I wish I knew where to find it again, for reference.

The main focus of the study seemed to be applying a new technology that allowed some brain measurements to identify the individual areas in the auditory cortex that responded to each different signal. Subjects were trained with a number of signals. Parameters for presentation were established where the subjects could reliably identify the different signals with a very high probability of success. The correlations between the specific signal and the specific auditory cortex neuron's response were found to be very consistent.

Then the subjects were led to expect a particular signal when they were in fact presented with one of the others, or sometimes with no signal at all. Subjects reported positively hearing what they expected to hear. This part has been know about for a long time. The new revelation was that the auditory cortex responded not in the spot/pattern  for the actual signal but in that for the expected signal. What was presented to the ear was irrelevant when it came to what was experienced by the subject's brain and by the subject's consciousness.

Has there been anything newer that cast doubt on these results?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: krabapple on 2012-08-26 18:53:16
Seriously though, I've been happy with the remastering work he did in the '80s and perhaps other work, but I don't typically get too caught up with that type of stuff unless the engineer was grossly incompetent beyond annoying stylistic trends. What's unfortunate is that while the man obviously has gifted ears, it would seem he has limited his potential by not embracing unbiased critical listening exercises (to Arny's point).



Aside from his demonstrated ignorance of the principles of digital audio, psychoacoustics, and testing methods (see his many posts on Steve Hoffman's forum)  and his reliable retreat to the I-hear-it-therefore-it's-real fallacy (ditto), there's the fact that Barry Diament  apparently couldn't hear that he'd reversed the channels on the first CD version of Led Zeppelin's Houses of the Holy.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-26 19:01:09
I'm not familiar with that but the channel reversal problem on IV appears to have been caused by a two-byte shift in the digital audio stream, and as such, most likely not caused by the mastering engineer and may not have been detected by the remastering engineer until it was too late to do anything about it (if ever).
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: krabapple on 2012-08-26 19:02:14
I'm not familiar with that but the channel reversal problem on IV appears to have been caused by a two-byte shift in the digital audio stream, and as such, most likely not caused by the mastering engineer and may not have been detected by the remastering engineer until it was too late to do anything about it (if ever).



Barry didn't do LZ IV.  That was a Zal Schreiber mastering.  He and Diament appear to have been Atlantic's two most prolific mastering engineers in the early days of CD.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: greynol on 2012-08-26 19:07:14
Is it possible that Houses may have suffered from the same issue?

PS: Apologies for the mastering/remastering inconsistency. I mean the same individual held responsible for digitizing the analog source.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Willakan on 2012-08-29 20:15:26
Quote
No amount of imagination could bring back something that was missing, and there was no other clue that something was missing.


This is not something I attempt to keep up with, but my memory says that I first learned about it in a psychology of perception university class decades ago: subjects can be made to believe/expect that they are listening to a signal that contains X and they will be certain they hear X when it is not being presented. This kind of result gets reported every now and then by the popular press in studies of wine tasting and "consumer" responses to expensive vs inexpensive goods.

Rather recently, perhaps not more than a year ago, I read an abstract. or report, about some new DBT and deception experiments. I wish I knew where to find it again, for reference.

The main focus of the study seemed to be applying a new technology that allowed some brain measurements to identify the individual areas in the auditory cortex that responded to each different signal. Subjects were trained with a number of signals. Parameters for presentation were established where the subjects could reliably identify the different signals with a very high probability of success. The correlations between the specific signal and the specific auditory cortex neuron's response were found to be very consistent.

Then the subjects were led to expect a particular signal when they were in fact presented with one of the others, or sometimes with no signal at all. Subjects reported positively hearing what they expected to hear. This part has been know about for a long time. The new revelation was that the auditory cortex responded not in the spot/pattern  for the actual signal but in that for the expected signal. What was presented to the ear was irrelevant when it came to what was experienced by the subject's brain and by the subject's consciousness.

Has there been anything newer that cast doubt on these results?


Any idea where you came across this report/paper?
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Dynamic on 2012-08-29 21:54:01
Any idea where you came across this report/paper?


I'm not AndyH-ha but I think I remember hearing about this paper too on one of the podcasts I listen to. Trouble is it could have been one of many, e.g.:

Skeptics Guide To The Universe (probably not that likely)
Scopes Monkey Choir (possibly one on audio pareidolia)
Naked Scientists (their general podcast and/or one of the topic-specific podcasts they make - most of which have a searchable transcript webpage)
Brain Science Podcast (also has transcripts available)

far less likely, but possible:
Little Atoms
or a sciency BBC podcast like
Material World
Infinite Monkey Cage (don't remember it being that jokey though)

It's possible that AndyH-ha heard it the same way and might narrow down the list.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: AndyH-ha on 2012-08-30 09:37:25
I'm guessing you mean the paper that found auditory neurons triggered by what was expected rather than by what was actually received at the ear. I know it was something I read and I'm fairly sure I read it on-line, but it could have been somewhere I picked up a journal or science related magazine to which I don't usually have access. It is aggravating when I can't recall or find something like that again.

I've also read about a study on wine tasting that found the brain stimulation of pleasure related areas was the same when people drank an expensive wine -- whether it was really an expensive wine or was a cheap wine passed off to them as something else.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Woodinville on 2012-08-30 09:55:20
I'm guessing you mean the paper that found auditory neurons triggered by what was expected rather than by what was actually received at the ear. I know it was something I read and I'm fairly sure I read it on-line, but it could have been somewhere I picked up a journal or science related magazine to which I don't usually have access. It is aggravating when I can't recall or find something like that again.

I've also read about a study on wine tasting that found the brain stimulation of pleasure related areas was the same when people drank an expensive wine -- whether it was really an expensive wine or was a cheap wine passed off to them as something else.


Aaaahhhhh....

I've given talks to that idea many times, and cited it on SGU, among other places, so yeah, you might have heard it.

There is a simple, trivial, wonderful example of that with so-called "backward masking", too.

Just show me the words, as it were...

Note: this doesn't show auditory neurons, it shows their results. That's good 'nuff for me.

Expectation bias exists, cross-sensory contamination is why our species is so successful, and in fact we shouldn't even call it contamination, it's integration of all senses, and its how humans survive.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Arnold B. Krueger on 2012-08-30 15:14:56
Me, I woud be happy if the audio industry moved focus (and R&D funding) away from snake-oil and towards the difficult stuff that everyone agree affects listening experience.



I happen to know that will be part of the subject of the Heyser lecture this coming AES convention.



Presumably the above will help unwind the innumerable revolutions of Heyser's body in its resting place that were added last year. ;-)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: splice on 2012-08-31 03:06:01
... Me, I woud be happy if the audio industry moved focus (and R&D funding) away from snake-oil and towards the difficult stuff that everyone agree affects listening experience.  ...


I doubt you'll be happy any time soon. Snake oil is much more profitable than making real improvements.
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: knutinh on 2012-08-31 08:02:56
So if "science" runs contradictory to how humans inherently work, to the solution that has helped us survive as a species for quite some time, where does that leave science? Is it a "harness" trying to force humans, against their nature (individually and collectively) to produce stuff that would otherwise be impossible?

Will the era of scientific dominance in the "truth-finding" tasks (such as: does punishment make for less crime in society, how do I calculate the area of this circle, what affects the quality of my hifi experience...) be but a short historical blimp? Preceded and succeeded by religious and intuitive explanations supported by anecdotes and authority.


Sorry about the philosophical rambling, I watched a debate about healers and alternative medicine yesterday. People seem to _really_ want to believe. And whenever there are believers, there will be people who give them what they want, either because they are cynically exploiting cancer patients, or because they are, themselves, believers.

Expectation bias exists, cross-sensory contamination is why our species is so successful, and in fact we shouldn't even call it contamination, it's integration of all senses, and its how humans survive.


If you think about it, many of the important problems that our ancestors had to solve in order to survive and reproduce can be seen as some sort of pattern recognition problem. "given what I am currently seeing and hearing, what I have learned about this place, is there or is there not a sabre-tooth tiger hiding in the bushes". The consequence of erroneous conclusions is either 1: running for no reason, 2: being eaten. Clearly, the problem is asymmetric, and the "nervous" individuals, or those who tended to see patterns even when there is none, would be better suited to survive such problems.

-k
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Porcus on 2012-08-31 09:14:28
I've also read about a study on wine tasting that found the brain stimulation of pleasure related areas was the same when people drank an expensive wine -- whether it was really an expensive wine or was a cheap wine passed off to them as something else.


Just saw this: http://www.bbc.co.uk/programmes/b00vhw1d (http://www.bbc.co.uk/programmes/b00vhw1d)

They fool people -- even chef students -- by adding the wrong colour to drinks.
The subjects also eat potato crisp, but wearing headphones, where the pitch of the chewing noises is altered. Higher pitch --> potato crisp rated as more crispy.

And there's the McGurk effect: http://www.youtube.com/watch?v=aFPtc8BVdJk (http://www.youtube.com/watch?v=aFPtc8BVdJk)
Title: DBT Is Flawed Because Bob Stuart Says So
Post by: Porcus on 2012-08-31 09:44:03
So if "science" runs contradictory to how humans inherently work, to the solution that has helped us survive as a species for quite some time, where does that leave science? Is it a "harness" trying to force humans, against their nature (individually and collectively) to produce stuff that would otherwise be impossible?


Does depend a bit on whether science resorts to being descriptive (and that includes the disciplines' use of words. For example, rationality is a normative word, and economics fail at being merely descriptive if it tries to define something as 'rational', I would say.)

Humans inherently work in a 3-dimensional colourspace. Physics has shown there are more than three base “colours” if colours are to mean “wavelengths”. That doesn't leave physics anywhere wrong, except that it may be contrary to the common interpretation to all it “colours”. People would likely accept that  “colours” that the human eye cannot see could still be referred to as “colours”, and “sounds” that the human ear cannot hear could still be referred to as “sounds”. At least I like(d) to think so. Call it naïve.




Will the era of scientific dominance in the "truth-finding" tasks [...] be but a short historical blimp? Preceded and succeeded by religious and intuitive explanations supported by anecdotes and authority.


Twenty years ago, the denialist industry still claimed there was no proof that tobacco smoking was dangerous. Obviously that ship was going down like an apple towards Newton's head. But it was a highly successful campaign (when you are essentially bound to die sooner or later, a long and prosperous life is a success). I am not much of an optimist, and my gut feeling is that mumbo-jumbo is on the rise, but that is not a scientific statement -- it could very well be the “everything was better in the old days” effect. It wasn't necessarily better then! Americans did not accept evolution twenty years ago, and I would not be surprised if someone dug up an old poll from the pre-internet era showing that the Europeans too were just as stupid then as they are now. Just because nowadays we face massive evidence-in-your-face for Einstein's claim of the infinitude of human stupidity, it may all boil down to the internet pill being, shock, the red one.

Blah blah blah, grrr, grrr, grrr ...