Hi,
Just wanted your thoughts about a recent ABX test done be a few members(users) of the norwegian audio forum www.avforum.no.
The test was an ABX-test with three participants, each participant did 10 trials.
The audio system was:
System Audio Explorer Master
2x Electrocompaniet AW180 monoblock
Electrocompaniet EC4.9
Electrocompaniet ECC 1 CD player
Hegel HD20 DAC
The cables tested was:
*Noname 4mm cable (2x3 meters) with banana plugs, price approx 100USD.
*Nordost Baldur (2x3 metres), price approx 1000USD
To get to the point, the result was as follows:
Participant 1 was able to correctly identify X 7 out of 10 times
Participant 2 was able to correctly identify X 8 out of 10 times
Participant 3 was able to correctly identify X 7 out of 10 times
I'm no expert in either statistics or mathematics (and I'm also a cable sceptic), and as far as I understand none of these results are statistically significant. But that all three tests should have a relatively high number of correct answers is interesting, and if we add the results together (I don't know if that is statistically correct), you would get 22 out of 30 correct, hvis IS statistically significant.
All in all it looks like a plausible indication that it is possible to discern loudspeaker cables from each other. There could of course be a number of weaknesses to the test itself and I'm awaiting some additional information on how the test was conducted, but preliminary information indicates that the test concept was reasonable.
Any immediate questions or comments?
I remain skeptical... I'm sure I wouldn't hear a difference. (But, I might guess 7 of 10 correctly!)
It's "interesting" that none of the participants did better than 8 of 10, or worse than 7 of ten.
I remain skeptical... I'm sure I wouldn't hear a difference. (But, I might guess 7 of 10 correctly!)
It's "interesting" that none of the participants did better than 8 of 10, or worse than 7 of ten.
I might guess 7/10 as well, but three times in a row? I'm sceptical as well, and it may well be a weakness in the test. That would be the first assumption.
But assuming the test methodology is correct, I think it's an interesting result. Hopefully the test methods will be confirmed, and the test repeated for verification.
Without more information, we're left to speculation, which to me makes these these types of discussions are a complete waste of time.
I respectfully suggest people refrain starting such threads when there is such a blatant lack of pertinent information in the future.
Please take my comments from the point of view of a regular member and not a moderator. AFAIC, there is nothing in violation of the TOS regarding the content of this discussion, though it is questionable that it rises to a level deserving placement in the listening tests sub-forum; hearkening back to my original concern.
Without more information, we're left to speculation, which to me makes these these types of discussions are a complete waste of time.
I respectfully suggest people refrain starting such threads when there is such a blatant lack of pertinent information in the future.
Please take my comments from the point of view of a regular member and not a moderator. AFAIC, there is nothing in violation of the TOS regarding the content of this discussion, though it is questionable that it rises to a level deserving placement in the listening tests sub-forum; hearkening back to my original concern.
I understand your concern, I just thought it would be interesting to get input from hydrogen audio with respect to possible weaknesses in the test. I also think it is an interesting topic, as documented ABX tests of loudspeaker cables (negative or positive) are quite few and far between, probably because they are quite time consuming. I also assume that there are a number of negative tests performed that we never hear about.
I also understand that information is somewhat lacking. I will attempt to get more information from the guys who did the test and supply it here (for those that are interested).
If a moderator thinks that the thread is better suited for another sub-forum, please move it.
I look forward to reading more information. Right now I can't take the results seriously; not when people can easily come up with multiple possible reasons that would invalidate the test.
We recently discussed (http://www.hydrogenaudio.org/forums/index.php?showtopic=89210) a similar statistical result. Yes, as long as the participants made their decisions independently, you can combine the results and improve the case for significance. If so, there is something going on here. It could be a flaw in the test or it could be a real audible difference. The sound of cables is also something we've recently discussed (http://www.hydrogenaudio.org/forums/index.php?showtopic=89642&hl=impedance). In the end, one should keep in mind that different doesn't necessarily indicate better. Determining which is better requires a different king of testing.
There is one characteristic of speaker cables that can definitely make an audible difference, and that is resistance. How closely matched in resistance were these cables?
Series inductance and parallel capacitance will have a low-pass effect. Whether the resulting corner is low enough for this to be audible is the real question.
We really need to determine the nature of the test before diving into hypotheticals.
For the discussion to meander like this is rather pointless.
Perhaps the way to look at this is that the listeners could not reliably distinguish between a no-name cable and an audiophile cable.
The term "reliably distinguish" is a bit vague. Do you mean with 100% accuracy?
Which "noname" cable costs 100US for 6 meters?
My thoughts exactly - that's about $5.25/ft.
4 mm^2 is right in between 12 and 11 AWG, and I think you'd be hard-pressed to find 12 gauge zip cord that cost much more than $0.60/ft.
Decent banana plugs are a few dollars apiece - $10/ea. if you really want to get fancy - so yes, there's quite a discrepancy there.
In any event, as greynol has observed, we have no idea how the test was actually conducted - first and foremost, how was the switching accomplished?
The "noname" cable was a 4mm^2 cable from a Norwegian company "Biltema" (www.biltema.no) that sells more or less everything. The cable can be bought in a roll of 10 metre for 219 NOK = 39USD with the current exchange rate. So 3.9 USD per metre. (link to the cable: http://www.biltema.no/no/Bil---Mc/Bilstere...oyttalerkabel/) (http://www.biltema.no/no/Bil---Mc/Bilstereo/Monteringsdeler/Hoyttalerkabel/)). It had identical nordost bananaplugs as the Nordost cable soldiered on.
So the actual cable 3.9USD X 6 metres (not counting the price of the banana plugs) was about 23.4USD. My mistake.
During the test they first listened to A, then B (without being informed wich was nordost and which was biltema), and then X, and were asked to determine if X was A or B. Each participant was allowed to choose what music to listen to, and for how long.
They were also allowed to listen to the cables sighted for a while before conducting the actual ABX test.
The participants did not do 10 listening tests in a row, the listeners alternated between them so that the listeners got some rest in between the tests.
PRELIMINARY INFORMATION:
Before the test it was claimed that it would be conducted as follows (this has not yet been confirmed after the test):
- The listener is alone in the listening room while doing the ABX (There are a separate room for waiting to do the test)
- The song that is played is selected by the listener, and played back at the same volume during each test
- The listener is forced to put on ear phones before cable switching - making him unable to hear the switch
- The listener is either blindfolded, or the change of cables is done behind a curtain, making him unable to see the switch
- The listener is (by his own discretion) allowed to listen to parts of or an entire song
- There is no talk between the listeners in between tests.
When it comes to the question about resistance, I would assume that both cables are thick enough to make this a small (non-existant?) issue at 3 metres. If not we should have a lot more positive tests out there.
More information will hopefully follow.
To get to the point, the result was as follows:
Participant 1 was able to correctly identify X 7 out of 10 times
Participant 2 was able to correctly identify X 8 out of 10 times
Participant 3 was able to correctly identify X 7 out of 10 times
I'm no expert in either statistics or mathematics (and I'm also a cable sceptic), and as far as I understand none of these results are statistically significant. But that all three tests should have a relatively high number of correct answers is interesting, and if we add the results together (I don't know if that is statistically correct), you would get 22 out of 30 correct, hvis IS statistically significant.
I find it interesting that all 3 listeners had such close results. Did they tend to get it right on the same trial numbers? If so, then look for a difference between trials instead of a difference between cables.
Was the choice of X random (ie coin flip) or a test administrator selecting? Did all listeners get the same order?
In format, did a trial consist of one listen to A, B, and X, or could the listener ask for repeats? If, for example, he asks to hear B and then X you have to take care that he doesn't get clues from a shorter transition time if X is B, so no cable change would be required (or a disconnect/reconnect takes less time).
As was discussed in a recent topic of analog vs digital, double-blind is important. That is, there is no exposure to any one who knows which is which.
The phones to prevent hearing the switch, are they sufficient? These are usually meant to protect hearing, not to keep you from hearing at all, typically between 20 and 30 dB of isolation.
One way to skew results is picking the best ones. So was there a total of 3 participants, or is this the best 3 results out of more?
Link to the original thread, maybe, even if it's in another language? Also the fact that it wasn't immediate switching makes this even more suspicious, IMO. Also, were the switches completely random (as in throwing a coin or similar, instead of being at discretion of the testers)?
Other considerations:
Looks like a 'homemade' cable ('banana plugs soldered on') versus a 'phile jewelry cable. Could use some measurement data on both cables themselves. Maybe the 'audiophile' cable is intentionally 'colored'. Maybe the soldering on the homemeade cable isn't good
.
Also, was there any plausible way the cable switching person could have cued the subjects (consciously or not)?
A little more information:
*A trial consisted of one listen to A, B and X, no repeats.
*There was a total of three participants, no results have been removed or hidden
*I'm not sure what the point was with soldering on banana plugs on the cheap cables, I have asked about that
*The order of X was not the same for each listener.
*It has not been fully explained how the change was made, but it does not appear to be full double blind.
The first tester guessed correct on the cheap cable 3 times and the expensive cable 4 times.
The second tester guessed correct on the cheap cable 5 times and the expensive cable 3 times
The third tester guessed correct on the cheap cable 4 times and the expensive cable 3 times
Link to the original thread (long):
http://avforum.no/forum/kabler/113328-skik...-avsluttet.html (http://avforum.no/forum/kabler/113328-skikkelig-blindtest-avsluttet.html)
Link to post with an excel sheet with the results. Probably understandable even though it is in norwegian, let me know if not:
http://avforum.no/forum/kabler/113328-skik...tml#post1702429 (http://avforum.no/forum/kabler/113328-skikkelig-blindtest-avsluttet-19.html#post1702429)
A complete explanation of the test methodology has not yet been posted, I am suspecting that the importance of all these factors is not entirely evident of the guys who did the test, but I'm doing my best to get the relevant information.
My assumption so far is that there is some kind of weakness present here. The most significant so far being that it appears that the guy doing the switch knew which cables were being played, and did the switch while the listener was in the room. In what sense the listener was deprived of sight/hearing while the switch was made has not yet been fully explained.
Some more information:
It was not a double blind, the guy who switched cables knew what was connected.
The cable switch took about 15 seconds each time, and the cables were "switched" even though it was the same cable that was about to be listened to again. As the cables had identical banana plugs the switch was identical both in timing and "sound" when plugged into the sockets.
The switch was done behind a curtain, and the listeners never saw any of the cables before or during the test (which were referred to as cable A or cable B at all times), so they had no practical way of identifying the cables.
The reason why the cheap cables was supplied with identical banana plugs as the expensive ones was to make the test (everything was the same except the actual cable) and switch as identical as possible.
The sequence of X was made up during the test by the guy who switched cables, it was not predetermined (but not randomized by flipping a coin either).
As far as I can see it seems to be a pretty well carried out test, but it was not double blind, so you do have the issue of someone who know what was connected being in the room with the listener during the cable switch (the guy who switched left the room during the actual listening, he just went in to switch cable between A, B, X and then left again between each one).
That doesn't seem very rigorous at all, nor very well thought out.
That doesn't seem very rigorous at all, nor very well thought out.
What are the obvious shortcomings, besides the fact that it wasn't double blind? Any reasonable suggestions for improvements are welcome, as another test is being planned for a later date to confirm the result.
The "noname" cable was a 4mm^2 cable from a Norwegian company "Biltema" (www.biltema.no) that sells more or less everything.
Actually, Biltema is a Swedish company
The term "reliably distinguish" is a bit vague. Do you mean with 100% accuracy?
Although the overall result was statistically significant none of the individual listeners could achieve a statistically significant result. This is at odds with the advertising blurb, e.g.
“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “
One would have hoped that a cable that surprises and delights would have been more detectable.
Suggestions for improvements would be for more than ten tests to see if individual listeners could achieve a statistically significant score and for listeners to attempt to describe the difference. For example, was cable X more delightful or was it a bit louder.
A bit simplified, but: p=.5, n=30, x=22 --> P(x) = .0055
I would say at the least that it merits positive correlation.
The term "reliably distinguish" is a bit vague. Do you mean with 100% accuracy?
Although the overall result was statistically significant none of the individual listeners could achieve a statistically significant result. This is at odds with the advertising blurb, e.g.
“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “
One would have hoped that a cable that surprises and delights would have been more detectable.
Suggestions for improvements would be for more than ten tests to see if individual listeners could achieve a statistically significant score and for listeners to attempt to describe the difference. For example, was cable X more delightful or was it a bit louder.
They should of course have more listeners and more tests, but they spent all day to do this many tests, so doing larger tests effectively is a challenge.
One of the listeners have provided their experience, in Norwegian - but a short summary of his impressions:
Before the test he did not think it was possible to tell the difference. If it was any difference, he assumed it would be that one of the cable sounded more "muddy" or less clear than the other.
Listening material: Hanne Boel, CD ”Black Wolf” – Medley Records. Track: ”Black Wolf”, the first two minutes.
During the sighted listening before the test, he did actually think there was a small difference, but it was small and it was hard to put a finger on exactly what was different. He had to really concentrate to tell the difference.
If he was pressed to explain what the difference was, he'd say that one of the cables had a more spacious sound, and somewhat crisper in some parts of the track. None of the cables sounded muddier than the other, and all instruments and the frequency response sounded the same.
During the test:
The first few times he felt fairly confident that he was able to guess correctly, but it became progressively harder to keep his concentration up. A break with some food helped. Even though he thought he could tell the difference during the test, he was surprised that he had been able to guess correct that many times.
His conclusion was that based on the test it appears that there may be a small difference, but it is so small that he does not find that it justfies the price of expensive cables compared to other investments as room treatment and other components.
If he was pressed to explain what the difference was, he'd say that one of the cables had a more spacious sound, and somewhat crisper in some parts of the track.
And even with that, maybe the better one is the cheaper one.
This is at odds with the advertising blurb, e.g.
“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “
One would have hoped that a cable that surprises and delights would have been more detectable.
Their literature also emphasizes the Baldur being good value for the money. You know, not that snake oil crazy expensive stuff.
A bit simplified, but: p=.5, n=30, x=22 --> P(x) = .0055
Calculation question: I got 0.8% for the probability of 22 or more hits in case of guessing (p=0.5). I think you calculated the probability for obtaining
exactly 22 hits. What is correct for p-level calculation?
octave-3.2.4.exe:15> 1 - binocdf(21, 30, .5)
ans = 0.0080624
Although the overall result was statistically significant none of the individual listeners could achieve a statistically significant result. This is at odds with the advertising blurb, e.g.
“ The high tolerance conductor spacing used in this cable means that musical performance reaches new heights. For detail, resolution and transparency Baldur will surprise and delight you. “
One would have hoped that a cable that surprises and delights would have been more detectable.
But honest marketing wouldn't sell cables.
Isn't "honest marketing" an oxymoron anyway?
Cheers,
David.
Being Norwegian, I've followed the discussion on AVForum with some interest, and to me it seems obvious that
1, the changes of cables were performed by a person being present at the same time and visible to the listeners.
2, the changes were not random, but decided by the 'changers' on a 50/50 basis.
Both these circumstances make the test useless..
The people criticizing the 'test' seems focused on the mathematical chances of guessing — but it's actually a matter of psychology and non-verbal communication. I'm actually surprised the result was not higher than 70-80%
Being Norwegian, I've followed the discussion on AVForum with some interest, and to me it seems obvious that
1, the changes of cables were performed by a person being present at the same time and visible to the listeners.
2, the changes were not random, but decided by the 'changers' on a 50/50 basis.
Both these circumstances make the test useless..
The people criticizing the 'test' seems focused on the mathematical chances of guessing — but it's actually a matter of psychology and non-verbal communication. I'm actually surprised the result was not higher than 70-80%
The fact that it wasn't a double blind test weakens the test, but as far as I know regular blind testing isn't totally useless. I would assume the added value of adding double blind depends on the nature of the test.
Bias in blind (not double blind) testing may occur for example if a doctor knows that the patient is getting placebo, and thus may without meaning to do so send out negative signals to the patient or some kind of body language indicating that the doctor doesn't believe that the pills will work. This may then be picked up by the patient, and affect the test.
I don't see how this issue is as relevant (I'm not saying it is totally irrelevant) in a test as the one discussed here. The listener isn't told to detect the most expensive cable, simply if X is A or B, and it is equally important that he guesses correctly if it is the cheap or expensive cable he is trying to identify.
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?
A bit simplified, but: p=.5, n=30, x=22 --> P(x) = .0055
Calculation question: I got 0.8% for the probability of 22 or more hits in case of guessing (p=0.5). I think you calculated the probability for obtaining exactly 22 hits. What is correct for p-level calculation?
octave-3.2.4.exe:15> 1 - binocdf(21, 30, .5)
ans = 0.0080624
?
Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)
Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)
It's easier to lean towards more trials when you've got some automation and you can pop them off on your computer in a few minutes.
If it involves arranging for a facility, people to do the switching, leaving and entering rooms, and (as recounted here) it takes all day to do 10 trials x 3 listeners maybe take what you can get.
?
Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)
Ok, bad wording on my side. I used "p" both for the probability for naming the right cable per trial (null hypothesis guessing: p=0.5) as well as for the outcome of the probability that the 22/30 result would be obtained. When judging whether if falls into the category "statistically significant" or not, which probabilty is to be calculated?
a) The probability for exactly 22 hits by chance?
b) The probability for 22 or more hits by chance?
?
Typically p=0.05 is set as the threshold for 'guessing'. But this is arbitrary, and there are reasons to argue that the 'guessing' threshold for such phenomena should be even more stringent (e.g., p=0.01)
Ok, bad wording on my side. I used "p" both for the probability for naming the right cable per trial (null hypothesis guessing: p=0.5) as well as for the outcome of the probability that the 22/30 result would be obtained. When judging whether if falls into the category "statistically significant" or not, which probabilty is to be calculated?
a) The probability for exactly 22 hits by chance?
b) The probability for 22 or more hits by chance?
try plugging the numbers in here
http://stattrek.com/tables/binomial.aspx (http://stattrek.com/tables/binomial.aspx)
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?
I've always been under the impression that double blind tests are necessary for experiments such as these due to exactly the fact that people are good at picking up on subconscious cues of those around them. However, I've just been going on the word of mouth that people have told me this. I would be rather interested to see experimental results that indicate that Double Blind Experiments are needed.
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?
I've always been under the impression that double blind tests are necessary for experiments such as these due to exactly the fact that people are good at picking up on subconscious cues of those around them. However, I've just been going on the word of mouth that people have told me this. I would be rather interested to see experimental results that indicate that Double Blind Experiments are needed.
I agree. It's one thing if the cable changer stands in the room and asks the guy if he thinks this cable sounds better than the previous one, while knowing that it is the expensive cable that is playing. But in this particular context (identify if X is A or B), I'm not sure how the cable changer should be able to subcounsciously help the tester.
I agree. It's one thing if the cable changer stands in the room and asks the guy if he thinks this cable sounds better than the previous one, while knowing that it is the expensive cable that is playing. But in this particular context (identify if X is A or B), I'm not sure how the cable changer should be able to subcounsciously help the tester.
We weren't there to see what interaction was possible, but as an example a more upbeat tone of voice or expression when he announces that the the high price cable is ready, whether it is A, B, or X. Maybe he sways to the music a bit when the high price cable is playing.
When the forum was discussing the bases they had to cover so the test would be unassailable I'm surprised double-blind didn't come up, since it is generally a standard for ABX testing. For that matter, so is random selection of X, but that part isn't so much in the public consciousness.
We weren't there to see what interaction was possible, but as an example a more upbeat tone of voice or expression when he announces that the the high price cable is ready, whether it is A, B, or X. Maybe he sways to the music a bit when the high price cable is playing.
When the forum was discussing the bases they had to cover so the test would be unassailable I'm surprised double-blind didn't come up, since it is generally a standard for ABX testing. For that matter, so is random selection of X, but that part isn't so much in the public consciousness.
But there's no immediate relevance as to wether the expensive or cheap cable is playing. The test isn't about figuring out which cable is best, it is as about successfully identifying X each time, regardless of wether X is the cheap or the expensive cable. I assume the guy could send "expensive signals" to the listeners when the expensive cable is playing, and if the listener gets this on B, but not A or X - he could assume that X is A. But I must admit I have my doubts as to how much this would affect this test in practice.
When it comes to swaying to the music: The person that changed the cable was not in the room while the listening was conducted. He exits and enters the room between each change. But I guess he may be walking faster in anticipation each time he has changed to the expensive cable, we have no real way of knowing how all this works. Which is the point of double blind in the first place I guess.
My point is just that it may be slightly unfair / rushed to say that the test / result is totally useless because it wasn't double blind.
It looks like reasonable care was taken in testing. The simplest explanation appears to be that there are audible differences between the cables. I suggest someone measure the the frequency response and attenuation of the cables while they are connected to the speakers used in the test. It's not a difficult test to do. You can even take a crack with a test CD and a digital volt meter if you don't have real audio test equipment.
The expensive cables are a peculiar geometry and it is quite possible they will affect the sound. Not necessarily better but audibly different.
In another audio forum, an older thread about DBT'ing just today had a link to an interconnect cable blind test:
http://www.nordost.com/pdf/hifiplus_issue34.pdf (http://www.nordost.com/pdf/hifiplus_issue34.pdf)
To me it seems rather lacking in details about just how blind of a test it was.
In another audio forum, an older thread about DBT'ing just today had a link to an interconnect cable blind test:
http://www.nordost.com/pdf/hifiplus_issue34.pdf (http://www.nordost.com/pdf/hifiplus_issue34.pdf)
To me it seems rather lacking in details about just how blind of a test it was.
After further review.
"Hi Fi Plus" magazine issue #34 has a 2004 date and I don't see this test listed on their web page.
What are the obvious shortcomings, besides the fact that it wasn't double blind?
The more obvious one to me, is that it wasn't random. Furthermore, I'm wondering if these two are true:
1, the changes of cables were performed by a person being present at the same time and visible to the listeners.
2, the changes were not random, but decided by the 'changers' on a 50/50 basis.
So, you are saying that #1 is not true at all, from your posts, right?
#2, it was decided on a 50/50 basis? Which means that not only it wasn't random, but also the "changers" made sure there was equal number of A's and B's?
1. The listener was not removed from the room while the change was taking place, so the listener could see the person who did the change. But the person that changed cables left the room before the listening was conducted, and the cable change was done behind a curtain, so it was of course not visible for the listener.
2. The changes was decided by the "changer" as the test went along, but X was the cheap and expensive cable 50% of the time. This is probably another weakness in the test, but as far as I know, the listeners were not aware of this fact during the test.
In precisely what way could the changer unconsciously project "B" or "A" to the listener while changing the cable?
The means need not be precise, just effective!
Our need for DBTs becomes clear if one considers the story of "Clever Hans" a horse that "did arithmetic" but was actually sensing the right answers from his handlers.
A number of means by which Clerver Hans' handlers communicated the right answers to the horse have been documented, if memory serves. You might want to study up on this.
Now that’s an interesting study; thanks!
A few quotes from Hans’ Wikipedia article (http://en.wikipedia.org/wiki/Clever_Hans):
the horse got the right answer only when the questioner knew what the answer was, and the horse could see the questioner. He observed that when von Osten knew the answers to the questions, Hans got 89 percent of the answers correct, but when von Osten did not know the answers to the questions, Hans only answered six percent of the questions correctly.
Pfungst then proceeded to examine the behaviour of the questioner in detail, and showed that as the horse's taps approached the right answer, the questioner's posture and facial expression changed in ways that were consistent with an increase in tension, which was released when the horse made the final, correct tap. This provided a cue that the horse could use to tell it to stop tapping.
…
Pfungst made an extremely significant observation. After he had become adept at giving Hans performances himself, and fully aware of the subtle cues which made them possible, he discovered that he would produce these cues involuntarily regardless of whether he wished to exhibit or suppress them. Recognition of this striking phenomenon has had a large effect on experimental design and methodology for all experiments whatsoever involving sentient subjects (including humans).
Interesting study.
Hopefully the test will be repeated as double blind later this year.
...with each and every trial chosen randomly by coin toss and hopefully a larger number of participants and trials.
Was there anything else called into question?
...with each and every trial chosen randomly by coin toss and hopefully a larger number of participants and trials.
Was there anything else called into question?
A larger number of participants and trials may prove difficult as it apparently is quite time consuming to conduct the tests, but time will tell.
The main issues that was suggested changed for the next round was:
a) ensure double blind testing
b) selecting X randomly (coin toss or similar)
c) determine wether level matching between cables are required (can difference in resistance in the cables produce audible differences in volume?)
d) Switch cables both on the loudspeaker and amplifier end. During the previous test, the cables were only changed on the loudspeaker end, as the amps (AW180) has a double set of speaker terminals. So theoretically the test may have proved that speaker terminal A and speaker terminal B on the AW180s doesn't sound the same (I don't know how plausible that is in practice).
And to repeat @Notat's suggestion, it would surely be (time)cost-effective to do some measurements on the expensive cables, to see if they are colouring the sound?
Our need for DBTs becomes clear if one considers the story of "Clever Hans" a horse that "did arithmetic" but was actually sensing the right answers from his handlers.
The handler and the horse never see one another in this test. I know that it can be subtle but I don't see a mechanism for conveying information to the participant.
Please do some measurements first. That will be much easier and objective than doing more listening. It is not difficult to build a cable that intentionally colors sound. Demonstrating that such a cable sounds different is not a great feat. I suspect that's what you've done here.
Is it possible to make recordings of the soundsystem with only the speakercable as variable?
Then use abx software to determine audible differences?
Our need for DBTs becomes clear if one considers the story of "Clever Hans" a horse that "did arithmetic" but was actually sensing the right answers from his handlers.
The handler and the horse never see one another in this test. I know that it can be subtle but I don't see a mechanism for conveying information to the participant.
From a page back:
1. The listener was not removed from the room while the change was taking place, so the listener could see the person who did the change.
I assume the subject can't see what the technician is doing. I assume the subject and technician are not in cahoots. This is not like the trainer's posture as the horse does math or the physician's demeanor as he writes a script for a placebo. The technician leaves the room while the subject listens and answers the ABX. I guess it is possible there's something I'm overlooking and better to be rigorous than to have to question. On the other hand, if you establish such a rigorous standard, you give yourself a mechanism to reject results you don't like. This introduces a whole different sort of bias.
I assume the subject can't see what the technician is doing. I assume the subject and technician are not in cahoots. This is not like the trainer's posture as the horse does math or the physician's demeanor as he writes a script for a placebo. The technician leaves the room while the subject listens and answers the ABX.
In neither of the other cases are the tester and subject in cahoots either.
Yes, it is like the trainer's posture or the physician's demeanor. If the tester has a different attitude toward the expensive cable that's reflected in something similar to what the horse was picking up then the damage is done before the listening starts.
The listener was not alone for a whole ABX trial. The tester had to enter the room to change cables for every switch between A, B, and X.
I assume the subject can't see what the technician is doing. I assume the subject and technician are not in cahoots.
Doesn't matter. Scientists don't assume that either. The DBT requirement rests on other grounds -- the fact that cueing can be *unconscious* or 'unintentional' to both 'sender' and 'recipient' .
This is not like the trainer's posture as the horse does math or the physician's demeanor as he writes a script for a placebo. The technician leaves the room while the subject listens and answers the ABX. I guess it is possible there's something I'm overlooking and better to be rigorous than to have to question. On the other hand, if you establish such a rigorous standard, you give yourself a mechanism to reject results you don't like. This introduces a whole different sort of bias.
Those who study perception (and the journals that publish their work) would not consider it over-rigorous; they would consider it standard operating procedure.
Is it possible to make recordings of the soundsystem with only the speakercable as variable?
This. It may be as trivial as a different frequency response in the cable. Perhaps there's some inductance and/or capacitance involved in the cable configuration. Coiled a certain way?
Also, how many people have tried ABXing loudspeaker cables? Has it happened 200 times (inverse of the p-val) before? We'd be due for a solid false positive if that were the case. The point of ABX is that you can get the probability arbitrarily low. I'd like to see them get a better result before I'm willing to accept this. It's too at-odds with what I've been able to demonstrate to be true. I can ABX MP3 at 320 vs. FLAC on at least a few samples. Cables have never made a significant difference, unless they're pretty obviously faulty.
Those who study perception (and the journals that publish their work) would not consider it over-rigorous; they would consider it standard operating procedure.
As long as that's applied uniformly to all results, we're good. This implies it is not fair to use cursory listening to decide whether DBT is required.
I prefer to be guided by Mr. Sagan, "Extraordinary claims require extraordinary evidence." If the two cables measure significantly differently, the test described here has not produced an extraordinary result and we should be able to accept them without demanding additional rigor.
Is it possible to make recordings of the soundsystem with only the speakercable as variable?
Then use abx software to determine audible differences?
Yes, this is possible. It has limited usefulness. If you can hear a difference in these recordings, esoteric arguments aside, you'd be safe to conclude that you would be able to hear a difference in the room. If you don't hear a difference on the recording, you might still be able to hear a difference in the room because limitations of the recording may prevent you from hearing them.
Making a recording is similar to making the measurement I've suggested. If we had such recordings using the same source material and both sets of cables, I'd not bother listening to them. I'd analyze them and look for frequency response differences.
I would think that this kind of ABX test would be almost impossible to conduct. The sound reaching the tester's ears would change if anything or anyone in the room moved, let alone small changes in the listener's position or orientation. Add this to the significant delay in switching among A, B and X, and it seems highly unlikely that anyone could successfully identify a subtle difference.
Those who study perception (and the journals that publish their work) would not consider it over-rigorous; they would consider it standard operating procedure.
As long as that's applied uniformly to all results, we're good. This implies it is not fair to use cursory listening to decide whether DBT is required.
I prefer to be guided by Mr. Sagan, "Extraordinary claims require extraordinary evidence." If the two cables measure significantly differently, the test described here has not produced an extraordinary result and we should be able to accept them without demanding additional rigor.
...and I suggested measuring the cables in my first thread post. But simply measuring 'differently' is still not a free pass for accepting an anecdote as accurate, and bypassing rigor. Measured differences, though 'real', are not necessarily audible. The magnitude and type of difference matters.
I would think that this kind of ABX test would be almost impossible to conduct. The sound reaching the tester's ears would change if anything or anyone in the room moved, let alone small changes in the listener's position or orientation. Add this to the significant delay in switching among A, B and X, and it seems highly unlikely that anyone could successfully identify a subtle difference.
Toole and Olive have deviced adequate tests for loudspeakers in rooms. Why would you think that testing cables is any harder?
-k
I would think that this kind of ABX test would be almost impossible to conduct. The sound reaching the tester's ears would change if anything or anyone in the room moved, let alone small changes in the listener's position or orientation. Add this to the significant delay in switching among A, B and X, and it seems highly unlikely that anyone could successfully identify a subtle difference.
Yet people report large sound quality differences in sighted evaluations, even though the exact same "unlikely that anyone could successfully identify a subtle difference" problems exist.
Let's be honest - if the difference becomes undetectable if you move your head by a few mm, it doesn't matter.
That's
not to say that the difference must be larger than that which occurs when you move your head a few mm. Moving your head a few mm can create a large measurable difference. But our brains can often "tune this out" to track down a much smaller (but consistent) difference. If it's really there and audible at all.
Cheers,
David.
I have no doubt that cable does make a "difference" in sound. Crossed that bridge a long time ago. What I do doubt is the ability to identify those differences (statistically significantly). I have tried myself to differentiate "differences" on demand and have come up short. Too many human variables at play
I have no doubt that cable does make a "difference" in sound. Crossed that bridge a long time ago. What I do doubt is the ability to identify those differences (statistically significantly). I have tried myself to differentiate "differences" on demand and have come up short. Too many human variables at play
Without reliability, repeatability, and statistical significance,
the "results" are useless... This is about economics & marketing, and if an expensive speaker cable doesn't sound consistently & repeatedly better than zip cord, it's a waste of money (assuming sound quality is the criteria). If I can secretly switch-out someone's expensive cables and they are not 100% sure something's changed, they have wasted their money.
What if I claim that drug A is more effective than drug B, but it's not
statistically better? (The FDA would tell me I cannot advertise that claim.)
Or, what if I say my Camero is faster around the track than your Mustang, but it's not statistically or repeatably faster? What does that even mean??? (If you are a Mustang owner, you are likely to strongly disagree with my conclusions!)
Statistically non-reliable information is of no use any use to anybody!
...If I can secretly switch-out someone's expensive cables and they are not 100% sure something's changed, they have wasted their money.
What if I claim that drug A is more effective than drug B, but it's not statistically better? (The FDA would tell me I cannot advertise that claim.)
...
Even if they are not 100% sure that something has changed, the difference might be worth something for somebody if they are 99% sure, or 51% sure. The question is if it is any larger than pure luck/chance.
In most cases I would probably not be able to tell the difference between my current flac collection of music, and the same converted to sensible mp3 files. However, the added cost for going flac is negligible for me, and the peace of mind knowing that all files are sample-for-sample identical to their CD original is worth something - it might even have practical value in a few corner cases.
The only difference between me and a lunatic audiophile in this case is that I am willing to spend a few dollars worth of hard drive space on something that might or might not audibly matter. Hard-core audiophiles might be willing to spend $10.000 on cables that probably wont audible matter. No big difference if you have the money.
-k
No big difference if you have the money.
It has a big difference in terms of it's effect on the audio industry though.
There can't be that many people on the planet with enough money to buy all the things that might improve sound, but don't really. And none of us has enough time to listen to them. That's not to mention all the great music that you're actually ignoring when trying to listen for the sound of the cable!
Cheers,
David.
I have no doubt that cable does make a "difference" in sound. Crossed that bridge a long time ago. What I do doubt is the ability to identify those differences (statistically significantly). I have tried myself to differentiate "differences" on demand and have come up short. Too many human variables at play
What this amounts to is a claim that something that cannot be reliably detected still nevertheless exists. Very shaky philosophical ground you are standing on.
"A difference that makes no difference
is no difference" is one way to phrase it. It's called "positivism" and is the philosophical ground that all modern science stands on. You are free to set yourself against the whole basis of science if you wish, of course, but if so one wonders how you can do anything but live in constant anxiety since if you believe what you seem to believe you cannot trust anything that science has produced. How can you be sure that you will not spontaneously explode at any moment? How can you trust that your words typed into your computer will appear in this forum? After all the whole worldwide web is based on science and if everything that has been developed using the scientific method in the last two centuries were to stop working a huge number of human beings would cease to exist in a very short time, me among them.
No, I am afraid that when it comes to a disagreement with competent scientists and technicians who put their insites into practical use, and your philosophically suspect assertions, I side with the scientists.
No big difference if you have the money.
It has a big difference in terms of it's effect on the audio industry though.
The biggest (negative) effect on the audio industry is that R&D resources are diverted from technology that affects audible qualities with large certainty, into technologies that cannot be proven to have any (significant) audible benefit.
But this is off topic. Any effort to prove the claims of the audiophiles should be applauded, and duely critizised if anything is weak.
-k
The biggest (negative) effect on the audio industry is that R&D resources are diverted from technology that affects audible qualities with large certainty, into technologies that cannot be proven to have any (significant) audible benefit.
How much R&D do you think is really needed (and actually used) in audio cable industry? (well, maybe a lot, in the field of marketing... )
The biggest (negative) effect on the audio industry is that R&D resources are diverted from technology that affects audible qualities with large certainty, into technologies that cannot be proven to have any (significant) audible benefit.
How much R&D do you think is really needed (and actually used) in audio cable industry? (well, maybe a lot, in the field of marketing... )
Maybe "diverted" isn't correct - maybe "nullified". Back in the day, more companies were doing more research into improving sound reproduction. It was worth it because at least a few people paid money for better sound.
Now "better sound" costs less, and the people spending silly money aren't really chasing it (though many believe they are).
Some people claim "but sound quality
is now good enough - the real innovation has moved into other areas" - on one level that's true, though I rarely hear audio systems that are "good enough" (for the price). And proper immersive hi-fi surround sound is very rare.
Cheers,
David.
Some people claim "but sound quality is now good enough - the real innovation has moved into other areas" - on one level that's true, though I rarely hear audio systems that are "good enough" (for the price). And proper immersive hi-fi surround sound is very rare.
Transducer technology is certainly an area where there is still plenty of room for improvement, but where a lot of R&D is needed even to archieve little gain, as laws of physic are difficult to circumvent. This, in business terms, means little return of investiment.
And in my opinion this is an area where the term "quality" has still a subjective and non ABXable meaning.
Just wanted your thoughts about a recent ABX test done be a few members(users) of the norwegian audio forum www.avforum.no.
.....
Any immediate questions or comments?
It is very easy to detect the difference in sound between two cables if one or both of them are making an audible contribution to the sound! For example, $1000 cables are sometimes 'so heroically misconceived' as to introduce a non-flat frequency response, or to resonate in the audio passband when driving into a low-capacitance or low-inductance load, etc. The directional arrows printed on the Nordost cables are an immediate source of worry.
Where is the engineering data on the cables tested?
Similarly, was the amplifier output unmodified from the manufacturer's spec? Audiophiles are notorious tweakers.
It would have been interesting to see the electrical FR and distortion of the two cables measured at the speaker terminals when connected to the speakers and amp.