ABSTRACT

A pilot study was conducted in order to investigate the application of descriptive analysis techniques to the evaluation of consumer audio equipment. A global seven-step methodology for the descriptive evaluation of the sound characteristics of consumer audio equipment was developed. In contrast to the commonly used forced choice discrimination methods for audio equipment evaluation, the proposed methodology enhances, rather than compromises, a subject's sensory input. A case study employing a minimally trained subject demonstrated the applicability of sensory evaluation techniques to sound. There has been some resistance in the audio community to the adoption of sensory science evaluation methods because such methods are erroneously thought to apply only to products that affect the senses of touch, taste, smell and sight.

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones. 

Quote from: pdq on 2010-09-30 23:27:12

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones. 

He says:

"Blind audio testing, which includes visually obscuring all or part of the sound stage, rapid switching of musical selections and off-axis and group
seating, impairs the listener's ability to localize sounds (seeing), to internalize and evaluate aural cues (hearing) and to
receive correct stereophonic tactile information (touching). Any stereophonic audio system testing methodology which
compromises and hinders the processes of human sensory perception will result in consistently inaccurate and often
absurd results."

This is one of the most confused paragraphs I've ever read about listening tests.  It is a total misrepresentation of the true facts.

Since when does ABX test include "visually obscuring all or part of the sound stage"? I guess the author thinks that blind testing necessarily means wearing blindfolds, poking one eyes out, or darkering the room.

As has been pointed out many times, rapid switching of musical selections is not part of ABX testing. What may be rapdily switched is which presentation of the musical lsitening selections is being listened to at any particular time. Or not.

Since ABX tests are frequently done with just one listener, and since headphone listening is commonly done during ABX tests, the comments about off-axix and group seating are equally irreleevant to the true factys.

This paper is bascially a straw man argument. The apparently athor makes up his own ideas about what a bad listening test might be like, and then names them "ABX". The artile looks to me like baldfaced libel. I'd be worried if the author had any influence.

That is a big a$$ compound sentence but it is actually true ihttp://www.hydrogenaudio.org/forums/index.php?act=Post&CODE=02&f=40&t=83979&qpid=725107f you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

Quote from: analog scott on 2010-10-02 01:12:20

That is a big a$$ compound sentence but it is actually true ihttp://www.hydrogenaudio.org/forums/index.php?act=Post&CODE=02&f=40&t=83979&qpid=725107f you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

'To say the least'?  Read the article before commenting please, Scott. Obviously you have not.  Go ahead, it's a free download.

The paragraph quoted is the *concluding paragraph* of the article.  If he was writing objectively, Dr. Smith would have written something like, 'blind tests have been conducted where in addition to being blind, ....", clearly separating the conditions which are intrinsic to a blind test, from those that are not.  As written, he appears to be trying to tar audio blind testing itself or 'at the very least' obfuscating the issue by the use of the ambiguous term 'include'.    This is on top of multiple risible claims elsewhere in the article, like "According to Lipshitz, Vanderkooy and Clark, It is better to blindfold or otherwise visually handicap music listeners".

And btw, your reflexive need to undermine anything Arny writes remains as pathetic as it is predictable.  Please be aware that some of us here know of the long history between you two.

Quote from: Arnold B. Krueger on 2010-10-01 16:51:14

Quote from: pdq on 2010-09-30 23:27:12

It appears that his main justification for rejecting ABX testing is that it was once used to test audio quality of telephones. 

He says:

"Blind audio testing, which includes visually obscuring all or part of the sound stage, rapid switching of musical selections and off-axis and group
seating, impairs the listener's ability to localize sounds (seeing), to internalize and evaluate aural cues (hearing) and to
receive correct stereophonic tactile information (touching). Any stereophonic audio system testing methodology which
compromises and hinders the processes of human sensory perception will result in consistently inaccurate and often
absurd results."

This is one of the most confused paragraphs I've ever read about listening tests.  It is a total misrepresentation of the true facts.

Since when does ABX test include "visually obscuring all or part of the sound stage"? I guess the author thinks that blind testing necessarily means wearing blindfolds, poking one eyes out, or darkering the room.

As has been pointed out many times, rapid switching of musical selections is not part of ABX testing. What may be rapdily switched is which presentation of the musical lsitening selections is being listened to at any particular time. Or not.

Since ABX tests are frequently done with just one listener, and since headphone listening is commonly done during ABX tests, the comments about off-axix and group seating are equally irreleevant to the true factys.

This paper is bascially a straw man argument. The apparently athor makes up his own ideas about what a bad listening test might be like, and then names them "ABX". The artile looks to me like baldfaced libel. I'd be worried if the author had any influence.

That is a big a$$ compound sentence but it is actually true if you break it down for what it is and only what it is. He is describing a specific subset of "blind audio testing." That subset includes a visual obscuring of the "sound stage" (not sure why this would be an issue) rapid switching of musical selctions, and "off-axis and group seating." The last part is an issue. You will mess up the imaging of stereo playback *if* you do test that way.

If the author makes any broad inferences about blind testing from this specific subset conditions then that would be problematic to say the least.

Dude, lighten up. I was making fun of the sentence/paragraph.  Oh well, once one has to explain the joke the joke dies.

Quote from: analog scott on 2010-10-02 07:33:26

Dude, lighten up. I was making fun of the sentence/paragraph.  Oh well, once one has to explain the joke the joke dies.

The dominant impression was one of contentious spirit and prevarication for me. You might add a bold-faced "Bazinga!" next time, so that the simple-minded among us also get the humor.

The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).

In other words, ABX is fundamentally flawed because it compares gear to gear rather than comparing gear to performance standards. This is ludicrous and demonstrates an ignorance of the purpose of stereophonic music reproduction. This seems to be a concept that the "objectivists" and "scientists" cannot, or will not, grasp

1. An ABX (duo-trio balanced reference) test generally requires a subject population of at least 16 persons. Optimum subject population is at least 32 or more persons.

2. An ABX (duo-trio balanced reference) test which employs a subject size of less than 28 persons generates high rates of beta error (false negatives or "no differences between samples") in the results.

3. An ABX (duo-trio balanced reference) test must compare samples which are unknown (unfamiliar) to the test subjects.

4. An ABX (duo-trio balanced reference) test must use untrained test subjects.

I pulled this gem from one of those polkaudio trainwrecks:

Quote

The fundamental flaw in applying ABX tests to audio evaluation is in that people participating in ABX tests are searching for differences between equipment rather than conducting a performance evaluation against standard performance criteria (metrics).

In other words, ABX is fundamentally flawed because it compares gear to gear rather than comparing gear to performance standards. This is ludicrous and demonstrates an ignorance of the purpose of stereophonic music reproduction. This seems to be a concept that the "objectivists" and "scientists" cannot, or will not, grasp

.


It's saddening to read because he employs academic sensory testing jargon, and mixes 'real' science with tendentious crap.
E.g.,  this amazing jumble of fact and fiction:

http://www.polkaudio.com/forums/showpost.p...mp;postcount=53

Quote

1. An ABX (duo-trio balanced reference) test generally requires a subject population of at least 16 persons. Optimum subject population is at least 32 or more persons.

2. An ABX (duo-trio balanced reference) test which employs a subject size of less than 28 persons generates high rates of beta error (false negatives or "no differences between samples") in the results.

3. An ABX (duo-trio balanced reference) test must compare samples which are unknown (unfamiliar) to the test subjects.

4. An ABX (duo-trio balanced reference) test must use untrained test subjects.