FYI - I've posted a new article called "Validation of a Binaural Room Scanning Measurement System for Evaluation of Automotive Audio Systems" This method makes it both practical and possible do to double-blind listening tests on different loudspeakers, listening rooms or automotive audio systems while controlling important nuisance variables (loudspeaker position, listener position, sighted biases,etc).

I think that audio reviewers could afford such a system for conducting controlled, double-blind evaluations of loudspeakers

Quote from: solive on 2009-06-14 22:48:56

FYI - I've posted a new article called "Validation of a Binaural Room Scanning Measurement System for Evaluation of Automotive Audio Systems" This method makes it both practical and possible do to double-blind listening tests on different loudspeakers, listening rooms or automotive audio systems while controlling important nuisance variables (loudspeaker position, listener position, sighted biases,etc).

I think that audio reviewers could afford such a system for conducting controlled, double-blind evaluations of loudspeakers

The obvious hang up will be the same as it was for the kinds of audio component evaluations that I posted on my old PCABX web site. BRS seems to put some equipment between the reviewer and the equipment. No matter how transparent you can prove that equipment to be, very many people will not feel right about it.

Compare and contrast the inherent audible imperfections of an excellent 24/96 ADC/DAC pair to those in BRS Please see the graphic labelled "BRS errors".  I think it is fair to say that BRS has far, far more of them than an excellent 24/96 ADC/DAC pair.

The listening test results showed there were no statistically significant differences in equalization preferences between the in situ and BRS playback methods. This was true for mono, stereo and multichannel playback modes (see slides 21-23). An interesting finding was that these results were achieved using a BRS calibration based on a single listener whose calibration tended to work well for the other listeners on the panel. This suggests that individualized listener calibrations for BRS-based listening tests may not be necessary, so long as the calibration and listeners are carefully selected.

Quote from: solive on 2009-06-14 22:48:56

FYI - I've posted a new article called "Validation of a Binaural Room Scanning Measurement System for Evaluation of Automotive Audio Systems" This method makes it both practical and possible do to double-blind listening tests on different loudspeakers, listening rooms or automotive audio systems while controlling important nuisance variables (loudspeaker position, listener position, sighted biases,etc).

I think that audio reviewers could afford such a system for conducting controlled, double-blind evaluations of loudspeakers

The obvious hang up will be the same as it was for the kinds of audio component evaluations that I posted on my old PCABX web site. BRS seems to put some equipment between the reviewer and the equipment. No matter how transparent you can prove that equipment to be, very many people will not feel right about it.

Compare and contrast the inherent audible imperfections of an excellent 24/96 ADC/DAC pair to those in BRS Please see the graphic labelled "BRS errors".  I think it is fair to say that BRS has far, far more of them than an excellent 24/96 ADC/DAC pair.

Quote from: Arnold B. Krueger on 2009-06-16 13:34:52

Quote from: solive on 2009-06-14 22:48:56

FYI - I've posted a new article called "Validation of a Binaural Room Scanning Measurement System for Evaluation of Automotive Audio Systems" This method makes it both practical and possible do to double-blind listening tests on different loudspeakers, listening rooms or automotive audio systems while controlling important nuisance variables (loudspeaker position, listener position, sighted biases,etc).

I think that audio reviewers could afford such a system for conducting controlled, double-blind evaluations of loudspeakers

The obvious hang up will be the same as it was for the kinds of audio component evaluations that I posted on my old PCABX web site. BRS seems to put some equipment between the reviewer and the equipment. No matter how transparent you can prove that equipment to be, very many people will not feel right about it.

Compare and contrast the inherent audible imperfections of an excellent 24/96 ADC/DAC pair to those in BRS Please see the graphic labelled "BRS errors".  I think it is fair to say that BRS has far, far more of them than an excellent 24/96 ADC/DAC pair.

BRS is not really required for listening tests on electronics - since they can be easily switched and evaluated in a controlled way. However, for  doing controlled evalutions of different loudspeakers, acoustical treatments/room designs and automotive audio sytems there is BRS is the only way I know for doing these tests (note: for loudspeaker tests you don't need to use BRS if you have a loudspeaker shuffler like this one). As we've shown in this paper. the differences between the these test objects are much larger than the BRS errors.

Quote from: solive on 2009-06-16 18:53:57

Quote from: Arnold B. Krueger on 2009-06-16 13:34:52

Quote from: solive on 2009-06-14 22:48:56

FYI - I've posted a new article called "Validation of a Binaural Room Scanning Measurement System for Evaluation of Automotive Audio Systems" This method makes it both practical and possible do to double-blind listening tests on different loudspeakers, listening rooms or automotive audio systems while controlling important nuisance variables (loudspeaker position, listener position, sighted biases,etc).

I think that audio reviewers could afford such a system for conducting controlled, double-blind evaluations of loudspeakers

The obvious hang up will be the same as it was for the kinds of audio component evaluations that I posted on my old PCABX web site. BRS seems to put some equipment between the reviewer and the equipment. No matter how transparent you can prove that equipment to be, very many people will not feel right about it.

Compare and contrast the inherent audible imperfections of an excellent 24/96 ADC/DAC pair to those in BRS Please see the graphic labelled "BRS errors".  I think it is fair to say that BRS has far, far more of them than an excellent 24/96 ADC/DAC pair.

BRS is not really required for listening tests on electronics - since they can be easily switched and evaluated in a controlled way. However, for  doing controlled evalutions of different loudspeakers, acoustical treatments/room designs and automotive audio sytems there is BRS is the only way I know for doing these tests (note: for loudspeaker tests you don't need to use BRS if you have a loudspeaker shuffler like this one). As we've shown in this paper. the differences between the these test objects are much larger than the BRS errors.

FWIW, the well-known editor of a high end magazine has specfically commented on the stiff costs of speaker-shuffling. However, he won't even do blind tests of the easy stuff, so his comments have to be discounted.

I agree that simple switching has a reasonable effort level associated with it for purely electronic devices that work at line level in and out. However, the number of blind test comparitors that work at the hardware level is vanishing compared to the virtually unlimited supply of free software-based comparators.

Departing  even slightly from the line-in, line-out situation to power amplifiers can be a big stumbling block for many because amplifier/speaker interfacing can be an issue. It bcomes non-trivial if the speakers at hand are easy loads or simply signficiantly different from all reasonable operating environments. 

When you start trying to evaluate music players, then the issues related to time synching can make things really overwhelming for most people who would otherwise be interested in doing blind tests. The time-related problems start when the UUT has audible latency, and are full-blown for almost all regular music players.

I suppose that is all true, and someone else is welcome to solve those problems. Our group's research and testing is focussed on more serious sound quality problems in the reproduction chain( i.e. loudspeakers and their acoustical interactions with small rooms including cars, lately headphones, and signal procesing algorithms aimed at fixing audible problemss).  Solving these problems (along with the quality of recordings ) will have the greatest benefit to consumers in terms of sound quality improvement.

Irrational objections from 'audiophiles' are just that.