Re: How are the listeners selected for a Listening Test?
Reply #5 – 2021-05-06 00:26:24
All of your comments are perfectly valid points requiring consideration. Which is why some formal listening test methodologies require things like
only so-called "othologically normal" subjects as participants (age 18-25 IIRC, without any hearing disorders; this part is called pre-screening) pretraining of the test subjects qualified to participate - the more experience, the better usually (incl. handout of instructions before the test) exclusion of test results based on criteria like "mistaken hidden reference condition for a codec condition" (this part is called post-screening) listening in a controlled environment (listening with specific loudspeakers or headphones, with background noise below a certain threshold). Plus what Kamedo2 said about anchors. The reasoning behind the training and those anchors is that they help the test participants understand the quality scale they are judging stimuli on. That, in turn, makes the judgments more consistent and, therefore, facilitates identification of quality differences between codecs (or encoders) after the test since the confidence intervals usually stay smaller than without training and anchors. Unfortunately, most of these things can only be realized in a lab, like the ones available to companies developing codecs. On this forum, some compomises have to be made, especially regarding controlled environment of course, since it can't be controlled by the test coordinator. Btw, there actually are formal test methodologies explicitly focusing on untrained listeners, especially in the speech coding area (e.g. for VoIP communication, see, e.g., section A.4 of ITU-T Rec. P.800). Chris
Last Edit: 2021-05-06 01:30:45 by C.R.Helmrich