Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: ABX testing with low anchor (Read 4264 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

ABX testing with low anchor

Adding anchors (especially low) is quite common for MUSHRA tests.
I wondered if including a low anchor in an ABX test would be a good idea.
When unsuccessfully searching for very small or non-existing audible differences, it can happen that participants question the validity of the test.
A low anchor might be able to validate the test setup.
I couldn't find info about this. Apparently it's not a common approach, so I am curious to hear your thoughts.

ABX testing with low anchor

Reply #1 –
Quote from: Kees de Visser on
I wondered if including a low anchor in an ABX test would be a good idea.

If the other test conditions are relatively far from transparency (e.g. below 128 kbps stereo), yes, certainly.
Quote
When unsuccessfully searching for very small or non-existing audible differences, it can happen that participants question the validity of the test.
A low anchor might be able to validate the test setup.

Yes, but in my experience its main purpose is to "span up" a reasonable grading range so that listeners grade more consistently. Without the
anchor I've often noticed that some listeners grade a stimulus e.g. with "annoying" while for others the same stimulus is "... but not annoying".

Chris
If I don't reply to your reply, it means I agree with you.

ABX testing with low anchor

Reply #2 –
Semieducated babble alert:

(+) Could serve a few purposes:
1) Expectation bias might lead people to think that there are no differences between a supposedly "good" encoding X and the lossless, and therefore not listen carefully. Expecting that there are differences to be found, might alert people to at least listen until they have found those. Not only sharpen up, but spend more time on the experiment.
2) If the underlying reality is that some people have good hearing and others do not, then you might want to answer both "Is encoding X transparent to most?" and "Is encoding X transparent to those who can tell it from the low anchor?". Then one could maybe easier (correctly!) reject the "X indistinguishable from high anchor" null.
3) And then Chris' point.

(-) Objections, though, to the same items:
1') People might expect that X has those artifacts most audible in the low anchor, and listen for these and ignore others.
1') Try-until-you-make-it --> cherrypicking bias.
2') Complexity from testing ABX with AB picked repeatedly randomly from ABC. People might be confused, people might drop out and response rate might decrease.
3') If X would normally be considered "annoying" by significantly many, you could easily get fewer "annoying" answers if you introduce an "even more annoying" baseline.

And of course, comparing with previous tests with a different design of experiment ... the science of metastudies. Could be interesting if one has sufficiently many listeners, to randomize them in "old" vs "new" setup and test whether there are significant differences across 1, 2, 3 above.
 

ABX testing with low anchor

Reply #3 –
Quote from: C.R.Helmrich on
Quote from: Kees de Visser on
When unsuccessfully searching for very small or non-existing audible differences, it can happen that participants question the validity of the test. A low anchor might be able to validate the test setup.
Yes, but in my experience its main purpose is to "span up" a reasonable grading range so that listeners grade more consistently. Without the anchor I've often noticed that some listeners grade a stimulus e.g. with "annoying" while for others the same stimulus is "... but not annoying".
I was thinking out loud for ABX tests, so there's no grading involved. Tests like DSD256 vs DSD64 or 24/192, brick wall filters at 96 kHz etc. that would probably qualify "impossible" in HA experience. I suppose redbook 16/44.1 wouldn't qualify as a low anchor
Perhaps the training period is a better place for anchors like this ?