Hi, I have a question.
From http://www.hydrogenaudio.org/forums/index....showtopic=31945 (http://www.hydrogenaudio.org/forums/index.php?showtopic=31945) :
You're on the wrong forum, buddy. ABX tests are infallible....
Infallible?? Well, if you say so. I stand corrected.
They are, and they are reproducible. And they are the most scientific way of proving such a statement.
One is entitled to his opinion, but once he decides to share it, it must be backed up by evidence.
I don't have a real firm grasp of statistics, but I remember hearing that there's a tradeoff between the possibility of type I errors and the possibility of type II errors. In other words, I thought there was a tradeoff between false positives and false negatives.
If it's true, how does the double-blind ABX method account for this? Or is that unnecessary?
Thank you in advance.
ABX tests can only prove there is a difference between two tested materials. Since it cannot prove there is no difference, the number of false negatives is infinite, and thus the number of false positives is null.
You divide by zero !
excellent post, Pio. May I suggest that it be added to the 'What is a blind ABX test?" pinned thread, in General Audio?
(actually it seems to me that that pinned thread should be in *this* forum too)
Pio's post should also be linked to in the FAQ, in my opinion.
I added this text to the ABX topic : http://www.hydrogenaudio.org/forums/index....showtopic=16295 (http://www.hydrogenaudio.org/forums/index.php?showtopic=16295)
This way, I don't think that it is necessary to add a link in the FAQ
Hi, I have a question.
From http://www.hydrogenaudio.org/forums/index....showtopic=31945 (http://www.hydrogenaudio.org/forums/index.php?showtopic=31945) :
I don't have a real firm grasp of statistics, but I remember hearing that there's a tradeoff between the possibility of type I errors and the possibility of type II errors. In other words, I thought there was a tradeoff between false positives and false negatives.
If it's true, how does the double-blind ABX method account for this? Or is that unnecessary?
Thank you in advance.
A Type I error is to reject the null hypothesis when the null hypothesis is true.
A Type II error is to accept the null hypothesis when it is false. I use accept, when really it is failure to reject.
More clearly:
You are listening to two tracks that are identical. Yet you manage to identify track A correctly a statistically significantly number of times. You are then able to say they are different, but they are not. This is a Type II error.
You are listening to two different tracks, but you cannot tell the difference in a statistically significant manner, this is a Type II error.
So before somebody claims that a designated encoding rate is transparent, they should be calculating the power of their test, this will give them some indication of how often, given the varibility of their results, they will commit a Type II error. If they have insufficient replication, they may be claiming transparency with only a (making up numbers here) 1/16 chance of detecting a difference.
' date='Apr 11 2006, 14:01' post='381104']
ABX tests can only prove there is a difference between two tested materials. Since it cannot prove there is no difference, the number of false negatives is infinite, and thus the number of false positives is null.
I agree we cannont prove that two things are the same, but we can calculate our chance of making an error when we do not reject the null.
We use a 1/20 (5%) chance of making a Type 1 error as our base rate, but generally fail to determine the Type II error rate. We should.
h