Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Probability formula (Read 8070 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Probability formula

Hi.

Does anyone can give me a formula of guessing probability in ABX test? For example: N - number of trials, C - number of correct trials.
For C = N it's very simple: P=0.5^N , but if С ≠ N...
🇺🇦 Glory to Ukraine!

Probability formula

Reply #1
The distribution which describes the statistics of an ABX test is the binomial distribution. It takes two parameters, the number of trials n and the probability of success p. To find out how likely it is that the result of your test is random, for a test with n trials of probability p and c correct guesses, you need to evaluate the cumulative distribution function of the binomial distribution at (n-c). In case of an ABX test p=0.5. For example, in python using scipy stats you'd get for n = 16 and c = 6:
Code: [Select]
>>> import scipy.stats as st
>>> p = 0.5
>>> n = 16
>>> c = 6
>>> print st.binom.cdf(n-c,n,p)
0.8949432373046875

You can also look up some values in the table at: ABX binomial probability table.

General stuff about ABX statistics.
It's only audiophile if it's inconvenient.

Probability formula

Reply #2
Thanks. Well, if so, the formula is:



We get right numbers for n=3, k=2, n=k... But look at this:





Something is wrong with this formula...
🇺🇦 Glory to Ukraine!

Probability formula

Reply #3
Something is wrong with this formula...


You have to accumulate. Add the probability of the n=5 k=5. You will get the right number.

Alternatively, you can do this pyramid to get the probability, and then accumulate:
Code: [Select]
 n=0     1
n=1    0.5 0.5
n=2  0.25 0.5 0.25
n=3 0.13 0.37 0.37 0.13





Probability formula

Reply #4
Something is wrong with this formula...


In your example, we are interested in probability of obtaining "four or more" correct guesses, not "exactly four". So the probability of obtaining five out of five should be added to your result.

Probability formula

Reply #5
To add to the other answers, here is the CDF of the binomial distribution, which is the sum of all probailities up to the one in question.
It's only audiophile if it's inconvenient.

Probability formula

Reply #6
The distribution which describes the statistics of an ABX test is the binomial distribution. It takes two parameters, the number of trials n and the probability of success p. To find out how likely it is that the result of your test is random, for a test with n trials of probability p and c correct guesses, you need to evaluate the cumulative distribution function of the binomial distribution at (n-c). In case of an ABX test p=0.5. For example, in python using scipy stats you'd get for n = 16 and c = 6:
Code: [Select]
>>> import scipy.stats as st
>>> p = 0.5
>>> n = 16
>>> c = 6
>>> print st.binom.cdf(n-c,n,p)
0.8949432373046875

You can also look up some values in the table at: ABX binomial probability table.

General stuff about ABX statistics.



I did the calculations for that table and an unpublished version of it that went out to 200 trials.  I used the formula in the CRC handbook, which is online in Google Books. It was very difficult to calculate because factorals get big for numbers > 50 or so.

These links have some relevant resources:

http://www.lsbaudio.com/ABX/index.html

http://lsbaudio.com/publications/AES127_ABX.pdf


Probability formula

Reply #8
You have to accumulate. Add the probability of the n=5 k=5. You will get the right number.

Alternatively, you can do this pyramid to get the probability, and then accumulate:
Code: [Select]
 n=0     1
n=1    0.5 0.5
n=2  0.25 0.5 0.25
n=3 0.13 0.37 0.37 0.13



Something is wrong with this formula...


In your example, we are interested in probability of obtaining "four or more" correct guesses, not "exactly four". So the probability of obtaining five out of five should be added to your result.



If so, the formula is:



Yes, it works for 4/5. But it will give a double result for n=k, and the wrong result for 6/8for example:



(The right value is 0.145)


The right formula is in my previous post. Thanks to all.
🇺🇦 Glory to Ukraine!

Probability formula

Reply #9
If so, the formula is:
[...]


No. How did you come to this conclusion?

Quote
The right formula is in my previous post.


Yes, and this is what we were talking about the entire time.

Once again, we are interested in probablity of obtaining k or more correct answers out of n. To find it, you have to sum the probablity of obtaining k out of n, the probability of obtaining k+1 out of n and so on.

Probability formula

Reply #10
I cringe every time I see these ABX programs call the p-value "the probability that one is guessing".


Probability formula

Reply #12
I did the calculations for that table and an unpublished version of it that went out to 200 trials.  I used the formula in the CRC handbook, which is online in Google Books. It was very difficult to calculate because factorals get big for numbers > 50 or so.
For calculations involving large factorials you should use Stirling's approximation.

Using scipy.stats calculations are still blazingly fast up to (arbitrarily?) large n. After checking, I found that scipy uses the Cephes mathematical functions library to compute the binomial CDF: https://github.com/scipy/scipy/blob/master/...l/cephes/bdtr.c. It uses the incomplete beta integral according to the comments in the code. According to wikipedia the beta integral can be approximated using Stirling's formula. So this is consistent with pdq's reply.
It's only audiophile if it's inconvenient.

Probability formula

Reply #13
It was very difficult to calculate because factorals get big for numbers > 50 or so.

For calculations involving large factorials you should use Stirling's approximation.


Why?

With a little thought I came up with a means for doing it with no approximations.  I bludgeoned it to death with one of the largest mainframes of the day, which became almost totally idle and unused after the end of the first shift. Of course today my Sansa Fuze may have more RAM and processing power. LOL!

 

Probability formula

Reply #14

http://zak.s206.xrea.com/bitratetest/abxtable.htm
This is probably the simplest way.
Code: [Select]
/////////////////////////////////////////////////////
//  Probability formula without factorial, power, approximation
//  Written in JavaScript.
//  FIRST, make pascal triangle.
//  SECOND, accumulate from right to left, adding itself to left cell,
//                  10/13 is more like the 10+/13, sum of 13/13, 12/13, 11/13, 10/13.
//  FINALLY, you get the binomial probability table.
/////////////////////////////////////////////////////

var i = 0;
var j = 0;


/////////////////////////////////////////////////////
//  YOU can change the size of the table by just rewriting below.
var max_trials = 30;
var max_correct_responses = 18;
/////////////////////////////////////////////////////



pascal_triangle = new Array(max_trials);
probability_table = new Array(max_trials);

/////////////////////////////////////////////////////
// Initialize tables.    Fill all zero.
/////////////////////////////////////////////////////
for(i=0;max_trials>=i;i++){
  pascal_triangle[i] = new Array(max_trials+1);
  probability_table[i] = new Array(max_trials+1);
  for(j=0;max_trials>=j;j++){
    pascal_triangle[i][j] = 0.0;
    probability_table[i][j] = 0.0;
  }

}
pascal_triangle[0][0] = 1.0;//              SEED

/////////////////////////////////////////////////////
// Propagate pascal triangle directly below
// Distribute evenly (half to down-left, half to down-right.)
/////////////////////////////////////////////////////
for(i=0;max_trials>i;i++){
  for(j=0;i>=j;j++){
    pascal_triangle[i+1][j] += pascal_triangle[i][j]*0.5;
    pascal_triangle[i+1][j+1] += pascal_triangle[i][j]*0.5;
  }
}

/////////////////////////////////////////////////////
//  IT'S SHOWTIME!!!!!!
//  ACCUMULATE!!!!!!!!!
/////////////////////////////////////////////////////

for(i=0;max_trials>=i;i++){
  var acum = 0.0;
  for(j=i;j>=0;j--){
    acum += pascal_triangle[i][j];
    probability_table[i][j] = acum;
  }
}


/////////////////////////////////////////////////////
// Output the result as HTML.
/////////////////////////////////////////////////////

var str = "<table border>";
str += "<tr>";
str += "<th> </th>";
str += '<th colspan="'+(max_correct_responses+1)+'">NUMBER OF TRIALS WITH CORRECT RESPONSES </th>'
str += "</tr>";
str += "<tr>";
str += "<th>TRIALS</th>";
for(i=0;max_correct_responses>=i;i++){
  str += "<th rowspan="+(i+1)+">"+i+"</th>";
}
str += "</tr>";


for(i=0;max_trials>=i;i++){
  str += "<tr><th>";
  str += i;
  str += "</th>";
  for(j=0;Math.min(i,max_correct_responses)>=j;j++){
    var num = probability_table[i][j];
    var num_str = num.toFixed(5);
    var popup_str = j+"+ of "+i+", p="+num;
    var significance_level = 'style="background-color:#fdd;"';            //red
    if(0.05>num)significance_level = 'style="background-color:#ffc;"';//yellow
    if(0.01>num)significance_level = 'style="background-color:#dfe;"';//green
                      //must be blueish green for colorblind to recognize difference between red.

    str += '<td title="'+popup_str+'" '+significance_level+'>'+num_str+'</td>';
  }
  str += "</tr>";
}
str += "<tr>";
str += "<th> </th>";
for(i=0;max_correct_responses>=i;i++){
  str += "<th>"+i+"</th>";
}
str += "</tr>";
str += "</table>";

document.write(str);