ABX questions
Reply #2 – 2004-03-27 17:59:38
The point about hiding results during the test is this: The p-values all recent ABX programs show, only give you the correct "probability to reach the current score (or better) by guessing" if the number of trials has been fixed before the test starts. If the tester is allowed to stop the test whenever he likes (e.g. when a certain p-value is reached), the real "probability ..." is bigger. I tried to explain this in the 1st post of this thread . One way to avoid this is to force the tester to enter the number of trials he's going to perform at the beginning of the test, another one is to hide the results. So the advantage of hiding the results is that the p-values are correct, the disadvantage is (obviously) that the tester doesn't have feedback and e.g. might not notice that he becomes fatigued quickly enough. A rule-of-thumb workarround would be to "perform one trial extra", i.e. if you want to reach a "probability ..." of 0.01, you don't stop after reaching 7/7 or 10/11 (-> p-value < 0.01) but perform one additional trial, i.e. you stop if 8/8 or 11/12 is reached. This method works good enough for < 30 trials. A safer method would be to use the little program I uploaded here to calculate "stop points" before you start the test (or use the examples from the post), e.g.: You want to reach a "probability ..." of 0.01. Now depending on how the stop points are calculated (you have the choice - are you aiming for quick success = 7/7 or do you expect some hard fight scoring e.g. 36/50 in the end?), the test will be finished successfully e.g. at1. Stop point: (8/8) C-Value: 0.00390625 2. Stop point: (11/12) C-Value: 0.00585938 3. Stop point: (13/15) C-Value: 0.00769043 4. Stop point: (16/19) C-Value: 0.00844574 5. Stop point: (18/22) C-Value: 0.00913858 6. Stop point: (21/26) C-Value: 0.00942713 7. Stop point: (23/29) C-Value: 0.00969638 8. Stop point: (26/33) C-Value: 0.00981075 9. Stop point: (29/37) C-Value: 0.00986504 10. Stop point: (31/40) C-Value: 0.00991874 11. Stop point: (34/44) C-Value: 0.0099426 12. Stop point: (36/47) C-Value: 0.00996601 Or you give up without reaching one of those scores and the test is failed. I hope this answers most of your questions, otherwise feel free to ask.