I've wrote a program for evaluating the corrected p-val for situation 2. It takes the maximum allowed number of ABX trials and the required confidence (i.e. traditional p-val) as arguments and returns the probability to pass the test with guessing.

Example:

(4, 0.95) -> 0 (impossible)

(5, 0.95) -> 0.03125 (5/5 -> pval=.96875)

(6, 0.95) -> 0.03125 (either the test is won at 5/5 or lost)

(16, 0.95) -> 0.08755493164

(100, 0.95) -> 0.2020580977 (!!!)

(16, 0.99) -> 0.01422119141

The idea is:

Construct a pascal triangle up to a certain level.

1

1 1

1 2 1

1 3 3 1

1 4 6 4 1

1 5 10 10 5 1

....

Now we can read it as follows: A(row=trials+1, column=correct+1) / 2^trials = P(abx=correct/trials | trials used) (e.g. the probability to score 2 times correct out of 3 is A(4, 3)/2^3 = 3/8)

The pval would be the sum of all those probabilities to the left of the chosen item.

My program does the following: seek for the earliest win condition (e.g. 5/5), calculate its probability (e.g. 0.03125) and set the corresponding item in the pascal triangle to **0** and recalculate, e.g.:

1 5 10 10 5 **0**

1 6 15 20 15 5 0

....

The last step is to make sure, that nothing is counted twice. Now start again. The next win condition with probability /= 0 is 7/8, with remaining probability of P(abx=7/8 | not 5/5) = 0,0195.., and so on...

Here's the Maple source code, I hope it's readable (# starts a comment):

CorrPVal:=proc(n,reqConfidence,Prob)

local k, Trial, LastResult, Result, Confidence:

Result:=array([1,seq(0,i=1..n)]): # initialize [1,0,...,0]

Prob:=0:

for Trial from 1 to n do # create new line of triangle

LastResult:=copy(Result): # only the last lines is required, so nothing

# more is stored / copy to help variable

Confidence:=0:

Result[1]:=1: # set first element to 1

k:=1: # now the rest

Confidence:=Confidence+binomial(Trial,0)*1/2^Trial:

while Confidence < eval(reqConfidence) and k <= Trial do

# check if target confidence is reached or all trials has been attempted

Result[k+1]:=LastResult[k]+LastResult[k+1]:

# calculate new element of Pascal triangle

Confidence:=Confidence+binomial(Trial,k)*1/2^Trial:

# increase Confidence (->more trials were correct)

k:=k+1:

end do:

if k<=Trial then # winning condition

Prob:=evalf(eval(Prob)+(LastResult[k]+LastResult[k+1])/2^Trial):

# add to sum of all winning probabilities

end if:

end do:

end proc:

# now follows the execution

# the result is stored in variable 'prob'

CorrPVal(16,0.95,prob):

prob;

# the result is displayed