Re: Bayes Factors for ABX tests
Reply #11 – 2017-10-06 16:48:50
EekWit, pedantically we can never arrive at truth using such tests and evaluations. But we can get to odds "beyond reasonable doubt" either way. That's just life ... where we cannot easily prove things such as in axiomatic systems. On the uniform prior: yeah. I was not precise enough when I spoke about "total ignorance". The flat or Beta(1, 1) prior contains the knowledge that a trial can both fail and succeed. I think that's a very reasonable and sensible assumption for an ABX test. It also follows the principle of indifference: from .45 to .55 you get the same probability as from .9 to 1 which is - big surprise - 10%. Beta(0, 0) could be interpreted as: either the trials always fail or they always succeed. This would make more sense e.g. for testing a whether a chemical reaction happens or not. 0% everywhere except for the 100% at both extremes. Beta(0.5, 0.5) could be interpreted as: we don't know that it's possible for trials to both fail and succeed. But that gets you 6% from .45 to .55 and 20% from .9 to 1. This prior could make sense in a situation where you didn't know what kind of proportion between 0 and 1 you're dealing with (could be linear, could be logarithmic ...) and try to minimize the effects of the prior. And there are many other attempts at "objective" or "uninformative" or "diffuse" priors since the Jeffreys prior is not without problems can can even lead to inconsistent results, but that's a complex topic.