Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Null Hypothesis? (Read 6524 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Null Hypothesis?

Lately, I've been reading a bunch of threads related to DAC's and ABX testing. Trying to get a handle on this. One of the forum admins, (greynol?), posted a bunch of links. I tried to read them all. It was a bit overwhelming. One thing caught my attention. The concept of the "null hypothesis".
So, I did a Wiki search. Now, I'm really confused.

Wiki: The term "null hypothesis" refers to a general statement that there is no relationship between two measured phenomena.

Me: My dog has four legs. My cat has four legs. Therefore, my dog is a cat.

Wiki: Rejecting or disproving the null hypothesis—and thus concluding that there are grounds for believing that there is a relationship between two phenomena is a central task in the modern practice of science, and gives a precise criterion for rejecting a hypothesis.

Me: Here's the part that's messing with my head.

Wiki: The null hypothesis is generally assumed to be true until evidence indicates otherwise.

Me: So, If I proclaim that my dog is a cat, it's assumed true until someone proves me wrong? This makes no logical sense to me. I must be misinterpreting the definition of the "null hypothesis."

Re: Null Hypothesis?

Reply #1 –
Quote
Rejecting or disproving the null hypothesis—and thus concluding that there are grounds for believing that there is a relationship between two phenomena

As i understand that sentence, if the null hypothesis is true, there is no relation. So you would have to prove that there is one, rather than the other way around.
 

Re: Null Hypothesis?

Reply #2 –
I guess that's whats messing with my head. How would one know if the NH is true? Before it's tested?

Re: Null Hypothesis?

Reply #3 –
The idea of a "null hypothesis" or rather "default position" has it's roots in logic and philosophy. It is reasonable to stick with the default position. Claims are rejected until sufficient evidence is available to convince one otherwise. Then the default position is abandoned and the alternative position, for which ideally overwhelming evidence was provided, is adopted. That's the rational way to do it anyway.


A null hypothesis in statistics is what you have posted as definition. In the case of null hypothesis significance testing (NHST) the null is not accepted, because of the way the statistical analysis is carried out:
Imagine the case of throwing a coin randomly 10 times. It lands tails 9 times. The null hypothesis (H0) would be that the coin is fair. An alternative hypothesis (H1) would be that the coin is biased towards tails.
In typical frequentist inference fashion, we'd do a NHST: we calculate the probability of achieving 9/10 tails or a more extreme result (that only leaves 10/10 in our case) assuming that the coin was fair (our null hypothesis):
P(X >= 9 | H0) = 0.01

Note that here we calculate the probability of a random variable (X .. the random outcome of tossing a coin, so either heads or tails) given the assumption that H0 is true.
This is called the p-value and it is compared against a more or less arbitrary significance level. 0.05 is often used. Since our p-value is below that we call the result statistically significant, i.e. the null should be rejected.


A Bayesian analysis of the situation would be a more natural approach imo, as it would allow to accept or reject the "null" hypothesis or any other hypothesis. It's more natural because Bayesian inference also makes use of a prior probability, that is, you wouldn't assume that the coin was fair if it was from a magician's shop. You would give the "null" hypothesis a lower prior probability than e.g. the alternative hypothesis H1 that the coin is not fair.
Then, every time you flip the coin you calculate P(H1|coin flipping result) and the resulting probability is then fed back into the same equation as prior probability for the next coin flip.

Note that here we actually calculate the probability of a hypothesis given the data we have gathered from our experiment.


But I'll stop my rambling here..
"I hear it when I see it."

Re: Null Hypothesis?

Reply #4 –
Quote from: Artie on
Me: My dog has four legs. My cat has four legs. Therefore, my dog is a cat.
That is simply an invalid logical argument. You don't need to go to statistics to show that.

Maybe you should find another example.
"I hear it when I see it."

Re: Null Hypothesis?

Reply #5 –
For more information on why your chosen example is invalid, read up on the fallacy of the undistributed middle.

In short, in order for you to conclude that your dog is a cat based on your previous (valid) assertions that both dogs and cats have four legs, you must first prove that all four-legged animals are cats.

Re: Null Hypothesis?

Reply #6 –
Quote from: Artie on
I must be misinterpreting the definition of the "null hypothesis."

The wiki said "no relationship" and you interpreted that by creating a 100% tight relationship between your dog and your cat.

The correct interpretation is, "They both have four legs, but there's no relationship until I've found evidence to suggest there is one."

Re: Null Hypothesis?

Reply #7 –
Quote from: Artie on
One of the forum admins, (greynol?), posted a bunch of links. I tried to read them all.
I sincerely apologize.  The links do provide some worth on a technical level and you did indeed hone in on perhaps the single most important element in the dialog which seems never to be understood by people "asking" to be convinced.  I have to admit that my choice of links was driven by the desire to create a parallel to the topic in which I dumped them. You are welcome to infer anything you wish into what constitutes a parallel. I have read far too many discussions initiated or hijacked by a religious zealot "seeking" a reason why he should "believe in" (aka accept) science.

Bravo for asking the right question here!

Re: Null Hypothesis?

Reply #8 –
Quote from: dhromed on
The correct interpretation is, "They both have four legs, but there's no relationship until I've found evidence to suggest there is one."
The four legs are the relationship. But since it's an invalid argument the conclusion does not follow. In other words the argument fails before we even get to any evidence.

P1) All elephants are pink.
P2) Nella is an elephant.
C) Therefore, Nella is pink.

That is a valid argument, i.e. if the premises are true then the conclusion has to be true, there is no way around that.
The next step would be to show that this is a sound argument. For that you need to demonstrate all premises to be true. That's where the evidence comes in.

Premise 1 makes/is a claim. The default position ("null hypothesis") would be the position that there is no relation between elephants and pink.
"I hear it when I see it."

Re: Null Hypothesis?

Reply #9 –
Thanks for the replies all. Good info and links. A lot to read and interpret.

Btw, the cats/dog thing was just what popped into my head while trying to compose the question. It's actually a line from an old British sitcom. (Yes, Prime Minister, to be exact.)

Re: Null Hypothesis?

Reply #12 –
And that, is the challenge.  ;)

Re: Null Hypothesis?

Reply #13 –
If dog and cat were the same, just different words for the same thing, then the "undistributed middle" fallacy would not matter (it would still be a fallacy though).

The deep and important question we want to find the answer to, is whether dog and cat are truly the same thing. What we test statistically, however, is whether a set of data can be used to tell the difference.
If we can reliably tell a difference from data, then we can reliably say they are not the same thing.
Until there is evidence for a difference, we would not claim there is any.

OK, assume that we found that a scientist with expensive equipment can tell cats from dogs. Yay! But maybe there is a cheaper way? The statistician (me) wants to make a test. Unfortunately, I can only pay one participant, you. And I cannot pay for bandwidth to show video either, but I do have a telephone. So what I really test would be the following:
H0: you cannot distinguish between "cat" and "dog" by the ear.
H1: you can.
Test: repeatedly play over my telephone barking and meowing. You pick "dog" or "cat" and count the right and wrong answers. You scored 199/200 (in one of the cases you were sneezing and could not hear). I do my calculations, and find that if the null hypothesis were true, you should really, really, really not have guessed so much better than fifty-fifty.

Good! Then I do not need the scientist with expensive equipment, I can just call you.  But notice that H1 would not win with 103/200 even though 103 is more than 100; a flipping a coin would very often score as good as 103/200, and I want to be confident that you are better than the coin.