Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 64 kbit/s test ended (Read 16067 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

64 kbit/s test ended

Reply #50 –
Regarding plotting standard deviations:
Quote
Originally posted by ff123

It's probably better to use the standard error of the mean (divide the standard deviation by the square root of N) instead of the standard deviation anyway.  This gets you a plot which almost looks like the 95% confidence plots I drew.


When comparing the means of different groups there is no question that the standard error is better to use than the standard deviation. And the Tukey's test is even better again.

The standard deviation does better show (for me at least) the variation in the data. At least you can look at the plot and say "approximately 68% of the data points lie within the error bars". Actually, this would only be true if the sample population were symmetrical and bell-shaped (i.e., normal). I suspect this is not the case because of an inordinate number of outliers (namely scores of 5). This makes the plot of standard deviations even less useful!

Presumably the Tukey's test also depends on a normally distributed population (I'm not sure about this). Are you aware if the extent of non-normality present had an adverse effect on the analysis?

On the other hand, I think that the Friedman non-parametric analysis that you did should be independent of the distribution.

64 kbit/s test ended

Reply #51 –
Quote
Originally posted by shday
Presumably the Tukey's test also depends on a normally distributed population (I'm not sure about this). Are you aware if the extent of non-normality present had an adverse effect on the analysis? 

On the other hand, I think that the Friedman non-parametric analysis that you did should be independent of the distribution.


The Tukey's does assume normal distribution.  However, when I compared the results against the results from the bootstrap program I also use, there was little difference (bootstrap method does not assume normal distribution).  There was a slight loss of power using Tukey's, but not enough to concern me.  And the advantage of the Tukey's is that it pops out a nice 95% confidence interval to use for pretty graphs.

I only used the Friedman/non-parametric Fisher LSD for the overall ranking.  This is because I thought it unfair to compare ratings from one sample with ratings from another sample.

ff123

64 kbit/s test ended

Reply #52 –
Holy shit Jon, you're everywhere on that /. article!
The sky is blue.

64 kbit/s test ended

Reply #53 –
Yeah, I was busy modding up every single one of his comments
And modding down some of the idiot comments.

64 kbit/s test ended

Reply #54 –
Quote
Holy shit Jon, you're everywhere on that /. article!

I don't know what you mean - only 12 of the comments are mine .
(It made for very good work avoidance)
I gave up after a while though -- with a /. story you seem to have about 30 minutes to have a discussion, and then everyone gets bored and moves onto the next story they know nothing about.

64 kbit/s test ended

Reply #55 –
Quote
Originally posted by ff123


The bitrates if I subtract 3500 bytes from each of the ogg files come out to be 1 to 2 kbit/s less than the published bitrates.  I don't know if wma or the quicktime have a header as well, but I'll make some sort of note in the results page.


Ah, hm, I redid the math and you're right ff123; the difference is fairly small neglecting the headers.  I obviously screwed up the arithmetic.

Apologies for a) sounding like I was finding fault (I thought the test was fair and evenhanded, even if slashdot sent a wave of crackmonkies your way ;-) and b) unneccessarily worrying you