Originally posted by Garf
Is it possible to merge all samples together and see if you can make global conclusions?
Here is my attempt at a global conclusion:
I eyeballed each graph and ranked the codecs from best to worst on a 5 point scale, where 5 is best and 1 is worst. Codecs that tied received an average of the rankings that tied.
mp3pro oggq0 ogg64 wma8 aac
2.5 4.5 4.5 2.5 1
4 1.5 4 4 1.5
4 2.5 2.5 5 1
5 3.5 3.5 1 2
4 5 2 2 2
4 5 3 1.5 1.5
3.5 5 3.5 1 2
4.5 1 2.5 2.5 4.5
4.5 4.5 3 1 2
5 3.5 3.5 1 2
3 4.5 4.5 1.5 1.5
5 3.5 3.5 1 2
Then I ran this through the Friedman calculator at:
http://ff123.net/friedman/stats.html
and got the following
FRIEDMAN version 1.24 (Jan 17, 2002) [url]http://ff123.net/[/url]
Friedman Analysis
Number of listeners: 12
Critical significance: 0.05
Significance of data: 3.76E-04 (highly significant)
Fisher's protected LSD for rank sums: 15.182
Ranksums:
mp3pro oggq0 ogg64 wma8 aac
49.00 44.00 40.00 24.00 23.00
---------------------------- p-value Matrix ---------------------------
oggq0 ogg64 wma8 aac
mp3pro 0.519 0.245 0.001* 0.001*
oggq0 0.606 0.010* 0.007*
ogg64 0.039* 0.028*
wma8 0.897
-----------------------------------------------------------------------
mp3pro is better than wma8, aac
oggq0 is better than wma8, aac
ogg64 is better than wma8, aac
So for the samples chosen for this test, mp3pro seems to have won purely on ranking points, while wma8 and aac are the definite losers. A different set of samples would have a different ranking.
ff123