Thanks guys! Interesting results.
One note though:
Read 5 treatments, 531 samples => 10 comparisons
Means:
Vorbis Nero_HE-AAC Apple_HE-AAC Opus AAC-LC@48k
3.513 3.547 3.817 3.999 1.656
For processing the result .txt files with chunky I organized them to sample folders. I removed the results that were marked "invalid" and results that apparently had a fixed newer version (marked as such). I had a duplicate problem with romor's results (a couple of duplicates in a subfolder), but I decided to keep the newer result files. I got 566 remaining result files. Assuming I did not make lots of mistakes, I wonder what can cause the difference. Did you disqualify more results after creating the rar package or does "531 samples" mean something else than the total number of result files?
Here's how chunky parses the 566 result files I have:
% Result file produced by chunky-0.8.4-beta
% ..\chunky.exe --codec-file=..\codecs.txt -n --ratings=results --warn -p 0.05
%
% Sample Averages:
Vorbis Nero Apple CELT Anchor
2.56 4.28 4.19 2.67 1.87
2.95 4.20 4.03 2.36 1.68
3.42 3.51 3.98 4.73 2.51
4.12 3.84 4.49 4.64 2.18
4.18 3.59 3.87 4.52 1.95
3.35 3.68 3.34 4.00 1.56
3.86 2.98 2.96 3.50 1.85
4.03 3.78 4.09 4.49 2.02
3.60 3.71 3.89 3.94 1.51
4.28 2.78 2.19 4.12 1.44
4.12 3.93 4.17 4.39 1.70
3.25 3.18 3.20 4.14 1.77
3.83 3.63 3.86 4.56 1.41
3.49 3.81 4.01 4.27 1.37
4.15 3.84 4.08 4.76 2.04
3.97 2.74 3.09 4.38 1.74
3.35 3.24 4.15 4.44 1.56
2.68 2.96 3.63 4.10 1.51
3.58 4.37 4.88 3.73 1.76
3.40 4.10 4.68 4.26 1.61
3.80 3.49 3.55 4.43 1.38
3.81 3.30 4.27 4.26 1.13
3.59 3.14 3.51 4.09 1.18
3.29 3.61 3.88 4.16 1.36
3.66 3.84 4.37 3.86 1.55
2.78 3.99 4.18 2.82 1.57
3.62 3.88 3.92 3.93 1.34
3.39 4.03 4.39 3.96 1.46
3.61 4.12 4.36 4.09 1.54
4.42 3.48 4.29 4.68 1.82
% Codec averages:
% 3.60 3.63 3.92 4.08 1.65