Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat Listening Test @ 128 kbps - FINISHED (Read 165651 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #125
Of course you are correct, guruboolez. My in-depth opinion is quite similar. Perhaps I simplified my answer a bit too much.

Actually, in the pretest discussion I tried to find some "low bitrate" human voice samples that would have been difficult for the encoders too, but with my limited experience in this I wasn't very successful. I posted a couple of jazz and opera voice samples (one of them was mono), but no one was impressed about them.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #126
Quote
If my memory serves me, Ivan promised to publish results of PEAQ analyses of encoded sound samples from the listening test. May we hope to see them?
[a href="index.php?act=findpost&pid=357501"][{POST_SNAPBACK}][/a]


Yes,

Here they are (XLS file) - files were processed with the reference commercial implementation of the Advanced PEAQ analysis (ITU BS.1387 / Advanced PEAQ) with automatic delay and gain compensation (pretty much default options)

So - Advanced PEAQ rated encoders like this:

iTunes: 4.46
Nero: 4.49
LAME: 4.33
Vorbis: 4.45
WMA: 4.44
Shine: 2.48

Here are raw results, starting from the last sample, down to the first:

Code: [Select]
    iTunes    Nero    LAME    Vorbis    WMA    Shine
Sample 18    -0.291379    -0.22438    -0.551136    -0.410874    -0.464171    -2.89086
Sample 17    -0.485661    -0.625404    -0.558785    -0.477524    -0.517271    -3.32223
Sample 16    -0.562372    -0.474273    -0.82978    -0.430512    -0.62892    -1.97767
Sample 15    -0.536917    -0.397598    -0.684565    -0.604991    -0.757444    -3.09543
Sample 14    -0.55296    -0.601416    -0.639036    -0.59632    -0.557496    -2.68244
Sample 13    -0.523003    -0.533149    -0.504188    -0.646469    -0.395212    -2.82543
Sample 12    -0.584267    -0.626797    -0.710563    -0.615299    -0.559485    -1.73998
Sample 11    -0.539979    -0.455684    -0.604449    -0.470218    -0.518138    -1.71468
Sample 10    -0.414742    -0.469117    -0.701445    -0.31328    -0.710211    -2.08605
Sample 09    -0.698941    -0.720779    -0.729232    -0.743117    -0.515512    -3.49285
Sample 08    -0.595305    -0.66342    -0.615311    -0.57183    -0.566153    -3.1158
Sample 07    -0.430192    -0.422099    -0.765232    -0.668022    -0.545455    -2.10136
Sample 06    -0.763366    -0.405315    -0.626802    -0.467821    -0.594331    -1.00229
Sample 05    -0.648616    -0.682204    -0.512671    -0.625941    -0.458714    -3.0605
Sample 04    -0.573599    -0.545224    -0.625719    -0.605796    -0.493474    -3.22439
Sample 03    -0.454201    -0.418986    -1.21393    -0.531274    -0.833721    -1.35914
Sample 02    -0.497593    -0.431844    -0.524538    -0.578111    -0.453721    -2.72985
Sample 01    -0.583718    -0.501613    -0.654598    -0.565685    -0.567475    -2.96624
ANOVA    -0.54    -0.51    -0.67    -0.55    -0.56    -2.52
Worst Item    -0.763366    -0.720779    -1.21393    -0.743117    -0.833721    -3.49285
Best Item    -0.291379    -0.22438    -0.504188    -0.31328    -0.395212    -1.00229

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #127
Quote
Quote
Sorry, no.
You can do it yourself and state that it's your plot - I have no problem with that. I am not responsible for the way people post that image and what they do and don't do. I doubt that image is going to appear on a forum out of the sudden and without any comments - people who post the plot can also create a link below or above. The encoder version is more than enough in the graphics.
Discussion about this is over.
[{POST_SNAPBACK}][/a]

ok then, can you post the xls/openoffice/whatever you used?
[a href="index.php?act=findpost&pid=357440"][{POST_SNAPBACK}][/a]


[a href="http://maresweb.de/listening-tests/mf-128-1/pandts.xls]http://maresweb.de/listening-tests/mf-128-1/pandts.xls[/url]

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #128
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #129
Quote
Very big congratulations to Sebastian for managing to produce interesting results after all the hardships he had to go through to discuss his test and later conduct it.

And here's to your upcoming tests! May the next test (64kbps?) be as successfull (but less stressful!) than this one. I know for sure I can hardly wait for it...
[a href="index.php?act=findpost&pid=357444"][{POST_SNAPBACK}][/a]


Ivan is doing some pre-tests to decide which Nero encoder to use (AFAIK). I would love to conduct the next low-bitrate test (maybe with Shade[ST] since he's interested in running a test, too).

Quote
How I can decrypt the results with the key.key file?
[a href="index.php?act=findpost&pid=357456"][{POST_SNAPBACK}][/a]


ABC/HR has an option like Tools --> Process results or something (sorry, the school PC I am currently using does not have Java installed).

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #130
Quote
So - Advanced PEAQ rated encoders like this:

As you have access to both standard and advanced PEAQ, does the advanced version really provide more real life correlation?

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #131
I think the PEAQ analysis is not reliable since it is based on psy-models like most lossy encoders. If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results. Of course, PEAQ uses a very highly tuned model, but still, it is not as reliable as real-world testing with human beings.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #132
Quote
If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results.

Of course, and it seems clear that Ivan is using advanced PEAQ often, based on the PEAQ results of Nero.
However, even with the same psymodel, in a bitrate limited scenario, there is the bit allocation part that you will impact results.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #133
Quote
Gabriel
As you have access to both standard and advanced PEAQ, does the advanced version really provide more real life correlation?


I would have to re-do the test with the Basic PEAQ, but today I have absolutely no time for that - as Basic PEAQ is publically available (AFsp package / PQEvalAudio)  someone could also do it - if not, I'll try over the week.

As for the Advanced PEAQ and correlation - I cannot comment this, as I have not performed too much correlation tests, but judging from the ITU BS.1387 papers on AES conferences, they indeed achieved higher correlation with the real-world listening test data by usign Advanced PEAQ, than with the Basic (FFT) model.  Correlation was one of the factors for forming the "basic" and "advanced" PEAQ out of the so-called "toolbox" (set of algorithmic tools).

Quote
Sebastian MaresI think the PEAQ analysis is not reliable since it is based on psy-models like most lossy encoders. If an encoder has the same (or at least very similar) psy-model as the tool used for testing, the PEAQ analysis is going to rate the respective codec as excellent, even though the psy-model of the tested encoder might not achieve good results. Of course, PEAQ uses a very highly tuned model, but still, it is not as reliable as real-world testing with human beings.


Hmm - I think the purpose of PEAQ is not to replace the subjective listening tests - but to assist developers / engineers in situations where subjective tests would be tpp slow (even impossible) and/or too expensive - for example, where a lot of tests need to be made in the short time for many tools,  doing subjective tests for each setting/tool/combination would be impossible.

Therefore PEAQ can only be "more or less" correlated to the real-world data - by no means they could replace real human being and its judgment, especially a lot of high trained humans in a properly done test

As for the psymodel - as Gabriel said, there is much more in the codec that generates noise than just the psymodel's estimation - bit allocation over time and frequency, quantization properties, stereo coding, lossless (noiseless) coding step, etc...  So, tweaking the psymodel just to match PEAQ values probably won't lead to the best ODG result.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #134
Quote
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
[a href="index.php?act=findpost&pid=357534"][{POST_SNAPBACK}][/a]


Hmm no idea at all  But I also have few samples exibiting similar behavior, but for AAC.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #135
Just quickly going back to AAC players - besides the ones already mentioned there is http://www.panasonic-europe.com/news_read.aspx?id=2091 this panasonic one (and an older model), the Sony Walkman W800 (and K750), and the PSP. If Sony are allowing AAC playback on their phones and PSP i wouldn't be suprised to see support on other stuff like their network walkmans or Hi-MD later on. I think the Hi-MD devices already playback mp3 natively (not that you'd ever want to buy one).


Multiformat Listening Test @ 128 kbps - FINISHED

Reply #137
Quote
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
[a href=\"index.php?act=findpost&pid=357534\"][{POST_SNAPBACK}][/a]
At first, I thought that sample 3 was similarly odd too:

Quote
iTunes Nero LAME Vorbis WMA Shine

Sample 03 -0.454201 -0.418986 -1.21393 -0.531274 -0.833721 -1.35914
but then I checked my test report and found out that I too ranked LAME and Shine similarly poorer than the others (which I couldn't differentiate):

Code: [Select]
ABC/HR for Java, Version 0.5b, 13 January 2006
Testname: Carbonelli

Tester:

1L = Sample03\Carbonelli_3.wav
2L = Sample03\Carbonelli_1.wav
3R = Sample03\Carbonelli_2.wav
4L = Sample03\Carbonelli_6.wav
5L = Sample03\Carbonelli_4.wav
6L = Sample03\Carbonelli_5.wav

---------------------------------------
General Comments:
---------------------------------------
3R File: Sample03\Carbonelli_2.wav
3R Rating: 3.8
3R Comment: Artifact in the very first long note
---------------------------------------
5L File: Sample03\Carbonelli_4.wav
5L Rating: 3.8
5L Comment: Sounds a little "unstable". Probably the low anchor.

Carbonelli is the only sample where low anchor is not obvious.
---------------------------------------

ABX Results:
Original vs Sample03\Carbonelli_4.wav
    11 out of 12, pval = 0.0030
Original vs Sample03\Carbonelli_2.wav
    17 out of 17, pval < 0.001


---- Detailed ABX results ----
Original vs Sample03\Carbonelli_4.wav
Playback Range: 14.227 to 15.868
    3:12:42 AM f 0/1 pval = 1.0
    3:12:45 AM p 1/2 pval = 0.75
    3:12:48 AM p 2/3 pval = 0.5
    3:12:51 AM p 3/4 pval = 0.312
    3:12:54 AM p 4/5 pval = 0.187
    3:12:57 AM p 5/6 pval = 0.109
    3:12:59 AM p 6/7 pval = 0.062
    3:13:02 AM p 7/8 pval = 0.035
    3:13:05 AM p 8/9 pval = 0.019
    3:13:08 AM p 9/10 pval = 0.01
    3:13:11 AM p 10/11 pval = 0.0050
    3:13:15 AM p 11/12 pval = 0.0030

Original vs Sample03\Carbonelli_2.wav
Playback Range: 00.000 to 01.329
    3:20:40 AM p 1/1 pval = 0.5
    3:20:48 AM p 2/2 pval = 0.25
    3:20:51 AM p 3/3 pval = 0.125
    3:20:54 AM p 4/4 pval = 0.062
    3:21:05 AM p 5/5 pval = 0.031
    3:21:32 AM p 6/6 pval = 0.015
    3:21:35 AM p 7/7 pval = 0.0070
    3:21:37 AM p 8/8 pval = 0.0030
    3:21:42 AM p 9/9 pval = 0.0010
    3:21:46 AM p 10/10 pval < 0.001
    3:21:54 AM p 11/11 pval < 0.001
    3:21:57 AM p 12/12 pval < 0.001
    3:22:01 AM p 13/13 pval < 0.001
    3:22:03 AM p 14/14 pval < 0.001
    3:22:06 AM p 15/15 pval < 0.001
    3:22:09 AM p 16/16 pval < 0.001
    3:22:12 AM p 17/17 pval < 0.001

As a comparison, guruboolez's results were these:

Code: [Select]
ABC/HR for Java, Version 0.5b, 07 décembre 2005
Testname: Carbonelli

Tester: guruboolez

1L = Sample03\Carbonelli_5.wav
2L = Sample03\Carbonelli_1.wav
3L = Sample03\Carbonelli_2.wav
4L = Sample03\Carbonelli_6.wav
5R = Sample03\Carbonelli_4.wav
6R = Sample03\Carbonelli_3.wav

---------------------------------------
General Comments:
---------------------------------------
1L File: Sample03\Carbonelli_5.wav
1L Rating: 4.8
1L Comment: very very small ringing (I'm surprised myself by the ABX score I get)
---------------------------------------
2L File: Sample03\Carbonelli_1.wav
2L Rating: 3.8
2L Comment: little ringing
---------------------------------------
3L File: Sample03\Carbonelli_2.wav
3L Rating: 2.0
3L Comment: tremolo effect; very minor kind of warbling also
---------------------------------------
4L File: Sample03\Carbonelli_6.wav
4L Rating: 4.3
4L Comment: minor distortion (unsure -> need ABXing)
---------------------------------------
5R File: Sample03\Carbonelli_4.wav
5R Rating: 1.3
5R Comment: severe ringing
---------------------------------------
6R File: Sample03\Carbonelli_3.wav
6R Rating: 4.3
6R Comment: same kind of problem than 4L
---------------------------------------

ABX Results:
Original vs Sample03\Carbonelli_6.wav
    8 out of 8, pval = 0.0030
Original vs Sample03\Carbonelli_5.wav
    7 out of 8, pval = 0.035
Original vs Sample03\Carbonelli_3.wav
    7 out of 8, pval = 0.035


---- Detailed ABX results ----
Original vs Sample03\Carbonelli_6.wav
Playback Range: 00.000 to 02.695
    8:15:57 PM p 1/1 pval = 0.5
    8:16:01 PM p 2/2 pval = 0.25
    8:16:07 PM p 3/3 pval = 0.125
    8:16:09 PM p 4/4 pval = 0.062
    8:16:12 PM p 5/5 pval = 0.031
    8:16:16 PM p 6/6 pval = 0.015
    8:16:21 PM p 7/7 pval = 0.0070
    8:16:25 PM p 8/8 pval = 0.0030

Original vs Sample03\Carbonelli_5.wav
Playback Range: 00.000 to 02.695
    8:18:32 PM p 1/1 pval = 0.5
    8:18:36 PM p 2/2 pval = 0.25
    8:18:41 PM p 3/3 pval = 0.125
    8:18:51 PM p 4/4 pval = 0.062
    8:18:55 PM p 5/5 pval = 0.031
    8:18:59 PM p 6/6 pval = 0.015
    8:19:03 PM p 7/7 pval = 0.0070
    8:19:16 PM f 7/8 pval = 0.035

Original vs Sample03\Carbonelli_3.wav
Playback Range: 00.000 to 02.695
    8:17:05 PM p 1/1 pval = 0.5
    8:17:09 PM p 2/2 pval = 0.25
    8:17:26 PM p 3/3 pval = 0.125
    8:17:31 PM p 4/4 pval = 0.062
    8:17:34 PM p 5/5 pval = 0.031
    8:17:38 PM f 5/6 pval = 0.109
    8:17:45 PM p 6/7 pval = 0.062
    8:17:49 PM p 7/8 pval = 0.035

Here's a summary:

PEAQ _ Alex B _ guruboolez

iTunes: -0.45 (~4.5) _ 5.0 _ 3.8
Lame: -1.2 (~3.8) _ 3.8 _ 2.0
Nero: -0.42 (~4.6) _ 5.0 _ 4.3
Shine: -1.36 (~3.6) _ 3.8 _ 1.3
Vorbis: -0.53 (~4.5) _ 5.0 _ 4.8
WMA: -0.83 (~4.2) _ 5.0 _ 4.3

All three testers found LAME and Shine clearly worse than the others with this sample. Advanced PEAQ found WMA Pro slightly worse than the AAC codecs or Vorbis. Guru found that iTunes had a bit more problems than Nero or WMA and ranked Vorbis to be the best.

EDIT

The overall results for this sample:



Artist: Giovanni Stefano Carbonelli
Title: Sonata Settima In La Minore
Genre: Baroque Chamber Music
Sumbitted by: guruboolez

Nero score: 4.90

iTunes, AoTuV and WMA Professional are tied on first place. LAME and Shine are tied on last place.

EDIT 2: added PEAQ resuts in scale 0.0 - 5.0

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #138
Quote
Ivan> PEAQ results for LAME with sample 16 "SongForGuy" is unusually low. The listening test result for this sample doesn't show anything unusual:
http://www.maresweb.de/listening-tests/mf-128-1/image019.png

Do you have an idea about this big difference?
[a href="index.php?act=findpost&pid=357534"][{POST_SNAPBACK}][/a]
Several individuals have mistaken LAME for the low anchor on this sample. Could be by coincidence or... LAME has an issue with SongForGuy.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #139
Perhaps this question has been asked many times, but I'd like to know what it is what makes this aoTuVb4.51 version so extremely good compared to the libvorbis 1.12 version? I even hear "the resurrection of Vorbis" is due to aoTuV!
If the aoTuVb4.51 has a score of 4,79 in this test, what average score would the libvorbis 1.12 have compared to the aoTuV and all the others!??

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #140
One note about Advanced PEAQ, and the ratings - its neural network was trained by the large listening test database up to 1997.

In addition,  in its tool set, PEAQ itself does not take into account stereo effects too much, and it might be the reason of the difference in real rating between, say, Vorbis - which is using lossy stereo above (if I am not mistaken) 10 kHZ and PEAQ estimation.

It could be that PEAQ's model treats distortion introduced by the lossy stereo as much more relevant as it is to the most of the human listeners due to a HF stereo masking not taken into account.

What would be interesting, if someone starts the project of extending the PEAQ by more elaborate binaural hearing models, as well as training on the more recent listening tests.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #141
Quote
Quote
Let me rephrase: How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
[a href="index.php?act=findpost&pid=357297"][{POST_SNAPBACK}][/a]


Geez, no idea. That would take too much time - time that I don't have right now. I could send you all results if you really want to do it yourself.
[a href="index.php?act=findpost&pid=357306"][{POST_SNAPBACK}][/a]

@Sebastian: I like to have a go at it.  Just tell me how to obtain the files.

[span style='font-size:8pt;line-height:100%']Edit: Speliing[/span]

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #142
Quote
Quote
Quote
Let me rephrase: How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
[a href="index.php?act=findpost&pid=357297"][{POST_SNAPBACK}][/a]


Geez, no idea. That would take too much time - time that I don't have right now. I could send you all results if you really want to do it yourself.
[a href="index.php?act=findpost&pid=357306"][{POST_SNAPBACK}][/a]

@Sebastian: I like to have a go at it.  Just tell me how to obtain the files.

[span style='font-size:8pt;line-height:100%']Edit: Speliing[/span]
[a href="index.php?act=findpost&pid=357613"][{POST_SNAPBACK}][/a]


Damn, while re-sorting all files for you now, I noticed a mistake caused by an improper naming of a result. Sample 8 (eric_clapton) has a result called "anon16-result08" which is in fact for Elizabeth. "anon16-result07" is identical to "anon16-result08". I am going to re-screen the results for sample 8 and then re-do the final plots. Sorry folks! I am sure all other results are in order.

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #143
The plot for sample 8 was updated - only minor changes, nothing that would change the overall results.

Still working on the final plots now. The only difference is the changed Tukey's HSD value (change of 0.001) and the lower ranking of Shine only.

Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #144
Quote
Vorbis did not finish 'at the front'.  The difference between AoTuV and iTunes AAC is within the margin of error and is not significant.
[a href="index.php?act=findpost&pid=357498"][{POST_SNAPBACK}][/a]


I didn't say it was alone at the front, did I?

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #145
Quote
Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

updated xls?

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #146
Quote
ABC/HR has an option like Tools --> Process results or something (sorry, the school PC I am currently using does not have Java installed).

Thanks, it worked. It shoud be explaned and linked in somewere, because with search is dificult to find that.

The decoded sample12 result that I don't send:
Code: [Select]
ABC/HR for Java, Version 0.5b, 13 Janeiro 2006
Testname: MysteriousTimes

Tester: Caroliano

1R = Sample12\MysteriousTimes_5.wav
2R = Sample12\MysteriousTimes_6.wav
3L = Sample12\MysteriousTimes_2.wav
4R = Sample12\MysteriousTimes_3.wav
5L = Sample12\MysteriousTimes_4.wav
6R = Sample12\MysteriousTimes_1.wav

---------------------------------------
General Comments:
---------------------------------------
1R File: Sample12\MysteriousTimes_5.wav
1R Rating: 4.0
1R Comment:
---------------------------------------
2R File: Sample12\MysteriousTimes_6.wav
2R Rating: 4.2
2R Comment:
---------------------------------------
3L File: Sample12\MysteriousTimes_2.wav
3L Rating: 4.2
3L Comment:
---------------------------------------
5L File: Sample12\MysteriousTimes_4.wav
5L Rating: 2.5
5L Comment: low-anchor
---------------------------------------

ABX Results:

Again I rated vorbis as the worst sounding. I must be sensible for Vorbis artfacts...

PS: I was anon26, but is ok to display my name. I only forgot to put my name in them...

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #147
Quote
Quote
Edit: All problems solved now. As stated, only the non-zoomed plot was affected (and there, only Shine is affected - it lost 0.01 points).

updated xls?
[a href="index.php?act=findpost&pid=357706"][{POST_SNAPBACK}][/a]


Everything - plots, the results page, the results RAR...

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #148
Quote
How many of the grades given were 5.0? And how much if you re-add the ranked references as meaning that codec got a 5.0 for that sample?

So, there's 403 valid test results times 5 codecs (Shine doesn't count), or about 2015 grades. How many of those are 5.0, i.e. perfectly transparent?
[a href="index.php?act=findpost&pid=357297"][{POST_SNAPBACK}][/a]

@Sebastian: Thanks for the test results. 

I managed to proces the results and produce this table:
Code: [Select]
Ranked refs      24     14     18      6     19     25
5.0's           304    260    299     36    313    302
5.0's %          75%    65%    74%     9%    78%    75%
4.0 and above   361    334    358     60    375    355
4.0 and above %  90%    83%    89%    15%    93%    88%

Multiformat Listening Test @ 128 kbps - FINISHED

Reply #149
Quote from: Sebastian Mares,Jan 17 2006, 07:00 AM

updated xls?


Everything - plots, the results page, the results RAR... [/quote]
i mean, do you have an updated xls to download?