Ivan Dimkovic wrote:Hi AllOk, WavRate is almost finished, only thing that needs to be done is multiple test analysis (but project file is already compatible with that).What's new:- Better look&feel Small glitches corrected, like playback info- Automatic randomization of the items prior to testing- Slider selection of the range of playback- Automatic encryption of the output files (security purposes)You can download wavrate from:http://www.psytel-research.co.yu/downloads/wavrate.exeOk, so first test iteration will be castanets test:http://www.audiocoding.com/listening_tests...tanets_test.zipCodec info for the Test #1 could be found here:http://www.psytel-research.co.yu/downlods/...s_codecinfo.zipThis zip file is enrypted - I will supply everyone with the password when listening test #1 is finished.Please unpack those LPAC files into the same directory and open those .wrf files from the wavrate. You can rank each codec with mark ranging from 1 (bad) to 5 (excellent) in 0.1 increments.Please perform those tests carefully, and comment your results for each item as much as you can - tell me what do you think what's wrong with particular test item and why you think it deserves particular mark.Please send saved results (File/Save...) to: email@example.com - subject "Castanets Test"Next test will incorporate larger sample (30 seconds)
Hmm... deviations between listeners are quite big. For example, some of them are ranking codecs in range of 1.0 to 3.5 and others are ranking them in range of 3.0 - 5.0
I have collected 7 results up to date. Maybe we will have more accurate results after 30 results are submitted.ff123 - what do you suggest for the statistical analysis?
FRIEDMAN version 1.20 (Oct 8, 2001) [url]http://ff123.net/[/url]Friedman AnalysisNumber of listeners: 9Critical significance: 0.05Significance of data: 7.17E-05 (highly significant)Fisher's protected LSD for rank sums: 20.369 PsyAAC12 ComAAC_A PsyFast2 MP3Enc LamePsy Lame390 FastEnc FhGAAC 19.50 25.00* 31.00* 34.50* 40.50* 41.50* 48.00* PsyAAC12 5.50 11.50 15.00 21.00* 22.00* 28.50* ComAAC_A 6.00 9.50 15.50 16.50 23.00* PsyFast2 3.50 9.50 10.50 17.00 MP3Enc 6.00 7.00 13.50 LamePsy 1.00 7.50 Lame390 6.50 FhGAAC PsyAAC12 ComAAC_A PsyFast2 MP3Enc LamePsy Lame390 FastEnc 70.50 51.00 45.50 39.50 36.00 30.00 29.00 22.50 FhGAAC is better than ComAAC_A, PsyFast2, MP3Enc, LamePsy, Lame390, FastEncPsyAAC12 is better than LamePsy, Lame390, FastEncComAAC_A is better than FastEnc
Critical significance of Fisher's LSD analysis: 5.000E-02 Fisher's LSD for rank sums: 2.037E+01
Originally posted by ff123 Hmmm. How do I do pre-formatted editing on this forum? doesn't seem to work.