Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Plotting listening tests graphs (Read 7668 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Plotting listening tests graphs

Here is the question:
How to plot the usual listening tests graphs, without buying software?

I know it can be done with Excel, but I would not buy Excel just for that. Anyone know how to do it under OpenOffice, GnuPlot or anything else?

Plotting listening tests graphs

Reply #3 –
Quote
Roberto mentioned in his guide that OpenOffice.org Calc 1.x has "Stock Chart 1" as a plot type

Ok, but I first need to have the data without the outliers. How to have such data?
Chunky is supposed to produce it, but it is complaining that "Sample directory names must end in a number" even if my folders are named "samplexx".

Did you used chunky yourself in your public test?

Plotting listening tests graphs

Reply #4 –
Did you try sample.xx ?
 

Plotting listening tests graphs

Reply #5 –
Quote
Quote
Roberto mentioned in his guide that OpenOffice.org Calc 1.x has "Stock Chart 1" as a plot type

Ok, but I first need to have the data without the outliers. How to have such data?
Chunky is supposed to produce it, but it is complaining that "Sample directory names must end in a number" even if my folders are named "samplexx".

Did you used chunky yourself in your public test?
[a href="index.php?act=findpost&pid=369366"][{POST_SNAPBACK}][/a]


Yes. Create a new folder called "Results" and inside that folder, create your 18 folders containing the results for the samples. Only those 18 folders with the corresponding results should be inside "Results". Then use cmd and navigate to the "Results" folder and call chunky from there (your best bet would be placing chunky into the Windows directory or something that is in your PATH).
I also had the same problem as you because I stored the chunky executable in the same directory where my folders for the results were located.

Plotting listening tests graphs

Reply #6 –
Quote
I also had the same problem as you because I stored the chunky executable in the same directory where my folders for the results were located.

Thanks, I didn't know that chunky can not be in the same folder as the results subfolders.

Now, it is going further, but still complaining:
Code: [Select]
E:\tmp\results\test>chunky.exe -c "1,test" -n --rating=results --warn -p 0.05
Parsing result files...
Traceback (most recent call last):
  File "chunky", line 639, in ?
  File "chunky", line 595, in main
  File "abchr_parser.pyc", line 634, in __init__
  File "abchr_parser.pyc", line 646, in _handleTargets
  File "abchr_parser.pyc", line 698, in __init__
  File "abchr_parser.pyc", line 634, in __init__
  File "abchr_parser.pyc", line 646, in _handleTargets
  File "abchr_parser.pyc", line 290, in __init__
  File "abchr_parser.pyc", line 503, in _parseFile
KeyError: 1

Plotting listening tests graphs

Reply #7 –
No idea - maybe something is wrong with that -c parameter. Did you really name the file "1,test" (with a comma instead of a dot)? Also, try specifying an absolute path.

Sorry, now I realize that it's not a file, but you are manually specifying the number and the corresponding encoder name.

How exactly are the results sorted?

The command line arguments I used weren't much different than yours, so I guess it has something to do with how you saved the files.
This is what I was using:

Code: [Select]
@echo off
chunky --codec-file="D:\Encoders.txt" -n -p 0.05 --ratings=results --warn
for %%f in (*.txt) do friedman -tp %%f > analysis-%%f

It also performs friedman for each result delivered by chunky.

Another edit... I just noticed that you use --rating; shouldn't that be --ratings?

Plotting listening tests graphs

Reply #8 –
Quote
Another edit... I just noticed that you use --rating; shouldn't that be --ratings?

Right. This is probably because of my many tries.

Perhaps there is something wrong with my results files? I uploaded some hypothetical result files (notation altered):
http://www.hydrogenaudio.org/forums/index....showtopic=42159
Can they be processed by chunky?

Plotting listening tests graphs

Reply #9 –
Now I see where the problem is. You shouldn't name the encoders A, B, C..., but 1, 2, 3...

If you have these two result sets:

Code: [Select]
ABC/HR for Java, Version 0.5b, 24 February 2006
Testname: aquatisme - sample1

Tester:

1L = sample1\sample1_2.wav
2L = sample1\sample1_6.wav
3L = sample1\sample1_4.wav
4R = sample1\sample1_7.wav
5R = sample1\sample1_1.wav
6R = sample1\sample1_3.wav
7L = sample1\sample1_5.wav

---------------------------------------
General Comments:
---------------------------------------
1L File: sample1\sample1_2.wav
1L Rating: 4.5
1L Comment:
---------------------------------------
3L File: sample1\sample1_4.wav
3L Rating: 4.1
3L Comment:
---------------------------------------
4R File: sample1\sample1_7.wav
4R Rating: 2.8
4R Comment:
---------------------------------------
5R File: sample1\sample1_1.wav
5R Rating: 2.5
5R Comment:
---------------------------------------
6R File: sample1\sample1_3.wav
6R Rating: 3.3
6R Comment:
---------------------------------------
7L File: sample1\sample1_5.wav
7L Rating: 4.5
7L Comment:
---------------------------------------

ABX Results:

Code: [Select]
ABC/HR for Java, Version 0.5b, 21 February 2006
Testname: aquatisme - sample1

Tester:

1R = sample1\sample1_7.wav
2R = sample1\sample1_2.wav
3L = sample1\sample1_3.wav
4R = sample1\sample1_4.wav
5L = sample1\sample1_6.wav
6L = sample1\sample1_5.wav
7L = sample1\sample1_1.wav

---------------------------------------
General Comments:
---------------------------------------
1R File: sample1\sample1_7.wav
1R Rating: 4.4
1R Comment:
---------------------------------------
2R File: sample1\sample1_2.wav
2R Rating: 4.0
2R Comment:
---------------------------------------
3L File: sample1\sample1_3.wav
3L Rating: 1.5
3L Comment:
---------------------------------------
4R File: sample1\sample1_4.wav
4R Rating: 3.5
4R Comment:
---------------------------------------
6L File: sample1\sample1_5.wav
6L Rating: 4.1
6L Comment:
---------------------------------------
7L File: sample1\sample1_1.wav
7L Rating: 2.1
7L Comment:
---------------------------------------

ABX Results:

Chunky produces this:

Code: [Select]
% Result file produced by chunky-0.8.4-beta
% chunky --codec-file=D:\Encoders.txt -n -p 0.05 --ratings=results --warn
%
% aquatisme - sample1:

First	Second	Third	Fourth	Fifth	Sixth	Seventh
2.50	4.50	3.30	4.10	4.50	5.00	2.80
2.10	4.00	1.50	3.50	4.10	5.00	4.40

% Codec averages:
% 2.30	4.25	2.40	3.80	4.30	5.00	3.60

And after running friedman:

Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) [url=http://ff123.net/]http://ff123.net/[/url]
Tukey HSD analysis

Number of listeners: 2
Critical significance:  0.05
Tukey's HSD:   2.974

Means:

Sixth    Fifth    Second   Fourth   Seventh  Third    First    
  5.00     4.30     4.25     3.80     3.60     2.40     2.30   

-------------------------- Difference Matrix --------------------------

         Fifth    Second   Fourth   Seventh  Third    First    
Sixth      0.700    0.750    1.200    1.400    2.600    2.700  
Fifth               0.050    0.500    0.700    1.900    2.000  
Second                       0.450    0.650    1.850    1.950  
Fourth                                0.200    1.400    1.500  
Seventh                                        1.200    1.300  
Third                                                   0.100  
-----------------------------------------------------------------------

I named the encoders simply First, Second, Third...

Plotting listening tests graphs

Reply #10 –
Quote
Now I see where the problem is. You shouldn't name the encoders A, B, C..., but 1, 2, 3...

Finally solved...
Thank you very much for the help.

Plotting listening tests graphs

Reply #11 –
Another question: in the usual plots, the bars represent the 95% confidence interval. I have the mid values, but how to properly obtain the low and high values of the 95% confidence?

Plotting listening tests graphs

Reply #12 –
Quote
Another question: in the usual plots, the bars represent the 95% confidence interval. I have the mid values, but how to properly obtain the low and high values of the 95% confidence?
[a href="index.php?act=findpost&pid=369770"][{POST_SNAPBACK}][/a]


Divide by two.

In your example, the HSD value is 2.974. When creating the confidence bars, use 1.487 for up and 1.487 for down.

Plotting listening tests graphs

Reply #13 –
Ok, now I have all the info to "easily" plot those graphs.

However, this is mainly a multi-steps manual process. I thought that Excel was able to produce such graphs directly from listening tests results, performing the anova itself and knowing how to plot the graphs. Is is able to do it itself, or is it also a manual process (like in OO.org case)?