Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat 48 kbps Listening Test (Read 47048 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Multiformat 48 kbps Listening Test

Reply #100 –
Quote from: guruboolez on
Quote from: rjamorim on
it's their decision wheter it should be tested or not, and it's our duty to respect their decision.

I 100% agree with you. Unfortunately, this desire hasn't been respected... by Microsoft itself:
http://www.hydrogenaudio.org/forums/index....showtopic=40589


That doesn't matter, for me. It's the developer decision wheter it should be tested or not, IMO. That's the least modicum of respect we should show for their works.

Multiformat 48 kbps Listening Test

Reply #101 –
Quote from: rjamorim on
That doesn't matter, for me. It's the developer decision wheter it should be tested or not, IMO. That's the least modicum of respect we should show for their works.

It would then be better to see some coherence. Imagine a friendly developer (Aoyumi, Ivan, Gabriel) publishing on one side a listening test including an alpha release of an encoder and of course claiming its clear superiority but who deny on the other side the right to people to publish themselves the results of their own tests... A bit weird
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

Multiformat 48 kbps Listening Test

Reply #103 –
Companies are bad and incoherent, yes we know this. When they start to use listening test results like this, I'm not even sure the results should be made public.

But I don't think anyone should put himself in potentially legal trouble if he doesn't feel like it. It is a personal decission I can respect and understand.
"We cannot win against obsession. They care, we don't. They win."

Multiformat 48 kbps Listening Test

Reply #104 –
Quote from: Sebastian Mares on
How long does it take until beta 5 is available? Also, the version you are going to send me, will the final product produce the same files as the pre-release? Maybe we can do something like in the AAC test like setting a deadline when the "final" encoder should become available, what do you think?

I cannot promise about a release fixed date of beta5.
But I can show final version about q-1 if you show the deadline.

Multiformat 48 kbps Listening Test

Reply #106 –
Quote from: Sebastian Mares on
Ah, right, I totally forgot about that guru, thanks for pointing out. WMA Standard then. Which settings do you wish folks?
I made a little test with WMA 9.1 standard and ten samples from Gabriel's test. I tried CBR 32kHz 48 kbps and VBR 2-pass 44.1 KHz 48 kbps. Bitrate managed VBR is not available at 32 kHz / 48 kbps. VBR q10 produces too high bitrates.

For encoding I used dbPowerAMP and had 2-pass enabled for both settings, but I doubt it does anything for CBR. The test conditions and samples were identical with my recent Vorbis 48 kbps test.

The results



ABC/HR logs

Code: [Select]
ABC/HR for Java, Version 0.5b, 11 June 2006
Testname: chanchanT WMA

Tester: Alex B

1R = WMA32kHz - 01 - chanchanT.wav
2R = WMA44kHz - 01 - chanchanT.wav

---------------------------------------
General Comments: 
---------------------------------------
1R File: WMA32kHz - 01 - chanchanT.wav
1R Rating: 1.8
1R Comment: 
---------------------------------------
2R File: WMA44kHz - 01 - chanchanT.wav
2R Rating: 1.5
2R Comment: 
---------------------------------------

ABX Results:
WMA32kHz - 01 - chanchanT.wav vs WMA44kHz - 01 - chanchanT.wav
    9 out of 10, pval = 0.01


---- Detailed ABX results ----
WMA32kHz - 01 - chanchanT.wav vs WMA44kHz - 01 - chanchanT.wav
Playback Range: 07.722 to 09.206
    2:20:25 AM p 1/1 pval = 0.5
    2:20:33 AM f 1/2 pval = 0.75
    2:20:45 AM p 2/3 pval = 0.5
    2:20:55 AM p 3/4 pval = 0.312
    2:21:14 AM p 4/5 pval = 0.187
    2:21:27 AM p 5/6 pval = 0.109
    2:21:37 AM p 6/7 pval = 0.062
    2:21:47 AM p 7/8 pval = 0.035
    2:22:07 AM p 8/9 pval = 0.019
    2:22:22 AM p 9/10 pval = 0.01

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 11 June 2006
Testname: fools WMA

Tester: Alex B

1R = WMA44kHz - 02 - fools.wav
2R = WMA32kHz - 02 - fools.wav

---------------------------------------
General Comments: Both are bad. 2 has less stereo separation but it may be more pleasant overall.
---------------------------------------
1R File: WMA44kHz - 02 - fools.wav
1R Rating: 1.8
1R Comment: 
---------------------------------------
2R File: WMA32kHz - 02 - fools.wav
2R Rating: 2.0
2R Comment: 
---------------------------------------

ABX Results:
WMA44kHz - 02 - fools.wav vs WMA32kHz - 02 - fools.wav
    12 out of 12, pval < 0.001


---- Detailed ABX results ----
WMA44kHz - 02 - fools.wav vs WMA32kHz - 02 - fools.wav
Playback Range: 19.641 to 23.172
    11:33:13 PM p 1/1 pval = 0.5
    11:33:29 PM p 2/2 pval = 0.25
    11:33:35 PM p 3/3 pval = 0.125
    11:33:40 PM p 4/4 pval = 0.062
    11:33:58 PM p 5/5 pval = 0.031
    11:34:03 PM p 6/6 pval = 0.015
    11:34:08 PM p 7/7 pval = 0.0070
    11:34:12 PM p 8/8 pval = 0.0030
    11:34:17 PM p 9/9 pval = 0.0010
    11:34:21 PM p 10/10 pval < 0.001
    11:34:26 PM p 11/11 pval < 0.001
    11:34:32 PM p 12/12 pval < 0.001

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: kraftwerk WMA

Tester: Alex B

1L = WMA32kHz - 03 - kraftwerk.wav
2L = WMA44kHz - 03 - kraftwerk.wav

---------------------------------------
General Comments: no comments...
---------------------------------------
1L File: WMA32kHz - 03 - kraftwerk.wav
1L Rating: 1.0
1L Comment: 
---------------------------------------
2L File: WMA44kHz - 03 - kraftwerk.wav
2L Rating: 1.0
2L Comment: 
---------------------------------------

ABX Results:

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: Liszt WMA

Tester: Alex B

1R = WMA32kHz - 04 - Liszt_in_B.wav
2R = WMA44kHz - 04 - Liszt_in_B.wav

---------------------------------------
General Comments: Not bad when compared with some other samples. Couldn't make difference between these two.
---------------------------------------
1R File: WMA32kHz - 04 - Liszt_in_B.wav
1R Rating: 2.8
1R Comment: 
---------------------------------------
2R File: WMA44kHz - 04 - Liszt_in_B.wav
2R Rating: 2.8
2R Comment: 
---------------------------------------

ABX Results:

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: orion_ii WMA

Tester: Alex B

1R = WMA44kHz - 05 - orion_ii.wav
2R = WMA32kHz - 05 - orion_ii.wav

---------------------------------------
General Comments: totally distorted, unusable
---------------------------------------
1R File: WMA44kHz - 05 - orion_ii.wav
1R Rating: 1.0
1R Comment: 
---------------------------------------
2R File: WMA32kHz - 05 - orion_ii.wav
2R Rating: 1.0
2R Comment: 
---------------------------------------

ABX Results:

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: sandman WMA

Tester: Alex B

1L = WMA32kHz - 06 - sandman.wav
2R = WMA44kHz - 06 - sandman.wav

---------------------------------------
General Comments: Terrible, in ABX test 1 was slighly better
---------------------------------------
1L File: WMA32kHz - 06 - sandman.wav
1L Rating: 1.3
1L Comment: 
---------------------------------------
2R File: WMA44kHz - 06 - sandman.wav
2R Rating: 1.1
2R Comment: 
---------------------------------------

ABX Results:
WMA32kHz - 06 - sandman.wav vs WMA44kHz - 06 - sandman.wav
    10 out of 10, pval < 0.001


---- Detailed ABX results ----
WMA32kHz - 06 - sandman.wav vs WMA44kHz - 06 - sandman.wav
Playback Range: 25.081 to 26.204
    12:08:18 AM p 1/1 pval = 0.5
    12:08:24 AM p 2/2 pval = 0.25
    12:08:29 AM p 3/3 pval = 0.125
    12:08:37 AM p 4/4 pval = 0.062
    12:08:42 AM p 5/5 pval = 0.031
    12:08:50 AM p 6/6 pval = 0.015
    12:09:00 AM p 7/7 pval = 0.0070
    12:09:10 AM p 8/8 pval = 0.0030
    12:09:30 AM p 9/9 pval = 0.0010
    12:09:39 AM p 10/10 pval < 0.001

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: Stravinsky_Capriccio WMA

Tester: Alex B

1L = WMA32kHz - 07 - Stravinsky_Capriccio.wav
2L = WMA44kHz - 07 - Stravinsky_Capriccio.wav

---------------------------------------
General Comments: Surprisingly good
---------------------------------------
1L File: WMA32kHz - 07 - Stravinsky_Capriccio.wav
1L Rating: 3.7
1L Comment: 
---------------------------------------
2L File: WMA44kHz - 07 - Stravinsky_Capriccio.wav
2L Rating: 3.7
2L Comment: 
---------------------------------------

ABX Results:
WMA32kHz - 07 - Stravinsky_Capriccio.wav vs WMA44kHz - 07 - Stravinsky_Capriccio.wav
    7 out of 8, pval = 0.035


---- Detailed ABX results ----
WMA32kHz - 07 - Stravinsky_Capriccio.wav vs WMA44kHz - 07 - Stravinsky_Capriccio.wav
Playback Range: 03.390 to 05.396
    12:21:22 AM p 1/1 pval = 0.5
    12:21:43 AM f 1/2 pval = 0.75
    12:22:04 AM p 2/3 pval = 0.5
    12:22:31 AM p 3/4 pval = 0.312
    12:22:47 AM p 4/5 pval = 0.187
    12:22:57 AM p 5/6 pval = 0.109
    12:23:24 AM p 6/7 pval = 0.062
    12:24:38 AM p 7/8 pval = 0.035

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: TomsDiner WMA

Tester: Alex B

1R = WMA44kHz - 08 - TomsDiner.wav
2L = WMA32kHz - 08 - TomsDiner.wav

---------------------------------------
General Comments: Very good. 1 has a bit more lowpass.
---------------------------------------
1R File: WMA44kHz - 08 - TomsDiner.wav
1R Rating: 3.6
1R Comment: 
---------------------------------------
2L File: WMA32kHz - 08 - TomsDiner.wav
2L Rating: 3.9
2L Comment: 
---------------------------------------

ABX Results:
WMA44kHz - 08 - TomsDiner.wav vs WMA32kHz - 08 - TomsDiner.wav
    10 out of 10, pval < 0.001


---- Detailed ABX results ----
WMA44kHz - 08 - TomsDiner.wav vs WMA32kHz - 08 - TomsDiner.wav
Playback Range: 08.601 to 12.424
    12:30:19 AM p 1/1 pval = 0.5
    12:30:34 AM p 2/2 pval = 0.25
    12:30:48 AM p 3/3 pval = 0.125
    12:31:20 AM p 4/4 pval = 0.062
    12:31:39 AM p 5/5 pval = 0.031
    12:31:45 AM p 6/6 pval = 0.015
    12:31:51 AM p 7/7 pval = 0.0070
    12:32:01 AM p 8/8 pval = 0.0030
    12:32:05 AM p 9/9 pval = 0.0010
    12:32:15 AM p 10/10 pval < 0.001

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: twist_shout WMA

Tester: Alex B

1R = WMA32kHz - 09 - twist_shout.wav
2L = WMA44kHz - 09 - twist_shout.wav

---------------------------------------
General Comments: 
---------------------------------------
1R File: WMA32kHz - 09 - twist_shout.wav
1R Rating: 1.5
1R Comment: 
---------------------------------------
2L File: WMA44kHz - 09 - twist_shout.wav
2L Rating: 1.2
2L Comment: 
---------------------------------------

ABX Results:
WMA32kHz - 09 - twist_shout.wav vs WMA44kHz - 09 - twist_shout.wav
    10 out of 10, pval < 0.001


---- Detailed ABX results ----
WMA32kHz - 09 - twist_shout.wav vs WMA44kHz - 09 - twist_shout.wav
Playback Range: 19.912 to 23.161
    12:37:00 AM p 1/1 pval = 0.5
    12:37:14 AM p 2/2 pval = 0.25
    12:37:40 AM p 3/3 pval = 0.125
    12:37:51 AM p 4/4 pval = 0.062
    12:38:06 AM p 5/5 pval = 0.031
    12:38:44 AM p 6/6 pval = 0.015
    12:38:56 AM p 7/7 pval = 0.0070
    12:39:09 AM p 8/8 pval = 0.0030
    12:39:24 AM p 9/9 pval = 0.0010
    12:39:51 AM p 10/10 pval < 0.001

____________________________________________________________________________________________________

ABC/HR for Java, Version 0.5b, 12 June 2006
Testname: waiting WMA

Tester: Alex B

1R = WMA32kHz - 10 - waiting.wav
2L = WMA44kHz - 10 - waiting.wav

---------------------------------------
General Comments: 2 has more lowpass and it is more distorded.
---------------------------------------
1R File: WMA32kHz - 10 - waiting.wav
1R Rating: 1.5
1R Comment: 
---------------------------------------
2L File: WMA44kHz - 10 - waiting.wav
2L Rating: 1.2
2L Comment: 
---------------------------------------

ABX Results:
WMA32kHz - 10 - waiting.wav vs WMA44kHz - 10 - waiting.wav
    10 out of 10, pval < 0.001


---- Detailed ABX results ----
WMA32kHz - 10 - waiting.wav vs WMA44kHz - 10 - waiting.wav
Playback Range: 04.376 to 06.668
    12:44:23 AM p 1/1 pval = 0.5
    12:44:31 AM p 2/2 pval = 0.25
    12:44:40 AM p 3/3 pval = 0.125
    12:44:43 AM p 4/4 pval = 0.062
    12:45:00 AM p 5/5 pval = 0.031
    12:45:16 AM p 6/6 pval = 0.015
    12:45:35 AM p 7/7 pval = 0.0070
    12:45:55 AM p 8/8 pval = 0.0030
    12:46:06 AM p 9/9 pval = 0.0010
    12:46:19 AM p 10/10 pval < 0.001

I would use CBR 32 kHz 48 kbps since it was a bit better.


EDIT

In general, I think that codecs that are this bad should be used in mono mode (I used stereo). If WMA and possibly some other codecs like AAC LC, MP3 (especially) were tested in mono mode that would produce more or less better audio quality (depending on the original amount of channel separation). Then the public test would also measure how important the stereo effect is for the users.

Multiformat 48 kbps Listening Test

Reply #112 –
For WMA, should I use CBR? 2-pass ABR leads to the same discussion we had during the 128 kbps listening test (useful only when encoding whole tracks, decoding back to WAV and then extracting the sample rather than encoding the sample only with WMA). Does anyone have experience with quality based VBR? If not, I could do some tests.

Multiformat 48 kbps Listening Test

Reply #113 –
Any estimate for the test date?
ruxvilti'a

Multiformat 48 kbps Listening Test

Reply #114 –
Well, I was held back because of Microsoft and lots of redirects (X tells me to talk to Y, Y tells me to talk to Z, Z points me back to X who then points me to A...). I will try to hurry up now.

Zambelli did his best to get in touch with some persons, but unfortunately didn't have much luck either.

Multiformat 48 kbps Listening Test

Reply #115 –
Sorry... But what if u test SSE2 Optimised Vorbis Encoder (Lancer) ?

That would be quite interesting I feel...
Go for Ogg

--------------------

It's Fast, Free, Open and has all stuffs that you need.

Multiformat 48 kbps Listening Test

Reply #116 –
I would rather test AoTuV since it recently got improved - quality wise.

I would like to end the discussion about codecs at this point. We should move on to the settings ASAP.

High and low anchor settings are set: -V5 --vbr-new for high anchor LAME and iTunes LC-AAC (CBR) at 48 kbps.
Nero is also clear (-b 48000), so is Vorbis (-q -1).

What is left is ATRAC3+ which I have to look into (still have to install VMware and Windows XP), as well as the two WMA encoders. Again, does anyone have information about WMA 9.1 Standard VBR and 48 kbps? Do you think CBR would be better?
Last but not least - do you think I should use WMA 10 from Vista or should I encode using XP and WMP 11?

Multiformat 48 kbps Listening Test

Reply #117 –
Quote
Last but not least - do you think I should use WMA 10 from Vista or should I encode using XP and WMP 11?

Should this make any difference?