hello
i just made an abx test with a little cut from a marilyn manson song. you can download here as self-extracting rar including aped-file plus winabx:
http://mitglied.lycos.de/pensionjaegerhoel...sontestfile.exe (http://mitglied.lycos.de/pensionjaegerhoelzl/hifi/abx_mmansontestfile.exe)
please can someone load the crap to his or her webspace because tripod/lycos is gonna delete it i'm quite sure.
what i did do to check out:
- sec. 9.5 to 12.0 (i listened the "hit")
- sec. 5.8 to 8.7 (the guitar doesn't sound like the original with mp3 and ogg vorbis; mpc does it [slightly different, but nearly unhearable at least at this nighttime])
my abx:
WinABX v0.42 test report
01/19/2004 21:26:10
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\ausreichend\mp3.wav
Start position 00:00.0, end position 00:22.7
21:27:05 1/1 p=50.0%
21:27:13 2/2 p=25.0%
21:27:19 3/3 p=12.5%
21:27:27 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 21:27:49
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\ausreichend\ogg.wav
Start position 00:00.0, end position 00:22.7
21:28:34 1/1 p=50.0%
21:28:48 2/2 p=25.0%
21:29:01 3/3 p=12.5%
21:29:05 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 21:47:51
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\mp3.wav
Start position 00:00.0, end position 00:22.7
21:48:11 1/1 p=50.0%
21:48:14 2/2 p=25.0%
21:48:17 3/3 p=12.5%
21:48:20 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 21:50:49
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\ogg.wav
Start position 00:00.0, end position 00:22.7
21:51:39 1/1 p=50.0%
21:51:46 2/2 p=25.0%
21:52:00 3/3 p=12.5%
21:52:54 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 21:53:17
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\mpc.wav
Start position 00:00.0, end position 00:22.7
21:54:18 0/1 p=100.0%
21:54:44 1/2 p=75.0%
21:55:17 2/3 p=50.0%
21:55:47 3/4 p=31.2%
WinABX v0.42 test report
01/19/2004 22:01:13
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\mp3.wav
Start position 00:05.8, end position 00:08.7
22:01:45 1/1 p=50.0%
22:02:02 2/2 p=25.0%
22:02:05 3/3 p=12.5%
22:02:09 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 22:02:42
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\ogg.wav
Start position 00:05.8, end position 00:08.7
22:03:44 1/1 p=50.0%
22:03:48 2/2 p=25.0%
22:03:52 3/3 p=12.5%
22:03:54 4/4 p=6.2%
WinABX v0.42 test report
01/19/2004 22:07:31
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\mpc.wav
Start position 00:09.5, end position 00:12.0
22:09:11 0/1 p=100.0%
22:09:27 1/2 p=75.0%
22:09:43 2/3 p=50.0%
22:09:47 3/4 p=31.2%
i used philips sbc 3375 headphones connected to my old (30 years or so) pioneer SX-300W receiver which was connected analogue (what else...?) to my onboardsoundcard (asus a7n8x deluxe rev. 2).
i was impressed by the quality mpc offers, mainly the "radio" profile did catch me and made me think.
if you speak german please visit this homepage to get more information.
thread in hifi-forum.de by me (http://www.hifi-forum.de/viewthread-33-1015.html)
hope you can confirm.
CU
fLOh
first of all, welcome to the forum!
but...
uh, abx result with 4 trials and thus a p-value of something FAR below 5% are, hmm, quite useless I guess... as they are not significant in any way.
edit:
[~ 96kbps / für Ausprobier- oder Testuser sowie Charthithörer]
- MP3
(Lame 3.92 MMX / -b 32 -m j -h -p --abr 96 -B 320 --resample 44.1 -q 0 -d)
- Ogg Vorbis
(q-2)
Preis: 0,29 Euro
I woudn't pay
anything for that kinf of quality
do you think 'ausprobier-users' (komisches wort ) would realy pay for this?
btw, you may want to read tihis link (http://www.hydrogenaudio.org/forums/index.php?showtopic=16295&) for more info on abxing
and the faqs here (http://www.hydrogenaudio.org/forums/index.php?showforum=15&) for recommended lame settings etc
cheers
I want to add/repeat some points:
About ABX testing
- If you only perform 4 trials, the result is quite unsecure. Better keep on until p> 1% (e.g. 7/7)
- All trials count! E.g. after getting 2/4 it's not allowed to start again from 0/0. This would spoil the p-value calculation and make the results meaningless (just in case ).
- Most likely it'll be easier if you listen to a short passage of the sample (2-3 seconds) often, trying to focus on certain details (e.g. transients like percussion, background noise, high-pitched sounds like cymbals)
About encoders:
- for lame try the --alt-preset commandlines (see Digga's link). Most likely the results will be better at comparable bitrates then with the commandlines you used.
- for ogg vorbis at high bitrates you might want to try gt3b1 encoder available at rarewares (http://rarewares.hydrogenaudio.org).
About playback/equipment:
- Many cheap soundcards resample 44.1kHz audio to 48kHz in a crappy way on playback. Try foobar2000's diskwriter for decoding to .wav using its high quality resampler to 48kHz - or directly use foobar2000's ABX tool with resampler enabled.
- Lossy Encoding -> Decoding causes clipping on loud music. To avoid this clipping affecting your ABX test, decrease volume. The easiest/most elegant way to do this, is to use foobar2000 for decoding with replaygain enabled or - again - foobar2000's ABX tool for performing ABX tests.
according to your suggestions i did another testing (identical source like the above).
WinABX v0.42 test report
01/20/2004 17:53:38
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und
Einstellungen\2800Plus\Desktop\abx\ausreichend\mp3.wav
Start position 00:00.0, end position 00:22.7
17:54:09 1/1 p=50.0%
17:54:11 2/2 p=25.0%
17:54:13 3/3 p=12.5%
17:54:15 4/4 p=6.2%
17:54:17 5/5 p=3.1%
17:54:21 6/6 p=1.6%
17:54:24 7/7 p=0.8%
17:54:27 8/8 p=0.4%
17:54:33 9/9 p=0.2%
WinABX v0.42 test report
01/20/2004 17:56:26
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und
Einstellungen\2800Plus\Desktop\abx\ausreichend\ogg.wav
Start position 00:05.8, end position 00:08.6
17:57:01 1/1 p=50.0%
17:57:04 2/2 p=25.0%
17:57:07 3/3 p=12.5%
17:57:09 4/4 p=6.2%
17:57:11 5/5 p=3.1%
17:57:14 6/6 p=1.6%
17:57:17 7/7 p=0.8%
17:57:20 8/8 p=0.4%
17:57:22 9/9 p=0.2%
WinABX v0.42 test report
01/20/2004 17:58:27
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\mp3.wav
Start position 00:05.8, end position 00:08.6
17:59:04 1/1 p=50.0%
17:59:06 2/2 p=25.0%
17:59:08 3/3 p=12.5%
17:59:10 4/4 p=6.2%
17:59:12 5/5 p=3.1%
17:59:14 6/6 p=1.6%
17:59:16 7/7 p=0.8%
17:59:18 8/8 p=0.4%
17:59:20 9/9 p=0.2%
WinABX v0.42 test report
01/20/2004 18:02:49
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\ogg.wav
Start position 00:17.5, end position 00:22.7
18:03:28 1/1 p=50.0%
18:03:31 2/2 p=25.0%
18:03:36 3/3 p=12.5%
18:03:40 4/4 p=6.2%
18:03:45 5/5 p=3.1%
18:03:49 5/6 p=10.9%
18:03:53 6/7 p=6.2%
18:03:57 7/8 p=3.5%
18:04:00 8/9 p=2.0%
WinABX v0.42 test report
01/20/2004 18:04:53
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\normal\mpc.wav
Start position 00:05.8, end position 00:08.6
18:06:49 0/1 p=100.0%
18:06:53 1/2 p=75.0%
18:06:59 2/3 p=50.0%
18:07:03 3/4 p=31.2%
18:07:06 4/5 p=18.8%
18:07:09 4/6 p=34.4%
18:07:16 4/7 p=50.0%
18:07:19 5/8 p=36.3%
18:07:33 6/9 p=25.4%
WinABX v0.42 test report
01/20/2004 18:09:13
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\mp3.wav
Start position 00:05.8, end position 00:08.6
18:10:49 1/1 p=50.0%
18:10:52 2/2 p=25.0%
18:10:55 3/3 p=12.5%
18:10:57 4/4 p=6.2%
18:11:01 5/5 p=3.1%
18:11:03 6/6 p=1.6%
18:11:05 7/7 p=0.8%
18:11:07 8/8 p=0.4%
18:11:11 9/9 p=0.2%
WinABX v0.42 test report
01/20/2004 18:11:43
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\ogg.wav
Start position 00:05.8, end position 00:08.7
18:13:30 1/1 p=50.0%
18:13:33 2/2 p=25.0%
18:13:35 3/3 p=12.5%
18:13:37 4/4 p=6.2%
18:13:41 5/5 p=3.1%
18:13:45 6/6 p=1.6%
18:13:48 7/7 p=0.8%
18:13:51 8/8 p=0.4%
18:13:53 8/9 p=2.0%
WinABX v0.42 test report
01/20/2004 18:24:23
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\hoch\mpc.wav
Start position 00:11.0, end position 00:13.0
18:26:15 0/1 p=100.0%
18:26:17 1/2 p=75.0%
18:26:19 1/3 p=87.5%
18:26:23 1/4 p=93.8%
18:26:25 2/5 p=81.2%
18:26:27 3/6 p=65.6%
18:26:29 3/7 p=77.3%
18:26:31 4/8 p=63.7%
18:26:34 4/9 p=74.6%
WinABX v0.42 test report
01/20/2004 18:36:31
A file: C:\Dokumente und Einstellungen\2800Plus\Desktop\abx\test.wav
B file: C:\Dokumente und
Einstellungen\2800Plus\Desktop\abx\special\lame_320CBR.wav
Start position 00:07.5, end position 00:08.7
18:38:14 1/1 p=50.0%
18:38:17 2/2 p=25.0%
18:38:20 2/3 p=50.0%
18:38:22 2/4 p=68.8%
18:38:24 3/5 p=50.0%
18:38:26 4/6 p=34.4%
18:38:28 5/7 p=22.7%
18:38:30 6/8 p=14.5%
18:38:32 7/9 p=9.0%
More on the clips:
- "ausreichend" (~ sufficient):
MP3: Lame 3.92 MMX / -b 32 -m j -h -p --abr 96 -B 320 --resample 44.1 -q 0 -d
Ogg Vorbis: oggdropXPd V1.7.6 - vorbis 1.0.1 / q-2
- "normal" (~ standard):
MP3: Lame 3.92 MMX / -b 32 -m j -h -p --abr 128 -B 320 -q 0 -d
Ogg Vorbis: oggdropXPd V1.7.6 - vorbis 1.0.1 / q-4
MusePack: MPC-Encoder 1.14 - Beta / --quality 4 --xlevel
- "hoch" (~ high):
MP3: Lame 3.92 MMX / -b 32 -m j -h -p --abr 192 -B 320 -q 0 -d
Ogg Vorbis: oggdropXPd V1.7.6 - vorbis 1.0.1 / q-6
MusePack: MPC-Encoder 1.14 - Beta / --quality 5 --xlevel
Every compressed file was decoded with the same program as it has been encoded. (MPC-Decoder V1.95z6 3D-Now / SSE)
More on the idea:
If you're asking yourself why the hell i did use such big lame syntaxes and didn't just use --alt-preset standard or perhaps --alt-preset extreme i'm gonna tell you why: the whole test-idea came to me as i wrote that hifi-forum thread where i was talking about selling music on the internet (like popfile.de) and how quality may be to which prices. but that isn't worth discussing because musiclabels won't sell mp3s or other non-drm-fileformats and if they'll do they won't do it with great quality, or am i wrong?
More on the abx:
i used "winabx". mode was set to abx. hide results was checked (until i had done 9 tests).
btw: wouldn't have my soundcard if it resampled the output to 48kHz (which i'm quite sure it doesn't) this also done on the original wav? wouldn't that mean that the particular error(s) must have been on all replays so it's meaningless?
the snippet i used can be downloaded from the link above. seems like even at 192kbps mp3 isn't transparent (but many people think it is).
i wouldn't pay for a 96kbps mp3 (as in "sufficient") or even for a 128kbps mp3 as in "standard". i'm not sure if i - if the price is ok - would buy a 128kbps ogg vorbis or -quality 4 --xlevel mpc. but the higher the quality the better. if i got the ability to chose between the formats i would buy the -quality 5 --xlevel mpc rather than the other formats. what about you?
thank you for your warm welcome. nice to be here.
i hope my results may be helpful by now.
CU
fLOh
according to your suggestions i did another testing (identical source like the above).
The ABX tests themselves look ok now, but there seem to be still a few (probably minor) issues:
- I can't see if replaygain was used for decoding to avoid clipping (or if you have checked manually (e.g. using a wave editor) if the decoded files clip.
- I suggested foobar2000 for decoding because, besides applying replaygain, it is able to decode mp3 encoded with lame without offset. With corrected offsets (can be done manually as well) ABX tests are more reliable. This only affects the mp3s IIRC.
- It would be good to provide a description of the differences you heard (not only which position in the sample, but also what it sounded like), so it's easier for others to repeat the test
If you're asking yourself why the hell i did use such big lame syntaxes and didn't just use --alt-preset standard or perhaps --alt-preset extreme i'm gonna tell you why: the whole test-idea came to me as i wrote that hifi-forum thread where i was talking about selling music on the internet (like popfile.de) and how quality may be to which prices. but that isn't worth discussing because musiclabels won't sell mp3s or other non-drm-fileformats and if they'll do they won't do it with great quality, or am i wrong?
I've read your post in the other forum. What I don't understand is: If you want to test settings based on your assumptions on what music labels would choose for download services, why do you test musepack and vorbis at all? BTW, there are already legal download services providing --alt-preset standard downloads. You'll find more information about it if you browse/search the "validated news" forum here.
btw: wouldn't have my soundcard if it resampled the output to 48kHz (which i'm quite sure it doesn't) this also done on the original wav? wouldn't that mean that the particular error(s) must have been on all replays so it's meaningless?
No. Crappy resampling can cause aliasing and distortion. Inaudible differences can become audible if 'processed' this way. A good (extreme) example is 'udial.wav' from this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=9772). (Read before playing back, especially the warning!)
... what about you?
I replied in the 1st place to help performing the ABX test correctly. I'm not sure if I would pay for music downloads unless they're losslessly compressed, and ATM I'm rather not interested in the topic. I can get the CDs I like for reasonable money e.g. used at ebay.