Hello!
I would like to announce the launch of my public, multiformat listening test @ 48 kbps.
The encoders featured are:
- iTunes AAC 7.0.2.16 (96 kbps High Anchor)
- Ogg Vorbis AoTuV 5 Beta
- WMA Professional 10
- Nero HE-AAC May 26 2006
- WMA Standard 9.2
- iTunes AAC 7.0.2.16 (48 kbps Low Anchor)
For further information on how to participate, please visit the test announcement page: http://www.maresweb.de/listening-tests/mf-48-1/ (http://www.maresweb.de/listening-tests/mf-48-1/)
The test is scheduled to end on December 8th, 2006.
For those using BitTorrent, statistics are available here: http://www.rarewares.org/tracker/mystats.php (http://www.rarewares.org/tracker/mystats.php)
Please use BT whenever possible.
Thanks to Roberto for sharing.
Thank you!
Regards,
Sebastian Mares
Weeee, a new test! I didn't follow the pre-test discussion, but I'm definitely gonna take part.
Was way better than the one for my last test.
I am out to deliver newspapers (not regarding the listening-test, though ).
Was way better than the one for my last test.
Haha
Damn, and I missed it!
Great...
But the last 3 links for Sample18/Sample19/Sample20 are the same on the ReadMe.html page...
Fixed, thanks!
I downloaded some samples (8,10,11,13), but there not any wma files at all?
Yes, the WMA files were decoded to WAV and then encoded to WavPack because there is no WMA Professional decoder for Linux and OS X.
Yes, the WMA files were decoded to WAV and then encoded to WavPack because there is no WMA Professional decoder for Linux and OS X.
Ahh, thanks.
Test announcement newsletter sent.
Those that are not yet in the newsletter: subscribe! (http://www.rjamorim.com/test/newsletter.html)
Thanks Roberto!
By the way, I already have results for 4 samples!
Thanks Roberto!
By the way, I already have results for 4 samples!
That was me
a little note about the .torrent files which are shown on readme.htm
whenever i try to download one of the .torrent files from there, for some reason, Opera renames the file to "SampleXX.zip.zip" instead of "SampleXX.zip.torrent". manually adding it from µTorrent worked though.
No idea why it happens. Works flawlessly with Firefox and µTorrent.
whenever i try to download one of the .torrent files from there, for some reason, Opera renames the file to "SampleXX.zip.zip" instead of "SampleXX.zip.torrent". manually adding it from µTorrent worked though.
Looks like the server is delivering the wrong MIME type. Firefox keeps the correct extension, but tries to open the file with the ZIP file handler.
I don't think I can fix that without admin access to the server (which I don't have)
Yep, problems with Opera as well. I'm not really keen on installing some vague BitTorrent client, so I guess it's HTTP download for me
edit: somehow the Java ABC-HR does't work, everything is greyed out, no sound, no sign of anything playing. (I have the sample folders as they are described, and decodeall worked fine). I´m now doing the test with the w32 binary of ABC-HR 1.0 stable, should be acceptable, right?
<sarcasm>the HTTP downloads should be web-seeded</sarcasm>
When I tried fot the first time to run ABC/HR I didn't get a sound. Then I check a sound blaster as a playback device in settings of ABC/HR. There is sound with both : Sound Blaster and primary sound driver.
I suppose I should check Sound Blaster, right?
Looks like the server is delivering the wrong MIME type. Firefox keeps the correct extension, but tries to open the file with the ZIP file handler.
I don't think I can fix that without admin access to the server (which I don't have)
Interestingly enough, my Firefox handles it fine. It shows the ZIP icon, yet opens the torrent files with the proper program (µTorrent).
Yep, problems with Opera as well. I'm not really keen on installing some vague BitTorrent client, so I guess it's HTTP download for me
That's OK.
edit: somehow the Java ABC-HR does't work, everything is greyed out, no sound, no sign of anything playing. (I have the sample folders as they are described, and decodeall worked fine). I´m now doing the test with the w32 binary of ABC-HR 1.0 stable, should be acceptable, right?
What version of the Java Runtime are you using?
I'm afraid the results from ABC-HR 1.0 won't be of much use. For instance, it doesn't account for delay. there must be some problem on your side, as it seems to be working for everyone else...
It didn't output any sound for me either, but when I changed the audio output to Primary Sound Driver it worked fine.
With a little luck schnofler is going to see this thread and come with a solution
Doing these tests makes me wonder how guruboolez can hear artifacts in some of this stuff.
For people without sound, simply change the playback device from the options. I don't know why ABC/HR does not default to the primary audio device like every other program.
Perhaps you can add that note to the readme, or at least to the first post/listening test page.
Also, I sent you my results.
i've saved the results from testing the first sample, but i can't get the program to show me which files are originals and which are encoded ones as the readme says it will. hmm... should the program be automatically showing me these results?
You won't find out the results until Sebastian posts the key so you can decrypt the results.
You're reading ABCHR's readme, which only says that for a normal, not encrypted test, not the encrypted ones that we're doing.
Perhaps you can add that note to the readme, or at least to the first post/listening test page.
Also, I sent you my results.
Check your mail, especially the second mail I sent.
Did so and replied.
You won't find out the results until Sebastian posts the key so you can decrypt the results.
You're reading ABCHR's readme, which only says that for a normal, not encrypted test, not the encrypted ones that we're doing.
ok, i get it now. thanks
What is real name of song from sample 11?
Thanks
You can see the sample name in ABC/HR. The sample is "Mysterious Times" by "Sash! feat. Tina Cousins".
You can even find it on YouTube.
Please consider offering a torrent with all samples next time
I told Roberto to create one (he's responsible for torrents), but then on the other hand, looking at the last statistics, a lot of people downloaded the whole package only for testing the first sample.
Problem in sample 10 !!! the offsets are erroneus for all but one sample! Please, fix the .ecf file.
What do you mean? The offsets were calculated by ABC/HR.
Just tested and I really don't see where the problem is. Are you sure you are not mixing up due to pre-echo or "Muting" because of the lowpass filter?
I am running all samples through the offset detector to see if I get different offsets for sample 10 (all encoders should have the same offset regardless of sample).
Edit: JAZ is right, I will update the ECF once I check the remaining 10 samples. Results should be still valid, though, since you can clearly differentiate between original and sample even when not relying on offset.
Fixed, thanks [JAZ]!
Learned something new for the next test: always check offsets and don't rely 100% on ABC/HR.
I had the same problem in the AAC48 test for one of the samples.
For people without sound, simply change the playback device from the options. I don't know why ABC/HR does not default to the primary audio device like every other program.
Java doesn't tell me anything about the audio devices besides their names and what they're supposed to be playing. From Java's viewpoint there is no "primary audio device". I just pick any one that appears to be able to play sound. Sometimes it works, sometimes it doesn't. The only sure-fire way to pick a device that works is not to use native devices at all, but use Java's software mixer instead, which sucks.
Learned something new for the next test: always check offsets and don't rely 100% on ABC/HR.
I had the same problem in the AAC48 test for one of the samples.
Just had a look at this. With the sample 10 from this test it can be fixed by just doubling the window size (which also increases processing time). If you're interested, Sebastian, I've uploaded a version with this fix. Offset detection might still fail when there's not much going on in the first few seconds of the sample, though.
Thanks for the offer schnofler, but now that I fixed the problem myself, I guess I don't need it - except for a future test maybe. You could upload that version so that others can download, too (in case anyone wants to conduct a listening test, too).
BTW, maybe someone with Bash knowledge or so can create a script for OS X. Are there decoders for Intel Macs already?
When trying to open Sample15.ecf I get a "file is not a supported file type" error.
Also, while doing Sample 16 Save/Open Session menu items where gone. Restarting ABC/HR solved that one.
EDIT: Redownloaded Sample15.zip via http and it's ok now.
Last sample As always I'm going to go through all samples one more time tomorrow...
I haven't had any problem with sample15 nor 16. Seems you had a problem with the download (or who knows..)
I had the same problem in the AAC48 test for one of the samples.
I bet it was bibilolo which is my sample10. Did you notice the problem and edit the offsets manually?
Fixed, thanks [JAZ]!
Learned something new for the next test: always check offsets and don't rely 100% on ABC/HR.
Should we download sample 10 again? Is this sample the only one affected?
It has nothing to do with the sample itself - only the ECF file was affected. You can download the ABC/HR package again which contains Sample10.ecf.
Here's another download mirror for you guys... let's see how well i own my quota
http://plonkmedia.org/001users/hydrogenaudio/ (http://plonkmedia.org/001users/hydrogenaudio/)
http://plonkmedia.org/001users/hydrogenaudio/AllSamples.zip (http://plonkmedia.org/001users/hydrogenaudio/AllSamples.zip) <--i think that's all of them "zip -0 Sam*.zip AllSamples.zip" from SSH command line gave me a 158mb Sample01.zip ... so i'm guessing it's all of the other ones in there...
I had the same problem in the AAC48 test for one of the samples.
I bet it was bibilolo which is my sample10. Did you notice the problem and edit the offsets manually?
Yes, I manually corrected the offsets very soon after the test start (a tester reported about the wrong offsets for this sample)
just sent you my results for 20 samples today. I hope they are useful.
Yep, received them, thanks! Some people on the IPS board also seem to be interested since I received some PMs and someone also replied to my thread.
I have to say that even using headphones (Senn HD555) for some samples I couldn't tell the difference from the original. For others the differences were very subtle.
Either codecs improved a lot or I am losing my hearing (or both!!)
High anchor for some samples is indistinguishable for me. As it's 48 kbit/s test I expect that the result is still valid if only high anchor isn't spotted.
As it's 48 kbit/s test I expect that the result is still valid if only high anchor isn't spotted.
The result is valid if you don't spot anything :B
Why should only people that can hear artifacts be allowed to participate?
The result is valid if you don't spot anything :B
Why should only people that can hear artifacts be allowed to participate?
Yep. It's real life test.
In extremal case listener can be totally deaf or has serious problem with hearing. But it's not common in this forum.
I'm getting a weird hang when adjusting the playback range in the ABX section (only happens occassionally, but it's catastrophic when it does happen)
Bummer, maybe schnofler can help.
I had it hang once or twice as well. Also, it seemed that whenever I saved my session, I was unable to change the rating sliders without first reloading the session, but everything else still works.
This is soo cool. If I have time I'm going to complete all of the samples.
I managed to decompress files on Linux using
wine cmd /c Sample05.bat
(instead of wine wcmd as it states in the docs)
Using Wine 0.9.22 on Ubuntu
I sent you my 20 test results also now, just wanted to make sure you got them. Fun test!
I received them, thank you!
Thanks to all involved! I'm not going to go into any extensive ABX testing...if it's imperceptible, then I'm voting 5. This is mainly due to my left ear hyperacusis, which tends to interfere with involved listening tests such as these. I hope these will be acceptable results!
I hope this doesn't skew the results too much but many times I think I have a good idea which sample is AAC+ based on that SBR graininess. :X
The samples are often pretty easy to pick out (though there's one or two contenders that do a good job!), so don't worry too much.
To be honest, I tested some samples myself just for fun (did not save the results of course) and for the majority of the samples, I was only able to pick out 3 codecs.
Drrrats! I totally completely utterly forgot about this page:
http://wiki.hydrogenaudio.org/index.php?title=Current_events (http://wiki.hydrogenaudio.org/index.php?title=Current_events)
Oh, didn't know about that one. I only edited the listening tests wiki page myself.
Sebastian, did u get results from trans at mchsi.com? ;) Just want to be sure they made it past any spam filters.
Yep, thanks!
Just out of curiosity, how many samples so far? Is the participation on this test being significant?
I have something around 10 results per sample making a total of 200 results so far. Keep 'em coming!
Good to hear that you're getting plenty of testers.
OK, I decrypted all result files, sorted them automatically using ABC/HR and the wonderful ReNamer with its regexp replacing method, eliminated all invalid results (ranked reference without ABX test) and then counted:
Sample01: 13
Sample02: 15
Sample03: 13
Sample04: 13
Sample05: 13
Sample06: 10
Sample07: 16
Sample08: 12
Sample09: 16
Sample10: 12
Sample11: 13
Sample12: 12
Sample13: 13
Sample14: 14
Sample15: 14
Sample16: 14
Sample17: 12
Sample18: 15
Sample19: 15
Sample20: 15
Three days left for testing and it looks like I don't have to extend the test. I would like to ask people to start sending results soon if they haven't already.
OK, I decrypted all result files, sorted them automatically using ABC/HR and the wonderful ReNamer with its regexp replacing method, eliminated all invalid results (ranked reference without ABX test) and then counted:
".. without ABX test".
Isn't enough just to move the sliders up and down and then save via "Save Test Results.." ?
It is as long as there are no ranked references.
It is as long as there are no ranked references.
Sorry to be stupid but what do you mean by that? I have a problem with english terminology
In abc-hr, you have, for each contender in a sample, a slider with the original, and another with the contender (codec, setting...).
If you lower the slider which is the original, instead of the contender, and don't provide an ABX (showing that you haven't randomly or unknowingly done that), the result, for that sample, is invalid.
Exactly. Anyways, ABC/HR automatically disables the reference slider once a successful ABX test finished. Therefore, I don't think it's possible to have ranked references with successful ABX.
Anyways, I updated the number of results I have so far: 368!
Is it possible to send results on friday or do I have to send them on thursday?
I have to apologize, I'm a bit late.
It's OK. I will process results on Saturday.
By the way, does anyone have friedman.exe? I found two versions on a CD of mine: one says that I have too many samples (10 ) and that only 8 are allowed, the other one hangs.
It's OK. I will process results on Saturday.
By the way, does anyone have friedman.exe? I found two versions on a CD of mine: one says that I have too many samples (10 ) and that only 8 are allowed, the other one hangs.
Only 8 codecs are allowed. I didn't think you were testing that many. Maybe there is a formatting problem? Make sure there is no space in the name of each of the codecs.
Ah, I see. That seems to be the problem then - I have a space in "WMA Professional" and "WMA Standard".
I would like to ask everybody who did not send the results yet to do so as soon as possible. I would like to have all results by 3 - 4 o'clock PM GMT.
I would like to ask everybody who did not send the results yet to do so as soon as possible. I would like to have all results by 3 - 4 o'clock PM GMT.
Would it be possible to postpone things a little bit? I would like to participate (and try to get some friends involved), but the problem is that I am almost without sleep all this week and the work just keeps pouring here.
If you could accept some late submissions, it would be quite nice.
Thanks, Rogério.
Sorry, no. I will accept results for the next 12 hours only (since I process tomorrow anyways).
Torrents deleted...
Many thanks for hosting samples and seeding Roberto!
Many thanks for hosting samples and seeding Roberto!
Mah pleejohr.
Just sent 7 results. Bummer, now everyone will have proof that my hearing sucks.
It's the equipment for sure - just like in my case...
~7 hours left so please send results if you didn't already.
I am hitting the sack.
I can't wait to see the results.
Uff, finally got to do the test. Results sent.
P.S.: I demand a better music selection next time!
I agree, the eurotwance has to go. Kraftwerk was a nice touch, though.
Sorry, no. I will accept results for the next 12 hours only (since I process tomorrow anyways).
Ok, no problems then...
I would like to have participated, but Real Life ? has kept me away from a keyboard. :-(
Regards, Rogério.
OK, test is now officially closed. No more results will be accepted. Many thanks to everyone who participated!
How many results did you end up with?
Bleh, just got back home from shopping with mom which always ends up being totally different than what I planned. Will process results ASAP - sorry for the delay.