Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Public MP3 Listening Test @ 128 kbps - CLOSED (Read 163258 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #100 –
Yes, extend it 2 weeks more. If it will lead to getting results from a few more users it will have impact at the total endresult. Participants should be made clear that not all samples have to be tested, we all have little spare time. It's better to send one ore a few results than none.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #103 –
I "did my part" and just contributed a sample (and ABX'd all the encoders, woot). I had a hard time ranking the encoders though, and my ears were not at all happy after everything was done. I think I'll limit myself to one sample a day in the future.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #104 –
Here's an update of the number of valid results I have so far:

Sample01: 14 (+3)
Sample02: 8 (+1)
Sample03: 9 (+3)
Sample04: 10 (+3)
Sample05: 8 (+1)
Sample06: 7 (+1)
Sample07: 9 (+2)
Sample08: 8 (+2)
Sample09: 9 (+2)
Sample10: 8 (+2)
Sample11: 8 (+2)
Sample12: 10 (+1)
Sample13: 8 (+2)
Sample14: 9 (+2)

The numbers in the brackets show the number of additional results received since extending the deadline. Thanks to everybody who participated already.

And yeah, so far, results are very suprising.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #105 –
Well, kwanbis got the word out, and Hydrogenaudio is now experiencing a Slashdotting... My my.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #106 –
Now that it has appeared on /.:

Most comments so far are complaining that it's too complicated to set up. This might be a reason for so few responses so far.

To quote anonymous coward:
Quote
You know what, I thought I'd be nice and give this a shot, but the amount of effort involved just isn't worth it. If it isn't 'click on this link, listen, rate', it's too much work. Download x, install x, email x - way, way, way too much work for what is being given in return.


And they even haven't tried conducting the test yet which is even more work...

I hope there are at least some people who take the effort.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #107 –
These tests are unusable in Linux, with jre-1.4, -1.5, and -1.6.  What's the point in using Java, then?  It seems filenames are inconsistent (expecting Sample01/Sample01.wav instead of Sample01/sample01.wv).  Fixing that, I see another error:

Quote
Error
Config file could not be loaded.
file is not a supported file type


That's 55MB you won't be getting back.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #108 –
Quote from: incripshin on
These tests are unusable in Linux, with jre-1.4, -1.5, and -1.6.  What's the point in using Java, then?  It seems filenames are inconsistent (expecting Sample01/Sample01.wav instead of Sample01/sample01.wv).  Fixing that, I see another error:

Quote
Error
Config file could not be loaded.
file is not a supported file type


That's 55MB you won't be getting back.


I don't understand what the problem is here. What do you mean with filenames are inconsistent? You have 14 ZIP files that you can download individually - nobody forces you to download all 55 MB and this is stated in the readmes. The ZIP consist of 6 MP3 files that represent the samples and one lossless WavPack (wv) file that is the reference. If your archiver doesn't mess up, you should end up with files that have the proper casing which is also used in the batch files.

Then what do you mean with that error? Where does that error come from and when? If you load an ecf file? Did you follow the readme file at all? Does the folder structure look like in folder-setup.png?

By the way, the reason why the whole test is not made in Flash is that Flash would require you to be connected to the Internet all the time. The current solution allows you to download the files at work or at an Internet cafe if you don't have broadband at home and do the test on your PC while you are offline. The resulting encrypted result files have only a few KB which should be no problem to submit from home then.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #109 –
I agree, the test is a little hard to figure out. You really have to read the "read me" manual very closely. Especially if English isn't your first language I suppose.

For example: The sentence "Navigate to the "bin" directory and run "DecodeXX.bat" trew me off. It took me a while to understand that you first have to decode the samples to wav-files. I have never done this kind of decoding. Not hard to do, I just haven't done it before.

Anyway, I never got any farther than loading the samples in the ABC/HR program. Nothing happened when clicking the start buttons, on any of the samples, that's where I gave up. Sorry.
//From the barren lands of the Northsmen

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #110 –
Quote
Please note: if you don't hear any sound when using ABC/HR, simply change the device from ABC/HR's settings. In some cases, ABC/HR might not default to the primary audio driver.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #111 –
Quote from: Sebastian Mares on
I don't understand what the problem is here. What do you mean with filenames are inconsistent? You have 14 ZIP files that you can download individually - nobody forces you to download all 55 MB and this is stated in the readmes. The ZIP consist of 6 MP3 files that represent the samples and one lossless WavPack (wv) file that is the reference. If your archiver doesn't mess up, you should end up with files that have the proper casing which is also used in the batch files.

Then what do you mean with that error? Where does that error come from and when? If you load an ecf file? Did you follow the readme file at all? Does the folder structure look like in folder-setup.png?

By the way, the reason why the whole test is not made in Flash is that Flash would require you to be connected to the Internet all the time. The current solution allows you to download the files at work or at an Internet cafe if you don't have broadband at home and do the test on your PC while you are offline. The resulting encrypted result files have only a few KB which should be no problem to submit from home then.


So I didn't know that wv was WavPack's extension, and this led to confusion.  This was the first error I got:
Quote
Config file could not be loaded.
File Sample01/Sample01.wav could not be opened.


My only thought here is to check the permissions for the directory, and it is indeed writable.  I downloaded all the samples because I wanted to, not because I'm stupid.  I followed the instructions to the letter.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #112 –
Guys, please understand that I am organizing these tests based on some given software. If the software is buggy, I have two options: either don't organize the tests at all or - like in this case - organize the test and inform people about possible problems. If I had the time, I would code something like ABC/HR myself. schnofler is busy with other things unfortunately and doesn't have time to look into the problem. My Java knowledge is very limited and I have no idea what is going wrong. I tried to build ABC/HR against JDK 1.6 but it didn't help - the hangs got shifted to other areas.

As for the complaints for decoding and downloading several packages: I could also make everything bundled in one package, but then people will complain that they have to DL 50 MB even if they only want to test two or three samples. Also, I could also provided the samples decoded to WAV already, but then the average package size won't be 4 MB, but 40 MB or even more. I also stated in my previous post why I don't offer an online based test environment. Beside the argument that you have to be online all the time, you also have to develop something like this first and I don't know Flash / Action Script at all.

Quote from: incripshin on
My only thought here is to check the permissions for the directory, and it is indeed writable.  I downloaded all the samples because I wanted to, not because I'm stupid.  I followed the instructions to the letter.


You have to open the ecf file not the wav file.

Quote
Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf" (again, "XX" being the number of the test you want to take - make sure you ran the batch file respective to that package before).

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #113 –
Quote from: Sebastian Mares on
You have to open the ecf file not the wav file.
Quote
Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf" (again, "XX" being the number of the test you want to take - make sure you ran the batch file respective to that package before).


This is exactly what I did.  The error message reads, 'Config file could not be loaded'.  And so, I left it implied that I was opening a config file.

Edit: I skipped step 4, which began 'WINDOWS USERS'.  There are two subsections, and I am stupid.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #114 –
Ah, OK, sorry. Hmm... And the folder structure is like on the picture? That is, you have a folder where abchr.jar is located and in that folder you have a subfolder called Sample01 where the decoded WAV files are?

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #115 –
Quote from: Canar on
Well, kwanbis got the word out, and Hydrogenaudio is now experiencing a Slashdotting... My my.

I hope it is for the good

Maybe we should be updating the 1st post with all the findings to help newbies.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #116 –
The /.ers are having a field day it seems. I am glad the word got out there, but the ... complaints. I just finished the test fine, but it seems people are having troubles with it.  I do think it would be a good idea doing a form of Flash application if at all possible in the future, even though you have a valid point of not having to be online to do the test. Maybe an option ?

I did like this reply though:

Quote
While popular music is acceptable at 128kbps with recent encoders, certain niche music genres like spectralist music clearly suffer at low bitrates. With pieces like Per Norgard's  Symphony No. 3 [amazon.com]  or Grisey's Les espaces acoustiques you can easily hear the difference between 256kbps and the original CD-quality on even average headphones or speakers. Any music which depends on a greater portion of the natural overtone series than just the first handful of partials will need higher bitrate encoding.


Oh well. Oh, and this one:

Quote
Umm... 128 Kbps? Seriously? And no Ogg Vorbis, AAC etc... If you're bothering to set up a listening test, why limit yourself to 128 Kbps MP3?

Also, this should really be set up as a blind test, you get to listen to two clips, and have to choose which is better. The clips are randomized, of course... I could go on, but I'd just make myself sound even more arrogant.


Arrogant indeed.

Thanks for putting the test together!

sizetwo.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #118 –
why don't you upload all to www.mediafire.com as a mirror?

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #120 –
Nobody's blaming you Sebastian, you are doing a great job organizing everything. Some of us are just pointing out that there are some things, or steps, that are a bit unclear, and maybe even mis-explained. You really want to get to the listening part as soon as possible, because you know that part alone is going to take a whole lot of time and effort. And when you run into a bunch of small trials and obstacles on the way there, it pisses you off, and discourages you. 

I would really like to see an online, pre-set, interactive test. You could probably make it look just like the ABC/HR-interface. That interface could probably be saved and used in another test, just having the samples replaced. Would that be possible? I don't know. I'm talking about a future test now, I'm well aware of the fact it's a little late for this particular test. 

Anyway, I have now passed all the hinders (thanx Seb!) and will hopefully be able to conduct a few tests pretty soon.
//From the barren lands of the Northsmen

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #121 –
I understand your complaints and I agree that for a newcomer, the testing process is somewhat complicated. I am looking forward to simplifying it for upcoming tests, either by making the documentation easier to read and follow or by developing / using other software.

Anyways, just wanted to let you know guys that after an average of 2000 hits per day today I ended up with 11666. At 22 o'clock German time I had over 5000 already. So thanks for spreading the news. I am also happy to let you know that I've already received several valid results and sample 1 is only one result away from the "magical" 20 results.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #122 –
Quote from: Sebastian Mares on
Is RapidShare an option?

RapidShare is an option for sure, problem is that is realy slooooooow for non-premium users.

I rahter recommend mediafire and megaupload.

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #123 –
@Sebastian
I really sympathise with you. There's a neat, tight project going on, but not quite enough numbers. So you publicise it, and in come the hordes, not having a clue what they're doing or why they're doing it, or why it has to be hard work to get results. And, yes, I'm sure there are glitches, but people who care will work through them because the results matter, or at least take the time to have specific questions. And if in doubt, blame the poor sod who's organising it and doing more work than everyone else.

In other words, if you don't like this, for God's sake don't work in higher education.

I am really grateful for your work, and persistence. The only reason I haven't contributed is that I am sure my ears couldn't distinguish between anything half-way satisfactory at ~128.

So hang in there. Or walk away from it all. Whichever feels right.
 

Public MP3 Listening Test @ 128 kbps - CLOSED

Reply #124 –
Some comments:

I found this test through slashdot. 

I use linux with JRE 1.6.  ABC/HR works flawlessly.

Many people are saying the test has recevied poor results due to difficulties with setup and the like. I disagree.  This is really very easy if you just take the time to read the short instructions.

That said, I don't believe this test is for everyone. It is VERY difficult to pick out aural differences in the samples.  I have tested 1, 2, 6 ,13, and 14, and have found only 4 or 5 samples that I can verify via an ABX test, not counting the low anchor.  I would not want someone casually performing these tests, as it takes time and patience to truly do this right.

I hope to finish the entire set before the 22nd, but this stuff is grueling!