Skip to main content

Topic: Public MP3 Listening Test @ 128 kbps - CLOSED (Read 128698 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Polar
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #25
Sebastian, keep the good work.
A listening test like this is so huge to prepare and execute, I even faint when thinking of it.
Looking forward to see the results.
I'll second that.  Hats off for Sebastian.

I wonder if LAME 3.98.2 will ever be threatened as the current recommended encoder!
I'm afraid I expect a statistical tie between LAME 3.97, 3.98, Fraunhofer, perhaps iTunes too.  Heck, maybe even Helix may not escape the tie's statistical error margin.  In other words: please contribute your results, people!

  • uart
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #26
I've just a had areally quick listen to the samples and I'll do the test soon.

Wow from the quick test just one encoder is really horrbile bad. I suppose it's the low anchor. Is that l3enc low anchor actually broken or something? I've never heard anything as bad as that.
  • Last Edit: 22 October, 2008, 12:55:43 PM by uart

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #27
Wow from the quick test just one encoder is really horrbile bad. ...

Just one? At least on some of the samples there is another one (not necessarily always the same one) which is pretty nasty as well.
As for the other encoders I'm having a hard time to differentiate them as far as the samples are concerned I tried so far. Differences are so subtle that I'm pretty uncertain whether they're just imagination. So my result often is 5.0.
  • Last Edit: 22 October, 2008, 01:04:44 PM by halb27
lame3995n -Q0.5

  • uart
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #28
Oh yeah I'm sure they'll be other artifacts, like I said I haven't had a full listen yet. But there is one encoder there that is just so bad it really stands out on every sample, like you can notice it immediately even if you're not listening with any attention. That surprised me a bit.
  • Last Edit: 22 October, 2008, 01:51:35 PM by uart

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #29
But there is one encoder there that is just so bad it really stands out on every sample, like you can notice it immediately even if you're not listening with any attention. That surprised me a bit.

ReallyRareWares states that it's "The First Ever publicly available MP3 Software Encoder" (Date: 1994-03-16). Legendary quality, you see.

  • /mnt
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #30
Oh yeah I'm sure they'll be other artifacts, like I said I haven't had a full listen yet. But there is one encoder there that is just so bad it really stands out on every sample, like you can notice it immediately even if you're not listening with any attention. That surprised me a bit.


Sounds like its l3enc, that encoder really sucks at 128; but impressive at 192 for its time. But hearing the l3enc encode of castnets really does hurt my ears .
"I never thought I'd see this much candy in one mission!"

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #31
First, I would like to thank Sebastian and Alex for their work in this test. Greatly appreciated. 
This is the first time I do an ABC/HR test (though, I'm familiar with the ABX one).
Right now, I've just tested the sample01 and saved my results.
I just want to know if there's a way I can see them?
In other words, is there a way I can read the content of the .erf file I've just created?

Anyhow, I plan on testing the 14 samples and will mail them asap.

I wonder if LAME 3.98.2 will ever be threatened as the current recommended encoder!

Me too! Even though version 3.97 is already a great one IMO.

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #32
First, I would like to thank Sebastian and Alex for their work in this test. Greatly appreciated. 
This is the first time I do an ABC/HR test (though, I'm familiar with the ABX one).
Right now, I've just tested the sample01 and saved my results.
I just want to know if there's a way I can see them?
In other words, is there a way I can read the content of the .erf file I've just created?

Anyhow, I plan on testing the 14 samples and will mail them asap.

Thanks for your kind words.

Once the test is over, Sebastian will publish the key file and you can process your result files, so keep them saved.


One thing that has not been separately mentioned or discusssed now is the tester's name. You can add your nickname to "Show name in results file". I'd recommend adding it unless you really want to send anonymous results. It would help finding your results if you have questions about some particular samples after the test.

I'd like to also remind that if you need to have a break and close the ABC/HR program you can save the session and reopen it, but you can't start from the beginning (i.e. reopen the ecf file) and quickly apply the ratings for the already tested samples because each time the ecf file is opened the samples will be in a different order.

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #33
While testing it happens rather often that I find differences subtle, and I'm not sure whether they really exist. So I'd like to try to ABX the difference for the encoder just under consideration.
The ABX button is in the program's head part so it doesn't invite me to use it in the situation described.
Can I use it? In case I can, how can I make sure that I ABX the encoder under consideration?
lame3995n -Q0.5

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #34
Once the test is over, Sebastian will publish the key file and you can process your result files, so keep them saved.

Thanks for your clarification.

Quote
You can add your nickname to "Show name in results file". I'd recommend adding it [...]

OK, will do. 

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #35
While testing it happens rather often that I find differences subtle, and I'm not sure whether they really exist. So I'd like to try to ABX the difference for the encoder just under consideration.
The ABX button is in the program's head part so it doesn't invite me to use it in the situation described.
Can I use it? In case I can, how can I make sure that I ABX the encoder under consideration?

You can easily select the sample:



After a succesful ABX test the original sample for the tested encoder will be locked to 5.0 in the main window.
  • Last Edit: 23 October, 2008, 06:40:24 AM by Alex B

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #36
You can easily select the sample: ...

Thanks a lot.
Just to be sure: 'sample' here refers not to the 14 listening test samples, but to the 6 mp3/original pairs under consideration with 1 listening test sample. Is this correct?
lame3995n -Q0.5

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #37
Just to be sure: 'sample' here refers not to the 14 listening test samples, but to the 6 mp3/original pairs under consideration with 1 listening test sample. Is this correct?

Yes. You are testing only the samples that are included in the opened test configuration file. The program does not access the 13 other test configuration files anyhow.

In my screenshot I was testing the configuration file #9. I was unsure about the contender/encoder #3. I didn't move the sliders in the main window before trying the ABX test.

After you have succesfully ABXed all contenders you can compare the encoders with each other and reconsider the final ratings. In case you can't ABX a particular encoder you should not move the sliders from 5.0.
  • Last Edit: 24 October, 2008, 03:07:59 PM by Alex B

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #38
Just to be sure: 'sample' here refers not to the 14 listening test samples, but to the 6 mp3/original pairs under consideration with 1 listening test sample. Is this correct?

Yes. You are testing only the samples that are included in the opened test configuration file. ... In case you can't ABX a particular encoder you shoud not move the sliders from 5.0.

Thanks a lot for making things totally clear. I'll throw away my results obtained so far (with 5.0 score most of the time), and start anew with ABXing taking heavy part in it.
lame3995n -Q0.5

  • Alexxander
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #39
As I had Java crashes when saving results or sessions, I wrote down the (slider) results on paper before saving, just in case I had a crash. I figured that in case of a crash I just could open again the ABC/HR Config of the same sample and save results again. Luckily only on the 5th sample I noticed that when you reload the ABC/HR Config with the same track the sample orders have changed so my results written on paper didn't correspond anymore.

So I downgraded the Java version and after 4 tests I noticed I forgot to fill in my name. But I have saved the sessions. So my question is if it's OK to just open/load the saved ABC/HR Session, put in my name and save the results again (and overwrite the former result file)?

Thx

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #40
So I downgraded the Java version and after 4 tests I noticed I forgot to fill in my name. But I have saved the sessions. So my question is if it's OK to just open/load the saved ABC/HR Session, put in my name and save the results again (and overwrite the former result file)?

As I said earlier, it is safe to save a session and reopen it. You can add your name and continue testing after reopening a saved session.

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #41
Thanks for the feedback you guys (Alex B and others) provide. I usually get home after 5 pm that is why I cannot reply to the questions myself.

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #42
First results arrived today!

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #43
Yay, first results for all samples received today. One encoder is quite surprising so far.

  • Bodhi
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #44
Yay, first results for all samples received today. One encoder is quite surprising so far.

Which one?

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #45
Patience, young padawan...

  • Bodhi
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #46

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #47
I've just posted my results and one encoder surprised me too (I mean for the potential third place) 
  • Last Edit: 02 November, 2008, 06:43:00 AM by melomaniac

  • jmartis
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #48
I downloaded ABC/HR, I also downloaded a sample package "Sample07.zip". I ran Sample07.bat, then executed abchr.jar, clicked Open ABC/HR config, loaded Sample07.ecf.

The problem: Clicking play on any sample doesn't produce any sound.

Problem #2. This sample package happened to be the eig sample, which produces easily noticable artifacts. Just out of curiosity, I tried to encode the sample with FHG surround encoder "-br 0 -m 4 -q 1 -vbri -ofl" and compared my encoding to the test samples. It sounded different from all of them, and it shouldn't.

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #49
The problem: Clicking play on any sample doesn't produce any sound.

Options > Settings > Playback > Playback Device  and then choose the right one for you.

EDIT:
I also downloaded a sample package "Sample07.zip [...] This sample package happened to be the eig sample.

Weird, mine is Suzanne Vega, not eig.
  • Last Edit: 26 October, 2008, 05:39:13 PM by melomaniac