Skip to main content

Topic: Public MP3 Listening Test @ 128 kbps - CLOSED (Read 128705 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Public MP3 Listening Test @ 128 kbps - CLOSED
Greetings!

After months of endless discussion and preparation, the MP3 listening test is finally open.

The featured encoders, as most of you know already, are:

LAME 3.97
LAME 3.98.2
iTunes 8.0.1.11
Fraunhofer IIS mp3surround CL encoder v1.5
Helix v5.1 2005.08.09

The anchor used in this test is:

l3enc 0.99a as low anchor

If you want to read more before starting, the address of the test is http://www.listening-tests.info/mp3-128-1/. If you want to start right away, have a look at the readme. Please note that you do NOT have to test all samples - it is a great help even if you test one or two!

Info: The people who did not start testing already but who would like to participate, it would be very nice if you could focus especially on samples: 5, 6, 8, 9, 10, 11, 13, 14. Sample 1 already has 25 valid results, so please do not test it (of course, if you already did, you can submit your results for it, but if you just started, maybe invest time and effort in testing the aforementioned samples first).

Update: The test is scheduled to end on November 22nd, 2008.

Warning: There seem to be problems with ABC/HR when JRE 1.6 is installed. If you are affected by hangs, crashes or any other difficulties, please download and install JRE 1.5 Update 15 from http://java.sun.com/products/archive/j2se/5.0_15/index.html. You do NOT have to uninstall your existing copy of JRE 1.6 since both versions can be installed at the same time. However, if you have JRE 1.5 and JRE 1.6 installed, please make sure you start ABC/HR with JRE 1.5 manually. Since the working directory has to be the location where abchr.jar is stored, the best thing you can do is to open a command prompt window, navigate to the location where you stored abchr.jar and the rest of the files and then call "C:\Program Files\Java\jre1.5.0_15\bin\java.exe -jar abchr.jar". Of course, this is only an example which you have to adapt according to where java.exe is stored on your PC.

Note: As Hydrogenaudio Forums members and I found out, there is a bug in the iTunes MP3 encoder that is present in all iTunes versions since 5.0.1.4 (including the tested version 8.0.1.11). The problem manifests itself only on multi-core CPUs (both PCs and Macs) and leads to a significantly lower bitrate of the encoded samples. Apple engineers confirmed our findings and assured me that the next iTunes release will contain a fix for this problem. Please notice that the only functioning work-around is to disable all but one core in your computer's BIOS or to start Windows with the /numproc=1 parameter. Setting the iTunes process affinity to one core does NOT help.

Happy testing!
  • Last Edit: 19 November, 2008, 08:19:08 AM by Sebastian Mares

  • Raiden
  • [*][*][*]
  • Developer
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #1
I've got problems with "Save test results" and "Save session". ABC/HR locks up and the only way I can close it is using the task manager. JDK 1.6.0_10 is installed.

  • rednyrg721
  • [*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #2
I've got problems with "Save test results" and "Save session". ABC/HR locks up and the only way I can close it is using the task manager. JDK 1.6.0_10 is installed.

Can it be connected with Google Chrome being open? I think Chrome's sandbox is to blame. Closing Chrome helped in my case.

EDIT: No, now it works even with Chrome being open, so this is not the case.

EDIT2: WinXP SP2 + JRE Version 6 Update 10 (build 1.6.0_10-b33) here.
  • Last Edit: 17 October, 2008, 12:27:27 PM by rednyrg721

  • Synthetic Soul
  • [*][*][*][*][*]
  • Global Moderator
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #3
FYI I have also had various problems with ABC/HR locking up.

Java Control Panel says "Version 6 Update 7 (build 1.6.0_07-b06)" (most recent update).

I can load in a config, but if I then try to load in another, or use any other menu item it seems, it hangs.
I'm on a horse.

  • Alexxander
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #4
I've got problems with "Save test results" and "Save session". ABC/HR locks up and the only way I can close it is using the task manager. JDK 1.6.0_10 is installed.

Me too on VistaSP1 and jre1.6.0_04. I did 3 samples, one went OK, an other caused a hang when opened Save Results (and therefor had to do listening test again) and the third sample hang ABC/HR after the result file was written.

  • Alex B
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #5
I am seeing the same issue.

I have XP and Java 6 update 7 (build 1.6.0_07-b06). It's updater does not offer to install "update 10".

When I create a new unencrypted session the save functions work normally.

  • stigc
  • [*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #6
Same problem here. Windows Xp and 1.6.0_10-rc-b28.

Update 10 is only a release candidate.

  • halb27
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #7
Same for me (Windows XP).
lame3995n -Q0.5

  • shakey_snake
  • [*][*][*][*][*]
  • FB2K Moderator
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #8
Can I post the news of this iTunes encoder bug on a news site like neowin?

Sounds quite embarrassing.
elevatorladylevitateme

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #9
shakey_snake, you can post it as the message is public anyways, but please consider that the MP3 developers at Apple were very cooperative and confirmed / fixed the bug within 24 hours. The next iTunes version should have the fix included.

As for the ABC/HR problems - I am afraid that there isn't much I can do except to contact schnofler once more, but since he didn't reply to my last PMs, I don't know if he can help or if he's too busy whatsoever. I can also confirm the problem but it doesn't happen all the time.

The problem seems to be caused by JRE 1.6. JRE 1.5 works fine here.
  • Last Edit: 18 October, 2008, 06:06:06 AM by Sebastian Mares

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #10
I just recompiled ABC/HR using the latest JDK, JDOM and JGoodies but the problem persists. With the "old" version included in the test, I can open the configuration files but cannot save them and with my compile, I cannot open the configuration files so I created new ones, I can save them, but once I try to open a new configuration, it hangs.

So yeah, unless schnofler comes up with a great idea, please use JRE 1.5.
  • Last Edit: 18 October, 2008, 06:19:54 PM by Sebastian Mares

  • westgroveg
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #11
Although it's great that the encoders can adapt to certain flaws by raising the bit rate, and it's a clear advantage when seeking transparency in the best listening conditions, I think encoders that can maintain quality and also keep a more constant bit rate should be noted in a 128kbps test, because they are superior, if I'm aiming for 128kbps and I get 200+ I switch encoder.

Hehe just curious, you didn't list the encoders by any chance in your opinion of best to worst did you?
  • Last Edit: 19 October, 2008, 05:10:08 PM by westgroveg

  • [JAZ]
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #12
I think encoders that can maintain quality and also keep a more constant bit rate should be noted in a 128kbps test, because they are superior, if I'm aiming for 128kbps and I get 200+ I switch encoder.


If one wants, needs, or things that wants or needs an ABR encoder, there are options out there for this task. This test is not about this, like you knew, right?


Also, if a setting does continually use a (much?) higher than expected bitrate, you have two options: Choose a lower one, or choose a different method/encoder.

That is not the case here, as you also knew, right?

  • westgroveg
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #13

I think encoders that can maintain quality and also keep a more constant bit rate should be noted in a 128kbps test, because they are superior, if I'm aiming for 128kbps and I get 200+ I switch encoder.

Like you knew, right?

No idea what you mean by this jerk off.

Quote
If one wants, needs, or things that wants or needs an ABR encoder, there are options out there for this task.


Some encoders don't allow ABR.

Quote
if a setting does continually use a (much?) higher than expected bitrate, you have two options: Choose a lower one, or choose a different method/encoder.


If 70% of your files are at the target bit rate but the other 30% use 40% or so higher bit rate you can't just use a lower VBR parameter, it's impractical to be switching back and forth VBR parameters, having to change encoders would prove my point


Moderators: Please change "[JAZ]" his user name breaks the forum system for quoting "[" "]" should be illegal.
  • Last Edit: 19 October, 2008, 08:32:33 PM by westgroveg

  • Ron Jones
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #14
Easy westgrove. Let's not start the name calling on page one.

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #15
westgroveg, the settings and encoders chosen for this test are based on a large number of samples and they result in ~128 kbps. As others pointed out, if your goal is to reach a target bitrate for each sample individually rather than a quality level, use ABR or even CBR.

  • Ron Jones
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #16
I went ahead and consolidated all the samples into a zip file, as having to download and extract multiple files is something of an annoyance. Feel free to download that here.

I can mirror the original sample files as well if you'd wish, Sebastian.

  • [JAZ]
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #17
No idea what you mean by this jerk off.

I was implying that you should know this test is about VBR encoders, not about ABR encoders.

Some encoders don't allow ABR.

And do they allow VBR? I mean: it probably is of no concern for this test.

If 70% of your files are at the target bit rate but the other 30% use 40% or so higher bit rate you can't just use a lower VBR parameter, it's impractical to be switching back and forth VBR parameters, having to change encoders would prove my point

You didn't get it. There are easy to encode files, and difficult to encode files. If your collection has mostly difficult to encode content, you would set yourself to a lower setting and be fine (*if your main concern is bitrate*). Else, you would have to accept that they are difficult to encode.

Again, the goal of VBR is to keep the quality constant, not the bitrate. If you need an ABR setting, use an ABR setting. We are evaluating VBR settings/encoders in this test.

Moderators: Please change "[JAZ]" his user name breaks the forum system for quoting "[" "]" should be illegal.

Isn't that a bit harsh on your side? I am a user of this site since 2002. I've been using this nickname since 1999 (or even 1998, can't remember). I wouldn't mind changing it to something else (Since I don't use it any longer on new places), but directly asking moderators to change it on my behalf because it annoys you when quoting me... It's not exactly keen, you know.
Also, recently this problem was brought up, and it is fixed in a more recent version of this board software, but the administrators won't update it yet (since it has many custom changes that have to be adapted)
  • Last Edit: 20 October, 2008, 01:24:57 PM by [JAZ]

  • Bodhi
  • [*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #18
Hi,

Why V5.7?
I've read it but I can't remember where nor what it said...

Nice work Sebastian et al
  • Last Edit: 20 October, 2008, 01:33:14 PM by Bodhi

  • saratoga
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #19


I think encoders that can maintain quality and also keep a more constant bit rate should be noted in a 128kbps test, because they are superior, if I'm aiming for 128kbps and I get 200+ I switch encoder.

Like you knew, right?

No idea what you mean by this jerk off.


Maybe if you didn't flip the order of all the sentences around you wouldn't have so much trouble reading them?

Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #20
Hi,

Why V5.7?
I've read it but I can't remember where nor what it said...

Nice work Sebastian et al


LAME 3.98 presets use a slightly higher bitrate compared to 3.97.

  • westgroveg
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #21



I think encoders that can maintain quality and also keep a more constant bit rate should be noted in a 128kbps test, because they are superior, if I'm aiming for 128kbps and I get 200+ I switch encoder.

Like you knew, right?

No idea what you mean by this jerk off.


Maybe if you didn't flip the order of all the sentences around you wouldn't have so much trouble reading them?

I didn't flip the order the sentence Micky, I left irrelevant information out

[JAZ] didn't mean to go a lil' crazy at you, the thing about your user name, it's just that people can't quote you without manually editing code, but the mods don't seem to mind, truth is the forum system shouldn't allow any user to system characters in their name.

  • Zarggg
  • [*][*][*][*][*]
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #22
Did you not read the post where we said that this exact issue has been addressed in a more recent version of this forum software? Please don't blame the "system" for this.

Back on-topic: I think I'll sit this test out because it's painfully obvious that I need more ABX training. I can only tell a difference on one sample, which is most likely the low anchor. Can't wait to read the results, though!
  • Last Edit: 21 October, 2008, 12:45:10 PM by Zarggg

  • Neasden
  • [*][*][*]
  • Banned
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #23
Sebastian, keep the good work.
A listening test like this is so huge to prepare and execute, I even faint when thinking of it.
Looking forward to see the results.
I wonder if LAME 3.98.2 will ever be threatened as the current recommended encoder!

Note: Unfortunately my ears will not reach the category of "gold" until I fix my Class II malloclusion, at least more 1 1/2 year to go (Currently a cross-bite is disturbing the eustachian tube wich disturbs hearing). So sorry for not participating in this test!
  • Last Edit: 23 October, 2008, 02:41:15 PM by Neasden

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Public MP3 Listening Test @ 128 kbps - CLOSED
Reply #24
Back on-topic: I think I'll sit this test out because it's painfully obvious that I need more ABX training. I can only tell a difference on one sample, which is most likely the low anchor.

Did you use headphones? If yes, what model?