Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat@128kbps listening test - FINISHED (Read 179194 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat@128kbps listening test - FINISHED

Hello.

I'd like to announce the results of the Multiformat at 128kbps listening test

Vorbis aoTuV is tied to Musepack at first place, Lame MP3 is tied to iTunes AAC at second place, WMA Standard is in third place and Atrac3 gets last place.

The results page is here:
http://www.rjamorim.com/test/multiformat128/results.html

For those in a hurry, here are the zoomed overall results:


Big thanks to everyone that helped and participated.

Best regards;
Roberto.

Multiformat@128kbps listening test - FINISHED

Reply #1 –
Now that was a surprise... Lame as good as AAC??? Anyone expected that?

Multiformat@128kbps listening test - FINISHED

Reply #2 –
Vorbis (aoTuV) and MPC tied for first place.  LAME and iTunes tied for second.  Then WMA-S in third, and ATRAC3 at the back of the pack.

Funny that there was no real consistency this time across music types with the formats tested.  Tends to oppose theories about certain formats excelling with certain types of music.  At least among these samples.

Multiformat@128kbps listening test - FINISHED

Reply #3 –
What the!  ... surprised!
myspace.com/borgei - last.fm/user/borgei

Multiformat@128kbps listening test - FINISHED

Reply #4 –
Whoa, look at aoTuV!! 

It is now as good as MPC.  Very good work, Aoyumi.  Vorbis is now back in the spotlight.

Multiformat@128kbps listening test - FINISHED

Reply #5 –
I believed Musepack would win the test especially such bitrate range(-q4.15).  Anyway,  it's very interesting result, good job Roberto and all participants.

Multiformat@128kbps listening test - FINISHED

Reply #6 –
Surprisingly, MPC 1.14 (same tested last year) isn't tied anymore with iTunes AAC, but “win”.
ATRAC3 (minidisc) is obviously a poor encoding solution.
aoTuV is without doubt a great step behind for Vorbis!

Multiformat@128kbps listening test - FINISHED

Reply #8 –
Very good results by aoTuV. It seems all the others have a new target for 128kbps quality now.
One thing which this test shows is that VBR coding (aoTuV, MPC) is definitely way to go for 128kbps, and with good enough VBR tweaking it's certainly possible to be clearly better than CBR (iTunes).
Juha Laaksonheimo

Multiformat@128kbps listening test - FINISHED

Reply #9 –
Quote
One thing which this test shows is that VBR coding (AoTuV, MPC) is definitely way to go for 128kbps, and with good enough VBR tweaking it's certainly possible to be clearly better than CBR (iTunes).

Yes. That is also true for Lame. With a very good VBR implementation, it got close to the best AAC implementation at that bitrate.

Let's hope Apple implements VBR in their codec, and Ahead improves their implementation considerably.

Multiformat@128kbps listening test - FINISHED

Reply #10 –
Quote
I believed Musepack would win the test especially such bitrate range(-q4.15).

I thought so too.

I anticipated a tie between MPC and QT-AAC, then Vorbis in second place, then LAME, then WMA-S and ATRAC at the back.  Vorbis and QT-AAC both surprised me.

Multiformat@128kbps listening test - FINISHED

Reply #11 –
My browsers(Firefox, MSIE) don't show test comments correctly. Also, the title of this page seems to be wrong.

Multiformat@128kbps listening test - FINISHED

Reply #12 –
Quote
My browsers(Firefox, MSIE) don't show test comments correctly.

It's XML. IE should show something like this:
http://esc17.midphase.com/~calmerc/screenshots/screen-1.jpg

XML is worse for readability but easier to be parsed. That's why Schnofler switched to XML results in recent versions of ABC/HR Java.

Quote
Also, the title of this page seems to be wrong.


Fixed. Thanks for reporting.

Multiformat@128kbps listening test - FINISHED

Reply #13 –
Quote
XML is worse for readability but easier to be parsed. That's why Schnofler switched to XML results in recent versions of ABC/HR Java.

I expected something like in raw *.txt format. Thanks for clarification.

Multiformat@128kbps listening test - FINISHED

Reply #14 –
Quote
I expected something like in raw *.txt format. Thanks for clarification.

Schnofler already has a converter from xml -> txt in ABC/HR. But it only works for encrypted results ATM. Hopefully he'll add support for already decrypted results.

Multiformat@128kbps listening test - FINISHED

Reply #15 –
Wow, what really impresses me is that I don't think there was one sample where the vorbis encoder did poorly.  This is a little shocking after last test.  Excellent work aoTuV!

Multiformat@128kbps listening test - FINISHED

Reply #16 –
Surprise surprise!
I hope this'll give vorbis development a new boost.

Multiformat@128kbps listening test - FINISHED

Reply #17 –
Oh! Joy!

Multiformat@128kbps listening test - FINISHED

Reply #18 –
Quote
Oh! Joy!



I'm happy my test is spreading happiness.

Multiformat@128kbps listening test - FINISHED

Reply #19 –
woow, now thats what i not expected

- vorbis aotuv: vorbis is back, and i am proud to have helped finding out what vorbis encoder should be used
- mpc vs aac: funny that mpc was that better than itunes (with a only 0.15 higher setting than in the last test)
- wma9: lol, worse than mp3! (and i even wonder that it got rated that high, even at 128 it had this metallic sound sometimes) -> go away m$
- atrac3: even worse than wma9 -> go away sony

and if you take this test as a comparison between some online music stores (itunes vs. wma9 based ones vs. sonys new store) itunes clearly comes out as the winner, leaving wma9 behind by far!
I know, that I know nothing (Socrates)

Multiformat@128kbps listening test - FINISHED

Reply #20 –
I see there is a very small margin between mpc and aoTuv, how would aoTuv react
in higher bitrates.?

Multiformat@128kbps listening test - FINISHED

Reply #21 –
Very interesting results ...

I think it could be an interesting addition to show the bitrate for each encoder in the specific diagrams for each sample ...
The name was Plex The Ripper, not Jack The Ripper

Multiformat@128kbps listening test - FINISHED

Reply #22 –
The more I think of it the more impressed I am with the performance of LAME. Very good work Gabriel (and consider changing -V 5 default --athaa-sensitivity).
"To understand me, you'll have to swallow a world." Or maybe your words.

Multiformat@128kbps listening test - FINISHED

Reply #23 –
How many results were discarded because of ranked refs?

Multiformat@128kbps listening test - FINISHED

Reply #24 –
Quote
How many results were discarded because of ranked refs?

54

Mind you that I didn't discard results that ranked the reference but on that sample pair ABXd the samples to a pval of 0.05 or less.