Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat 128 kbps Listening Test (Read 275178 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat 128 kbps Listening Test

Greetings!

I am planning to start a multiformat listening test at 128 kbps on November, 30th which should end on December, 11th. The test start can be postponed if necessary.

Edit: The test has been postponed: Stard date is December 5th and end date is December 25th if everything goes well.

Edit: The discussion about codecs is now over.

The following codecs are going to be included:
  • iTunes / QuickTime AAC
  • Nero Digital 4.2.1.0
  • LAME 3.97
  • Ogg Vorbis, AoTuV 4.5
  • WMA Professional 9.1
  • Shine as low anchor
I'm open to suggestions regarding the settings to use. The only thing that's for sure is iTunes / QT AAC which will be used at 128 kbps VBR.
I'd like to ask for Garf's or Ivan's opinion regarding Nero Digital (CBR or VBR, etc.).
As for the rest of the formats, suggestions from the community are welcome, too.

I have some samples in mind already and we'll start discussing them soon. While at it, would you guys prefer 18 or 12 samples?

Regards,
Sebastian

Multiformat 128 kbps Listening Test

Reply #1
IMHO
1. MusePack - on the last test it performed very well at that bitrate.
2. WMA - lot of people use it - it would be good to prove WMA is not the best codec as lot of people in the world think.
So my vote for MusePack and WMA.
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

 

Multiformat 128 kbps Listening Test

Reply #2
I'm with de Mon.

Multiformat 128 kbps Listening Test

Reply #3
wma standard and pro

Multiformat 128 kbps Listening Test

Reply #4
Hi. Thanks for your efforts in organizing the test.

I would suggest skipping musepack as well for the reasons already mentioned and also because it is rarely used at this bitrate.

WMA standard would be the low anchor here I think.If WMA Pro is finding hardware support these days it might be useful to have it blind tested as well.

It breaks my open source loving heart that FAAC is not more actively developed. If it has not changed significantly then testing is probably not warranted at this time.

$.02

Multiformat 128 kbps Listening Test

Reply #5
Quote
There is room for two other competitors or one competitor and a low anchor. Maybe MusePack and some version of WMA (Professional or Standard)?


There should be one WMA at least, otherwise the retards populating Slashdot will claim it wasn't tested because we are afraid our beloved codecs would be owned badly by it.

I wouldn't feature MPC. But that's what you should expect from an MPC hater like me.

Quote
While at it, would you guys prefer 18 or 12 samples?[a href="index.php?act=findpost&pid=341355"][{POST_SNAPBACK}][/a]


18. Otherwise a well know person around here will use the few samples as an excuse for his format of choice not winning, in case it loses (and if it wins, he'll stay quiet)...

Multiformat 128 kbps Listening Test

Reply #6
Quote
WMA standard would be the low anchor here I think.
[a href="index.php?act=findpost&pid=341373"][{POST_SNAPBACK}][/a]


For the low anchor, I suggest using something that sounds bad for sure, like l3enc.

Multiformat 128 kbps Listening Test

Reply #7
Quote
Quote
WMA standard would be the low anchor here I think.
[a href="index.php?act=findpost&pid=341373"][{POST_SNAPBACK}][/a]


For the low anchor, I suggest using something that sounds bad for sure, like l3enc.
[a href="index.php?act=findpost&pid=341388"][{POST_SNAPBACK}][/a]


Atrac?

Multiformat 128 kbps Listening Test

Reply #8
ATRAC3 has certainly improved since SonicStage 2.0 (especially in 2.2 I think) but would still be the low anchor. Probably more interesting than including another MPEG layer 3 codec like l3enc. So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.3.

New Nero AAC LC/HE Encoder is 4.2, not 3.0...

Multiformat 128 kbps Listening Test

Reply #9
Quote
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]



If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRAC superiority which is quite widespread in other communities.
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

Multiformat 128 kbps Listening Test

Reply #10
Quote
Quote
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]



If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRACK superiority which is quite widespread in other communities.
[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]


Maybe. The last time the atrac community just attacked the testing methodology used.

Die Hards........

Multiformat 128 kbps Listening Test

Reply #11
Quote
Quote
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]



If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRACK superiority which is quite widespread in other communities.
[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]

This sounds like a good idea, and at the bitrate that the test will be done, the results should be very interesting.
we was young an' full of beans

Multiformat 128 kbps Listening Test

Reply #12
Quote
If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRAC superiority which is quite widespread in other communities.[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]


Didn't we do it already more than an year ago?

Testing ATRAC3 is beating a dead horse, IMO, even if it improved since SS 2.0. I doubt it got much better (Sony has other things to worry about, like DRMing their format to the limit and spreading rootkits throughout the world), it's forcedly DRMd - DRM is not an option like in WMA - and even though it was bad, it wasn't bad enough to act like an anchor. The anchor is there to avoid people ranking samples too low just because there isn't anything worse, even though said samples are prefectly acceptable otherwise.


IMO, good anchors would be old or weird encoders from RRW (like l3dec, MBaacenc, QDesign, AUPECg2), or a lowpass.

Multiformat 128 kbps Listening Test

Reply #13
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.

Quote
... (Sony has other things to worry about, like DRMing their format to the limit and spreading rootkits throughout the world) ...


The name was Plex The Ripper, not Jack The Ripper

Multiformat 128 kbps Listening Test

Reply #14
Quote
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.
[a href="index.php?act=findpost&pid=341413"][{POST_SNAPBACK}][/a]


I agree with that.

Multiformat 128 kbps Listening Test

Reply #15
I don't - retards from Slashdot (which rjamorin mentioned) will claim that Pro isn't included because of its superiority (and that it will Soon™ be supported by portables anyway...)

Multiformat 128 kbps Listening Test

Reply #16
Quote
since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.[a href="index.php?act=findpost&pid=341413"][{POST_SNAPBACK}][/a]


Erm... you could use pretty much the same argument for MPC. And that didn't keep it from being tested on my older tests.

Quote
I don't - retards from Slashdot (which rjamorin mentioned) will claim that Pro isn't included because of its superiority (and that it will Soon™ be supported by portables anyway...)


Indeed, I agree. F*** SlashDot. We're not testing for the clueless hordes populating that place.


I just thought of another reason not to use ATRAC3: Sebastian is a nice guy. He doesn't deserve to feel the infinite pain involved with installing and using SonicStage 

Multiformat 128 kbps Listening Test

Reply #17
By the way, has WMA Pro ever really been compared in a listening test like this?  If so, could I be provided with a link?  If not, could it be included in this test?  I'm curious to see how good it really is.

Multiformat 128 kbps Listening Test

Reply #18
Quote
If so, could I be provided with a link?[{POST_SNAPBACK}][/a]


[a href="http://www.rjamorim.com/test/128extension/results.html]http://www.rjamorim.com/test/128extension/results.html[/url]

Quite outdated though.

Multiformat 128 kbps Listening Test

Reply #19
L3enc would be an interesting low anchor, as it would allow to check evolution of "state of the art" encoders. However, it might be a little too good for a low anchor.

Multiformat 128 kbps Listening Test

Reply #20
Quote
However, it might be a little too good for a low anchor.[{POST_SNAPBACK}][/a]


Even 1.0?

[a href="http://www.rjamorim.com/rrw/l3enc.html]http://www.rjamorim.com/rrw/l3enc.html[/url]

Multiformat 128 kbps Listening Test

Reply #21
A low anchor would be nice, but personally I would probably prefer WMA Standard and WMA Pro.

I think ATRAC would be a waste of time because:
1.  It's already pretty well known by most people who actually do any real research on the topic that it's inferior to just about all modern alternatives.
2.  It's fading in popularity, and will continue to do so quite rapidly in the near future.
3.  The people that feel that ATRAC are superior seem quite resistant to persuasion by evidence.

So, ATRAC isn't really a worthwhile choice.

I don't think MPC is really a good choice either, because:
1.  It isn't really used at this bitrate (not explicitly that is, although VBR may sometimes cause it to average out at this bitrate), even if it can perform well.
2.  I don't know if it's fading in popularity, but it's not really growing either.  In terms of relevancy for the results, I think MPC would be less useful for a large audience.
3.  It hasn't seen any serious development in a long time, whereas other alternatives might have, and would be more interesting from a results perspective -- MPC's performance is pretty well established either way, so it could serve possibly as some sort of anchor, but beyond that it's not going to provide much new data.

I understand that a low anchor is important, especially in order to provide a reference point illustrating just how much some of the codecs in this test have improved either upon recent iterations, or older/inferior codec designs in general.

Maybe the best choice would be to add WMA Standard and WMA Pro, AND a low anchor.  That has the danger of making the test too tedious and complex, but maybe it is still doable...

Multiformat 128 kbps Listening Test

Reply #22
I don't understand why everyone insist to inlude WMA Standard instead of the Pro version... You don't cripple other formats the same way, but wma everybody wants WMA to be bad and just to make sure it is, only test the ancient WMA Std. Ask Woodinville which version to use, and I'm pretty sure he suggest using Pro instead of Std.

Multiformat 128 kbps Listening Test

Reply #23
I definitely would like to see both WMA Standard and Pro included. I think testing against a lower anchor can be done in an another test. You can say whatever you want about WMA, but it at least had gapless support way before some of the other codecs had it (and some still don't have it, yes, I'm looking at you Apple). And I'm very much interested in the differences between WMA Standard and Pro.

Multiformat 128 kbps Listening Test

Reply #24
Quote
QUOTE(JeanLuc @ Nov 13 2005, 12:23 AM)
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.
*



I agree with that.


I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results

Unlike iTunes, Nero, LAME...they are all improved

WMA Pro vbr 128 kbps 2-pass is not in the previous test, it is interesting and more meaningful to add WMA Pro this time