Skip to main content ↑ ↓
Topic: MP3 at 128kbps public listening test (Read 41306 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 at 128kbps public listening test

Reply #25 –
Quote
Opinions? Maybe make the test all CBR?

Considering a lot of people still use the fact that the first AAC test was CBR only as an argument against it, I'd say that's a bad idea. IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.

Quote
Yes, I'm quite sure it is SoundJam.

If SoundJam indeed used an alternative MP3 encoder, then it's quite likely that's what iTunes uses since the encoding dialog pretty much hasn't changed since SoundJam. Plus, iTunes 1.0 was basically a scaled down version of SoundJam with a different GUI. It'd be very interesting to know how it performs - I've never seen any tests of it

How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim!

MP3 at 128kbps public listening test

Reply #26 –
what about using mp3pro@64 as anchor for the mp3 test?
i think it would fit as an anchor qualitywise and the results would also be really interesting...


for the aac test i wouldnt drop sorenson (it wasnt that bad at all)


and i would vote for a clip with sound and human voice (as from movies)
I know, that I know nothing (Socrates)

MP3 at 128kbps public listening test

Reply #27 –
My opinion:

*using Xing as low anchor seems to be a good idea, and more usefull than dist10.
*comparing the 3 main encoding engines used nowadays is a good thing (Fhg, ITunes, Lame)
*Including the old (but still widely used because of the Radium hack) AudioActive is a good thing.

Now, we could ask why only using current FhG engines in vbr only. The same (reversed) question could be asked about Lame, although I also think that it is very likely that cbr/abr would be better than vbr in those bitrates.

In order not to increase too much the number of candidates, I'd suggest an all cbr test. I know that this would penalize Lame (compared to abr), but otherwise there is the risk of endless debates.

Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.

Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?

MP3 at 128kbps public listening test

Reply #28 –
Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating"
@Gabriel... I think , your question about LAME versions is interesting. At least there must be a difference between 3.90.3 and latest stable release (otherwise I don'tknow what you did all this time)... maybe, we could include both? And if so, what settings would you suggest? alt-preset CBR?

Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.

MP3 at 128kbps public listening test

Reply #29 –
i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr)

imho the bad codecs should be punished, not the good ones 
I know, that I know nothing (Socrates)

MP3 at 128kbps public listening test

Reply #30 –
Quote
And if so, what settings would you suggest? alt-preset CBR?

For a cbr encoding that would be

*3.90.3: --alt-preset cbr 128
*3.93.1: --preset cbr 128
*3.94: -b 128

MP3 at 128kbps public listening test

Reply #31 –
I can agree on skipping VBR since it is difficult to use that in a fair comparison with other encoders, but I don't like the idea of skipping ABR for Lame. I don't that would implicate an unfair comparison.

If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.

BTW, thanks rjamorim for doing all these tests. Really great work. I am really looking forward to the march test, to see how MP3 compares to WMA std.

MP3 at 128kbps public listening test

Reply #32 –
Quote
I am not quite sure I understand you correctly, but if you are talking about the fact that you can't find any Lame, FhG or dist10 on any warez sites, the explanation is quite straightforward. Warez sites contain cracked and illegal software, but both Lame and FhG can be obtained legally for free, so why would anyone publish them on a warez site?

He was talking about the three official WMP codecs: CyberLink, InterVideo and (whatever was the third one).

Quote
How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim!


If Ivan managed to fix EnolaGay, it might well surpass some of the MP3 encoders

Still, I'm pretty fond of the idea of ruining Xing's reputation once and for all

Quote
Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.


Well, if most people agree on making this test CBR only, I might well later conduce a VBR FhG vs. ABR Lame test. I'm working for you guys, so it's your decision the way the test will go.

Quote
Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?


To be very sincere, I didn't make up my mind yet on that subject. There are reasons to use the HA version - the very same reasons why HA recommends that fork instead of latest official release.

And then again, there are several reasons to test latest official release: check how quality is progressing, help the developers, etc.

So, which one you recommend, as a lame developer?

Quote
Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating"


I gotta admit that appeals a lot to me, particularly :B

Quote
maybe, we could include both?


We are at a point where, if you suggest another encoder, please suggest what should be replaced. 6 codecs is the maximum, period. Even at 64kbps testing 8 codecs was a mistake and I don't want it to happen again.

Quote
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.


I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

Quote
If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.


OK. What FhG encoder, specifically?

Thanks a HUGE bunch for all your comments

Best regards;

Roberto.

MP3 at 128kbps public listening test

Reply #33 –
Quote
OK. What FhG encoder, specifically?

To be honest I didn't consider that, really. I just thought it would be more interesting to see the possible progress of Lame rather than three FhG. But maybe Gabriel can say something about what kind of improvements he expects for the 128 kbps settings in 3.94, before considering having both any further?

OK, here is my very personal wishlist for what I would like to see compared:
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

If I were to add something to this list it would be:
1. Gogo
2. Lame 3.94

These are all just suggestions...

MP3 at 128kbps public listening test

Reply #34 –
Quote
There would be 5 codecs and an anchor:

-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.

MP3 at 128kbps public listening test

Reply #35 –
Quote
So, which one you recommend, as a lame developer?

My own choice would be to test the latest beta or release (of course alpha are excluded).
In my mind, this is the more usefull, as it is representative of the current state of the codec, and could be helpfull in order to fix potential issues.

However, I can already hear those numerous people that will argue that the "HA recommended version" should be used, and nothing else.
(please note that in order to enforce this argument, it means that the "recommended version" has to be tested against latest one)

The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.

MP3 at 128kbps public listening test

Reply #36 –
Quote
i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.


This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

MP3 at 128kbps public listening test

Reply #37 –
Quote
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

MP3 at 128kbps public listening test

Reply #38 –
Since we're targeting how well MP3 codecs perform at 128kbps then it makes sense to use CBR for all encoding tests. Since CBR is what most average non-HA users encode with. Giving LAME use of ABR and forcing the rest to use CBR seems an unfair test.

MP3 at 128kbps public listening test

Reply #39 –
Quote
Quote
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

The problem with APS testing is that 1) you would have fewer people testing (and then rjamorim would run into the same problem that I did with MY listening test) and 2) you would have to have highly suggested/optimized settings for all the other codecs, which is simply impossible.  Like what's the APS equivalent on the FhG encoders?

Now if you'd like, I can arrange a personal test for you so that you see if you can hear the differences between APS and anything else, but it's not really good for a public test Even at 128, there will be people that aren't able to ABX it from the original.

MP3 at 128kbps public listening test

Reply #40 –
Quote
The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.


Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version. THEN... when all three tests are done, and we (hopefully) have LAME 3.94 it would be the right time for testing it against 3.93.1 and 3.90.3. This test has been needed for a long time and if I knew how to conduct it, I would make it a top priority.

Quote
IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.


Indeed! Let's not cripple good codecs.

Finally, I can see why XING would make an interesting anchor but considering the ones who will actually take time examining the results, I feel HE-AAC would be more interesting. Sure, it would be somewhat misplaced but then again it would provide useful information. In my opinion XING wouldn't.
 

MP3 at 128kbps public listening test

Reply #41 –
Quote
Quote
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.

I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

I'm still a bit sore from the results (or lack thereof) from my last listening test, otherwise, I'd be happy to volunteer.  Besides, you'll get that accusation in your 3rd test anyways, when either Nero or Quicktime (the expected winners of the AAC test, IMO) does nominally better than Vorbis (my expectations for your 128kbps extension).

It's a test that ought to be done, dammit. 

MP3 at 128kbps public listening test

Reply #42 –
Quote
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors", Lame and FhG, and that would also mean allowing FhG to use VBR. Using only CBR doesn't make a lot of sense in my opinion. Even if it is the most common newbie setting, so what? We already have Xing in the test, which should suffice to prove that the typical "newbie settings" are not the optimum. Limiting all the other codecs to CBR would make the test very one-sided.
I think one of the main goals of an MP3 128kbit/s test should be to find the best codec at this bitrate. We will never find this out if we limit the codecs artificially. Additionally, using CBR would penalize the MP3 format in the upcoming multiformat test (if the winner of the MP3 test is used) because it doesn't give an accurate image of what the format is capable of at that bitrate.
As for the sixth codec, I would suggest Lame 3.94 (or 3.93.1 if 3.94 is not ready by then). I guess a good result of a current Lame version would be a solid basis for a new HA recommendation (as "lack of proper testing" seems to be the main argument against recommending the newer Lame versions).

MP3 at 128kbps public listening test

Reply #43 –
Quote
I was under the impression that FhG Current was the slow codec. In this case, I'll do current and legacy slow. Thanks for the tip. I'll update my first post now.

You would know for sure which codec is the slow one.  It's super slow.

Quote
Quote
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).


Yes, I'm quite sure it is SoundJam.


Yep.  It is apparently quite good.

Quote
Quote
There's also the FhG slow codec (high quality codec within fastencc.exe)


Huh? So fastencc isn't the fast codec?


It contains both the fast and slow codecs.  The fast codec within fastencc.exe is buggy (loss of stereo separation).  The slow one (-hq) works fine.

ff123

MP3 at 128kbps public listening test

Reply #44 –
Quote
well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

MP3 at 128kbps public listening test

Reply #45 –
Quote
Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version.

I hope that I wasn't misunderstood. My own choice would be to use 3.94, but I pointed that other people might have different opinions.

MP3 at 128kbps public listening test

Reply #46 –
Quote
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?

MP3 at 128kbps public listening test

Reply #47 –
Quote
Quote
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?

I'm in favor of this. Making some VBR or ABR is rather unfair, IMO, because in my experience, most VBR and ABR modes tend to have, on average, a bitrate that is higher or lower than 128kbps.

From what I learned at school (in science ), for best results in an experiment, only one variable should be modified. In this case, that would be the encoder.  By using VBR and ABR, the bitrate is also modified, as well as the encoder's specific implementation of VBR/ABR. For example, in my experience, LAME ABR 128 consistently produces ABRs below 128.
Happiness - The agreeable sensation of contemplating the misery of others.

MP3 at 128kbps public listening test

Reply #48 –
Quote
i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr) imho the bad codecs should be punished, not the good ones

I TOTALLY AGREE!
//From the barren lands of the Northsmen

MP3 at 128kbps public listening test

Reply #49 –
Quote
Quote
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors" Lame and FhG and that would also mean allowing FhG to use VBR.

Add another vote for this selection! Two FhGs, one "default" the other "best quality", seem enough.


Quote
Quote
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?

You have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.
 
SimplePortal 1.0.0 RC1 © 2008-2019