HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: rjamorim on 2003-12-10 01:17:05

Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 01:17:05
Hello.

As I already mentioned here some times, I plan to conduce an MP3 @ 128kbps listening test in January.

The reason for the early discussion thread is that I'll travel for the holidays to a place with difficult internet connection. So, I'd rather have everything sorted out in advance because I can then start the test soon after I return.

So, here is what I am planning:

As usual, the test would run for 11 days, starting on a wednesday.

There would be 5 codecs and an anchor:

-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

[span style='font-size:8pt;line-height:100%']EDIT: Audition Legacy Slow -> Legacy Fast
Radium -> iTunes[/span]

Of course, these are open to debate and critics.

The sample suite would be the same as the 64kbps test. Again, if someone has some constructive criticism on it, speak out loud.

If things go as I plan, the test starts Jan 14th 2004 and ends Jan 25th.


Also, it's worth mentioning my future plans: in february, I plan to conduce another AAC@128kbps test, that would compare:

-Nero AAC
-Compaact
-Faac
-QuickTime
-NCTU-Faac (if they sort out their situation until then. Otherwise, I'll find something else)
-Anchor

And, in march, another multiformat test:

-AAC winner
-MP3 winner
-Vorbis GT3b2 or 1.0.1
-Musepack
-WMA standard
-Anchor

Please comment and give your ideas.

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: sthayashi on 2003-12-10 02:29:30
Roberto, I have a request for your 2nd test.  Include Ogg Vorbis in your AAC test.  Ogg Vorbis's direct competition in terms of format is AAC.  In your last test, Vorbis did not perform quite as well as the best AAC at the time.  Since Vorbis has not improved significantly since then (Garf's tunings notwithstanding, since they're not applicable at 128kbps), the results will not be as interesting.

However, seeing Ogg Vorbis compared to other AAC implementations will be more interesting (at least to me).  For example, to the end user, OggEnc is more like FAAC than Quicktime or Nero, and a comparison between the two should be most interesting to observe.

This is just my opinion though, and you're welcome to conduce your tests however you like.
Title: MP3 at 128kbps public listening test
Post by: saratoga on 2003-12-10 02:29:56
Quote
-Lame - --alt-preset 128
-FhG Audition Legacy Fast - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-Radium - 128kbps
-Xing 128kbps as anchor


Which of this is closest to iTunes? I'd sort of like to know how good its MP3 encoder is since a lot of people still use it.
Title: MP3 at 128kbps public listening test
Post by: elmar3rd on 2003-12-10 03:01:28
Well, I see the theoretical sense of a 128kbps-mp3-test regarding the upcoming tests.
But 1. can we gather enough participants and 2.  is 128kbps still a valid "border"?

What will we proove with these tests?  Is the 128kbps-range still that important to make three tests?

Let's talk about the targets first.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 03:04:19
Quote
Roberto, I have a request for your 2nd test.  Include Ogg Vorbis in your AAC test.  Ogg Vorbis's direct competition in terms of format is AAC.  In your last test, Vorbis did not perform quite as well as the best AAC at the time.  Since Vorbis has not improved significantly since then (Garf's tunings notwithstanding, since they're not applicable at 128kbps), the results will not be as interesting.

However, seeing Ogg Vorbis compared to other AAC implementations will be more interesting (at least to me).  For example, to the end user, OggEnc is more like FAAC than Quicktime or Nero, and a comparison between the two should be most interesting to observe.

This is just my opinion though, and you're welcome to conduce your tests however you like.

Well, I would gladly include vorbis. The problem is, I already have 6 codecs there (IMO, the maximum for a 128kbps test), and even if NCTU dropped, there is already another one in line.

Besides, calling it an AAC test would no longer make sense. (Although I'll probably have to consider another format for anchor anyway)

So, I probably won't include Vorbis in this test. But I would gladly help people interested in conducing such AAC vs. Vorbis test.

Quote
Which of this is closest to iTunes? I'd sort of like to know how good its MP3 encoder is since a lot of people still use it.


Veeery good point. I think it's worthing dropping one of the FhG codecs in favour of SoundJam/iTunes.

Any suggestion? Personally, I would drop Radium.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 03:18:16
Quote
Well, I see the theoretical sense of a 128kbps-mp3-test regarding the upcoming tests.
But 1. can we gather enough participants

I hope so

Keep in mind that MP3@128 is still what the majority of people consider CD quality. So, it is of interest, even if only to see what encoder is best at this mythical bitrate.

Quote
and 2.  is 128kbps still a valid "border"?


Anything above this is almost untestable due to near-transparency.
And below this, you start getting what people consider low bitrates (96kbps-)

Quote
What will we proove with these tests?


The best encoders at 128kbps.

You can't really extrapolate to higher bitrates, but since anything above that will flood my mailbox with ranked references, 128 will have to do.

Also, regarding AAC, it'll be a good measure to find out how codecs improved since the latest AAC test.

Quote
Is the 128kbps-range still that important to make three tests?


Do you suggest another range?

Quote
Let's talk about the targets first.


OK. My targets are the non-audiophile users (I.E, most people that don't visit to HA, and even some HA visitors) that are happy with MP3 at 128kbps but are interested in broadening their horizons, trying out the new batch of codecs that is being developed.

And my targets are the HA users that use medium bitrates for portable usage or simply because they admittedly don't have golden ears.

Also, one could say my targets are the format developers, that will probably be interested in finding out the flaws people detected, in order to improve their codecs. (Some won't give a damn, like Microsoft. But some probably will - Ahead, Apple, Xiph...)

So there you have my reasoning

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: CosmoKramer on 2003-12-10 03:47:44
What about --alt-preset cbr 128? I for one would like to see that one.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 04:09:07
Quote
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-10 04:44:44
Quote
-Lame - --alt-preset 128
-FhG Audition Legacy Fast - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-Radium - 128kbps
-Xing 128kbps as anchor

I believe that Audioactive and Radium are pretty much the same thing.

What's the difference between FhG Audition Legacy Fast and Audition Current?  It's my (wild-assed) guess that both are based on FastEnc (which is what FhG VBR has been based on in the past), and that the faster one should be dropped, if it's anything like the difference between the codecs in Cool Edit Pro 2.

I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).

There's also the FhG slow codec (high quality codec within fastencc.exe)

ff123
Title: MP3 at 128kbps public listening test
Post by: Cey on 2003-12-10 04:46:59
Quote
OK. My targets are the non-audiophile users (I.E, most people that don't visit to HA, and even some HA visitors) that are happy with MP3 at 128kbps but are interested in broadening their horizons, trying out the new batch of codecs that is being developed.

If you are targeting the 'average' non-HA user, then perhaps you should use some of the codecs *they* typically use.  That way you can show them how much better other codecs are.  If you use codecs they don't use, then they would have nothing to relate your results to their experience.

Such as MusicMatch (with several settings for the processing level), RealOne, iTunes, and so on.

Those three (or 5, if you count the three levels in MusicMatch) among the more common codecs that average people use.

(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft....  Although it would be interesting to find out the quality of those!)


And for the record, I do fall into the category of HA visitors who don't have "golden ears" except for the occasional song.  (Better sound card & better headphones would probably help me a lot...[grin])
Title: MP3 at 128kbps public listening test
Post by: Cey on 2003-12-10 04:50:18
Quote
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).

I think I remembering reading in the LAME mailing list archives that a developer for that mp3 encoder said that it was originally based on the original ISO, but that it was very very VERY heavily modified.  So much so that it is pretty much a new codec.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 04:57:14
Quote
I believe that Audioactive and Radium are pretty much the same thing.

Good to know. I'll drop Radium then, since latest Audioactive version is newer.

Quote
What's the difference between FhG Audition Legacy Fast and Audition Current?  It's my (wild-assed) guess that both are based on FastEnc (which is what FhG VBR has been based on in the past), and that the faster one should be dropped, if it's anything like the difference between the codecs in Cool Edit Pro 2.


I was under the impression that FhG Current was the slow codec. In this case, I'll do current and legacy slow. Thanks for the tip. I'll update my first post now.

Quote
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).


Yes, I'm quite sure it is SoundJam.

Quote
There's also the FhG slow codec (high quality codec within fastencc.exe)


Huh? So fastencc isn't the fast codec?

Anyway, fastencc is buggy and AFAIK this has been fixed on later releases of that library. Where can I find it? Audition? Musicmatch?

Quote
I think I remembering reading in the LAME mailing list archives that a developer for that mp3 encoder said that it was originally based on the original ISO, but that it was very very VERY heavily modified. So much so that it is pretty much a new codec.


Yeah, I'm pretty confident it's SoundJam. I'll replace Radium with iTunes then \
Title: MP3 at 128kbps public listening test
Post by: Todesengel on 2003-12-10 05:02:37
Someone tell me what is Anchor??? Is it new AAC encoder or, maybe, new format???
Title: MP3 at 128kbps public listening test
Post by: harashin on 2003-12-10 05:03:18
Ordinary people in Japan still prefers Gogo (2, 3)...
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 05:15:33
Quote
Such as MusicMatch (with several settings for the processing level), RealOne, iTunes, and so on.

Musicmatch = Audition, I would guess (both use latest FhG libs)
Real = Xing :-P
iTunes is already in

And I'll mention these associations at the results page.

Quote
(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft....  Although it would be interesting to find out the quality of those!)


Well, it's not like there's a huge amount of codecs in the market. You have Lame, FhG and dist10. iTunes uses a proprietary one, what else do we have?

I would guess these WMP codecs are all based on FhG.

Quote
And for the record, I do fall into the category of HA visitors who don't have "golden ears" except for the occasional song.  (Better sound card & better headphones would probably help me a lot...[grin])


From the results of my 128kbps multiformat test, transparency in the terms of the ITU (creators of teh ABC/HR test) - higher than 4 points - can easily be reached with modern codecs.

The reason of using 4 as the transparency level is that the ITU takes in consideration that their tests are done in a very controlled environment, equipment is top notch and the listeners are trained and extremely attentive - very far from the everyday situation.

Even considering my tests aren't done with the same level of profissionalism as the ITU ones and we cut some slack - say, .5 point more to reach transparency - one can say MPC reaches it and QuickTime gets pretty close.

(More about the ITU transparency border here (http://www.hydrogenaudio.org/forums/index.php?showtopic=13612&view=findpost&p=138333) and at ITU R.BS 1116-1 document)

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 05:18:19
Quote
Someone tell me what is Anchor??? Is it new AAC encoder or, maybe, new format???

Nono, anchor is a compressed stream encoded at low quality, or maybe with a lowpass. It is there to put the results into perspective and to avoid bad codecs getting too low rankings because there was nothing worse.

Quote
Ordinary people in Japan still prefers Gogo (2, 3)...


Well, ordinary people in the west still use Xing (AudioCatalyst, RealJukebox, RealOne...)

So, it's hard to please all markets
Title: MP3 at 128kbps public listening test
Post by: Cey on 2003-12-10 05:27:41
Quote
Quote
(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft....  Although it would be interesting to find out the quality of those!)


Well, it's not like there's a huge amount of codecs in the market. You have Lame, FhG and dist10. iTunes uses a proprietary one, what else do we have?

I would guess these WMP codecs are all based on FhG.

I don't know.  I've never heard anybody say what they are based on or how good they are.

I even got curious once and checked some warez sites just to see if I could find one and look through it with a hex editor (I never install warez... Way too much chance of virus or trojan) and I couldn't even find a copy of any of the three official mp3 codecs on the warez sites!

So I've kind of gotten the feeling that they are pretty rare.

I did read a comment here in HA from one user who had bought one and was very disappointed with it.  But that's about all I've heard.
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-10 05:36:41
I think LAME will perform better with ABR  @ 128 so you should use --alt-preset 128 to answer, does highest quality FhG out perform highest quality LAME @ 128?


I would like to see Xing with full weapons, Newest encoder, VBR, JS, etc.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 05:42:34
Quote
I would like to see Xing with full weapons, Newest encoder, VBR, JS, etc.

No problem.

I'll have to play around with Xing settings some first, though. For some strange reason, I never played with them much


(And yes, "newest weapon" would mean installing RealOne)
The things I do for you guys...
Title: MP3 at 128kbps public listening test
Post by: Cey on 2003-12-10 05:51:04
Quote
Quote
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?

[grimace]

I realize that many newbies do use CBR (I used to) but I think you need to draw the line somewhere, and that's a good place.

Maybe do one common encoder at CBR and VBR, just to give newbies an example of how much better the VBR is.

For a newbie friendly test like this, I don't see any point to ABR.  It should be VBR and one CBR for comparison.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 05:52:50
Lame VBR doesn't work well at bitrates smaller than ~160 :/

That's why I chose ABR.
Title: MP3 at 128kbps public listening test
Post by: harashin on 2003-12-10 06:17:09
Quote
Well, ordinary people in the west still use Xing (AudioCatalyst, RealJukebox, RealOne...)

So, it's hard to please all markets

Right. 

And Xing and Blade, we already know its quality in previous testing, are still popular here, too.
Title: MP3 at 128kbps public listening test
Post by: Cey on 2003-12-10 06:43:54
Quote
(And yes, "newest weapon" would mean installing RealOne)
The things I do for you guys...

The full 320kbps mp3 encoder is still linked on the RealOne page:

http://realone.real.com/ (http://realone.real.com/)

it goes to:

http://realone.real.com/r/url/?10442991173750 (http://realone.real.com/r/url/?10442991173750)

Which redirects to:

http://forms.real.com/real/jukebox/320k_re...l?src=r1central (http://forms.real.com/real/jukebox/320k_register.html?src=r1central)

I assume the one that comes with the free RealOne is probably limited below 128k, which I why I mention this...

I assume, of course, that the paid version of RealOne (which probably includes a full mp3 encoder) and the free downloadable version are the same encoder, rather than the possibility of the paid version including a decent encoder.

I mention that because it's still the same old RealJukebox link, which makes me wonder if they've updated it since the days of RJB.
Title: MP3 at 128kbps public listening test
Post by: magic75 on 2003-12-10 08:27:26
Some of my opinions:

I don't think all should be set to CBR. I do however vote for a mixture of typical defaults, like Xing & FhG @ 128 CBR, which would be what the typical non-HA user would get when using MusicMatch, Real or WMP. BTW, I am almost 100% sure that WMP uses FhG. Anyway, these "standard" encodings of a typical non-HA person would be interesting to compare with encodings done with more advanced codecs and settings, like lame -preset 128 and the FhG high quality settings, so that we get a comparision of how much better quality you would get by using these more advanced codecs/settings.

Personally I would really like to se Gogo in the test as well as a fast and free alternative (I have very old computer...) There are already three FhG codecs in the test, and personally I could drop one of them in favour of Gogo. But I guess the majority may very well be more interested in FhG so... And I guess fastenc is pretty fast as well...

To sum up: What I would like to see is a test that shows the newbie or non-HA person why dropping software like Musicmatch, Real or WMP can be a good idea.
Title: MP3 at 128kbps public listening test
Post by: magic75 on 2003-12-10 08:32:29
Quote
I even got curious once and checked some warez sites just to see if I could find one and look through it with a hex editor (I never install warez... Way too much chance of virus or trojan) and I couldn't even find a copy of any of the three official mp3 codecs on the warez sites!

So I've kind of gotten the feeling that they are pretty rare.

I am not quite sure I understand you correctly, but if you are talking about the fact that you can't find any Lame, FhG or dist10 on any warez sites, the explanation is quite straightforward. Warez sites contain cracked and illegal software, but both Lame and FhG can be obtained legally for free, so why would anyone publish them on a warez site?
Title: MP3 at 128kbps public listening test
Post by: danchr on 2003-12-10 09:09:50
Quote
Opinions? Maybe make the test all CBR?

Considering a lot of people still use the fact that the first AAC test was CBR only as an argument against it, I'd say that's a bad idea. IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.

Quote
Yes, I'm quite sure it is SoundJam.

If SoundJam indeed used an alternative MP3 encoder, then it's quite likely that's what iTunes uses since the encoding dialog pretty much hasn't changed since SoundJam. Plus, iTunes 1.0 was basically a scaled down version of SoundJam with a different GUI. It'd be very interesting to know how it performs - I've never seen any tests of it

How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim!
Title: MP3 at 128kbps public listening test
Post by: bond on 2003-12-10 09:22:14
what about using mp3pro@64 as anchor for the mp3 test?
i think it would fit as an anchor qualitywise and the results would also be really interesting...


for the aac test i wouldnt drop sorenson (it wasnt that bad at all)


and i would vote for a clip with sound and human voice (as from movies)
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 09:59:22
My opinion:

*using Xing as low anchor seems to be a good idea, and more usefull than dist10.
*comparing the 3 main encoding engines used nowadays is a good thing (Fhg, ITunes, Lame)
*Including the old (but still widely used because of the Radium hack) AudioActive is a good thing.

Now, we could ask why only using current FhG engines in vbr only. The same (reversed) question could be asked about Lame, although I also think that it is very likely that cbr/abr would be better than vbr in those bitrates.

In order not to increase too much the number of candidates, I'd suggest an all cbr test. I know that this would penalize Lame (compared to abr), but otherwise there is the risk of endless debates.

Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.

Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?
Title: MP3 at 128kbps public listening test
Post by: SacRat on 2003-12-10 10:52:21
Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating"
@Gabriel... I think , your question about LAME versions is interesting. At least there must be a difference between 3.90.3 and latest stable release (otherwise I don'tknow what you did all this time)... maybe, we could include both? And if so, what settings would you suggest? alt-preset CBR?

Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.
Title: MP3 at 128kbps public listening test
Post by: bond on 2003-12-10 11:01:58
i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr)

imho the bad codecs should be punished, not the good ones 
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 11:02:48
Quote
And if so, what settings would you suggest? alt-preset CBR?

For a cbr encoding that would be

*3.90.3: --alt-preset cbr 128
*3.93.1: --preset cbr 128
*3.94: -b 128
Title: MP3 at 128kbps public listening test
Post by: magic75 on 2003-12-10 11:09:51
I can agree on skipping VBR since it is difficult to use that in a fair comparison with other encoders, but I don't like the idea of skipping ABR for Lame. I don't that would implicate an unfair comparison.

If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.

BTW, thanks rjamorim for doing all these tests. Really great work. I am really looking forward to the march test, to see how MP3 compares to WMA std.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 13:46:23
Quote
I am not quite sure I understand you correctly, but if you are talking about the fact that you can't find any Lame, FhG or dist10 on any warez sites, the explanation is quite straightforward. Warez sites contain cracked and illegal software, but both Lame and FhG can be obtained legally for free, so why would anyone publish them on a warez site?

He was talking about the three official WMP codecs: CyberLink, InterVideo and (whatever was the third one).

Quote
How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim!


If Ivan managed to fix EnolaGay, it might well surpass some of the MP3 encoders

Still, I'm pretty fond of the idea of ruining Xing's reputation once and for all

Quote
Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.


Well, if most people agree on making this test CBR only, I might well later conduce a VBR FhG vs. ABR Lame test. I'm working for you guys, so it's your decision the way the test will go.

Quote
Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?


To be very sincere, I didn't make up my mind yet on that subject. There are reasons to use the HA version - the very same reasons why HA recommends that fork instead of latest official release.

And then again, there are several reasons to test latest official release: check how quality is progressing, help the developers, etc.

So, which one you recommend, as a lame developer?

Quote
Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating"


I gotta admit that appeals a lot to me, particularly :B

Quote
maybe, we could include both?


We are at a point where, if you suggest another encoder, please suggest what should be replaced. 6 codecs is the maximum, period. Even at 64kbps testing 8 codecs was a mistake and I don't want it to happen again.

Quote
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.


I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

Quote
If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.


OK. What FhG encoder, specifically?

Thanks a HUGE bunch for all your comments

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: magic75 on 2003-12-10 14:27:44
Quote
OK. What FhG encoder, specifically?

To be honest I didn't consider that, really. I just thought it would be more interesting to see the possible progress of Lame rather than three FhG. But maybe Gabriel can say something about what kind of improvements he expects for the 128 kbps settings in 3.94, before considering having both any further?

OK, here is my very personal wishlist for what I would like to see compared:
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

If I were to add something to this list it would be:
1. Gogo
2. Lame 3.94

These are all just suggestions...
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-10 14:41:25
Quote
There would be 5 codecs and an anchor:

-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 14:47:27
Quote
So, which one you recommend, as a lame developer?

My own choice would be to test the latest beta or release (of course alpha are excluded).
In my mind, this is the more usefull, as it is representative of the current state of the codec, and could be helpfull in order to fix potential issues.

However, I can already hear those numerous people that will argue that the "HA recommended version" should be used, and nothing else.
(please note that in order to enforce this argument, it means that the "recommended version" has to be tested against latest one)

The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 14:49:23
Quote
i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.


This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-10 15:03:37
Quote
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?
Title: MP3 at 128kbps public listening test
Post by: Yaztromo on 2003-12-10 15:13:43
Since we're targeting how well MP3 codecs perform at 128kbps then it makes sense to use CBR for all encoding tests. Since CBR is what most average non-HA users encode with. Giving LAME use of ABR and forcing the rest to use CBR seems an unfair test.
Title: MP3 at 128kbps public listening test
Post by: sthayashi on 2003-12-10 15:13:56
Quote
Quote
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

The problem with APS testing is that 1) you would have fewer people testing (and then rjamorim would run into the same problem that I did with MY listening test) and 2) you would have to have highly suggested/optimized settings for all the other codecs, which is simply impossible.  Like what's the APS equivalent on the FhG encoders?

Now if you'd like, I can arrange a personal test for you so that you see if you can hear the differences between APS and anything else, but it's not really good for a public test Even at 128, there will be people that aren't able to ABX it from the original.
Title: MP3 at 128kbps public listening test
Post by: KristianT on 2003-12-10 15:27:51
Quote
The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.


Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version. THEN... when all three tests are done, and we (hopefully) have LAME 3.94 it would be the right time for testing it against 3.93.1 and 3.90.3. This test has been needed for a long time and if I knew how to conduct it, I would make it a top priority.

Quote
IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.


Indeed! Let's not cripple good codecs.

Finally, I can see why XING would make an interesting anchor but considering the ones who will actually take time examining the results, I feel HE-AAC would be more interesting. Sure, it would be somewhat misplaced but then again it would provide useful information. In my opinion XING wouldn't.
Title: MP3 at 128kbps public listening test
Post by: sthayashi on 2003-12-10 15:28:44
Quote
Quote
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.

I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

I'm still a bit sore from the results (or lack thereof) from my last listening test, otherwise, I'd be happy to volunteer.  Besides, you'll get that accusation in your 3rd test anyways, when either Nero or Quicktime (the expected winners of the AAC test, IMO) does nominally better than Vorbis (my expectations for your 128kbps extension).

It's a test that ought to be done, dammit. 
Title: MP3 at 128kbps public listening test
Post by: schnofler on 2003-12-10 15:30:58
Quote
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors", Lame and FhG, and that would also mean allowing FhG to use VBR. Using only CBR doesn't make a lot of sense in my opinion. Even if it is the most common newbie setting, so what? We already have Xing in the test, which should suffice to prove that the typical "newbie settings" are not the optimum. Limiting all the other codecs to CBR would make the test very one-sided.
I think one of the main goals of an MP3 128kbit/s test should be to find the best codec at this bitrate. We will never find this out if we limit the codecs artificially. Additionally, using CBR would penalize the MP3 format in the upcoming multiformat test (if the winner of the MP3 test is used) because it doesn't give an accurate image of what the format is capable of at that bitrate.
As for the sixth codec, I would suggest Lame 3.94 (or 3.93.1 if 3.94 is not ready by then). I guess a good result of a current Lame version would be a solid basis for a new HA recommendation (as "lack of proper testing" seems to be the main argument against recommending the newer Lame versions).
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-10 15:33:16
Quote
I was under the impression that FhG Current was the slow codec. In this case, I'll do current and legacy slow. Thanks for the tip. I'll update my first post now.

You would know for sure which codec is the slow one.  It's super slow.

Quote
Quote
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).


Yes, I'm quite sure it is SoundJam.


Yep.  It is apparently quite good.

Quote
Quote
There's also the FhG slow codec (high quality codec within fastencc.exe)


Huh? So fastencc isn't the fast codec?


It contains both the fast and slow codecs.  The fast codec within fastencc.exe is buggy (loss of stereo separation).  The slow one (-hq) works fine.

ff123
Title: MP3 at 128kbps public listening test
Post by: danchr on 2003-12-10 15:40:06
Quote
well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 15:40:22
Quote
Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version.

I hope that I wasn't misunderstood. My own choice would be to use 3.94, but I pointed that other people might have different opinions.
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-10 15:45:13
Quote
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?
Title: MP3 at 128kbps public listening test
Post by: rpop on 2003-12-10 15:47:48
Quote
Quote
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?

I'm in favor of this. Making some VBR or ABR is rather unfair, IMO, because in my experience, most VBR and ABR modes tend to have, on average, a bitrate that is higher or lower than 128kbps.

From what I learned at school (in science ), for best results in an experiment, only one variable should be modified. In this case, that would be the encoder.  By using VBR and ABR, the bitrate is also modified, as well as the encoder's specific implementation of VBR/ABR. For example, in my experience, LAME ABR 128 consistently produces ABRs below 128.
Title: MP3 at 128kbps public listening test
Post by: DigitalDictator on 2003-12-10 15:50:43
Quote
i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr) imho the bad codecs should be punished, not the good ones

I TOTALLY AGREE!
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2003-12-10 16:07:52
Quote
Quote
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors" Lame and FhG and that would also mean allowing FhG to use VBR.

Add another vote for this selection! Two FhGs, one "default" the other "best quality", seem enough.


Quote
Quote
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?

You have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.
Title: MP3 at 128kbps public listening test
Post by: DigitalDictator on 2003-12-10 16:21:06
Quote
I'm in favor of this. Making some VBR or ABR is rather unfair, IMO, because in my experience, most VBR and ABR modes tend to have, on average, a bitrate that is higher or lower than 128kbps

So what? I want to know how good LAME is at APPROXIMATELY 128 kbps, doing its best. I don't give a rat's ass about CBR 128, because I know it's not as good! The same goes for Frauhofer's FastEnc and the rest of the bunch. How good can they perform around 128 kbps? Let it show me what they're made of!

Saying that VBR and ABR are not representative because the oblivious masses know only of CBR is pure BS! Heck, with that argument there's no point in having a test among different codecs! Not a single soul I know of have heard of MPC, AAC or OGG.

So maybe it'll be hard finding settings that will come close to 128 kbps on every encoder, so be it. I think every encoder should be allowed to give it its absolute best.

I know this has been beaten to death before but if CBR is the way to go on this test, people will always wonder how different encoders perform not being "crippled".
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-10 16:30:15
Quote
Add another vote for this selection! Two FhGs, one "default" the other "best quality", seem enough.

If the test is to include just two FhG's, I think one of them should be FastEnc based (i.e., default in MMJB).  This one can also be VBR if desired, since FhG VBR is FastEnc based.

I think the other should be Audioactive/Radium based.

The super-slow codec has two disadvantages:  1) it is super slow  2) it can sound both good and bad (incosistent).  For people with good high frequency hearing, it will probably have less of the "mp3" quality to it, but more low frequency glitching and dropouts.  Audioactive/Radium should be more consistent.

ff123
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 16:41:22
Are we sure that Fastenc is better in vbr than cbr?
Title: MP3 at 128kbps public listening test
Post by: de Mon on 2003-12-10 17:11:21
So many candidates and I like almost all of them. But I would like to see specially:
1. MPC
2. The latest LAME or LAME 3.93.1
Some words about LAME:
I would like to see it ABR, but what we'll do if encodings will be 118, 132 or 135 Kbs? So I think LAME must go CBR despite it  has been developed at this bitrate as ABR codec.
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?
What hard samples will be used?
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-10 17:18:04
Quote
1. MPC

I do not think that mpc would be a good candidate in a "128kbps MP3 listening test"
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 18:06:45
Quote
i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.

God, no. Besides, what would be the point? All that it would show is that APS is much better than the rest. But that would be no surprise to anyone here.

Using different bitrates in a multiformat test is already a difficult idea. In a "monoformat" one, it's almost unjustifiable.

Quote
The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.


I agree. And testing lame twice at the same setting, only changing version, would look like bias (much like people complained about ff123 testing Vorbis twice at his 64kbps test).

Quote
Since we're targeting how well MP3 codecs perform at 128kbps then it makes sense to use CBR for all encoding tests. Since CBR is what most average non-HA users encode with.


Good point.

Quote
Giving LAME use of ABR and forcing the rest to use CBR seems an unfair test.


Right, it would definitely smell like bias. Specially if you consider HA is pretty biased towards Lame (for very valid reasons, I should add).

Quote
Finally, I can see why XING would make an interesting anchor but considering the ones who will actually take time examining the results, I feel HE-AAC would be more interesting. Sure, it would be somewhat misplaced but then again it would provide useful information. In my opinion XING wouldn't.


Nope, HE AAC has already been tested against MP3 at 128kbps. Maybe in my next low bitrate test, but not now.

Besides, IMO Xing would provide very useful information. To start with, I don't know of any public test featuring Xing. So, it might be that it isn't even as bad as people paint it. Besides, if it really is bad, then we'll have the hard proof.

Quote
Besides, you'll get that accusation in your 3rd test anyways, when either Nero or Quicktime (the expected winners of the AAC test, IMO) does nominally better than Vorbis (my expectations for your 128kbps extension).


My tests are completely open for peer review. I offer all the results, and they can see for themselves that I'm not cheating in the encodes by looking at the batch files and checking that I'm using standard compiles of the codecs.

So, no, I will personally tell everyone that doubt the honesty of my tests to go f*ck themselves.

What I meant is that people will believe I'm doing provocation. "neener-neener, I just did this to prove Vorbis suxxxorz and AAC rulz!"

Since the 3rd test will be multiformat, it's not like I'm comparing Vorbis directly against AAC. It's everyone against everyone. Can't be seen as provocation.

Quote
It's a test that ought to be done, dammit.


Start choosing samples and settings. I'll remind you of this test a few months from now

I'll continue in my next post.. (getting too big)
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 18:23:56
Quote
I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors", Lame and FhG, and that would also mean allowing FhG to use VBR. Using only CBR doesn't make a lot of sense in my opinion. Even if it is the most common newbie setting, so what? We already have Xing in the test, which should suffice to prove that the typical "newbie settings" are not the optimum. Limiting all the other codecs to CBR would make the test very one-sided.

Right.

About FhG: Problem is that some say fastenc is actually better quality than HQ sometimes.

Anyone willing to do a quick parallel test comparing both? (using Audition or MMJ)

Quote
It contains both the fast and slow codecs. The fast codec within fastencc.exe is buggy (loss of stereo separation). The slow one (-hq) works fine.


Good to know. So maybe it's recommended to use fastencc -hq instead of Audition legacy HQ? I'm not sure, but I would expect Audition to use newer encoding libraries.

Quote
For example, in my experience, LAME ABR 128 consistently produces ABRs below 128.


Hrm... true, but still it behaves very well.
http://audio.ciara.us/test/128extension/results.html (http://audio.ciara.us/test/128extension/results.html)

The average Lame bitrate there was 124.5, never going above 133 or under 121. I think that's an acceptable error margin.

Also, quality in Adobe Audition goes from 1 to 100 in steps of 1, so it would be easy to find a VBR setting that outputs results close to 128.

But we must keep in mind what users want. And I'm still not sure if people are more interested in knowing CBR results or ABR/VBR results.

Quote
I TOTALLY AGREE!


OMG! Caps lock.

Quote
The super-slow codec has two disadvantages: 1) it is super slow 2) it can sound both good and bad (incosistent). For people with good high frequency hearing, it will probably have less of the "mp3" quality to it, but more low frequency glitching and dropouts. Audioactive/Radium should be more consistent.


That's very valuable information. Thanks for mentioning it, ff123.

Definitely something worth considering.

BTW: super-slow = mp3enc 3.1?

Quote
I would like to see it ABR, but what we'll do if encodings will be 118, 132 or 135 Kbs?


We would have to find out an acceptable error margin (I think something like 8kbps would be OK) and make sure no sample goes above or below this margin.

Quote
I do not think that mpc would be a good candidate in a "128kbps MP3 listening test"




Quote
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?


Default.

And I think the only really crappy Joint Stereo in FhG codecs is in the fast fastencc encoder. And that one won't be used.

Quote
What hard samples will be used?


Do you guys think it's needed to replace one of the 64kbps test samples with a difficult one?

Again, thanks a lot for your input.

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: sthayashi on 2003-12-10 18:28:11
Quote
Quote
It's a test that ought to be done, dammit.

Start choosing samples and settings. I'll remind you of this test a few months from now

Since the samples for my previous test went virtually unused, I'll use them.  As for settings, I've got the Vorbis one in mind already, but will open a new discussion thread concerning AAC settings.  Remind me later, and I'll start the thread.  Alternatively, if someone else has a pressing desire to conduce that test, then I'll be more than happy to supply them with my ideas.
Title: MP3 at 128kbps public listening test
Post by: amano on 2003-12-10 18:43:46
I'd rather use Radium itself than Audioactive. I think the Radium ACM is widely spread so that should be used in the test. I know that the Audioactive codec might be a bit newer, but in everyday life radium is more widespread and therefore more interesting to know about for most of us.

The same applied IMHO to WMA and WMAPRo. I wish you had tested WMA for the 128 kbps test, rather than WMAPro, because WMAPro isn't widespread, whereas WMA is interesting for hardware use.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-10 18:50:12
Quote
Good to know. So maybe it's recommended to use fastencc -hq instead of Audition legacy HQ? I'm not sure, but I would expect Audition to use newer encoding libraries.

I don't know what Audition legacy HQ is, but if it's super slow, then it should be the same as fastencc.exe -hq.  If it encodes in realtime or faster, then it's the fast codec.

Quote
BTW: super-slow = mp3enc 3.1?


Yes.  fastencc.exe -hq is pretty much the same as mp3enc 3.1, except that fastencc.exe has a higher cutoff frequency.

Quote
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?


At 128 kbit/s, they should all use joint-stereo.

ff123

Edit:  just to make it very clear what the various flavors of FhG codecs are, here they are as best as I know:

1.  fast codec (FastEnc); FhG VBR, which is based on FastEnc.
2.  super slow codec (i.e., mp3enc31 and its successors); same as fastencc.exe -hq
3.  Audioactive/Radium

FhG made two variants of the fast codec in the last Cool Edit Pro I looked at (version 2.1): the normal fast version, and the super fast version.  I presume the super fast version sounds worse.  The had dropped the super slow codec.  Perhaps in Adobe Audition, the super slow codec was reinstated as the "legacy HQ" setting.

iTunes/Soundjam is not FhG, as far as I can tell.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 19:12:11
Quote
The same applied IMHO to WMA and WMAPRo. I wish you had tested WMA for the 128 kbps test, rather than WMAPro, because WMAPro isn't widespread, whereas WMA is interesting for hardware use.

I know. For that reason, next multiformat@128kbps test will feature WMA Std

Quote
Remind me later, and I'll start the thread.


Well, somewhere around April, if you don't mind waiting that much.

I think it's better if we don't have several tests happening simultaneously. Things could go messy. And my schedule is pretty tight until march.


@ff123: thanks for all the info.
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-10 19:27:18
Quote
bYou have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.

well, then why do a public test? it's obvious that it allways depends on YOU, but this is more like a "generic" test ... also, where are the blind tests taht prove your saying?
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-10 19:28:58
Quote
God, no. Besides, what would be the point? All that it would show is that APS is much better than the rest. But that would be no surprise to anyone here.

well, i think that is the public asumption, but have we ever seen a (abx) test to prove it?
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2003-12-10 19:39:37
Quote
Quote
bYou have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.

well, then why do a public test? it's obvious that it allways depends on YOU, but this is more like a "generic" test ... also, where are the blind tests taht prove your saying?

You don't understand what I was trying to say: The group test is there to find if A is better than B, not to decide whether the quality gain is worth the additional bitrate. The latter really has to be done individually.

Quote
also, where are the blind tests taht prove your saying?

This is a well-known fact on this board. If you're interested in my tests, I can tell you that in my small private listening test of 5 different samples (no problem samples), I could easily pick out abr-130, while the abr-160 was nearly transparent.


Quote
Do you guys think it's needed to replace one of the 64kbps test samples with a difficult one?

Yes! Well, not for this but for the next test, so the evil Ahead AAC gang can't do special tuning for the standard Roberto-samples-set.   
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 19:56:14
Quote
Yes! Well, not for this but for the next test, so the evil Ahead AAC gang can't do special tuning for the standard Roberto-samples-set.   

MWAHAHAHA. OK then, I'll keep some of the samples and replace others, and I'll keep the list of what stays and what goes secret until the date of the test

I agree with you, that should avoid claims of tampering.
Title: MP3 at 128kbps public listening test
Post by: jrp on 2003-12-10 21:19:32
There are quite a few 128k tests out there, but they are several years old.

I would prefer to see new ground being broken, now that space is less of an issue, with test of the latest LAME -aps (which is said to be untested (at least in comparison to 3.90.x) but also works to some extent with Windows Media Player 9) compared to iTunes at 192k VBR, say, and the FhG encoders available for Windows Media Player 9 such as the Cyberlink or Intervideo versions (See, eg, http://www.wmplugins.com/SearchResults.asp...eyword=&type=4) (http://www.wmplugins.com/SearchResults.aspx?keyword=&type=4)).
Title: MP3 at 128kbps public listening test
Post by: fairyliquidizer on 2003-12-10 22:10:06
Quote
-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

rjamorim you star!

Great idea.  How about an "astro anchor" of Lame "APS -Y" I know it's a bit different but it will put them into perspective.  Something that the HA users can easily relate to.

As for LAME ABR is far more interesting than CBR and VBR at this bit rate is a mistake.  If you want to use CDR 128 please do ABR too.

I think ABR is the forgotten star of LAME, especially for people looking for medium bit rate encodes for portable use.

Which LAME build are u gonna use dude?

As for low anchor.... surely that'll be Xing!

Cheers,
Fairy

PS- please start after the Hogmanay hangovers have lifted!
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 22:14:26
Quote
Great idea.  How about an "astro anchor" of Lame "APS -Y" I know it's a bit different but it will put them into perspective.  Something that the HA users can easily relate to.

No. Once and for all: There will be no different bitrates at this test.

Quote
I would prefer to see new ground being broken, now that space is less of an issue, with test of the latest LAME -aps (which is said to be untested (at least in comparison to 3.90.x) but also works to some extent with Windows Media Player 9) compared to iTunes at 192k VBR, say, and the FhG encoders available for Windows Media Player 9 such as the Cyberlink or Intervideo versions


Well, I definitely won't be testing high bitrates. If anyone feel up to it, I can help. But I won't conduce anything in that fashion.

About the WMP codecs: I doubt they use different libraries than the ones used in MusicMatch and Audition.
Title: MP3 at 128kbps public listening test
Post by: jrp on 2003-12-10 22:55:12
Quote
Quote
Well, I definitely won't be testing high bitrates. If anyone feel up to it, I can help. But I won't conduce anything in that fashion.

About the WMP codecs: I doubt they use different libraries than the ones used in MusicMatch and Audition.

I'm not being argumentative, but could you please explain why?

Are the MusicMatch/Audition codecs any good?  Or would one not be able to tell the difference at 192k VBR, say?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-10 23:14:50
Quote
I'm not being argumentative, but could you please explain why?

At high bitrates, the samples become too transparent. And only a handful of golden ears can pick up artifacts - not enough to generate statistically valid results.

I mean, look at my former multiformat @ 128kbps test. All modern codecs were already ranked at scores higher than 4.  And the higher the bitrate, the higher the scores will go, eventually reaching 5.

Also, with all codecs ranking very high, you would need hundreds of listeners to avoid getting the confidence margins overlapping. And, as I already told you, there are only a handful of golden-ears.

So, IMO and from my experience, a high bitrate test would be a waste of time and effort. I don't want to burst anyone's motivation here, but that is my belief, and that's why I won't conduce such test.

Quote
Are the MusicMatch/Audition codecs any good?  Or would one not be able to tell the difference at 192k VBR, say?


Given they are FhG (which is good), they would probably be transparent on most samples for most people at that bitrate.

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: jrp on 2003-12-10 23:29:25
Thanks.  Helpful.  [I can't claim golden ears, but I am extremely pleased with my Shure e2c headphones. Having baulked at the price, originally, they are worth every £. My problem is getting a ripper/encoder that has good organisational capabilty.  If everything is likely to be +/- the same at 192k, I'll go back to WMP9.]
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-11 06:18:21
I think fastenc should be CBR to compare the quality against lame -ap128 most of what I have read & tested shows fastenc is the best FhG encoder & performs better using CBR.

f123 what are your thoughts about an optimal FhG encoder & which settings?


Oh, which compile of lame is going to be used? could 3.92 or 3.93.1 be more sutible for 128kbps?
Title: MP3 at 128kbps public listening test
Post by: frosty on 2003-12-11 07:05:25
I think the important thing that was brought up earlier is that some of the most widely used codecs should be tested.  In that case, Lame 3.90.3 might be the right choice as well as what MusicMatch and iTunes use.

I also vote for using ABR.  Does the average user really bother with ABR 128?  I know I would, but I can't think of a single friend who would.  Furthermore, it does help limit the number of variables in the test.
Title: MP3 at 128kbps public listening test
Post by: magic75 on 2003-12-11 07:16:13
Ok, one last post from me about the CBR vs ABR issue. I actually feel that not letting Lame use ABR is unfair to Lame. I mean what is the main difference between VBR and CBR from a user point of view? Bitrate predictability. With ABR we have that (or at least within +-5kbps), but better quality than CBR. So I don't think it is unfair, we are comparing the best quality settings of different codecs at a predictable bitrate of ~128 kbps.

But I agree with rjamorim, the test will look biased towards Lame if it is allowed to use ABR. But I still feel that the comparison would be really interesting! So I have actaully developed a wishlist 2.0...

Lame 128 ABR
Lame 128 CBR
FhG 128 CBR ~standard
FhG 128 CBR ~HQ
iTunes 128 CBR
Xing 128 CBR
Title: MP3 at 128kbps public listening test
Post by: saratoga on 2003-12-11 07:36:54
Would you consider Gogo as well?  I've heard a lot of people recommend it because its fast and probably better then fhg, but i have no idea how good it is.

Quote
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps


Are you absolutely sure iTunes does not use one of the fhg codecs listed above?  encspot reports that its fhg and the iTunes "about box" also lists fhg as having licensed software used in iTunes.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-11 08:25:54
Quote
Would you consider Gogo as well? I've heard a lot of people recommend it because its fast and probably better then fhg, but i have no idea how good it is.

I have no doubt that Gogo is worst than FhG.

Quote
Are you absolutely sure iTunes does not use one of the fhg codecs listed above?

I think that iTunes is using an evolution of SoundJam. But if there are doubts, we could ask developpers.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-11 16:18:01
Quote
I think fastenc should be CBR to compare the quality against lame -ap128 most of what I have read & tested shows fastenc is the best FhG encoder & performs better using CBR.

f123 what are your thoughts about an optimal FhG encoder & which settings?

Perhaps a pre-test would be in order to try to determine if FastEnc CBR is significantly better or worse than FastEnc VBR.

If anybody has the iTunes encoder, try encoding the clip:

http://lame.sourceforge.net/download/sampl.../main_theme.wav (http://lame.sourceforge.net/download/samples/main_theme.wav)

with both iTunes and FastEnc.  FastEnc has a problem with this sample, and so should iTunes, if it is FastEnc based.

ff123
Title: MP3 at 128kbps public listening test
Post by: DigitalDictator on 2003-12-11 18:21:48
Quote
I have no doubt that Gogo is worst than FhG

So let's test it and see then...
Title: MP3 at 128kbps public listening test
Post by: DigitalDictator on 2003-12-11 18:23:33
Quote
QUOTE 
I TOTALLY AGREE!



OMG! Caps lock.

You are very observant:) But don't worry, eventually you'll have keyboards with that ability in Brazil too...
Title: MP3 at 128kbps public listening test
Post by: odious malefactor on 2003-12-11 18:26:34
Hmm... can't seem to get this quote thing to work...
Title: MP3 at 128kbps public listening test
Post by: odious malefactor on 2003-12-11 18:27:56
Quote
Quote
I have no doubt that Gogo is worst than FhG

So let's test it and see then...

Yeah, let's do Gogo!!
Title: MP3 at 128kbps public listening test
Post by: music_man_mpc on 2003-12-11 19:11:27
I was under the impression that it was a well known fact that GOGO was made for speed as opposed to quality.  I for one think it should be left out.  However for the sake of #8 TOS I will conduct a quick listening test if anyone argues this.
Title: MP3 at 128kbps public listening test
Post by: saratoga on 2003-12-11 19:16:38
I've had people tell me 192 gogo sounds the same as LAME.  I think it should be tested.  Particularly if its so bad, it'd be quite easy to test then.
Title: MP3 at 128kbps public listening test
Post by: music_man_mpc on 2003-12-11 19:19:17
OK I'll start testing right now.
Title: MP3 at 128kbps public listening test
Post by: mdmuir on 2003-12-11 20:00:15
Thinking about uploading a sample from one of my Caruso cylinder recordings. After all, we need to know which 128 bit rate mp3 encoder encodes the hiss and distortion of these recordings with the most faithfulness.   
Title: MP3 at 128kbps public listening test
Post by: music_man_mpc on 2003-12-11 20:14:39
Both encodes were very easy to tell from the origonal WAV once I got used to it, but to be honest I couldn't tell the one encoded with LAME --alt-preset cbr 128 from the one encoded with Gogo.  Perhaps Gogo has more merit than I thought or, perhaps, I wasn't testing a good clip, I don't know.

A file: E:\Program Files\Encoding\schism.wav
B file: E:\Program Files\Encoding\Samples\Tool - Lateralus - 05 - Schism - 2001 - Metal.wav

15:47:59    1/1  p=50.0%
15:48:34    2/2  p=25.0%
15:49:04    3/3  p=12.5%
15:49:19    4/4  p= 6.2%
15:49:29    5/5  p= 3.1%
15:49:40    6/6  p= 1.6%
15:49:50    7/7  p= 0.8%
15:50:00    8/8  p= 0.4%
15:50:12    9/9  p= 0.2%
15:50:28  10/10  p< 0.1%

A file: E:\Program Files\Encoding\schismlame.wav
B file: E:\Program Files\Encoding\Samples\Tool - Lateralus - 05 - Schism - 2001 - Metal.wav

15:55:12    1/1  p=50.0%
15:55:19    2/2  p=25.0%
15:55:51    3/3  p=12.5%
15:56:07    4/4  p= 6.2%
15:57:54    5/5  p= 3.1%
15:58:07    6/6  p= 1.6%
15:58:20    7/7  p= 0.8%
15:58:27    8/8  p= 0.4%
15:58:35    9/9  p= 0.2%
15:58:43  10/10  p< 0.1%

WinABX v0.23 test report
12/11/2003 15:59:36

A file: E:\Program Files\Encoding\schism.wav
B file: E:\Program Files\Encoding\schismlame.wav

16:04:24    1/1  p=50.0%
16:04:40    1/2  p=75.0%
16:05:57    2/3  p=50.0%
16:06:13    2/4  p=68.8%
16:10:11    2/5  p=81.2%
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-11 22:08:17
Quote
Both encodes were very easy to tell from the origonal WAV once I got used to it, but to be honest I couldn't tell the one encoded with LAME --alt-preset cbr 128 from the one encoded with Gogo.  Perhaps Gogo has more merit than I thought or, perhaps, I wasn't testing a good clip, I don't know.

i don't need to respond then  ... this is why i think we should compare lame -aps agains 128, and against the others ...
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-11 22:30:42
OK, let me try to say this one more time:

I won't feature different bitrates at this listening test.


Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-11 22:38:17
Quote
Quote
Both encodes were very easy to tell from the origonal WAV once I got used to it, but to be honest I couldn't tell the one encoded with LAME --alt-preset cbr 128 from the one encoded with Gogo.  Perhaps Gogo has more merit than I thought or, perhaps, I wasn't testing a good clip, I don't know.

i don't need to respond then  ... this is why i think we should compare lame -aps agains 128, and against the others ...

I don't understand how it follows that we should test alt-preset standard (aps) against 128 kbit/s settings from music_man_mpc's test.

Let's get serious here:  --alt preset standard is heads and shoulders above --alt-preset cbr 128, and it would be a gigantic waste of time and effort to show this.  Try any sample with sharp transients (eg., castanets.wav).  Convince yourself.

ff123
Title: MP3 at 128kbps public listening test
Post by: music_man_mpc on 2003-12-12 02:30:14
Quote
Let's get serious here:  --alt preset standard is heads and shoulders above --alt-preset cbr 128, and it would be a gigantic waste of time and effort to show this.  Try any sample with sharp transients (eg., castanets.wav).  Convince yourself.

ff123

I agree entirely, my test was conducted simply to see if Gogo is, as Mike Giacomelli implied, in the same ballpark as LAME.  My conclusion was this:  to my ears, on the sample I tested (Tool - Schism 0:26 - 0:35) Gogo sound virtually identical to LAME at 128kbit/s.  What does this have to do with --alt-preset standard kwanbis?

edit: typo
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-12 05:41:58
Quote
If anybody has the iTunes encoder, try encoding the clip:

http://lame.sourceforge.net/download/sampl.../main_theme.wav (http://lame.sourceforge.net/download/samples/main_theme.wav)

with both iTunes and FastEnc.  FastEnc has a problem with this sample, and so should iTunes, if it is FastEnc based.

ff123

Currently I don't have access to the iTunes encoder & the d/l is 20mb but just in case someone does here (http://www.geocities.com/westgroveg/fastenc_-hq_main_theme.zip) is the fastenc encoded version.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-12 05:54:05
And here is the iTunes encode:
http://pessoal.onda.com.br/rjamorim/main_t..._iTunes_128.mp3 (http://pessoal.onda.com.br/rjamorim/main_theme_iTunes_128.mp3)
Title: MP3 at 128kbps public listening test
Post by: harashin on 2003-12-12 06:05:47
Quote
    * From: Bill Kincaid
    * Subject: Re: [mp3encoder] MP3 encoder for iTunes?
    * Date: Tue, 21 Oct 2003 09:46:34 -0700

Don't bother.  The iTunes MP3 encoder was written pretty much from scratch,
based on the ISO reference source in distribution 10, as LAME was.  I know,
I was the principal author (it was originally shipped in SoundJam, for
anyone who remembers that far back).

It has been heavily tweaked over the years and doesn't bear much resemblance
to dist 10 anymore...

-Bill Kincaid
iTunes


http://www.mail-archive.com/mp3encoder@min...g/msg01901.html (http://www.mail-archive.com/mp3encoder@minnie.tuhs.org/msg01901.html)
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 06:10:49
Quote
Currently I don't have access to the iTunes encoder & the d/l is 20mb but just in case someone does here (http://www.geocities.com/westgroveg/fastenc_-hq_main_theme.zip) is the fastenc encoded version.

This encode is actually what I refer to as the super slow FhG codec (the -hq option within fastencc.exe).  When I say "FastEnc" I really mean the fast FhG codec.  I'll post an encode of that later.

ff123
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-12 06:11:46
Thanks for the info, harashin


Edit: I think Harashin pretty much clarified, but here's a real fastenc encode. The command line used was:
fastencc main_theme.wav main_theme.mp3 -br 128000

http://pessoal.onda.com.br/rjamorim/main_t...astencc_128.mp3 (http://pessoal.onda.com.br/rjamorim/main_theme_fastencc_128.mp3)
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 06:38:11
Quote
Thanks for the info, harashin


Edit: I think Harashin pretty much clarified, but here's a real fastenc encode. The command line used was:
fastencc main_theme.wav main_theme.mp3 -br 128000

http://pessoal.onda.com.br/rjamorim/main_t...astencc_128.mp3 (http://pessoal.onda.com.br/rjamorim/main_theme_fastencc_128.mp3)

LOL,

Will the real FastEnc encode please stand up?

Roberto, you used fastencc.exe with default quality, which is the buggy (loss of stereo separation) fast codec.

Here is the real fastenc encoding:

http://ff123.net/export/main_theme_fastenc.mp3 (http://ff123.net/export/main_theme_fastenc.mp3)

which I made with Cool Edit Pro 2.1, using cbr, high quality option.

ff123
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-12 06:43:24
Quote
Thanks for the info, harashin


Edit: I think Harashin pretty much clarified, but here's a real fastenc encode. The command line used was:
fastencc main_theme.wav main_theme.mp3 -br 128000

http://pessoal.onda.com.br/rjamorim/main_t...astencc_128.mp3 (http://pessoal.onda.com.br/rjamorim/main_theme_fastencc_128.mp3)

your right rjamorim.

I rated:

main_theme_fastenc__128_-hq, Good

main_theme_iTunes_128, Bad

main_theme_fastencc_128, Worst

Both iTunes & fastenc have a problem with this sample. Would the -hq switch be used with fastenc in your test?
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 06:44:54
And just for fun, here are the listening results showing that FastEnc (FhG fast codec) messes up on this sample:



ABC/HR Version 0.9b, 30 August 2002
Testname:

1R = D:\junk\main_theme_iTunes_128.wav
2R = D:\junk\main_theme_fastenc.wav
3R = D:\junk\main_theme_slowenc.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: D:\junk\main_theme_iTunes_128.wav
1R Rating: 4.0
1R Comment:
---------------------------------------
2R File: D:\junk\main_theme_fastenc.wav
2R Rating: 2.8
2R Comment: very flangy near the beginning
---------------------------------------
3R File: D:\junk\main_theme_slowenc.wav
3R Rating: 3.5
3R Comment: a bit fluttery in the beginning; slightly worse than 1
---------------------------------------
ABX Results:

Edit:  "slowenc" is fastencc.exe using the -hq switch.  I see that my listening results differ from westgroveg's on the iTunes version, but perhaps that's because my high frequency hearing is not all that good.  iTunes spectral view looks like it has dropouts between 13 and 15 kHz, which typically translate into stuff that some people hear.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 06:52:49
Quote
Both iTunes & fastenc have a problem with this sample. Would the -hq switch be used with fastenc in your test?

fastencc.exe -hq (what I call "slowenc") might be the best-sounding codec for people with good high-frequency hearing.  It would be interesting to include in a 128 kbit/s mp3 test, but then that would mean either including 3 FhG codecs, or dropping either the fast codec or the Audioactive/Radium codec.

ff123
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-12 07:14:11
Just to be sure I used your abchr (first time),  f123.

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: main_theme

1R = C:\main theme iTunes 128.wav
2R = C:\fastenc -hq main theme.wav

---------------------------------------
General Comments:
Sample 1 has more ringing but I think has higher cut-off
---------------------------------------
ABX Results:
C:\main theme iTunes 128.wav vs C:\fastenc -hq main theme.wav
   10 out of 10, pval < 0.001


As I said fastenc -hq (or slowenc) sounds much better to me.

Edit:

Quote
fastencc.exe -hq (what I call "slowenc") might be the best-sounding codec for people with good high-frequency hearing. It would be interesting to include in a 128 kbit/s mp3 test, but then that would mean either including 3 FhG codecs, or dropping either the fast codec or the Audioactive/Radium codec.

Well rjamorim will have the last word but I think it would be good to compare with lame, one too many times I hear people say "why use FhG (at 128) when lame is better" I'd love to stick it in thier face.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 07:29:27
Quote
Just to be sure I used your abchr (first time),  f123.

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: main_theme

1R = C:\main theme iTunes 128.wav
2R = C:\fastenc -hq main theme.wav

---------------------------------------
General Comments:
Sample 1 has more ringing but I think has higher cut-off
---------------------------------------
ABX Results:
C:\main theme iTunes 128.wav vs C:\fastenc -hq main theme.wav
   10 out of 10, pval < 0.001


As I said fastenc -hq (or slowenc) sounds much better to me.

I'm almost sure when you use the "ringing" term that you're hearing higher frequency stuff in iTunes that I can't.

I'd bet that you'd personally like the slow FhG codec best overall, preferring it over the fast FhG codec and lame --alt preset 128.  However, it has annoying low frequency glitches (not apparent in main_theme.wav, though) which make it just about the worst mp3 codec for me.

It would definitely be an interesting codec to include, but I doubt that many people use it these days because I think FhG phased it out of their mp3 library.

ff123

Edit:  for an example of the type of low frequency glitching that occurs, try using fastencc.exe -hq on the blackbird.wav sample:

http://lame.sourceforge.net/download/sampl...s/BlackBird.wav (http://lame.sourceforge.net/download/samples/BlackBird.wav)
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-12 09:53:09
Mhh... that BlackBird is rather nasty. Is the hq switch limited? I tried to test BlackBird with -vbr -hq & -br192 -hq, in both situations -hq won't work with the other command
Title: MP3 at 128kbps public listening test
Post by: Loke on 2003-12-12 10:55:01
sorry to go a little offtopic but...

I'm confused
There seems to be so many different FhG encoders and many diff programs with various settings using them...
Could someone give my a list or a link to a list with all the encoders and which program using it. And what year they are from.
And what is supposedly to be the newest/best encoder?
(I know FhG encs used to have a phase shift problem in JS before, but I believe it to be solved now)

1. L3enc (1994-1997)
2. Producer Pro 1.0 (1996)  this is also the radium hack
3. Producer Pro 2.0 (1997)
3. Mp3enc (1997- ?)
4. Fastenc (?)      What versions/settings were there problems with? Is it still developed?
5. Musicmacht x.xx  What is the enc the newest version is using?
6. FhG Audition Legacy Slow/Current  Never heard of this, is the prog or the encoder called audition? Is it using fastenc?
7. WMP
8. Cool edit
9. Audioactive
10. ?

Isn't every FhG encoder based on each other(L3enc)?
Anybody help?
Title: MP3 at 128kbps public listening test
Post by: [proxima] on 2003-12-12 11:39:42
Quote
fastencc.exe -hq (what I call "slowenc") might be the best-sounding codec for people with good high-frequency hearing.  It would be interesting to include in a 128 kbit/s mp3 test, but then that would mean either including 3 FhG codecs, or dropping either the fast codec or the Audioactive/Radium codec.

ff123

I have not tested in depth slowenc but in at least one sample (rebel.wav) the codec fails quite badly with high frequencies (ringing, some frequencies are completely throwed out). Both "good" fastenc (CEP2.1) and iTunes have similar problems, LAME with nspsytune is much better even if not perfect. I think this sample is quite interesting, even new generation AAC encoders (expecially Fhg) have troubles with it. Reading this (http://www.hydrogenaudio.org/forums/index.php?showtopic=15839&) previous post you can download the clip and see some spectral views concerning AAC.
I'm of the idea of dropping slowenc, the codec is too slow and abandoned by Fraunhofer. The pre-test about CBR vs. VBR with Fraunhofer encoders is a right idea, what samples and encoders should be tested ? IIRC in your site there are some tests showing that Fhg CBR is better than VBR but i don't remember which Fhg codec.
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-12 13:37:08
Quote
I don't understand how it follows that we should test alt-preset standard (aps) against 128 kbit/s settings from music_man_mpc's test.


Quote
Let's get serious here:  --alt preset standard is heads and shoulders above --alt-preset cbr 128, and it would be a gigantic waste of time and effort to show this.  Try any sample with sharp transients (eg., castanets.wav).  Convince yourself.


Quote
I agree entirely, my test was conducted simply to see if Gogo is, as Mike Giacomelli implied, in the same ballpark as LAME.  My conclusion was this:  to my ears, on the sample I tested (Tool - Schism 0:26 - 0:35) Gogo sound virtually identical to LAME at 128kbit/s.  What does this have to do with --alt-preset standard kwanbis?


are you sure about what you are saying? i do encode in -aps, but is it really that better? or is it as music_man_mp thought about lame vs gogo? why is it a waste of time to test it? what abx do you have to prove it? (rule #8)? that you just don't agree with me, does not mean that i'm not being serious ... also, is just my friendly thought
Title: MP3 at 128kbps public listening test
Post by: 2Bdecided on 2003-12-12 14:07:28
I've read every single post in this thread, but I'm now a little lost as to where we're at.


FWIW I think it's essential that you include the _good_ _real_ version of FastEnc at CBR 128kbps, using either Cool Edit Pro (last version of both CEP and the plug-in), Adobe Audition (what is the real FastEnc called in that?), or Music Match Jukebox (recent version). Not fastenc.exe or old Music Match because both are buggy.

I think this is essential because it's such a newbie default mp3 encoding.


To my ears, FhG audio active, FhG FastEnc, and FhG SlowEnc all have their own sound. They all have their own faults too - pick samples carefully!


I don't know how you deal with CBR or VBR/ABR. Probably stick with one or the other. Seems unfair to mix both, but then I can't see why you would use lame CBR when ABR is so good, whereas I've never seen anyone claim that FhG VBR beats FhG CBR around this bitrate (maybe no one around here uses it). Difficult to answer all questions in six samples, so decide: do you want to compare encoders, or modes (ABR/CBR) - because I don't think you can comprehensively to both in one test.

btw recent Xing isn't as bad as very old Xing. Very Old Xing makes better low quality anchor, but current Xing is more relevant.

Cheers,
David.

P.S. lame -h -b 128 beats lame --alt-preset cbr 128 on at least two samples, so nothing is clear cut and predictable!
Title: MP3 at 128kbps public listening test
Post by: 2Bdecided on 2003-12-12 14:09:08
Quote
are you sure about what you are saying? i do encode in -aps, but is it really that better? or is it as music_man_mp thought about lame vs gogo? why is it a waste of time to test it? what abx do you have to prove it? (rule #8)? that you just don't agree with me, does not mean that i'm not being serious ... also, is just my friendly thought

Because there are about a million samples where lame cbr 128 is ABXable vs the original, and about ten where lame aps is ABXable vs the original.

Cheers,
David.
Title: MP3 at 128kbps public listening test
Post by: kwanbis on 2003-12-12 14:33:28
Quote
Because there are about a million samples where lame cbr 128 is ABXable vs the original, and about ten where lame aps is ABXable vs the original.

but you don't provide any supporting facts ... you just keep repeating a "known" (or assumed) fact ...

8. Any statement about sound quality must be supported by the author responsible for such statements by a double blind listening test demonstrating that he can hear a difference, together with a test sample. Graphs, non-blind listening tests, subtracting two files and so on are definetely not considered as valid evidences of sound quality
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-12 14:51:13
Quote
are you sure about what you are saying? i do encode in -aps, but is it really that better? or is it as music_man_mp thought about lame vs gogo? why is it a waste of time to test it? what abx do you have to prove it? (rule #8)? that you just don't agree with me, does not mean that i'm not being serious ... also, is just my friendly thought

Did you actually try castanets.wav, or florida_seq.wav, or death2.wav, or any sample at all?  Don't throw rule 8 at me until you've done a little homework!

ff123
Title: MP3 at 128kbps public listening test
Post by: 2Bdecided on 2003-12-12 14:55:35
Quote
Quote
Because there are about a million samples where lame cbr 128 is ABXable vs the original, and about ten where lame aps is ABXable vs the original.

but you don't provide any supporting facts ... you just keep repeating a "known" (or assumed) fact ...

8. Any statement about sound quality must be supported by the author responsible for such statements by a double blind listening test demonstrating that he can hear a difference, together with a test sample. Graphs, non-blind listening tests, subtracting two files and so on are definetely not considered as valid evidences of sound quality

I'm talking from personal experience.

(Not the "millions" part obviously! I was extrapolating from the samples I've encoded at 128kbps, and assuming that other people have similar experiences.)


The first sample I played with at 128kbps was youcantdothat.wav:
http://lame.sourceforge.net/gpsycho/quality.html (http://lame.sourceforge.net/gpsycho/quality.html)


The reason you're not getting a serious answer to your question is because it's probably harder (though certainly not impossible) to find a sample where aps and 128cbr are equal. It's easier to find a sample where 128cbr is audibly worse than aps - just go out and rip a random CD! You have to listen carefully to the result, but it's there much of the time.


Cheers,
David.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-12 16:19:16
Quote
Don't throw rule 8 at me until you've done a little homework!

I can't believe someone is actually trying to throw rule 8 at ff123, of all people! 
Title: MP3 at 128kbps public listening test
Post by: Atlantis on 2003-12-12 23:07:28
Quote
I don't know how you deal with CBR or VBR/ABR. Probably stick with one or the other. Seems unfair to mix both, but then I can't see why you would use lame CBR when ABR is so good, whereas I've never seen anyone claim that FhG VBR beats FhG CBR around this bitrate (maybe no one around here uses it). Difficult to answer all questions in six samples, so decide: do you want to compare encoders, or modes (ABR/CBR) - because I don't think you can comprehensively to both in one test.

----

P.S. lame -h -b 128 beats lame --alt-preset cbr 128 on at least two samples, so nothing is clear cut and predictable!

Hi,
I think it could be useful (or, at least I am really interested ) to have a listening test to compare CBR 128 versus ABR 128, and which encoder sounds better.
It could be split in four parts: the first one would be aimed at choosing the best commandline for ABR, the second would be aimed at finding the best encoder for ABR, the third aimed at finding the best CBR and the fourth will be a direct comparison of ABR vs CBR.

It is obviously a long and difficult test, but maybe it can help to clear some questions.

What do you think: is it worth trying?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-13 00:39:22
Quote
What do you think: is it worth trying?

What I wonder is if there is still that much interest in MP3 these days that would justify giving it so much attention. The people that are after listening tests and comparisions (I.E, the target audience of my tests) probably already know about modern formats, so they would probably only use MP3 for hardware compatibility.
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-13 01:29:56
Considering that 99% of downloadable audio is 128kbps mp3 (old & new material) & hardware mp3 support is being stuck in almost every DVD player, a lot of discmans, car head units, mobile phones & even digital cameras I think it's highly relevant. I also would want really fastenc -hq (slowenc) in the test because it is very different to other encoders & as f123 has said for people with better higher frequency hearing it is a great performer.

I also vote for a pre-test for fastenc settings (lowpass?) & CBR+VBR.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-13 01:37:06
Quote
I also vote for a pre-test for fastenc settings (lowpass?) & CBR+VBR.

Well, anyone willing to organize this quick pre-test?

I would do so myself, but I'll travel soon for the holidays and will only be back by mid-January.

I don't think several samples and countless listeners are needed. Just half dozen samples would be enough, since we only need to have an idea of what include in the test.

Ideas?
Title: MP3 at 128kbps public listening test
Post by: danchr on 2003-12-13 13:24:03
Quote
I also vote for a pre-test for fastenc settings (lowpass?) & CBR+VBR.

The same should be done for iTunes, really. If anyone's interested, I could encode a few samples using iTunes VBR & CBR.

First, I'll encode a couple of CD's of different genres to see what setting matches 128kbps VBR best.
Title: MP3 at 128kbps public listening test
Post by: de Mon on 2003-12-14 11:29:07
Quote
MWAHAHAHA. OK then, I'll keep some of the samples and replace others, and I'll keep the list of what stays and what goes secret until the date of the test

I agree with you, that should avoid claims of tampering.

I think vice versa. The list of samples MUST BE publicated so nobody will say there were tampering.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-14 15:11:18
Quote
I think vice versa. The list of samples MUST BE publicated so nobody will say there were tampering.

Right, but if the list is publicated before the test start, people might say some developer tweaked his encoder exactly for those samples.
Title: MP3 at 128kbps public listening test
Post by: de Mon on 2003-12-14 16:45:45
Quote
Quote
I think vice versa. The list of samples MUST BE publicated so nobody will say there were tampering.

Right, but if the list is publicated before the test start, people might say some developer tweaked his encoder exactly for those samples.

Hmmmm...
You can try to do the list so difficult so tuning to 1-3 samples will distune the codec for the others  . Isn't it possible? If it isn't. You could publicate the list and fix all the participants current versions. I think there can't be any REVOLUTION in 2-3 monthes. Huh? By the way how many samples will you use?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-14 16:55:38
Quote
Isn't it possible?

Might be, if I were a developer and knew exactly what feature in a sample would break others when trying to tune for it.

Quote
I think there can't be any REVOLUTION in 2-3 monthes. Huh?


Yeah, I think I'm being overcautious.

But I believe Ivan will try to fix the issues in EnolaGay

Quote
By the way how many samples will you use?


12, as usual.

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: Loke on 2003-12-14 17:45:41
Just wondering
Since different encoders use different frequency-cutoff , is the comparison fair? I mean, a clip isn't the same clip anymore when each encoder is lowpassing it before encoding. How about finding out what frequency the most frequency-cutting-encoder is lowpassing at, and then lowpassing each sample in cooledit or some prog., according to the found frequency. Then mp3encode the samples. Would that be a better/worse comparison?

Just a thought....
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-14 17:53:23
Quote
Just wondering
Since different encoders use different frequency-cutoff , is the comparison fair? I mean, a clip isn't the same clip anymore when each encoder is lowpassing it before encoding. How about finding out what frequency the most frequency-cutting-encoder is lowpassing at, and then lowpassing each sample in cooledit or some prog., according to the found frequency. Then mp3encode the samples. Would that be a better/worse comparison?

If an encoder is lowpassing too much or too few, it's his fault. And therefore it should be punished for it, and not bring every other codec down to the same (bad) level.
Title: MP3 at 128kbps public listening test
Post by: de Mon on 2003-12-14 22:31:32
Quote
Quote
I think there can't be any REVOLUTION in 2-3 monthes. Huh?


Yeah, I think I'm being overcautious.

But I believe Ivan will try to fix the issues in EnolaGay


You didn't understood me on this statement.
I mean you can publicate the list _today_ and use the last for _today_ encoders versions. They can't evolute much till testing starts so _today_ versions will be almost up to date versions in 2-3 monthes and the test will be almost recent too.
Or even we the developer of encoder will make tuning it speaks of good support of encoder 
Title: MP3 at 128kbps public listening test
Post by: westgroveg on 2003-12-17 08:27:57
Quote
Quote
Just wondering
Since different encoders use different frequency-cutoff , is the comparison fair? I mean, a clip isn't the same clip anymore when each encoder is lowpassing it before encoding. How about finding out what frequency the most frequency-cutting-encoder is lowpassing at, and then lowpassing each sample in cooledit or some prog., according to the found frequency. Then mp3encode the samples. Would that be a better/worse comparison?

If an encoder is lowpassing too much or too few, it's his fault. And therefore it should be punished for it, and not bring every other codec down to the same (bad) level.

I think test results would be more accurate if all encoders used the same lowpass where possible (e.g if the encoder allows, Cool Edit 2.1). After all lame is allowed to use the alt-preset command line other encoders should also be tweaked where possible. Aren't we trying to find the best performing encoder? then we should tweak all the encoders the best possible.

Edit: Someone might ABX an encoder "dull sounding" just because it uses 16khz lowpass
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-17 08:44:54
You should not tweak yourself the encoders, otherwise the result won't be representative of the usual encoder results.

Moreover, it is likely that the codec developers know better than you which lowpass value to use.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-22 15:55:10
Hello.

Some last points before I go on a trip (I'll only visit the boards seldomly until Jan 10th).

About VBR vs. CBR:

There are good reasons for either method. They are:

CBR is what newbies use once they start playing with MP3. Only later some of them come to know about VBR and ABR. Also, it's the default mode in most MP3 encoders: Lame, FhG, Xing... (or the only mode in some cases: Radium, Audioactive, iTunes)

Also, CBR is best to avoid criticism on test fairness, since there is no bitrate deviation.

VBR/ABR is what should be used to take the most out of each codec at the chosen bitrate. Lame is known for performing quite better at ABR than CBR, and I believe the same applies to FhG. In this case, Lame, FhG and Xing would use VBR encoding, and Audioactive/Radium and iTunes would use CBR.

No-CBR for Lame only is definitely out of the question. The test would look scandalously biased if Lame was tested at ABR and everything else at CBR.

IMO, the bottom point is if we want the test results to appeal to newbies or enlightened HA users. If we go for newbies, the test should be CBR. If we go for veterans, the test should be ABR.

Any thoughts?


About Lame version:

I think it's closely related to the newbies vs. veterans issue. If the test is done for veterans, Lame 3.90.3 should be featured. If it is for newbies, Lame 3.93.1, since this is the version you'll find in CDex, Winamp, and at Mitiok's site. I think that actually the only popular Lame binary distribution point featuring 3.90.3 is RareWares.


About Audioactive vs. Radium

It is said that both use similar encoding routines, so I would definitely test only one of them. The appeal of Radium is that it is very widely used in DivX rips. The appeal of Audioactive is that it seems to be based on more recent  - and therefore more tweaked - libraries, so quality is probably better.


About FhG version:

I'm inclined to test FhG Current and Legacy Fast from Adobe Audition, both at some VBR mode that comes close to 128kbps (in case we decide to go for VBR). Legacy Slow is - as the name implies - too slow and, from what it is said, fast is actually better (!).

If only one FhG codec is featured (that would be Audition Current), I might as well replace the other with Gogo 3.x


So, the codec list would be as follows:

Lame 3.90.3 or 3.93.1
FhG Current in Audition
FhG Legacy Fast or Gogo 3.x
iTunes 4.2
Audioactive 2.04j or Radium
RealOne MP3 encoder (Xing)


And the settings would depend on the choice for VBR or CBR.


Please post your comments and suggestions.

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: Latexxx on 2003-12-22 16:45:43
I'd like to see lame -br 128 as an anchor because all lamers just set their cdex to rip at that bitrate without touching the other setteings. The quality difference (???) between the br-switch and preset might even force some of those nasty pirate gruops to switch to presets. (But on the other hand lame 3.94 will force them to switch to presets but they'll still use stereo instead of joint stereo because they are stupid  )
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-22 16:59:49
Good point. But the low anchor is meant to sound bad, and I'm afraid Lame CBR could sound better than iTunes and Radium/Audioactive...

That's why I plan to use Xing as anchor.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2003-12-22 17:33:51
Please, leave Xing as anchor.

Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-22 18:01:48
Quote
Please, leave Xing as anchor.

I will, most probably.

Quote
Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94


But 3.94 is still beta. It's not a good idea to use beta stuff in listening tests, IMO, because people can easily claim the codec performed badly (if that's the case) because if some beta flaw. I personally would prefer to use a release version, either 3.90.3 or 3.93.1

Unless you plan to make Lame 3.94 stable until ~Jan 11th
Title: MP3 at 128kbps public listening test
Post by: schnofler on 2003-12-22 19:48:16
I'm still in favor of an ABR/VBR test.
Quote
IMO, the bottom point is if we want the test results to appeal to newbies or enlightened HA users. If we go for newbies, the test should be CBR. If we go for veterans, the test should be ABR.

I don't agree with you on this point. I think if you're a newbie and you're reading listening test results, that's probably because you want to find out what to use to make your encodings sound as good as possible (with MP3 at that specific bitrate). And if you want your encodings to sound good, you should use ABR/VBR. So I don't think CBR would be interesting for newbies. There is one reason I can think of for using CBR, though. The results of such a test might be useful for people who want to find out the quality of MP3s they didn't encode themselves (I probably don't have to mention where those MP3s come from, usually).

As for the question what Lame version to use, I'd go with 3.90.3, although I don't think it makes much of a difference, because both versions will likely be superceded by 3.94 in the near future.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-22 21:41:43
Quote
About FhG version:

I'm inclined to test FhG Current and Legacy Fast from Adobe Audition, both at some VBR mode that comes close to 128kbps (in case we decide to go for VBR). Legacy Slow is - as the name implies - too slow and, from what it is said, fast is actually better (!).

If both Legacy Fast and Legacy Slow actually encode with VBR, then they are probably the same codec -- the FhG fast codec, because that's the only one which does VBR.

In Cool Edit Pro 2.1, they had actually removed the FhG slow codec from the library, and made quality settings based on only the fast codec.

If that is really the case, then I would choose legacy slow for the FhG codec.  Remember that you can try blackbird.wav to find out if it has glitches in it or not.  That would indicate whether Legacy Slow really is the FhG slow codec or not.

ff123
Title: MP3 at 128kbps public listening test
Post by: Jojo on 2003-12-22 22:28:07
Quote
Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94

hmm, what about LAME 3.92 :confused:
Title: MP3 at 128kbps public listening test
Post by: bidz on 2003-12-22 22:37:23
Quote
CBR is what newbies use once they start playing with MP3. Only later some of them come to know about VBR and ABR. Also, it's the default mode in most MP3 encoders: Lame, FhG, Xing... (or the only mode in some cases: Radium, Audioactive, iTunes)

FYI, iTunes supports MP3 VBR encoding. Press "Custom" and it'll pop-up a configuration dialog, and you can check "Use Variable Bitrate Encoding (VBR)"

When in VBR mode, the bitrate you choose is the lowest bitrate the encoder will use. And you can select between these presets:

Lowest
Low
Medium Low
Medium
Medium High
High
Highest

and also select stereo/joint stereo, sample rate and lowpass filter.

Also EncSpot detects the iTunes files as: FhG (fastenc or mp3enc)
Title: MP3 at 128kbps public listening test
Post by: Steve999 on 2003-12-22 23:11:48
As an actual real-life newbie, I'd most want to see comparisons of VBR recordings that average or approximate 128 kbps.  That's what I could best use to make practical decisions. 
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-22 23:40:12
Quote
I don't agree with you on this point. I think if you're a newbie and you're reading listening test results, that's probably because you want to find out what to use to make your encodings sound as good as possible (with MP3 at that specific bitrate).

Very good point. Maybe I'll go with ABR/VBR after all.

Quote
If both Legacy Fast and Legacy Slow actually encode with VBR, then they are probably the same codec -- the FhG fast codec, because that's the only one which does VBR.


oh, I see. I thought Legacy Slow was mp3enc. Thanks for this information. I'll test blackbird.wav when I return home, and if it isn't the FhG Slow codec, I'll replace Legacy Fast with it.

Quote
hmm, what about LAME 3.92 :confused:


I chose Lame 3.90.3 because it's the HA branch, and 3.93.1 because it's the latest stable release. Since 3.93.1 came after 3.92, I see no point in using a deprecated version.

Quote
FYI, iTunes supports MP3 VBR encoding. Press "Custom" and it'll pop-up a configuration dialog, and you can check "Use Variable Bitrate Encoding (VBR)"


Ah, that's useful information. Thank-you for mentioning it.

Quote
Also EncSpot detects the iTunes files as: FhG (fastenc or mp3enc)


EncSpot is wrong. It has already been proven earlier in this thread (look for a link to the lame mailing list) that iTunes is based on SoundJam, that on it's turn is an independent tweak line from dist10.

Thank-you for your replies so far.

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2003-12-31 17:51:53
OK, I'm getting close to freezing the encoders and settings.

My ideas:

-FhG Audition Current VBR
-FhG Audition Legacy Fast or Slow VBR (more on that later)
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR

I'll test the VBR settings of Audition, Xing and iTunes to find out what comes closer to 128kbps overall.

About FhG Legacy Fast vs. Legacy Slow: I'll use slow if it's not the MP3enc branch. But I need someone to test it using Blackbird.wav (like ff123 suggested). I can't test now because I'm on a trip.
More info on identifying MP3enc: http://ff123.net/identify.html (http://ff123.net/identify.html)

About Lame version: I plan to test Lame 3.90.3, unless 3.94 is released until the test date (since 3.94 seems to feature major improvements compared to 3.90.X)

If everything goes as planned, the test will start on January 14th.

Please comment.

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: ff123 on 2003-12-31 19:58:54
Ok,

I acquired Adobe Audition 1.0 so I could take a look at the various mp3 codecs in it.

1.
Preset:  VBR - Good Quality Stereo
VBR, MP3
70 - (125-175 Kbps), High Quality
Max Bandwidth 14112 Hz
VBR Quality 70
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is unchecked
Allow Intensity Joint Stereo is unchecked
Allow Narrowing of Stereo Image is unchecked

Result:  not the super slow codec, as found within fastencc.exe 1.02 using the -hq switch.

2.
Preset:  128 Kbps Stereo (Internet)
CBR, MP3
128 Kbps, 44100 Hz, Stereo (11.0:1)
Max Bandwidth 22050
CBR Bitrate 128 Kbps
Sample Rate 44100 Hz
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is checked
Allow Intensity Joint Stereo is checked
Allow Narrowing of Stereo Image is unchecked

Result:  not the super slow codec

3.
Preset:  128 Kbps Stereo (Internet)
CBR, MP3
128 Kbps, 44100 Hz, Stereo (11.0:1)
Max Bandwidth 22050
CBR Bitrate 128 Kbps
Sample Rate 44100 Hz
Codec:  Current - Best Quality
Allow Mid-Side Joint Stereo is checked
Allow Intensity Joint Stereo is checked
Allow Narrowing of Stereo Image is unchecked

Result:  not the super slow codec

4.
Preset:  VBR - High Quality Stereo
VBR, MP3
Changed setting to 70 - (125-175 Kbps), High Quality
Max Bandwidth 17660 Hz
VBR Quality 70
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is unchecked
Allow Intensity Joint Stereo is unchecked
Allow Narrowing of Stereo Image is unchecked

Result:  not the super slow codec

So, as I suspected, the super slow code no longer exists in the FhG library.  I would choose either "128 Kpbs Stereo (Internet)" or "VBR - High Quality Stereo" as the base presets.

Using "VBR - Good Quality Stereo" will set the Max Bandwidth too low, in my opinion.  If using High Quality VBR, the default VBR setting is 100 and the default codec is Legacy - Medium Quality (Fast).  You'll probably have to at least change the VBR setting.  Probably either vbr 50 or 60 is closest to 128 kbps.

On main_theme.wav, at least, vbr 50 is an improvement over cbr 128.  It didn't matter whether I used legacy fast or current for the vbr encode.  Either one sounded better than cbr 128 (current).

So my suggestion would be to use the "VBR - High Quality Stereo" preset, and change the VBR setting to either 50 or 60, and change the codec to "Current - Best Quality."  I think that all the codecs within Audition are variations on the Basic FastEnc, so you should just choose one.

ff123

Edit:  Possibly the reason why VBR sounds better on main_theme.wav is because joint-stereo is disallowed (although I didn't verify with EncSpot that this is true for VBR 50).
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-02 16:14:22
Thanks for the help, ff123. As usual, I owe a lot to you


So, the updated codec list:

-FhG Audition Current VBR 50-60
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR - anchor


Now, the usual request for opinions:

Do you guys prefer that I leave the codec list at that (only 5 encoders featured) or that I replace Audition Legacy with Gogo, or some other encoder?

Also, since this is a mid-bitrate test, it might be a good idea to replace one or two of the samples with problem cases. Do you agree?

Thanks a lot.

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-02 17:31:14
I like the idea of a killer-sample. This will show to everybody a good exemple of failure of mp3 at this bitrate. Why not the good old friend of mp3: fatboy?
Title: MP3 at 128kbps public listening test
Post by: DigitalDictator on 2004-01-02 19:19:37
I'd like to see how much quality was sacrificed for speed with GOGO. IMO it's always been left out in tests, not among the best encoders but not really bad either. Kind of in the middle, almost boring:) The quality is basically (again IMO) estimated (i.e. it SHOULD perform this... and IS LIKELY to sound like that...)
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2004-01-02 20:26:52
Quote
-FhG Audition Current VBR 50-60
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR - anchor
I'd love to have a Lame ABR-CBR comparison, this way we would know, how large the difference (which I think is undisputed) is compared to codec differences. And there are still scenarios where the CBR compatibility is needed.

Quote
Also, since this is a mid-bitrate test, it might be a good idea to replace one or two of the samples with problem cases. Do you agree?
I'm not sure. Could a few problem cases skew the total ratings? Problem sample rating differences will probably be larger (as more of the scale is used), and hence have more influence on the overall scores. I am not sure if this is a desired effect.
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-02 20:42:31
Quote
Could a few problem cases skew the total ratings? Problem sample rating differences will probably be larger (as more of the scale is used), and hence have more influence on the overall scores. I am not sure if this is a desired effect.

As long as the test is comparing various encoders of one single format, I think that there are no problems to incude a killer-sample. Fatboy for exemple is destroyed by each encoders at this bitrate. It's not a lame-killer only, or a typical fhg-problem sample.

Of course, bad notations will affect the overall scores at the end of the test. On the other side, bad surprises may happen in real life with mp3 at this bitrate; so, I think that introducing a killer in the arena isn't a bad thing.
Title: MP3 at 128kbps public listening test
Post by: LoFiYo on 2004-01-02 22:36:19
Quote
I'd like to see how much quality was sacrificed for speed with GOGO. IMO it's always been left out in tests, not among the best encoders but not really bad either. Kind of in the middle, almost boring:) The quality is basically (again IMO) estimated (i.e. it SHOULD perform this... and IS LIKELY to sound like that...)

Why don't we test GOGO 3.12 once and for all, and see if it's really not that good in quality? If it's no good, then we can stop talking about it (until an improved version is released).
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-02 22:55:05
Quote
Why don't we test GOGO 3.12 once and for all, and see if it's really not that good in quality? If it's no good, then we can stop talking about it (until an improved version is released).

It's not a bad idea, but if we include gogo, there will be 6 encoders to rate. Five encoders are enough for a lot of people, maybe too much!

Nevertheless, if Roberto launch a mp3 general test, why not imagine a complete one. After all, will there be enough progress with MP3 to expect any similar test in the next years? I'm not sure.
With 6 challengers, we'll have :

- the good : lame
- the bad : Audioactive Pro
- the ugly : Xing
- the fast : gogo
- the alternative : iTunes (MAC users)
- the expensive : Audition

Good script, no? As sample, why not Morricone?
Title: MP3 at 128kbps public listening test
Post by: bond on 2004-01-02 22:59:49
and mp4pro at 64kbps
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2004-01-03 08:16:04
Quote
and mp4pro at 64kbps

Hmm. Doesn't the 64kbit Test (http://audio.ciara.us/test/64test/results.html) tell you everything you want to know about it? (HE-AAC 64 against Lame 128)
Title: MP3 at 128kbps public listening test
Post by: tycho on 2004-01-03 10:27:00
Just wondering what settings you planned for iTunes VBR.  Obviously you should set Quality to "Highest" and leave "Joint Stereo". But the bitrate settings is for the lower bound. I.e. 128 will give an avg. about 140 or so. and 112, produces an avg. around 120 (strangely this is independent of Quality settings). So which one to choose?

guruboolez: iTunes is not for MAC users only - its for PCs, too.
Title: MP3 at 128kbps public listening test
Post by: AstralStorm on 2004-01-03 10:48:58
It'd be fun to test Lame 3.90.3/3.93.1 vs 3.94b, but I think this isn't the scope of this test...

(mabe with just plain -b 128 to test the effectiveness of automatic presets)
Title: MP3 at 128kbps public listening test
Post by: danchr on 2004-01-03 12:44:27
Quote
guruboolez: iTunes is not for MAC users only - its for PCs, too.

<nitpick mode>
It's Mac, not MAC. MAC (http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=MAC+Address) is something your ethernet card has, while Mac is short for Macintosh (http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=Macintosh).
</nitpick mode>
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2004-01-03 12:53:24
OT:
Quote
<nitpick mode>
It's Mac, not MAC. MAC (http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=MAC+Address) is something your ethernet card has, while Mac is short for Macintosh (http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=Macintosh).
</nitpick mode>

Then why do you use "mac" in your signature?
Really, all-caps is just a way of emphasizing...
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-03 15:14:33
Good... lots of replies

So, I reckon Gogo is wellcome? In this case, it'll be featured instead of Audition Legacy.

@Guruboolez: I don't think 6 codecs is too much considering some of them will (theoretically) sound bad. And I agree, I don't think there will be more MP3 tests happening, so it's better if this one covers most significative encoders.

Also, I agree with the idea of adding Fatboy. But, in this case, what sample will be replaced?

For those that don't remember the sample list:
http://www.hydrogenaudio.org/forums/index....howtopic=12358& (http://www.hydrogenaudio.org/forums/index.php?showtopic=12358&)

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-03 16:42:10
I still think that you should consider new Lame. Perhaps a new release could be available soon...
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-03 16:43:18
How "soon"? 
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-03 17:01:07
Well, probably between 1 and 2 weeks.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-03 17:09:38
I plan to start my test on the 14th.

So please try to hurry
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-03 17:51:55
According to the fact that 3.90.3 is the same as 3.90 for abr encodings, I'd like to see the newest lame encoder in the test. After all, lame 3.90 is more than two years old now. I know it was heavily tested; but then, why not considering this public listening test as a first step for a complete checking? Even if 3.94 is beta... (recommanded version of musepack is still alpha...)
Title: MP3 at 128kbps public listening test
Post by: LoFiYo on 2004-01-03 18:57:05
Quote
According to the fact that 3.90.3 is the same as 3.90 for abr encodings, I'd like to see the newest lame encoder in the test. After all, lame 3.90 is more than two years old now. I know it was heavily tested; but then, why not considering this public listening test as a first step for a complete checking? Even if 3.94 is beta... (recommanded version of musepack is still alpha...)

I would like to see whether the technical improvements since 3.90x (substep quantization and all the other quality improvement changes detailed in history.html) will really result in better quality than the time-tested version like 3.90.3  .
Title: MP3 at 128kbps public listening test
Post by: eltoder on 2004-01-03 20:10:11
Quote
According to the fact that 3.90.3 is the same as 3.90 for abr encodings, I'd like to see the newest lame encoder in the test. After all, lame 3.90 is more than two years old now. I know it was heavily tested; but then, why not considering this public listening test as a first step for a complete checking? Even if 3.94 is beta... (recommanded version of musepack is still alpha...)

Guruboolez, may be you could do a presonal listening test between 3.90.3 and 3.94 like you did for WMA and Vorbis before previous test? This will not only help with test decision, but also provide some usefull information.

-Eugene
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2004-01-03 20:17:35
I think before a build from the 3.94 branch can be considered, all known problems (e.g. this one (http://www.hydrogenaudio.org/forums/index.php?showtopic=16585&st=50&&#entry165553) ) should be resolved, and a short comparison test (perhaps only individual) should be performed.
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-03 22:02:10
eltoder> if I find time enough and some motivation, why not.
Continuum> I don't see any problem in the mentioned thread with abr/cbr settings ??
Title: MP3 at 128kbps public listening test
Post by: Continuum on 2004-01-04 07:06:49
Quote
quickly ABXed fsol.flac
with preset 128
(3.90.3 and 3.94b1)

both 14/14 but 3.90.3 is (much) better

-lazka

But I don't know the status of that.
Title: MP3 at 128kbps public listening test
Post by: AstralStorm on 2004-01-04 07:40:58
That opinion on quality can be biased, he ought to use HR program for quality comparison.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-12 06:35:44
Well, the test is almost upon us.

I already selected the switches for each encoder. This is what we will have:

-Lame 3.95 --preset 128
-FhG Audition VBR quality 40, only mid/side stereo enabled, Current - Best codec
-iTunes VBR Highest quality, 112kbps base bitrate, joint stereo, smart encoding
-AudioActive CBR 128kbps, high quality
-Gogo 3.12 -b 128 -a
-Xing 1.5 VBR quality normal

That list can be considered soft-frozen, it will only change if some major issue is brought to my attention.

Also, it is my pleasure to inform you that all VBR codecs are in the range of 120-128kbps. So, no issues should arise about bitrate deviation.


About why Xing instead of RealOne:
RealOne and RealPlayer10 beta are two flaming pieces of garbage that refuse to install on my PC. Surprisingly, the installation process goes well and at the end it claims installation succesful, down to the requirement of restarting the PC (#%$&@!). But when I check the installation folder (whether before or after restarting), it's nearly empty and, not surprisingly, double clicking the .exe has no effect.

Since I can't be bothered to deal with such shitty software for an anchor, and I also won't risk a friendship asking a friend to try installing it on his PC, I gave up RealOne altogether and went back to Xing 1.5, that installed and ran perfectly. Real should take some hints from XingTech...


I didn't replace any sample with fatboy because noone suggested what sample to replace. So this test will have no problem cases, except the very evil and very interesting Waiting.

Next test will definitely feature evil samples.


Also, I would like to thank Gabriel Bouvigne and the rest of the (active) Lame team for rushing the 3.95 release on time for this test.

The schedule is kept, if there's no catastrophe until then, it will start on the 14th, next wednesday.

Best regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-12 07:24:13
Quote
I didn't replace any sample with fatboy because noone suggested what sample to replace. So this test will have no problem cases, except the very evil and very interesting Waiting.


I would have liked fatboy...
Why no setting up a poll for which sample to replace?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-12 12:29:33
Quote
I would have liked fatboy...
Why no setting up a poll for which sample to replace?

Because it's already too late. :/

When I asked about it (Jan 3rd) there was plenty of time. Now there are only two days.
Title: MP3 at 128kbps public listening test
Post by: danchr on 2004-01-12 12:48:46
What about LAME 3.90.3 <-> 3.95? Will that be "only" testing & tuning by those interested in it?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-12 12:52:30
Quote
What about LAME 3.90.3 <-> 3.95? Will that be "only" testing & tuning by those interested in it?

No 3.90.3 anymore. There are already too many codecs as it is.

And since 3.90.3 has already been tested a lot (128kbps extension and 64kbps test), I decided it would be better to try something newer this time.
Title: MP3 at 128kbps public listening test
Post by: AstralStorm on 2004-01-12 16:35:36
As the samples are same in 64kbps test and this one,
we can compare 3.95 vs 3.90.3 indirectly.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-12 17:51:00
Erm.. actually, I prefer you don't even try to do that :B

In the 64kbps test Lame was the higher anchor, so it was meant to win. Comparing lame to 64kbps samples, it's obvious it'll receive high scores. OTOH, in this test Lame will be amongst it's peers, so it's score will probably be lower even if 3.95 isn't worse in reality.

An example. The exact same Lame version, compile and settings were used in the 64kbps test and the 128 extension test. Still, at the 128 test Lame got a score of 3.66 points, while in the 64kbps test it got a score of 4.29.

Regards;

Roberto.
Title: MP3 at 128kbps public listening test
Post by: LoFiYo on 2004-01-12 19:37:14
Quote
-Gogo 3.12 -b 128 -a

Since Lame is using the optimal 128kbps setting (--preset 128), gogo might as well use -b 128 -a -q 0. Would you allow that?
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-12 20:47:35
Quote
Would you allow that?

No problem.

Thanks for the suggestion.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-13 08:12:06
I'd like to point that with 3.95, targetting 128kbps I think that -q1 and -q0 can really improve encoding quality.

On the other hand, they might not be representative of real-world use due to their slowness.
Title: MP3 at 128kbps public listening test
Post by: Jojo on 2004-01-13 13:01:40
Quote
I'd like to point that with 3.95, targetting 128kbps I think that -q1 and -q0 can really improve encoding quality.

is it possible to use this setting along with aps? How much longer would that take? I mean it says in the ChangeLog that the new release is 10% faster, so what's the deal? I rather wait a bit longer and have a smaller file or a file with better quality...
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-13 13:27:39
Quote
is it possible to use this setting along with aps?

I think that this is off-topic and would lead to endless debates.
I was speaking about 128kbps.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-13 14:31:44
I think its not a good idea tweaking the Lame settings for my test. It goes against the HA mantra of "NEVAH tweak the presets", so it would not be representative of what forum readers use.
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-13 14:37:12
Well, it's not that much tweaking, after all in 3.95 --preset 128 is just --abr 128. But I agree that it is not very representative of real use.
Title: MP3 at 128kbps public listening test
Post by: rossthiof on 2004-01-13 14:51:58
Gabriel, could you please tell me the tweaked preset for a bitrate around 128 kbps ?
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-13 15:03:47
Quote
Gabriel, could you please tell me the tweaked preset for a bitrate around 128 kbps ?


--preset 128
Title: MP3 at 128kbps public listening test
Post by: guruboolez on 2004-01-13 15:31:00
Quote
Well, it's not that much tweaking, after all in 3.95 --preset 128 is just --abr 128. But I agree that it is not very representative of real use.

It would be nice if some users will perform individual tests with --abr 128 and --abr 128 -q0, before the end of the test. Then Roberto may post something like an appendix, or a comment as conclusion about the (possible) benefits of the -q0 switch. What do you think?
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-13 15:43:22
I think that at 128kbps the benefits of q0 regarding quality are clear. But it does not represent how users will be using Lame.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-13 15:51:25
I agree with Guruboolez' idea, an appendix would indeed be better, specially considering usage representativity.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-13 20:03:42
Hello.

I'd like to ask if anyone here is experiencing problems to reach the listening tests site -
http://audio.ciara.us/test/ (http://audio.ciara.us/test/)

It's been weeks since I could last log into it. If it's only a DNS problem in my ISP there are no worries, it should get fixed sooner or later. But if more people are having issues, I'll need to move it again.

Verloren, can you help? I can't login through my FTP account either.

Also, I couldn't login using HA's SSH server + lynx.
Title: MP3 at 128kbps public listening test
Post by: evereux on 2004-01-13 20:06:05
rjamorim, No problem reaching the page here.
Title: MP3 at 128kbps public listening test
Post by: Atlantis on 2004-01-13 20:42:13
All fine from here, Roberto.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-13 21:10:27
Weird...

What's worse, I can't use the FTP account to upload the presentation page. So I guess I'll have to move the site to rjamorim.com anyway
Title: MP3 at 128kbps public listening test
Post by: AstralStorm on 2004-01-13 21:24:52
Quote
Quote
I'd like to point that with 3.95, targetting 128kbps I think that -q1 and -q0 can really improve encoding quality.

is it possible to use this setting along with aps? How much longer would that take? I mean it says in the ChangeLog that the new release is 10% faster, so what's the deal? I rather wait a bit longer and have a smaller file or a file with better quality...

Yes, of course it is. Just add -q ??? after --preset standard.
But the test should be based on standard setting.

Very short speed test:

Code: [Select]
C:\Music\LAMEsamples>c:\Lame3-90-3\lame.exe --preset standard fatboy.wav
LAME version 3.90.3 MMX  (http://www.mp3dev.org/)
CPU features: i387, MMX (ASM used), 3DNow!, SIMD
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding fatboy.wav to fatboy.wav.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=2
   Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  191/193    (99%)|    0:02/    0:02|    0:02/    0:02|   2.1150x|    0:00
32 [  1] *
128 [  2] %%
160 [ 24] %%%%%%%%%%********
192 [ 14] %%%%%%%%***
224 [ 13] %%********
256 [ 47] %%********************************
320 [ 92] ******************************************************************
average: 265.3 kbps   LR: 30 (15.54%)   MS: 163 (84.46%)

Writing LAME Tag...done

C:\Music\LAMEsamples>c:\Lame\lame.exe --preset standard -q 0 fatboy.wav
LAME version 3.95 MMX  (http://www.mp3dev.org/)
CPU features: MMX (ASM used), 3DNow! (ASM used), SSE
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding fatboy.wav to fatboy.wav.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=0
   Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  191/193    (99%)|    0:04/    0:04|    0:04/    0:04|   1.1611x|    0:00
32 [  1] *
96 [  2] %*
112 [  3] %%
128 [ 11] %%******
160 [ 11] %%%%****
192 [  6] %***
224 [ 16] %%%%*******
256 [ 40] %%%%%%%*******************
320 [103] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%************************************
average: 267.7 kbps   LR: 73 (37.82%)   MS: 120 (62.18%)

Writing LAME Tag...done
ReplayGain: -2.2dB

C:\Music\LAMEsamples>c:\Lame\lame.exe --preset standard -q 1 fatboy.wav
LAME version 3.95 MMX  (http://www.mp3dev.org/)
CPU features: MMX (ASM used), 3DNow! (ASM used), SSE
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding fatboy.wav to fatboy.wav.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=1
   Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  191/193    (99%)|    0:02/    0:02|    0:02/    0:02|   2.0867x|    0:00
32 [  1] *
96 [  2] %*
112 [  3] %*
128 [ 13] %%%******
160 [  7] %%%**
192 [  8] ******
224 [ 15] %%%%******
256 [ 39] %%%%%%%******************
320 [105] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%***********************************
average: 268.5 kbps   LR: 73 (37.82%)   MS: 120 (62.18%)

Writing LAME Tag...done
ReplayGain: -2.2dB

C:\Music\LAMEsamples>c:\Lame\lame.exe --preset standard -q 2 fatboy.wav
LAME version 3.95 MMX  (http://www.mp3dev.org/)
CPU features: MMX (ASM used), 3DNow! (ASM used), SSE
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding fatboy.wav to fatboy.wav.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=2
   Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  191/193    (99%)|    0:01/    0:01|    0:01/    0:01|   4.4350x|    0:00
32 [  1] *
96 [  2] %*
112 [  6] %%**
128 [  6] %%**
160 [ 11] %%%*****
192 [  4] %%*
224 [ 24] %%%%************
256 [ 36] %%%%%%******************
320 [103] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%***********************************
average: 268.1 kbps   LR: 73 (37.82%)   MS: 120 (62.18%)

Writing LAME Tag...done
ReplayGain: -2.2dB

C:\Music\LAMEsamples>c:\Lame\lame.exe --preset standard fatboy.wav
LAME version 3.95 MMX  (http://www.mp3dev.org/)
CPU features: MMX (ASM used), 3DNow! (ASM used), SSE
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding fatboy.wav to fatboy.wav.mp3
Encoding as 44.1 kHz VBR(q=2) j-stereo MPEG-1 Layer III (ca. 7.3x) qval=3
   Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  191/193    (99%)|    0:01/    0:01|    0:01/    0:01|   4.4350x|    0:00
32 [  1] *
96 [  2] %*
112 [  6] %%**
128 [  6] %%**
160 [ 11] %%%*****
192 [  4] %%*
224 [ 24] %%%%************
256 [ 36] %%%%%%******************
320 [103] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%***********************************
average: 268.1 kbps   LR: 73 (37.82%)   MS: 120 (62.18%)

Writing LAME Tag...done
ReplayGain: -2.2dB


Both compiles are ICL 7.1 by john33, most probably same optimisations...
Title: MP3 at 128kbps public listening test
Post by: Jojo on 2004-01-14 15:23:59
Quote
Well, it's not that much tweaking, after all in 3.95 --preset 128 is just --abr 128. But I agree that it is not very representative of real use.

does this actually mean that the standard setting is an abr (JS) mode instead of cbr (Stereo) like it used to be?
Title: MP3 at 128kbps public listening test
Post by: Gabriel on 2004-01-14 15:35:47
Quote
does this actually mean that the standard setting is an abr (JS) mode instead of cbr (Stereo) like it used to be?

The default encoding mode in Lame is still cbr/128/JS.

Btw if you want to know what is the default mode, you can try it.
Title: MP3 at 128kbps public listening test
Post by: S_O on 2004-01-14 16:54:31
Quote
-Xing 1.5 VBR quality normal
Is that the newest version? I just read that (by Karl Lillevold: http://www.hydrogenaudio.org/forums/index....ndpost&p=173122 (http://www.hydrogenaudio.org/forums/index.php?showtopic=17401&view=findpost&p=173122)):
Quote
P.S. I have been told the Xing encoder is much improved since its first incarnation, which is usually not well spoken of

You can also download a free plug-in supporting all bitrates encoding for RealPlayer. I would really like to see (confirm ) how bad it performs against Lame & other encoders using short blocks. I would be rather suprised not to see Xing at the last position, but this test could show the truth.
Title: MP3 at 128kbps public listening test
Post by: rjamorim on 2004-01-14 16:58:03
As you can read some posts above this one, I had no luck installing RealOne. So, no, sorry, this isn't the latest Xing. There's nothing I can do about it

BTW, test is now OPEN.
http://www.hydrogenaudio.org/show.php/showtopic/17563 (http://www.hydrogenaudio.org/show.php/showtopic/17563)