Skip to main content ↑ ↓
Topic: MP3 at 128kbps public listening test (Read 40592 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 at 128kbps public listening test

Reply #50 –
Quote
I'm in favor of this. Making some VBR or ABR is rather unfair, IMO, because in my experience, most VBR and ABR modes tend to have, on average, a bitrate that is higher or lower than 128kbps

So what? I want to know how good LAME is at APPROXIMATELY 128 kbps, doing its best. I don't give a rat's ass about CBR 128, because I know it's not as good! The same goes for Frauhofer's FastEnc and the rest of the bunch. How good can they perform around 128 kbps? Let it show me what they're made of!

Saying that VBR and ABR are not representative because the oblivious masses know only of CBR is pure BS! Heck, with that argument there's no point in having a test among different codecs! Not a single soul I know of have heard of MPC, AAC or OGG.

So maybe it'll be hard finding settings that will come close to 128 kbps on every encoder, so be it. I think every encoder should be allowed to give it its absolute best.

I know this has been beaten to death before but if CBR is the way to go on this test, people will always wonder how different encoders perform not being "crippled".
//From the barren lands of the Northsmen

MP3 at 128kbps public listening test

Reply #51 –
Quote
Add another vote for this selection! Two FhGs, one "default" the other "best quality", seem enough.

If the test is to include just two FhG's, I think one of them should be FastEnc based (i.e., default in MMJB).  This one can also be VBR if desired, since FhG VBR is FastEnc based.

I think the other should be Audioactive/Radium based.

The super-slow codec has two disadvantages:  1) it is super slow  2) it can sound both good and bad (incosistent).  For people with good high frequency hearing, it will probably have less of the "mp3" quality to it, but more low frequency glitching and dropouts.  Audioactive/Radium should be more consistent.

ff123

MP3 at 128kbps public listening test

Reply #52 –
Are we sure that Fastenc is better in vbr than cbr?

MP3 at 128kbps public listening test

Reply #53 –
So many candidates and I like almost all of them. But I would like to see specially:
1. MPC
2. The latest LAME or LAME 3.93.1
Some words about LAME:
I would like to see it ABR, but what we'll do if encodings will be 118, 132 or 135 Kbs? So I think LAME must go CBR despite it  has been developed at this bitrate as ABR codec.
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?
What hard samples will be used?
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

MP3 at 128kbps public listening test

Reply #54 –
Quote
1. MPC

I do not think that mpc would be a good candidate in a "128kbps MP3 listening test"

MP3 at 128kbps public listening test

Reply #55 –
Quote
i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.

God, no. Besides, what would be the point? All that it would show is that APS is much better than the rest. But that would be no surprise to anyone here.

Using different bitrates in a multiformat test is already a difficult idea. In a "monoformat" one, it's almost unjustifiable.

Quote
The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.


I agree. And testing lame twice at the same setting, only changing version, would look like bias (much like people complained about ff123 testing Vorbis twice at his 64kbps test).

Quote
Since we're targeting how well MP3 codecs perform at 128kbps then it makes sense to use CBR for all encoding tests. Since CBR is what most average non-HA users encode with.


Good point.

Quote
Giving LAME use of ABR and forcing the rest to use CBR seems an unfair test.


Right, it would definitely smell like bias. Specially if you consider HA is pretty biased towards Lame (for very valid reasons, I should add).

Quote
Finally, I can see why XING would make an interesting anchor but considering the ones who will actually take time examining the results, I feel HE-AAC would be more interesting. Sure, it would be somewhat misplaced but then again it would provide useful information. In my opinion XING wouldn't.


Nope, HE AAC has already been tested against MP3 at 128kbps. Maybe in my next low bitrate test, but not now.

Besides, IMO Xing would provide very useful information. To start with, I don't know of any public test featuring Xing. So, it might be that it isn't even as bad as people paint it. Besides, if it really is bad, then we'll have the hard proof.

Quote
Besides, you'll get that accusation in your 3rd test anyways, when either Nero or Quicktime (the expected winners of the AAC test, IMO) does nominally better than Vorbis (my expectations for your 128kbps extension).


My tests are completely open for peer review. I offer all the results, and they can see for themselves that I'm not cheating in the encodes by looking at the batch files and checking that I'm using standard compiles of the codecs.

So, no, I will personally tell everyone that doubt the honesty of my tests to go f*ck themselves.

What I meant is that people will believe I'm doing provocation. "neener-neener, I just did this to prove Vorbis suxxxorz and AAC rulz!"

Since the 3rd test will be multiformat, it's not like I'm comparing Vorbis directly against AAC. It's everyone against everyone. Can't be seen as provocation.

Quote
It's a test that ought to be done, dammit.


Start choosing samples and settings. I'll remind you of this test a few months from now

I'll continue in my next post.. (getting too big)

MP3 at 128kbps public listening test

Reply #56 –
Quote
I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors", Lame and FhG, and that would also mean allowing FhG to use VBR. Using only CBR doesn't make a lot of sense in my opinion. Even if it is the most common newbie setting, so what? We already have Xing in the test, which should suffice to prove that the typical "newbie settings" are not the optimum. Limiting all the other codecs to CBR would make the test very one-sided.

Right.

About FhG: Problem is that some say fastenc is actually better quality than HQ sometimes.

Anyone willing to do a quick parallel test comparing both? (using Audition or MMJ)

Quote
It contains both the fast and slow codecs. The fast codec within fastencc.exe is buggy (loss of stereo separation). The slow one (-hq) works fine.


Good to know. So maybe it's recommended to use fastencc -hq instead of Audition legacy HQ? I'm not sure, but I would expect Audition to use newer encoding libraries.

Quote
For example, in my experience, LAME ABR 128 consistently produces ABRs below 128.


Hrm... true, but still it behaves very well.
http://audio.ciara.us/test/128extension/results.html

The average Lame bitrate there was 124.5, never going above 133 or under 121. I think that's an acceptable error margin.

Also, quality in Adobe Audition goes from 1 to 100 in steps of 1, so it would be easy to find a VBR setting that outputs results close to 128.

But we must keep in mind what users want. And I'm still not sure if people are more interested in knowing CBR results or ABR/VBR results.

Quote
I TOTALLY AGREE!


OMG! Caps lock.

Quote
The super-slow codec has two disadvantages: 1) it is super slow 2) it can sound both good and bad (incosistent). For people with good high frequency hearing, it will probably have less of the "mp3" quality to it, but more low frequency glitching and dropouts. Audioactive/Radium should be more consistent.


That's very valuable information. Thanks for mentioning it, ff123.

Definitely something worth considering.

BTW: super-slow = mp3enc 3.1?

Quote
I would like to see it ABR, but what we'll do if encodings will be 118, 132 or 135 Kbs?


We would have to find out an acceptable error margin (I think something like 8kbps would be OK) and make sure no sample goes above or below this margin.

Quote
I do not think that mpc would be a good candidate in a "128kbps MP3 listening test"




Quote
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?


Default.

And I think the only really crappy Joint Stereo in FhG codecs is in the fast fastencc encoder. And that one won't be used.

Quote
What hard samples will be used?


Do you guys think it's needed to replace one of the 64kbps test samples with a difficult one?

Again, thanks a lot for your input.

Regards;

Roberto.

MP3 at 128kbps public listening test

Reply #57 –
Quote
Quote
It's a test that ought to be done, dammit.

Start choosing samples and settings. I'll remind you of this test a few months from now

Since the samples for my previous test went virtually unused, I'll use them.  As for settings, I've got the Vorbis one in mind already, but will open a new discussion thread concerning AAC settings.  Remind me later, and I'll start the thread.  Alternatively, if someone else has a pressing desire to conduce that test, then I'll be more than happy to supply them with my ideas.

MP3 at 128kbps public listening test

Reply #58 –
I'd rather use Radium itself than Audioactive. I think the Radium ACM is widely spread so that should be used in the test. I know that the Audioactive codec might be a bit newer, but in everyday life radium is more widespread and therefore more interesting to know about for most of us.

The same applied IMHO to WMA and WMAPRo. I wish you had tested WMA for the 128 kbps test, rather than WMAPro, because WMAPro isn't widespread, whereas WMA is interesting for hardware use.

MP3 at 128kbps public listening test

Reply #59 –
Quote
Good to know. So maybe it's recommended to use fastencc -hq instead of Audition legacy HQ? I'm not sure, but I would expect Audition to use newer encoding libraries.

I don't know what Audition legacy HQ is, but if it's super slow, then it should be the same as fastencc.exe -hq.  If it encodes in realtime or faster, then it's the fast codec.

Quote
BTW: super-slow = mp3enc 3.1?


Yes.  fastencc.exe -hq is pretty much the same as mp3enc 3.1, except that fastencc.exe has a higher cutoff frequency.

Quote
What stereo modes will be used with non LAME encoders? As I know none of them has good Joint Stereo?


At 128 kbit/s, they should all use joint-stereo.

ff123

Edit:  just to make it very clear what the various flavors of FhG codecs are, here they are as best as I know:

1.  fast codec (FastEnc); FhG VBR, which is based on FastEnc.
2.  super slow codec (i.e., mp3enc31 and its successors); same as fastencc.exe -hq
3.  Audioactive/Radium

FhG made two variants of the fast codec in the last Cool Edit Pro I looked at (version 2.1): the normal fast version, and the super fast version.  I presume the super fast version sounds worse.  The had dropped the super slow codec.  Perhaps in Adobe Audition, the super slow codec was reinstated as the "legacy HQ" setting.

iTunes/Soundjam is not FhG, as far as I can tell.

MP3 at 128kbps public listening test

Reply #60 –
Quote
The same applied IMHO to WMA and WMAPRo. I wish you had tested WMA for the 128 kbps test, rather than WMAPro, because WMAPro isn't widespread, whereas WMA is interesting for hardware use.

I know. For that reason, next multiformat@128kbps test will feature WMA Std

Quote
Remind me later, and I'll start the thread.


Well, somewhere around April, if you don't mind waiting that much.

I think it's better if we don't have several tests happening simultaneously. Things could go messy. And my schedule is pretty tight until march.


@ff123: thanks for all the info.

MP3 at 128kbps public listening test

Reply #61 –
Quote
bYou have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.

well, then why do a public test? it's obvious that it allways depends on YOU, but this is more like a "generic" test ... also, where are the blind tests taht prove your saying?

MP3 at 128kbps public listening test

Reply #62 –
Quote
God, no. Besides, what would be the point? All that it would show is that APS is much better than the rest. But that would be no surprise to anyone here.

well, i think that is the public asumption, but have we ever seen a (abx) test to prove it?

MP3 at 128kbps public listening test

Reply #63 –
Quote
Quote
bYou have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.

well, then why do a public test? it's obvious that it allways depends on YOU, but this is more like a "generic" test ... also, where are the blind tests taht prove your saying?

You don't understand what I was trying to say: The group test is there to find if A is better than B, not to decide whether the quality gain is worth the additional bitrate. The latter really has to be done individually.

Quote
also, where are the blind tests taht prove your saying?

This is a well-known fact on this board. If you're interested in my tests, I can tell you that in my small private listening test of 5 different samples (no problem samples), I could easily pick out abr-130, while the abr-160 was nearly transparent.


Quote
Do you guys think it's needed to replace one of the 64kbps test samples with a difficult one?

Yes! Well, not for this but for the next test, so the evil Ahead AAC gang can't do special tuning for the standard Roberto-samples-set.   

MP3 at 128kbps public listening test

Reply #64 –
Quote
Yes! Well, not for this but for the next test, so the evil Ahead AAC gang can't do special tuning for the standard Roberto-samples-set.   

MWAHAHAHA. OK then, I'll keep some of the samples and replace others, and I'll keep the list of what stays and what goes secret until the date of the test

I agree with you, that should avoid claims of tampering.

MP3 at 128kbps public listening test

Reply #65 –
There are quite a few 128k tests out there, but they are several years old.

I would prefer to see new ground being broken, now that space is less of an issue, with test of the latest LAME -aps (which is said to be untested (at least in comparison to 3.90.x) but also works to some extent with Windows Media Player 9) compared to iTunes at 192k VBR, say, and the FhG encoders available for Windows Media Player 9 such as the Cyberlink or Intervideo versions (See, eg, http://www.wmplugins.com/SearchResults.asp...eyword=&type=4).

MP3 at 128kbps public listening test

Reply #66 –
Quote
-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

rjamorim you star!

Great idea.  How about an "astro anchor" of Lame "APS -Y" I know it's a bit different but it will put them into perspective.  Something that the HA users can easily relate to.

As for LAME ABR is far more interesting than CBR and VBR at this bit rate is a mistake.  If you want to use CDR 128 please do ABR too.

I think ABR is the forgotten star of LAME, especially for people looking for medium bit rate encodes for portable use.

Which LAME build are u gonna use dude?

As for low anchor.... surely that'll be Xing!

Cheers,
Fairy

PS- please start after the Hogmanay hangovers have lifted!

MP3 at 128kbps public listening test

Reply #67 –
Quote
Great idea.  How about an "astro anchor" of Lame "APS -Y" I know it's a bit different but it will put them into perspective.  Something that the HA users can easily relate to.

No. Once and for all: There will be no different bitrates at this test.

Quote
I would prefer to see new ground being broken, now that space is less of an issue, with test of the latest LAME -aps (which is said to be untested (at least in comparison to 3.90.x) but also works to some extent with Windows Media Player 9) compared to iTunes at 192k VBR, say, and the FhG encoders available for Windows Media Player 9 such as the Cyberlink or Intervideo versions


Well, I definitely won't be testing high bitrates. If anyone feel up to it, I can help. But I won't conduce anything in that fashion.

About the WMP codecs: I doubt they use different libraries than the ones used in MusicMatch and Audition.

MP3 at 128kbps public listening test

Reply #68 –
Quote
Quote
Well, I definitely won't be testing high bitrates. If anyone feel up to it, I can help. But I won't conduce anything in that fashion.

About the WMP codecs: I doubt they use different libraries than the ones used in MusicMatch and Audition.

I'm not being argumentative, but could you please explain why?

Are the MusicMatch/Audition codecs any good?  Or would one not be able to tell the difference at 192k VBR, say?

MP3 at 128kbps public listening test

Reply #69 –
Quote
I'm not being argumentative, but could you please explain why?

At high bitrates, the samples become too transparent. And only a handful of golden ears can pick up artifacts - not enough to generate statistically valid results.

I mean, look at my former multiformat @ 128kbps test. All modern codecs were already ranked at scores higher than 4.  And the higher the bitrate, the higher the scores will go, eventually reaching 5.

Also, with all codecs ranking very high, you would need hundreds of listeners to avoid getting the confidence margins overlapping. And, as I already told you, there are only a handful of golden-ears.

So, IMO and from my experience, a high bitrate test would be a waste of time and effort. I don't want to burst anyone's motivation here, but that is my belief, and that's why I won't conduce such test.

Quote
Are the MusicMatch/Audition codecs any good?  Or would one not be able to tell the difference at 192k VBR, say?


Given they are FhG (which is good), they would probably be transparent on most samples for most people at that bitrate.

Regards;

Roberto.

MP3 at 128kbps public listening test

Reply #70 –
Thanks.  Helpful.  [I can't claim golden ears, but I am extremely pleased with my Shure e2c headphones. Having baulked at the price, originally, they are worth every £. My problem is getting a ripper/encoder that has good organisational capabilty.  If everything is likely to be +/- the same at 192k, I'll go back to WMP9.]

MP3 at 128kbps public listening test

Reply #71 –
I think fastenc should be CBR to compare the quality against lame -ap128 most of what I have read & tested shows fastenc is the best FhG encoder & performs better using CBR.

f123 what are your thoughts about an optimal FhG encoder & which settings?


Oh, which compile of lame is going to be used? could 3.92 or 3.93.1 be more sutible for 128kbps?

MP3 at 128kbps public listening test

Reply #72 –
I think the important thing that was brought up earlier is that some of the most widely used codecs should be tested.  In that case, Lame 3.90.3 might be the right choice as well as what MusicMatch and iTunes use.

I also vote for using ABR.  Does the average user really bother with ABR 128?  I know I would, but I can't think of a single friend who would.  Furthermore, it does help limit the number of variables in the test.

MP3 at 128kbps public listening test

Reply #73 –
Ok, one last post from me about the CBR vs ABR issue. I actually feel that not letting Lame use ABR is unfair to Lame. I mean what is the main difference between VBR and CBR from a user point of view? Bitrate predictability. With ABR we have that (or at least within +-5kbps), but better quality than CBR. So I don't think it is unfair, we are comparing the best quality settings of different codecs at a predictable bitrate of ~128 kbps.

But I agree with rjamorim, the test will look biased towards Lame if it is allowed to use ABR. But I still feel that the comparison would be really interesting! So I have actaully developed a wishlist 2.0...

Lame 128 ABR
Lame 128 CBR
FhG 128 CBR ~standard
FhG 128 CBR ~HQ
iTunes 128 CBR
Xing 128 CBR

MP3 at 128kbps public listening test

Reply #74 –
Would you consider Gogo as well?  I've heard a lot of people recommend it because its fast and probably better then fhg, but i have no idea how good it is.

Quote
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps


Are you absolutely sure iTunes does not use one of the fhg codecs listed above?  encspot reports that its fhg and the iTunes "about box" also lists fhg as having licensed software used in iTunes.
 
SimplePortal 1.0.0 RC1 © 2008-2018