MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Topic: MPC vs VORBIS vs MP3 vs AAC at 180 kbps (Read 339994 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #50 – 2005-08-22 22:14:51

Quote

I could do the same test, present the results in the same fancy manner, and come up with completely different results.[a href="index.php?act=findpost&pid=321838"][{POST_SNAPBACK}][/a]

Could you really? Have you done so? Or is that just speculation?

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #51 – 2005-08-22 23:20:03

Quote

Quote
I could do the same test, present the results in the same fancy manner, and come up with completely different results.[a href="index.php?act=findpost&pid=321838"][{POST_SNAPBACK}][/a]

Could you really? Have you done so? Or is that just speculation?
[a href="index.php?act=findpost&pid=321860"][{POST_SNAPBACK}][/a]

Speculation of course, primarily because differeing results are certinaly within the realm of possibility. Every one is different after all. I wasn't just talking about 180k tests by the way, my query applies to any bitrate. I'm lazy so I wouldn't really bother of course. I only bought it up because one of the things I see people posting here at HA.org all the time is that you have to ABX test for yourself in order to get any truly meaningful results, and that makes perfect sense of course. Thus, any testing I might do is only relevent to me, and is why I feel it would be a waste of time to post such results even if some folks may find said results interesting, though I honestly doubt anyone would actually care all that much. After all, I'm just a nobody and guruboolez is the one with the golden ears which, oddly enough, seem to have attained a strange sort of celebrity status around here, hehe. More power to him too. Like I said, despite what seems like (admitedly very minor) hypocrisy, I have to admit I find the tests fun and interesting to read just like everyone else. Thanks again for all your hard work guru! Again, I hope I have not offended anyone, since that was never my intent. Just find the whole thing rather amusing is all, given what is normally posted whenever someone new comes along and asks "what is the best codec/bitrate?".

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #52 – 2005-08-22 23:33:33

I think the flaw in your logic is that you see this as a black/white issue. Tests by other people - especially if you do know about their skill relative to yours - can be a helpful indication. Basically, statistics...... the more of those indications, the more robust the overall picture. However, that picture is only an "average" - and you may be above or below it... or somewhere else ;)

Basically, other peoples tests *are* helpful because they are an indication. Here's an example: I do know that guru's skills are far above mine(both in terms of training as well as ears). Thus, when listening to classical music i could asume that V3 should be transparent to me most of the time. However, if i will ABX the sample-set myself, then i may find out that even v4 is sufficient for me. Thus, other people's tests to give one an indication...... but for fine-tuning, it is necessary to ABX oneself. Some people may decide to skip the additional time for fine-tuning and instead "encode slightly higher than probably needed" - thus, trading space for time & effort.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #53 – 2005-08-22 23:52:23

Quote

Speculation of course, primarily because differeing results are certinaly within the realm of possibility. Every one is different after all.

Yes but blue is blue for everyone except maybe for daltonic persons. Everyone is different, but the possible profiles are not infinite. And would be the gap between these profiles really important?

Quote

Thus, any testing I might do is only relevent to me, and is why I feel it would be a waste of time to post such results

It's an admirable disinterestedness! I'm sincerly admirative.
But this board is a common place for sharing its own experience. When several individual experience are going along the same lines, then it starts to become trustable. Some formats are considered as superior to others: AAC over VQF, MPC over MP3, etc... These affirmations are based on similar individual experience. If everybody would keep their results secret, no knowledge is possible.

Now I could also keep my listening test for myself. But what will it imply? If I'll answer "Vorbis" to someone interesting to know what should be an ideal format for ~200 kbps encoding with classical, you know what will happen... Most people will ask me to backup my claim because MPC is well-known to have no rivality at this bitrate. One will brandish TOS#8, another one request my message to be deleted, I could get warn, etc...

Quote

After all, I'm just a nobody and guruboolez is the one with the golden ears which, oddly enough, seem to have attained a strange sort of celebrity status around here, hehe. More power to him too.

I don't like this kind of charisma either. I don't want people following blindly my conclusions. I'm not a guru [span style='font-size:8pt;line-height:100%'](yes, I know, my nickname could imply the opposite - but if there's one guru, it's rather Pierre Boulez than me)[/span]. That's why I often request additionnal listening tests and certainly not to set up an altar for my holy conclusions

Quote

Again, I hope I have not offended anyone, since that was never my intent.

For me it's OK. You're just questioning the way 'trustable knowledge' of something based on individual experience is possible. Well, it's just epistemology.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #54 – 2005-08-23 00:27:51

It is always very enjoyable and informative to read a listening test by guruboolez. In fact, I haven't visited HA for a while but once I saw a listening test done by guruboolez, I headed straight for this thread

What can I say? I am extremely happy with Ogg Vorbis! Many thanks to Aoyumi for fixing the problems and maturing Vorbis to such an advanced state that it can even rival the once-dominant MPC. Has anyone told Monty about these results? Monty needs to know!!

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #55 – 2005-08-23 01:28:02

I have to agree with Guru. Either these artifacts exist or they don't. The level at which various artifacts become bothersome to the individual is the only issue in question (and it's really only significant when dealing with higher bitrates...lower bitrates produce much more noticeable artifacts). Anyone with a golden pair of ears and artifact awareness (such as guru) can make a valid assessment when they blindly compare to the reference, IMO.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #56 – 2005-08-23 03:19:51

nice listening test guruboolez great to see the progress that vorbis has made in the last year, and its neat knowing the format still has a great deal of room for future improvements if necessary.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #57 – 2005-08-23 05:25:19

Quote

What can I say? I am extremely happy with Ogg Vorbis! Many thanks to Aoyumi for fixing the problems and maturing Vorbis to such an advanced state that it can even rival the once-dominant MPC. Has anyone told Monty about these results? Monty needs to know!!

He drops in occasionally, but I presume he is busy doing other more important stuff.

Quote

I hope I have not offended anyone, since that was never my intent. Just find the whole thing rather amusing is all, given what is normally posted whenever someone new comes along and asks "what is the best codec/bitrate?". wink.gif

all the listening tests show to me is a small or substantial improvement in the actual codec. "Best" is whatever is best for your ears and whatever you prefer. Don't forget that most things outside this community are irrelivent (unless websites point here) and unless they actually get merged into the mainline, much like in 1.1.1. It's just too bad their aren't any more developers working on getting multichannel coupling up to snuff, etc or who have the ability to do so. Taming that beast is another esotreric task

. Aoyumi work is always greatly appreciated though.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #58 – 2005-08-23 12:04:01

Quote

Quote
Regarding MPC, the result is split, somehow for classical samples there seems to be a degration in quality and size-effectivity (bitrate boost), comparing MPC v1.14b with MPC v1.15v
I cannot dare to ask Guru, to compare this sample set again between 1.14b and 1.15v to get a ranking of this new multiformat test including not only 1.15v but also 1.14.
[a href="index.php?act=findpost&pid=321715"][{POST_SNAPBACK}][/a]

There are two different problems in my opinion:

- bitrate consumption: mpc --standard has increased by more than 10 kbps compared to former release of mppenc (I can post a full bitrate table if you want). 1.15 series is by far the version which presents the highest bitrate (not only with classical: people listening different kind of music have confirmed it - but how much is something I can't say). But this inflation is not necessary a problem: some users don't really care about few additionnal kbps, and it could bring higher quality (higher bitrate doesn't necessary mean lower efficiency).

-quality (with classical): I'm not sure that 1.15 presents regression compared to 1.14 beta. The problem maybe occur earlier. I compared 1.15u to a much older release of mppenc (1.01j) which clearly revealed issues with the latest version of mppenc (+ inflated bitrate). Now I can't tell when exactly the problem happened, or if the quality (with classical) has slowly decreased with version > 1.0 stable.

yeah, I know it from posts, I used to use MPC 1.01j once, too. And 1.14 for some time, until I switched to 1.15u,v.
The bitrate increase is not only for classic or your impressive 150 samples, but also clearly known, reported here in forum, too, when comparing 1.14 with 1.15r,s,t,u,v. The bitrate increase happens in all genres, classic, jazz, pop, rock, metal probably, too.
I agree, that we don't need to discuss the bitrate as such.
I am using 1.15 myself, as it improved other problem samples. But I was pointing to the facts, just to make it not forgotten..

One remark to the validity of this test and others:

If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.
Of course, primarily you have to say, that this test is primarily valid only for this 1 test-person.
But we have some history here, and the reason why Guruboulez tests are very welcomed here, is following: Other persons repeated gurus tests (with or without posting results) and eitehr confirmed them, or admitted, not to hear the difference, but most important: no contradiction.
By experience some people here know, that guruboulez results are valid not only to him, but also to other persons, normally worded "made same experience".
So, if somebody above tries to play "devils advocate", and questions validity of this test, he should not ridicule himself. The only way to question results is in presenting listening test with opposite/different results.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #59 – 2005-08-23 14:43:07

Quote

If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.

Actually the deaf group adds significance, as the world isn't populated only by golden ears.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #60 – 2005-08-23 14:43:34

Quote

Quote
Probably, change of ATH, noise/tone masking, and noise compander etc. is useful to the improvement of roughness.

Magic does not exist there.

I was digging through the code before just to try and get a better understanding of how things work? what exactly is noise companding and noise biasing? I mean what specific roles do they play? I understand the ATH and noise/tone masking routines using the FFT and MDCT, but I don't really understand the other two that well. It's my understanding that when you are changing the impulse_noisetune if you are using advanced-command line switch you are actually adjusting the noise bias?
[a href="index.php?act=findpost&pid=321779"][{POST_SNAPBACK}][/a]

Noisebias expresses a frequency noise curve, and noise compand expresses a loudness noise curve. These are bearing the role important for determining a final noise masking value.
Moreover, impulse_noisetune operates noisebias of impulse block (short block).

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #61 – 2005-08-23 14:49:14

Quote

The bitrate increase is not only for classic or your impressive 150 samples, but also clearly known, reported here in forum, too, when comparing 1.14 with 1.15r,s,t,u,v. The bitrate increase happens in all genres, classic, jazz, pop, rock, metal probably, too.
[{POST_SNAPBACK}][/a]

I found an old message posted by Frank Klemm about this increase:

Quote

Bitrate is increased by typically 3.1% compared with 1.14. Most of this bitrate increase is related to
some modificitions in the tonality estimation model (which was introduced with 1.15g).

Source [a href=\"http://www.hydrogenaudio.org/forums/index.php?showtopic=6558&view=findpost&p=66098]here[/url].

If you're interested, I also compared the bitrate of six different mppenc version, from Buschmann 1.7.8c to latest 1.15v:

(a zoomed version here.)

If you can't see 1.78c it's simply because 0.90o (my oldest mppenc version) is ultra-close to it.
The 1.04 version is by far the most efficient version of mppenc - but remember how much people were frighten about this bitrate drop! Therefore, bitrate started to increase again with 1.06 (not present here).
1.14 is still close to native --standard bitrate (+3,5 kbps on average for my 150 full tracks).
1.15v is now far from what Buschmann 1.78c / Klemm 1.00 encoders used to output. The difference between the extreme is 15 kbps! Does it really justify some gain with a limited set of samples?

If you want to play with the datas, here's the complete table (sorted by samples):

Code: [Select]

CATALOG	1.7.8c	0.90o	1.01j	1.04	1.14	1.15v
NUMBER								
A01	153,2	153,1	157,0	152,7	157,9	164,1
A02	176,8	176,8	182,3	179,2	189,1	192,2
A03	185,6	185,6	189,7	189,4	200,4	205,1
A04	173,5	173,5	177,3	178,4	185,6	188,2
A05	172,8	172,8	176,2	162,2	170,0	177,9
E01	117,1	117,1	118,0	110,0	116,8	121,8
E02	173,8	173,8	175,5	171,6	178,8	186,7
E03	174,6	174,7	178,0	175,7	183,8	188,8
E04	159,7	159,6	160,2	153,1	159,6	167,6
E05	157,5	157,5	161,6	157,0	165,3	167,2
E06	173,8	173,8	175,3	171,7	178,9	187,2
E07	169,1	169,1	169,9	163,1	169,5	178,2
E08	164,4	164,4	170,1	154,4	165,0	168,0
E09	176,0	176,0	177,2	173,0	180,1	188,3
E10	168,1	168,1	170,1	158,5	166,3	173,3
E11	174,1	174,1	177,0	173,6	181,1	188,3
E12	150,3	150,3	152,6	145,2	151,5	157,9
E13	183,3	183,3	183,3	178,3	183,7	193,3
E14	169,9	169,9	172,6	169,2	177,3	185,2
E15	177,7	177,6	177,4	171,9	179,1	190,8
E16	186,0	186,0	187,8	185,0	191,9	199,7
E17	167,6	167,6	170,9	168,0	174,3	178,2
E18	149,8	149,8	151,8	146,3	151,7	157,4
E19	147,3	147,3	148,3	142,1	148,1	155,2
E20	151,0	151,0	152,0	141,0	148,9	157,3
E21	153,8	153,8	156,0	150,1	156,0	162,6
E22	173,5	173,4	175,3	174,1	180,2	187,6
E23	161,9	161,9	163,5	158,4	164,4	172,8
E24	159,1	159,1	160,3	153,1	160,6	169,1
E25	173,9	173,8	176,1	176,0	182,2	188,6
E26	165,1	165,0	167,9	161,7	168,9	174,6
E27	169,2	169,2	171,1	164,9	173,3	180,8
E28	151,7	151,7	156,0	153,9	159,6	159,6
E29	158,1	158,1	160,9	153,6	162,2	166,9
E30	164,2	164,2	168,2	166,0	172,7	175,9
E31	185,1	185,0	186,3	186,6	193,0	201,4
E32	189,7	189,7	189,9	188,0	193,7	204,7
E33	170,9	170,8	172,9	169,1	179,9	185,7
E34	162,3	162,3	164,9	159,0	163,3	168,8
E35	204,5	204,4	203,6	199,6	203,8	220,9
E36	201,5	201,4	201,3	199,9	205,9	216,3
E37	181,4	181,3	182,7	177,7	185,3	193,7
E38	174,9	174,8	176,0	171,8	178,4	187,1
E39	175,6	175,5	176,2	170,4	176,7	186,3
E40	157,7	157,7	160,3	152,2	160,7	168,6
E41	213,5	213,5	212,0	207,2	211,8	225,0
E42	207,7	207,6	207,3	203,1	209,7	220,6
E43	190,0	190,0	190,6	187,6	193,4	205,4
E44	169,8	169,8	171,1	161,9	168,7	178,9
E45	201,5	201,5	201,6	201,2	207,3	219,2
E46	168,6	168,5	171,0	168,0	173,2	182,2
E47	180,2	180,2	181,5	178,1	182,9	189,8
E48	188,3	188,3	189,5	186,8	194,5	203,1
E49	164,6	164,6	166,1	163,2	169,3	177,0
E50	194,4	194,4	196,4	196,5	202,4	210,0
E51	178,3	178,3	179,4	177,5	183,9	193,4
E52	183,5	183,5	183,7	182,3	187,7	196,9
E53	174,7	174,7	176,8	175,5	181,9	189,5
E54	171,0	171,0	172,7	169,4	176,3	181,7
E55	176,0	176,0	176,9	174,7	181,2	188,9
E56	173,8	173,8	177,2	172,6	179,9	185,1
E57	173,5	173,5	176,9	175,5	181,7	185,2
E58	173,1	173,1	177,4	178,1	183,9	185,5
E59	187,7	187,7	189,3	187,7	195,6	207,3
E60	176,3	176,3	179,0	178,7	185,5	191,2
S01	173,2	173,2	175,4	155,5	162,9	171,7
S02	153,6	153,6	155,9	148,9	156,4	163,8
S03	171,6	171,6	172,7	165,6	172,0	181,0
S04	182,8	182,8	184,0	178,7	186,9	193,4
S05	189,5	189,5	190,3	184,5	189,9	201,9
S06	178,1	178,1	178,5	170,0	179,9	187,6
S07	206,9	206,9	206,7	203,7	212,8	221,6
S08	173,9	173,9	176,0	170,1	179,3	186,8
S09	188,7	188,7	189,3	182,1	191,3	200,0
S10	176,7	176,7	178,0	170,4	177,8	190,9
S11	228,5	228,5	227,0	222,0	226,8	246,3
S12	229,3	229,2	228,0	223,9	228,2	248,1
S13	203,5	203,4	203,3	201,6	209,3	220,1
S14	189,4	189,4	191,2	192,0	197,9	207,5
S15	205,4	205,4	204,7	201,3	206,9	221,4
S16	233,9	233,8	232,4	227,7	233,6	251,2
S17	167,0	167,0	169,7	167,4	173,9	180,9
S18	180,8	180,7	180,2	172,3	177,2	184,7
S19	167,9	167,9	168,7	162,0	168,5	177,8
S20	173,2	173,2	172,9	169,3	175,1	185,0
S21	176,8	176,7	178,2	172,2	176,2	184,6
S22	196,3	196,3	195,4	189,7	195,9	207,3
S23	163,5	163,5	165,8	162,2	169,3	177,0
S24	139,6	139,6	141,7	136,2	143,4	149,5
S25	162,8	162,8	165,2	155,9	163,7	172,5
S26	124,4	124,4	125,8	118,9	124,2	132,3
S27	126,4	126,4	127,5	119,2	125,9	134,1
S28	136,8	136,8	137,6	130,6	135,0	145,8
S29	149,0	149,0	151,4	144,3	149,7	158,5
S30	162,8	162,8	164,0	151,8	161,0	168,3
S31	162,8	162,8	164,0	151,8	161,0	168,3
S32	187,4	187,4	187,7	180,7	191,1	201,4
S33	158,6	158,6	161,3	153,4	162,5	170,2
S34	196,2	196,2	197,6	192,5	202,2	212,6
S35	202,4	202,3	203,0	196,9	201,7	213,6
S36	167,8	167,8	173,1	168,3	174,8	180,6
S37	165,0	165,0	169,8	158,2	168,0	172,4
S38	169,7	169,7	172,0	167,1	174,1	186,2
S39	152,7	152,7	155,2	147,2	153,9	167,7
S40	178,2	178,2	181,2	180,8	184,4	202,0
S41	162,0	162,0	165,6	159,4	165,2	171,2
S42	171,2	171,1	172,3	162,4	170,4	178,0
S43	170,7	170,7	173,1	169,2	173,5	185,3
S44	226,6	226,6	225,9	220,0	224,6	236,6
S45	157,2	157,2	159,1	146,9	153,1	164,1
S46	166,6	166,5	169,3	159,9	167,8	178,2
S47	178,6	178,6	181,6	172,7	180,8	189,3
S48	164,0	164,0	166,0	160,8	167,9	175,6
S49	162,3	162,3	163,8	155,7	163,1	169,6
S50	194,5	194,4	196,6	194,2	205,9	219,1
S51	185,4	185,4	187,8	181,3	189,5	193,2
S52	185,3	185,2	186,6	184,1	191,8	199,5
S53	163,3	163,3	164,7	152,4	160,6	169,0
S54	120,9	120,9	122,7	110,5	121,0	131,6
S55	141,2	141,2	142,8	134,6	142,6	151,1
V01	175,1	175,1	176,3	173,8	179,0	188,3
V02	162,7	162,7	165,5	164,5	169,9	174,8
V03	109,5	109,5	110,5	100,4	108,2	113,8
V04	173,4	173,4	176,7	176,7	183,7	186,6
V05	163,3	163,2	165,8	163,4	169,6	175,6
V06	169,2	169,2	172,2	173,0	178,2	182,4
V07	163,4	163,4	166,2	165,0	170,8	176,3
V08	146,4	146,4	148,2	139,9	147,7	153,7
V09	177,2	177,1	179,5	174,1	182,6	188,7
V10	190,8	190,7	191,2	187,8	193,5	203,1
V11	156,1	156,1	159,5	151,9	159,3	164,7
V12	178,2	178,2	181,3	178,7	185,4	194,5
V13	161,9	161,9	165,8	159,4	166,0	174,5
V14	160,6	160,6	163,6	162,3	168,0	176,5
V15	173,7	173,7	175,2	170,3	178,1	190,7
V16	174,2	174,2	176,1	172,5	179,4	187,5
V17	162,1	162,1	163,7	153,9	160,9	171,8
V18	177,1	177,1	178,8	176,5	182,2	190,8
V19	167,6	167,6	169,9	166,8	173,0	180,0
V20	172,8	172,8	175,3	171,8	178,7	186,0
V21	164,6	164,6	167,2	158,9	166,5	171,3
V22	169,3	169,2	173,7	167,5	175,4	179,1
V23	183,3	183,3	184,5	176,8	184,9	192,3
V24	198,7	198,7	199,9	198,1	203,9	214,8
V25	180,4	180,4	183,1	179,1	186,2	194,0
V26	167,5	167,4	168,9	163,5	170,7	178,4
V27	175,9	175,8	178,0	175,3	181,0	189,5
V28	165,4	165,4	167,0	159,4	166,0	176,9
V29	169,1	169,1	173,0	172,4	179,7	185,4
V30	181,0	180,9	182,1	178,9	184,8	194,8

	172,71	172,69	174,51	169,37	176,28	184,54

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #62 – 2005-08-23 14:59:16

Quote

Quote
If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.

Actually the deaf group adds significance, as the world isn't populated only by golden ears.
[a href="index.php?act=findpost&pid=322047"][{POST_SNAPBACK}][/a]

True, but what user meant is -I suppose- that results of people unable to tell a difference are not really helpful if the purpose of the test is to know how the difference is between these encoders.
A lot of 5.0 everywhere are telling us that these encoders are transparent to a lot of people (I guess that there's no need to conduce such tests to confirm that ), but not how different are these encoders. That's why critical listeners could be needed here.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #63 – 2005-08-23 15:52:28

Quote

True, but what user meant is -I suppose- that results of people unable to tell a difference are not really helpful if the purpose of the test is to know how the difference is between these encoders.
A lot of 5.0 everywhere are telling us that these encoders are transparent to a lot of people (I guess that there's no need to conduce such tests to confirm that ), but not how different are these encoders. That's why critical listeners could be needed here.
[a href="index.php?act=findpost&pid=322053"][{POST_SNAPBACK}][/a]

Indeed. Deaf users are good to know how formats sound to everybody. But not necessarily how formats sound against each other.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #64 – 2005-08-30 15:27:20

Awesome piece of work and information, simply awesome.

Thanks a lot.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #65 – 2005-08-31 11:19:20

Here is the bitrate table (based on 150 full tracks):

Code: [Select]

FILE	faac	Nero	LAME	MPC	Vorbis

A01	175,0	199,2	194,7	164,1	201,7
A02	241,6	156,8	210,8	192,2	206,2
A03	230,6	165,7	230,3	205,1	293,1
A04	203,5	176,1	217,8	188,2	207,3
A05	183,3	161,9	175,8	177,9	182,6
E01	151,3	183,6	135,2	121,8	154,7
E02	179,1	185,6	182,0	186,7	178,8
E03	190,9	179,4	187,0	188,8	184,3
E04	172,9	179,8	178,0	167,6	174,8
E05	215,8	159,5	188,6	167,2	194,4
E06	173,6	186,9	188,9	187,2	179,0
E07	163,6	189,9	180,8	178,2	168,7
E08	182,4	147,8	191,5	168,0	199,4
E09	176,3	187,2	181,4	188,3	178,0
E10	176,4	170,1	167,9	173,3	166,7
E11	211,3	175,8	201,6	188,3	188,3
E12	163,4	184,1	169,5	157,9	167,3
E13	169,6	176,8	172,6	193,3	184,5
E14	163,6	190,7	184,9	185,2	176,5
E15	159,9	198,5	180,5	190,8	176,5
E16	188,0	181,6	194,0	199,7	202,3
E17	188,2	180,7	196,2	178,2	185,7
E18	169,5	197,0	174,3	157,4	177,6
E19	167,9	180,4	167,0	155,2	168,3
E20	166,0	180,2	179,4	157,3	161,8
E21	166,8	185,7	181,3	162,6	170,4
E22	176,5	182,8	187,9	187,6	177,1
E23	158,5	181,8	182,4	172,8	163,0
E24	161,9	189,5	175,9	169,1	163,3
E25	191,3	178,6	194,4	188,6	182,6
E26	176,3	178,0	184,5	174,6	171,3
E27	178,8	175,5	184,9	180,8	168,4
E28	178,8	183,0	191,4	159,6	183,8
E29	189,4	166,1	177,0	166,9	170,9
E30	183,7	184,2	196,9	175,9	174,9
E31	222,0	154,1	192,0	201,4	199,6
E32	190,2	180,9	186,9	204,7	196,8
E33	187,2	162,5	177,1	185,7	183,2
E34	160,6	181,1	174,6	168,8	167,5
E35	186,6	169,0	181,3	220,9	206,2
E36	208,6	173,5	187,3	216,3	216,5
E37	172,0	183,7	175,6	193,7	180,3
E38	174,3	183,6	182,5	187,1	177,2
E39	174,3	184,2	184,0	186,3	188,3
E40	163,7	181,9	170,6	168,6	157,1
E41	175,6	169,9	174,7	225,0	197,2
E42	182,5	177,6	174,3	220,6	190,6
E43	188,7	178,0	188,0	205,4	188,4
E44	170,0	167,8	180,8	178,9	167,9
E45	218,6	163,5	194,6	219,2	212,2
E46	185,3	178,0	187,9	182,2	179,7
E47	177,5	192,4	187,6	189,8	193,3
E48	182,0	182,8	187,7	203,1	187,5
E49	171,2	179,2	180,5	177,0	175,1
E50	226,3	179,6	214,9	210,0	211,6
E51	180,3	180,6	187,9	193,4	185,1
E52	182,3	176,4	186,0	196,9	188,7
E53	189,6	174,4	190,0	189,5	185,2
E54	173,6	177,1	183,4	181,7	179,0
E55	175,1	184,8	188,9	188,9	181,6
E56	213,3	163,2	186,4	185,1	184,4
E57	214,6	165,5	195,5	185,2	181,7
E58	206,1	177,5	202,4	185,5	181,9
E59	189,0	166,4	184,5	207,3	199,1
E60	202,4	173,8	193,0	191,2	203,9
S01	175,7	134,1	138,7	171,7	153,4
S02	173,1	175,1	173,4	163,8	166,1
S03	172,4	185,9	181,0	181,0	165,9
S04	175,0	172,1	180,1	193,4	174,0
S05	169,5	172,1	178,0	201,9	182,9
S06	183,4	166,3	166,8	187,6	173,7
S07	181,6	175,8	179,8	221,6	192,5
S08	188,3	174,8	181,3	186,8	171,0
S09	185,8	174,5	178,0	200,0	180,5
S10	230,6	158,1	185,5	190,9	181,7
S11	204,8	166,8	185,4	246,3	224,4
S12	217,5	160,1	185,0	248,1	232,2
S13	246,6	160,2	206,6	220,1	226,7
S14	229,1	162,5	216,5	207,5	226,6
S15	221,2	166,1	193,1	221,4	214,3
S16	204,5	164,6	174,6	251,2	220,2
S17	172,9	174,9	188,9	180,9	175,9
S18	161,0	175,0	184,8	184,7	169,5
S19	160,6	190,0	180,2	177,8	168,6
S20	164,6	188,5	204,3	185,0	174,5
S21	176,3	172,5	175,6	184,6	169,6
S22	171,9	172,0	181,6	207,3	189,6
S23	163,0	198,4	190,3	177,0	175,3
S24	176,2	182,3	174,3	149,5	176,7
S25	158,6	166,9	182,0	172,5	165,9
S26	151,9	189,4	150,4	132,3	156,8
S27	156,6	176,1	161,4	134,1	163,2
S28	153,4	198,1	151,9	145,8	163,0
S29	157,9	196,6	175,9	158,5	168,0
S30	161,3	171,6	141,7	168,3	156,6
S31	161,3	171,6	141,7	168,3	156,6
S32	187,8	168,6	159,2	201,4	192,2
S33	188,1	169,2	173,8	170,2	169,9
S34	165,3	175,0	185,7	212,6	178,7
S35	197,1	166,5	178,3	213,6	190,5
S36	152,9	197,1	177,2	180,6	180,6
S37	176,6	171,9	175,2	172,4	191,6
S38	163,8	194,2	193,0	186,2	185,7
S39	162,9	194,7	177,3	167,7	171,1
S40	172,1	180,1	184,9	202,0	196,2
S41	183,2	165,0	180,4	171,2	182,9
S42	164,1	163,9	174,8	178,0	163,3
S43	159,7	182,6	185,3	185,3	179,0
S44	193,3	169,0	157,4	236,6	205,3
S45	154,0	170,9	157,0	164,1	150,8
S46	164,0	169,1	172,0	178,2	165,9
S47	175,1	167,8	176,9	189,3	174,3
S48	163,3	190,7	180,2	175,6	167,7
S49	165,1	177,3	174,0	169,6	170,2
S50	186,6	165,1	182,2	219,1	194,1
S51	166,3	164,1	183,0	193,2	175,3
S52	190,5	178,7	183,2	199,5	182,8
S53	160,4	160,8	140,4	169,0	162,5
S54	163,9	158,7	146,5	131,6	162,4
S55	161,5	173,4	152,6	151,1	166,6
V01	172,7	184,1	188,1	188,3	179,8
V02	165,1	189,9	194,8	174,8	169,9
V03	171,1	172,0	141,5	113,8	147,0
V04	214,6	169,8	202,8	186,6	198,0
V05	171,3	186,0	189,1	175,6	174,3
V06	184,3	185,8	192,7	182,4	182,8
V07	175,5	181,8	199,8	176,3	171,5
V08	172,2	173,7	178,6	153,7	165,5
V09	180,2	171,6	173,3	188,7	175,7
V10	183,7	172,2	181,3	203,1	179,1
V11	168,5	178,9	178,8	164,7	154,9
V12	175,5	178,0	192,4	194,5	182,0
V13	183,1	177,1	189,3	174,5	171,4
V14	167,6	186,6	198,5	176,5	173,2
V15	173,4	178,1	190,9	190,7	179,2
V16	164,1	180,3	189,5	187,5	169,6
V17	177,3	185,1	187,3	171,8	174,6
V18	182,6	181,5	188,0	190,8	187,2
V19	179,1	168,9	180,2	180,0	170,2
V20	191,0	175,2	191,8	186,0	204,4
V21	194,9	163,7	178,2	171,3	175,1
V22	185,3	174,2	180,9	179,1	176,9
V23	165,3	178,9	181,5	192,3	168,7
V24	232,5	162,0	195,8	214,8	219,7
V25	192,8	163,2	184,1	194,0	180,9
V26	184,6	173,0	172,6	178,4	173,7
V27	196,9	172,5	186,6	189,5	182,4
V28	167,7	184,6	180,2	176,9	170,5
V29	189,0	177,9	187,6	185,4	178,3
V30	181,9	182,8	184,8	194,8	178,9

	180,92	176,14	181,79	184,54	181,48

	faac	Nero	LAME	MPC	Vorbis

Graphicaly and sorted by size:

I tried with older version of aacenc32.dll: identical - identical - identical...
In short, -streaming 'high' encodings are bit-to-bit identical from aacenc32.dll version 2.9.9.999 (dated from 2004.12.22) to 3.2.0.20! The difference start to appear with 2.9.9.998 dated from 2004.11.27. In other words, no progress in this area since eight months. What happened with Nero Digital team? Have Ivan only worked on SBR and low bitrate stuff during this time?

examples:
files are not available anymore for download
Here is the bitrate table (based on 150 full tracks):

Code: [Select]

FILE	faac	Nero	LAME	MPC	Vorbis

A01	175,0	199,2	194,7	164,1	201,7
A02	241,6	156,8	210,8	192,2	206,2
A03	230,6	165,7	230,3	205,1	293,1
A04	203,5	176,1	217,8	188,2	207,3
A05	183,3	161,9	175,8	177,9	182,6
E01	151,3	183,6	135,2	121,8	154,7
E02	179,1	185,6	182,0	186,7	178,8
E03	190,9	179,4	187,0	188,8	184,3
E04	172,9	179,8	178,0	167,6	174,8
E05	215,8	159,5	188,6	167,2	194,4
E06	173,6	186,9	188,9	187,2	179,0
E07	163,6	189,9	180,8	178,2	168,7
E08	182,4	147,8	191,5	168,0	199,4
E09	176,3	187,2	181,4	188,3	178,0
E10	176,4	170,1	167,9	173,3	166,7
E11	211,3	175,8	201,6	188,3	188,3
E12	163,4	184,1	169,5	157,9	167,3
E13	169,6	176,8	172,6	193,3	184,5
E14	163,6	190,7	184,9	185,2	176,5
E15	159,9	198,5	180,5	190,8	176,5
E16	188,0	181,6	194,0	199,7	202,3
E17	188,2	180,7	196,2	178,2	185,7
E18	169,5	197,0	174,3	157,4	177,6
E19	167,9	180,4	167,0	155,2	168,3
E20	166,0	180,2	179,4	157,3	161,8
E21	166,8	185,7	181,3	162,6	170,4
E22	176,5	182,8	187,9	187,6	177,1
E23	158,5	181,8	182,4	172,8	163,0
E24	161,9	189,5	175,9	169,1	163,3
E25	191,3	178,6	194,4	188,6	182,6
E26	176,3	178,0	184,5	174,6	171,3
E27	178,8	175,5	184,9	180,8	168,4
E28	178,8	183,0	191,4	159,6	183,8
E29	189,4	166,1	177,0	166,9	170,9
E30	183,7	184,2	196,9	175,9	174,9
E31	222,0	154,1	192,0	201,4	199,6
E32	190,2	180,9	186,9	204,7	196,8
E33	187,2	162,5	177,1	185,7	183,2
E34	160,6	181,1	174,6	168,8	167,5
E35	186,6	169,0	181,3	220,9	206,2
E36	208,6	173,5	187,3	216,3	216,5
E37	172,0	183,7	175,6	193,7	180,3
E38	174,3	183,6	182,5	187,1	177,2
E39	174,3	184,2	184,0	186,3	188,3
E40	163,7	181,9	170,6	168,6	157,1
E41	175,6	169,9	174,7	225,0	197,2
E42	182,5	177,6	174,3	220,6	190,6
E43	188,7	178,0	188,0	205,4	188,4
E44	170,0	167,8	180,8	178,9	167,9
E45	218,6	163,5	194,6	219,2	212,2
E46	185,3	178,0	187,9	182,2	179,7
E47	177,5	192,4	187,6	189,8	193,3
E48	182,0	182,8	187,7	203,1	187,5
E49	171,2	179,2	180,5	177,0	175,1
E50	226,3	179,6	214,9	210,0	211,6
E51	180,3	180,6	187,9	193,4	185,1
E52	182,3	176,4	186,0	196,9	188,7
E53	189,6	174,4	190,0	189,5	185,2
E54	173,6	177,1	183,4	181,7	179,0
E55	175,1	184,8	188,9	188,9	181,6
E56	213,3	163,2	186,4	185,1	184,4
E57	214,6	165,5	195,5	185,2	181,7
E58	206,1	177,5	202,4	185,5	181,9
E59	189,0	166,4	184,5	207,3	199,1
E60	202,4	173,8	193,0	191,2	203,9
S01	175,7	134,1	138,7	171,7	153,4
S02	173,1	175,1	173,4	163,8	166,1
S03	172,4	185,9	181,0	181,0	165,9
S04	175,0	172,1	180,1	193,4	174,0
S05	169,5	172,1	178,0	201,9	182,9
S06	183,4	166,3	166,8	187,6	173,7
S07	181,6	175,8	179,8	221,6	192,5
S08	188,3	174,8	181,3	186,8	171,0
S09	185,8	174,5	178,0	200,0	180,5
S10	230,6	158,1	185,5	190,9	181,7
S11	204,8	166,8	185,4	246,3	224,4
S12	217,5	160,1	185,0	248,1	232,2
S13	246,6	160,2	206,6	220,1	226,7
S14	229,1	162,5	216,5	207,5	226,6
S15	221,2	166,1	193,1	221,4	214,3
S16	204,5	164,6	174,6	251,2	220,2
S17	172,9	174,9	188,9	180,9	175,9
S18	161,0	175,0	184,8	184,7	169,5
S19	160,6	190,0	180,2	177,8	168,6
S20	164,6	188,5	204,3	185,0	174,5
S21	176,3	172,5	175,6	184,6	169,6
S22	171,9	172,0	181,6	207,3	189,6
S23	163,0	198,4	190,3	177,0	175,3
S24	176,2	182,3	174,3	149,5	176,7
S25	158,6	166,9	182,0	172,5	165,9
S26	151,9	189,4	150,4	132,3	156,8
S27	156,6	176,1	161,4	134,1	163,2
S28	153,4	198,1	151,9	145,8	163,0
S29	157,9	196,6	175,9	158,5	168,0
S30	161,3	171,6	141,7	168,3	156,6
S31	161,3	171,6	141,7	168,3	156,6
S32	187,8	168,6	159,2	201,4	192,2
S33	188,1	169,2	173,8	170,2	169,9
S34	165,3	175,0	185,7	212,6	178,7
S35	197,1	166,5	178,3	213,6	190,5
S36	152,9	197,1	177,2	180,6	180,6
S37	176,6	171,9	175,2	172,4	191,6
S38	163,8	194,2	193,0	186,2	185,7
S39	162,9	194,7	177,3	167,7	171,1
S40	172,1	180,1	184,9	202,0	196,2
S41	183,2	165,0	180,4	171,2	182,9
S42	164,1	163,9	174,8	178,0	163,3
S43	159,7	182,6	185,3	185,3	179,0
S44	193,3	169,0	157,4	236,6	205,3
S45	154,0	170,9	157,0	164,1	150,8
S46	164,0	169,1	172,0	178,2	165,9
S47	175,1	167,8	176,9	189,3	174,3
S48	163,3	190,7	180,2	175,6	167,7
S49	165,1	177,3	174,0	169,6	170,2
S50	186,6	165,1	182,2	219,1	194,1
S51	166,3	164,1	183,0	193,2	175,3
S52	190,5	178,7	183,2	199,5	182,8
S53	160,4	160,8	140,4	169,0	162,5
S54	163,9	158,7	146,5	131,6	162,4
S55	161,5	173,4	152,6	151,1	166,6
V01	172,7	184,1	188,1	188,3	179,8
V02	165,1	189,9	194,8	174,8	169,9
V03	171,1	172,0	141,5	113,8	147,0
V04	214,6	169,8	202,8	186,6	198,0
V05	171,3	186,0	189,1	175,6	174,3
V06	184,3	185,8	192,7	182,4	182,8
V07	175,5	181,8	199,8	176,3	171,5
V08	172,2	173,7	178,6	153,7	165,5
V09	180,2	171,6	173,3	188,7	175,7
V10	183,7	172,2	181,3	203,1	179,1
V11	168,5	178,9	178,8	164,7	154,9
V12	175,5	178,0	192,4	194,5	182,0
V13	183,1	177,1	189,3	174,5	171,4
V14	167,6	186,6	198,5	176,5	173,2
V15	173,4	178,1	190,9	190,7	179,2
V16	164,1	180,3	189,5	187,5	169,6
V17	177,3	185,1	187,3	171,8	174,6
V18	182,6	181,5	188,0	190,8	187,2
V19	179,1	168,9	180,2	180,0	170,2
V20	191,0	175,2	191,8	186,0	204,4
V21	194,9	163,7	178,2	171,3	175,1
V22	185,3	174,2	180,9	179,1	176,9
V23	165,3	178,9	181,5	192,3	168,7
V24	232,5	162,0	195,8	214,8	219,7
V25	192,8	163,2	184,1	194,0	180,9
V26	184,6	173,0	172,6	178,4	173,7
V27	196,9	172,5	186,6	189,5	182,4
V28	167,7	184,6	180,2	176,9	170,5
V29	189,0	177,9	187,6	185,4	178,3
V30	181,9	182,8	184,8	194,8	178,9

	180,92	176,14	181,79	184,54	181,48

	faac	Nero	LAME	MPC	Vorbis

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #66 – 2005-08-31 15:26:57

Wow!

Thank you, guruboolez!

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #67 – 2005-09-01 22:34:15

Thank you for the test.
I hope it will encourage other people to perform tests at high bitrates, which have been long considered as out of reach.
I'd like to test some metal samples with the same encoders but I'm so short of time in order to do all what I'd like to do. In fact, I've got a lot of time, but too much interesting things to do !

To remarks about statistics : MPC and Vobris seem tied, but the results don't take into account the ABX results, which adds a lot of significance. Maybe if we could analyse all the data, some encoders that look tied wouldn't be so anymore.
And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #68 – 2005-09-01 22:50:58

Quote

And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.
[a href="index.php?act=findpost&pid=324341"][{POST_SNAPBACK}][/a]

I believe that is correct. Friedmann can be used in the case of ties, but some modifications to the calculation of the F-value have to be made, if I remember it correctly.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #69 – 2005-09-02 00:58:53

I feel great urge to thank Guruboolez and Aoyumi for their very different, yet so coherent work on Vorbis. The next big thing is aoTuV beta 5 (or will it be the official release, like 1.2 or something?), which I hope will reduce some ugly noise issues (I can upload a few samples which are ABX-able @ -q5 and, supposedly, -q6).

Guru, your tests will always be greatly appreciated by me and my friends. Thank you, again.

Quote

What I'm trying to say is that, unlike most other genres, classical samples can be very different and so when you have a thorough test like this one, you can get a pretty good overall picture of each codec (encoder).
[a href="index.php?act=findpost&pid=321772"][{POST_SNAPBACK}][/a]

It depends on what do you call a difference. Music can be very different even while being in the same genre, let alone “guitar” or “electronic” music…

Quote

In fact, I haven't visited HA for a while but once I saw a listening test done by guruboolez, I headed straight for this thread
[a href="index.php?act=findpost&pid=321901"][{POST_SNAPBACK}][/a]

Hey, me too! I've just returned from my vacation, heh.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #70 – 2005-09-02 06:14:48

And in my opinion, the handling pre-echo mechanism which entered in beta3 are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)
Waiting for next great work of Aoyumi

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #71 – 2005-09-02 10:34:42

Quote

And in my opinion, the handling pre-echo mechanism which entered in beta3 are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)
Waiting for next great work of Aoyumi
[a href="index.php?act=findpost&pid=324395"][{POST_SNAPBACK}][/a]

I came to the same conclusion with beta3 using castanets and hihat. Beta2 does a better job at handling pre-echo in these extreme samples.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #72 – 2005-09-02 21:37:59

Quote

Quote
And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.
[a href="index.php?act=findpost&pid=324341"][{POST_SNAPBACK}][/a]

I believe that is correct. Friedmann can be used in the case of ties, but some modifications to the calculation of the F-value have to be made, if I remember it correctly.
[a href="index.php?act=findpost&pid=324350"][{POST_SNAPBACK}][/a]

I believe I made the required modifications (it's been a long while since I wrote the program) to take into account ties for a Friedman analysis

Consider 3 codecs, 2 of them tied for 1st, and the 3rd last. The rankings would be valued at 2.5, 2.5, 1

BTW, Guru's graph shows that he used parametric analyses (Fisher's LSD and Tukey's HSD), not a Friedman anlysis. The executable which runs all the analyses is named friedman.exe, though, because the friedman was the first one I wrote.

ff123

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #73 – 2005-09-03 04:22:29

Quote

And in my opinion, the handling pre-echo mechanism which entered in beta3 are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)

Can you exhibit the log of concrete ABX and HR test? It is more good if there is a detailed comment. In my test range(example - some castanets samples), a result worse than 1.1.0(1) hardly comes out, and, in many cases, the way of beta4 is accepted.

Quote

I came to the same conclusion with beta3 using castanets and hihat. Beta2 does a better job at handling pre-echo in these extreme samples.

Were you tested by beta4 in them? hihat of -q4 and castanets2 of -q6 have improved from beta3.

EDIT:An expression is corrected.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #74 – 2005-09-05 08:17:46

I tested castanets2 with beta4
Now I have no ABX log, but I remember result in castanets2 at q5

wav vs beta4 - 21/25

wav vs 1.1.1 - 15/25

I have the screenshots from my visual analyse encoded samples
I can mail it to you.

edit: my home acoustic is poor and q5 near transparent for me
but beta4 pre-echo is large...

Notice