Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MPC vs VORBIS vs MP3 vs AAC at 180 kbps (Read 325253 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #50
Quote
I could do the same test, present the results in the same fancy manner, and come up with completely different results.[a href="index.php?act=findpost&pid=321838"][{POST_SNAPBACK}][/a]


Could you really? Have you done so? Or is that just speculation?

 

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #51
Quote
Quote
I could do the same test, present the results in the same fancy manner, and come up with completely different results.[a href="index.php?act=findpost&pid=321838"][{POST_SNAPBACK}][/a]


Could you really? Have you done so? Or is that just speculation?
[a href="index.php?act=findpost&pid=321860"][{POST_SNAPBACK}][/a]

Speculation of course, primarily because differeing results are certinaly within the realm of possibility. Every one is different after all. I wasn't just talking about 180k tests by the way, my query applies to any bitrate. I'm lazy so I wouldn't really bother of course. I only bought it up because one of the things I see people posting here at HA.org all the time is that you have to ABX test for yourself in order to get any truly meaningful results, and that makes perfect sense of course. Thus, any testing I might do is only relevent to me, and is why I feel it would be a waste of time to post such results even if some folks may find said results interesting, though I honestly doubt anyone would actually care all that much. After all, I'm just a nobody and guruboolez is the one with the golden ears which, oddly enough, seem to have attained a strange sort of celebrity status around here, hehe. More power to him too. Like I said, despite what seems like (admitedly very minor) hypocrisy, I have to admit I find the tests fun and interesting to read just like everyone else. Thanks again for all your hard work guru! Again, I hope I have not offended anyone, since that was never my intent. Just find the whole thing rather amusing is all, given what is normally posted whenever someone new comes along and asks "what is the best codec/bitrate?".

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #52
I think the flaw in your logic is that you see this as a black/white issue. Tests by other people - especially if you do know about their skill relative to yours - can be a helpful indication. Basically, statistics...... the more of those indications, the more robust the overall picture. However, that picture is only an "average" - and you may be above or below it... or somewhere else ;)

Basically, other peoples tests *are* helpful because they are an indication. Here's an example: I do know that guru's skills are far above mine(both in terms of training as well as ears). Thus, when listening to classical music i could asume that V3 should be transparent to me most of the time. However, if i will ABX the sample-set myself, then i may find out that even v4 is sufficient for me. Thus, other people's tests to give one an indication...... but for fine-tuning, it is necessary to ABX oneself. Some people may decide to skip the additional time for fine-tuning and instead "encode slightly higher than probably needed" - thus, trading space for time & effort.
I am arrogant and I can afford it because I deliver.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #53
Quote
Speculation of course, primarily because differeing results are certinaly within the realm of possibility. Every one is different after all.

Yes but blue is blue for everyone except maybe for daltonic persons. Everyone is different, but the possible profiles are not infinite. And would be the gap between these profiles really important?

Quote
Thus, any testing I might do is only relevent to me, and is why I feel it would be a waste of time to post such results

It's an admirable disinterestedness! I'm sincerly admirative.
But this board is a common place for sharing its own experience. When several individual experience are going along the same lines, then it starts to become trustable. Some formats are considered as superior to others: AAC over VQF, MPC over MP3, etc... These affirmations are based on similar individual experience. If everybody would keep their results secret, no knowledge is possible.

Now I could also keep my listening test for myself. But what will it imply? If I'll answer "Vorbis" to someone interesting to know what should be an ideal format for ~200 kbps encoding with classical, you know what will happen... Most people will ask me to backup my claim because MPC is well-known to have no rivality at this bitrate. One will brandish TOS#8, another one request my message to be deleted, I could get warn, etc...

Quote
After all, I'm just a nobody and guruboolez is the one with the golden ears which, oddly enough, seem to have attained a strange sort of celebrity status around here, hehe. More power to him too.


I don't like this kind of charisma either. I don't want people following blindly my conclusions. I'm not a guru [span style='font-size:8pt;line-height:100%'](yes, I know, my nickname could imply the opposite - but if there's one guru, it's rather Pierre Boulez than me)[/span]. That's why I often request additionnal listening tests and certainly not to set up an altar for my holy conclusions

Quote
Again, I hope I have not offended anyone, since that was never my intent.
For me it's OK. You're just questioning the way 'trustable knowledge' of something based on individual experience is possible. Well, it's just epistemology.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #54
It is always very enjoyable and informative to read a listening test by guruboolez.  In fact, I haven't visited HA for a while but once I saw a listening test done by guruboolez, I headed straight for this thread

What can I say?  I am extremely happy with Ogg Vorbis!  Many thanks to Aoyumi for fixing the problems and maturing Vorbis to such an advanced state that it can even rival the once-dominant MPC.  Has anyone told Monty about these results?  Monty needs to know!!

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #55
I have to agree with Guru.  Either these artifacts exist or they don't.  The level at which various artifacts become bothersome to the individual is the only issue in question (and it's really only significant when dealing with higher bitrates...lower bitrates produce much more noticeable artifacts).  Anyone with a golden pair of ears and artifact awareness (such as guru) can make a valid assessment when they blindly compare to the reference, IMO.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #56
nice listening test guruboolez  great to see the progress that vorbis has made in the last year, and its neat knowing the format still has a great deal of room for future improvements if necessary.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #57
Quote
What can I say? I am extremely happy with Ogg Vorbis! Many thanks to Aoyumi for fixing the problems and maturing Vorbis to such an advanced state that it can even rival the once-dominant MPC. Has anyone told Monty about these results? Monty needs to know!!


He drops in occasionally, but I presume he is busy doing other more important stuff. 

Quote
I hope I have not offended anyone, since that was never my intent. Just find the whole thing rather amusing is all, given what is normally posted whenever someone new comes along and asks "what is the best codec/bitrate?". wink.gif


all the listening tests show to me is a small or substantial improvement in the actual codec. "Best" is whatever is best for your ears and whatever you prefer.  Don't forget that most things outside this community are irrelivent (unless websites point here) and unless they actually get merged into the mainline, much like in 1.1.1.  It's just too bad their aren't any more developers working on getting multichannel coupling up to snuff, etc or who have the ability to do so. Taming that beast is another esotreric task ;D.  Aoyumi work is always greatly appreciated though.
budding I.T professional

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #58
Quote
Quote
Regarding MPC, the result is split, somehow for classical samples there seems to be a degration in quality and size-effectivity (bitrate boost), comparing MPC v1.14b with MPC v1.15v
I cannot dare to ask Guru, to compare this sample set again between 1.14b and 1.15v to get a ranking of this new multiformat test including not only 1.15v but also 1.14.
[a href="index.php?act=findpost&pid=321715"][{POST_SNAPBACK}][/a]

There are two different problems in my opinion:

- bitrate consumption: mpc --standard has increased by more than 10 kbps compared to former release of mppenc (I can post a full bitrate table if you want). 1.15 series is by far the version which presents the highest bitrate (not only with classical: people listening different kind of music have confirmed it - but how much is something I can't say). But this inflation is not necessary a problem: some users don't really care about few additionnal kbps, and it could bring higher quality (higher bitrate doesn't necessary mean lower efficiency).

-quality (with classical): I'm not sure that 1.15 presents regression compared to 1.14 beta. The problem maybe occur earlier. I compared 1.15u to a much older release of mppenc (1.01j) which clearly revealed issues with the latest version of mppenc (+ inflated bitrate). Now I can't tell when exactly the problem happened, or if the quality (with classical) has slowly decreased with version > 1.0 stable.



yeah, I know it from posts, I used to use MPC 1.01j once, too. And 1.14 for some time, until I switched to 1.15u,v.
The bitrate increase is not only for classic or your impressive 150 samples, but also clearly known, reported here in forum, too, when comparing 1.14 with 1.15r,s,t,u,v. The bitrate increase happens in all genres, classic, jazz, pop, rock, metal probably, too.
I agree, that we don't need to discuss the bitrate as such.
I am using 1.15 myself, as it improved other problem samples. But I was pointing to the facts, just to make it not forgotten..


One remark to the validity of this test and others:

If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.
Of course, primarily you have to say, that this test is primarily valid only for this 1 test-person.
But we have some history here, and the reason why Guruboulez tests are very welcomed here, is following: Other persons repeated gurus tests (with or without posting results) and eitehr confirmed them, or admitted, not to hear the difference, but most important: no contradiction.
By experience some people here know, that guruboulez results are valid not only to him, but also to other persons, normally worded "made same experience".
So, if somebody above tries to play "devils advocate", and questions validity of this test, he should not ridicule himself. The only way to question results is in presenting listening test with opposite/different results.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #59
Quote
If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.


Actually the deaf group adds significance, as the world isn't populated only by golden ears.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #60
Quote
Quote
Probably, change of ATH, noise/tone masking, and noise compander etc. is useful to the improvement of roughness.

Magic does not exist there.


I was digging through the code before just to try and get a better understanding of how things work? what exactly is noise companding and noise biasing? I mean what specific roles do they play? I understand the ATH and noise/tone masking routines using the FFT and MDCT, but I don't really understand the other two that well.  It's my understanding that when you are changing the impulse_noisetune if you are using advanced-command line switch you are actually adjusting the noise bias?
[a href="index.php?act=findpost&pid=321779"][{POST_SNAPBACK}][/a]

Noisebias expresses a frequency noise curve, and noise compand expresses a loudness noise curve. These are bearing the role important for determining a final noise masking value.
Moreover, impulse_noisetune operates noisebias of impulse block (short block). 

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #61
Quote
The bitrate increase is not only for classic or your impressive 150 samples, but also clearly known, reported here in forum, too, when comparing 1.14 with 1.15r,s,t,u,v. The bitrate increase happens in all genres, classic, jazz, pop, rock, metal probably, too.
[{POST_SNAPBACK}][/a]
I found an old message posted by Frank Klemm about this increase:

Quote
Bitrate is increased by typically 3.1% compared with 1.14. Most of this bitrate increase is related to
some modificitions in the tonality estimation model (which was introduced with 1.15g).
Source [a href=\"http://www.hydrogenaudio.org/forums/index.php?showtopic=6558&view=findpost&p=66098]here[/url].

If you're interested, I also compared the bitrate of six different mppenc version, from Buschmann 1.7.8c to latest 1.15v:


(a zoomed version here.)

If you can't see 1.78c it's simply because 0.90o (my oldest mppenc version) is ultra-close to it.
The 1.04 version is by far the most efficient version of mppenc - but remember how much people were frighten about this bitrate drop! Therefore, bitrate started to increase again with 1.06 (not present here).
1.14 is still close to native --standard bitrate (+3,5 kbps on average for my 150 full tracks).
1.15v is now far from what Buschmann 1.78c / Klemm 1.00 encoders used to output. The difference between the extreme is 15 kbps! Does it really justify some gain with a limited set of samples?

If you want to play with the datas, here's the complete table (sorted by samples):
Code: [Select]
CATALOG	1.7.8c	0.90o	1.01j	1.04	1.14	1.15v
NUMBER
A01 153,2 153,1 157,0 152,7 157,9 164,1
A02 176,8 176,8 182,3 179,2 189,1 192,2
A03 185,6 185,6 189,7 189,4 200,4 205,1
A04 173,5 173,5 177,3 178,4 185,6 188,2
A05 172,8 172,8 176,2 162,2 170,0 177,9
E01 117,1 117,1 118,0 110,0 116,8 121,8
E02 173,8 173,8 175,5 171,6 178,8 186,7
E03 174,6 174,7 178,0 175,7 183,8 188,8
E04 159,7 159,6 160,2 153,1 159,6 167,6
E05 157,5 157,5 161,6 157,0 165,3 167,2
E06 173,8 173,8 175,3 171,7 178,9 187,2
E07 169,1 169,1 169,9 163,1 169,5 178,2
E08 164,4 164,4 170,1 154,4 165,0 168,0
E09 176,0 176,0 177,2 173,0 180,1 188,3
E10 168,1 168,1 170,1 158,5 166,3 173,3
E11 174,1 174,1 177,0 173,6 181,1 188,3
E12 150,3 150,3 152,6 145,2 151,5 157,9
E13 183,3 183,3 183,3 178,3 183,7 193,3
E14 169,9 169,9 172,6 169,2 177,3 185,2
E15 177,7 177,6 177,4 171,9 179,1 190,8
E16 186,0 186,0 187,8 185,0 191,9 199,7
E17 167,6 167,6 170,9 168,0 174,3 178,2
E18 149,8 149,8 151,8 146,3 151,7 157,4
E19 147,3 147,3 148,3 142,1 148,1 155,2
E20 151,0 151,0 152,0 141,0 148,9 157,3
E21 153,8 153,8 156,0 150,1 156,0 162,6
E22 173,5 173,4 175,3 174,1 180,2 187,6
E23 161,9 161,9 163,5 158,4 164,4 172,8
E24 159,1 159,1 160,3 153,1 160,6 169,1
E25 173,9 173,8 176,1 176,0 182,2 188,6
E26 165,1 165,0 167,9 161,7 168,9 174,6
E27 169,2 169,2 171,1 164,9 173,3 180,8
E28 151,7 151,7 156,0 153,9 159,6 159,6
E29 158,1 158,1 160,9 153,6 162,2 166,9
E30 164,2 164,2 168,2 166,0 172,7 175,9
E31 185,1 185,0 186,3 186,6 193,0 201,4
E32 189,7 189,7 189,9 188,0 193,7 204,7
E33 170,9 170,8 172,9 169,1 179,9 185,7
E34 162,3 162,3 164,9 159,0 163,3 168,8
E35 204,5 204,4 203,6 199,6 203,8 220,9
E36 201,5 201,4 201,3 199,9 205,9 216,3
E37 181,4 181,3 182,7 177,7 185,3 193,7
E38 174,9 174,8 176,0 171,8 178,4 187,1
E39 175,6 175,5 176,2 170,4 176,7 186,3
E40 157,7 157,7 160,3 152,2 160,7 168,6
E41 213,5 213,5 212,0 207,2 211,8 225,0
E42 207,7 207,6 207,3 203,1 209,7 220,6
E43 190,0 190,0 190,6 187,6 193,4 205,4
E44 169,8 169,8 171,1 161,9 168,7 178,9
E45 201,5 201,5 201,6 201,2 207,3 219,2
E46 168,6 168,5 171,0 168,0 173,2 182,2
E47 180,2 180,2 181,5 178,1 182,9 189,8
E48 188,3 188,3 189,5 186,8 194,5 203,1
E49 164,6 164,6 166,1 163,2 169,3 177,0
E50 194,4 194,4 196,4 196,5 202,4 210,0
E51 178,3 178,3 179,4 177,5 183,9 193,4
E52 183,5 183,5 183,7 182,3 187,7 196,9
E53 174,7 174,7 176,8 175,5 181,9 189,5
E54 171,0 171,0 172,7 169,4 176,3 181,7
E55 176,0 176,0 176,9 174,7 181,2 188,9
E56 173,8 173,8 177,2 172,6 179,9 185,1
E57 173,5 173,5 176,9 175,5 181,7 185,2
E58 173,1 173,1 177,4 178,1 183,9 185,5
E59 187,7 187,7 189,3 187,7 195,6 207,3
E60 176,3 176,3 179,0 178,7 185,5 191,2
S01 173,2 173,2 175,4 155,5 162,9 171,7
S02 153,6 153,6 155,9 148,9 156,4 163,8
S03 171,6 171,6 172,7 165,6 172,0 181,0
S04 182,8 182,8 184,0 178,7 186,9 193,4
S05 189,5 189,5 190,3 184,5 189,9 201,9
S06 178,1 178,1 178,5 170,0 179,9 187,6
S07 206,9 206,9 206,7 203,7 212,8 221,6
S08 173,9 173,9 176,0 170,1 179,3 186,8
S09 188,7 188,7 189,3 182,1 191,3 200,0
S10 176,7 176,7 178,0 170,4 177,8 190,9
S11 228,5 228,5 227,0 222,0 226,8 246,3
S12 229,3 229,2 228,0 223,9 228,2 248,1
S13 203,5 203,4 203,3 201,6 209,3 220,1
S14 189,4 189,4 191,2 192,0 197,9 207,5
S15 205,4 205,4 204,7 201,3 206,9 221,4
S16 233,9 233,8 232,4 227,7 233,6 251,2
S17 167,0 167,0 169,7 167,4 173,9 180,9
S18 180,8 180,7 180,2 172,3 177,2 184,7
S19 167,9 167,9 168,7 162,0 168,5 177,8
S20 173,2 173,2 172,9 169,3 175,1 185,0
S21 176,8 176,7 178,2 172,2 176,2 184,6
S22 196,3 196,3 195,4 189,7 195,9 207,3
S23 163,5 163,5 165,8 162,2 169,3 177,0
S24 139,6 139,6 141,7 136,2 143,4 149,5
S25 162,8 162,8 165,2 155,9 163,7 172,5
S26 124,4 124,4 125,8 118,9 124,2 132,3
S27 126,4 126,4 127,5 119,2 125,9 134,1
S28 136,8 136,8 137,6 130,6 135,0 145,8
S29 149,0 149,0 151,4 144,3 149,7 158,5
S30 162,8 162,8 164,0 151,8 161,0 168,3
S31 162,8 162,8 164,0 151,8 161,0 168,3
S32 187,4 187,4 187,7 180,7 191,1 201,4
S33 158,6 158,6 161,3 153,4 162,5 170,2
S34 196,2 196,2 197,6 192,5 202,2 212,6
S35 202,4 202,3 203,0 196,9 201,7 213,6
S36 167,8 167,8 173,1 168,3 174,8 180,6
S37 165,0 165,0 169,8 158,2 168,0 172,4
S38 169,7 169,7 172,0 167,1 174,1 186,2
S39 152,7 152,7 155,2 147,2 153,9 167,7
S40 178,2 178,2 181,2 180,8 184,4 202,0
S41 162,0 162,0 165,6 159,4 165,2 171,2
S42 171,2 171,1 172,3 162,4 170,4 178,0
S43 170,7 170,7 173,1 169,2 173,5 185,3
S44 226,6 226,6 225,9 220,0 224,6 236,6
S45 157,2 157,2 159,1 146,9 153,1 164,1
S46 166,6 166,5 169,3 159,9 167,8 178,2
S47 178,6 178,6 181,6 172,7 180,8 189,3
S48 164,0 164,0 166,0 160,8 167,9 175,6
S49 162,3 162,3 163,8 155,7 163,1 169,6
S50 194,5 194,4 196,6 194,2 205,9 219,1
S51 185,4 185,4 187,8 181,3 189,5 193,2
S52 185,3 185,2 186,6 184,1 191,8 199,5
S53 163,3 163,3 164,7 152,4 160,6 169,0
S54 120,9 120,9 122,7 110,5 121,0 131,6
S55 141,2 141,2 142,8 134,6 142,6 151,1
V01 175,1 175,1 176,3 173,8 179,0 188,3
V02 162,7 162,7 165,5 164,5 169,9 174,8
V03 109,5 109,5 110,5 100,4 108,2 113,8
V04 173,4 173,4 176,7 176,7 183,7 186,6
V05 163,3 163,2 165,8 163,4 169,6 175,6
V06 169,2 169,2 172,2 173,0 178,2 182,4
V07 163,4 163,4 166,2 165,0 170,8 176,3
V08 146,4 146,4 148,2 139,9 147,7 153,7
V09 177,2 177,1 179,5 174,1 182,6 188,7
V10 190,8 190,7 191,2 187,8 193,5 203,1
V11 156,1 156,1 159,5 151,9 159,3 164,7
V12 178,2 178,2 181,3 178,7 185,4 194,5
V13 161,9 161,9 165,8 159,4 166,0 174,5
V14 160,6 160,6 163,6 162,3 168,0 176,5
V15 173,7 173,7 175,2 170,3 178,1 190,7
V16 174,2 174,2 176,1 172,5 179,4 187,5
V17 162,1 162,1 163,7 153,9 160,9 171,8
V18 177,1 177,1 178,8 176,5 182,2 190,8
V19 167,6 167,6 169,9 166,8 173,0 180,0
V20 172,8 172,8 175,3 171,8 178,7 186,0
V21 164,6 164,6 167,2 158,9 166,5 171,3
V22 169,3 169,2 173,7 167,5 175,4 179,1
V23 183,3 183,3 184,5 176,8 184,9 192,3
V24 198,7 198,7 199,9 198,1 203,9 214,8
V25 180,4 180,4 183,1 179,1 186,2 194,0
V26 167,5 167,4 168,9 163,5 170,7 178,4
V27 175,9 175,8 178,0 175,3 181,0 189,5
V28 165,4 165,4 167,0 159,4 166,0 176,9
V29 169,1 169,1 173,0 172,4 179,7 185,4
V30 181,0 180,9 182,1 178,9 184,8 194,8

172,71 172,69 174,51 169,37 176,28 184,54

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #62
Quote
Quote
If a (the!) trained person tests samples, the results will be more consistent than you carry out a test with a bigger group which includes "deaf" people, who just lower the p-values, who don't add any significance.


Actually the deaf group adds significance, as the world isn't populated only by golden ears.
[a href="index.php?act=findpost&pid=322047"][{POST_SNAPBACK}][/a]

True, but what user meant is -I suppose- that results of people unable to tell a difference are not really helpful if the purpose of the test is to know how the difference is between these encoders.
A lot of 5.0 everywhere are telling us that these encoders are transparent to a lot of people (I guess that there's no need to conduce such tests to confirm that ), but not how different are these encoders. That's why critical listeners could be needed here.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #63
Quote
True, but what user meant is -I suppose- that results of people unable to tell a difference are not really helpful if the purpose of the test is to know how the difference is between these encoders.
A lot of 5.0 everywhere are telling us that these encoders are transparent to a lot of people (I guess that there's no need to conduce such tests to confirm that ), but not how different are these encoders. That's why critical listeners could be needed here.
[a href="index.php?act=findpost&pid=322053"][{POST_SNAPBACK}][/a]


Indeed. Deaf users are good to know how formats sound to everybody. But not necessarily how formats sound against each other.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #64
Awesome piece of work and information, simply awesome.

Thanks a lot.
WavPack 5.6.0 -b384hx6cmv / qaac64 2.80 -V 100

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #65
Here is the bitrate table (based on 150 full tracks):


Code: [Select]
FILE	faac	Nero	LAME	MPC	Vorbis

A01 175,0 199,2 194,7 164,1 201,7
A02 241,6 156,8 210,8 192,2 206,2
A03 230,6 165,7 230,3 205,1 293,1
A04 203,5 176,1 217,8 188,2 207,3
A05 183,3 161,9 175,8 177,9 182,6
E01 151,3 183,6 135,2 121,8 154,7
E02 179,1 185,6 182,0 186,7 178,8
E03 190,9 179,4 187,0 188,8 184,3
E04 172,9 179,8 178,0 167,6 174,8
E05 215,8 159,5 188,6 167,2 194,4
E06 173,6 186,9 188,9 187,2 179,0
E07 163,6 189,9 180,8 178,2 168,7
E08 182,4 147,8 191,5 168,0 199,4
E09 176,3 187,2 181,4 188,3 178,0
E10 176,4 170,1 167,9 173,3 166,7
E11 211,3 175,8 201,6 188,3 188,3
E12 163,4 184,1 169,5 157,9 167,3
E13 169,6 176,8 172,6 193,3 184,5
E14 163,6 190,7 184,9 185,2 176,5
E15 159,9 198,5 180,5 190,8 176,5
E16 188,0 181,6 194,0 199,7 202,3
E17 188,2 180,7 196,2 178,2 185,7
E18 169,5 197,0 174,3 157,4 177,6
E19 167,9 180,4 167,0 155,2 168,3
E20 166,0 180,2 179,4 157,3 161,8
E21 166,8 185,7 181,3 162,6 170,4
E22 176,5 182,8 187,9 187,6 177,1
E23 158,5 181,8 182,4 172,8 163,0
E24 161,9 189,5 175,9 169,1 163,3
E25 191,3 178,6 194,4 188,6 182,6
E26 176,3 178,0 184,5 174,6 171,3
E27 178,8 175,5 184,9 180,8 168,4
E28 178,8 183,0 191,4 159,6 183,8
E29 189,4 166,1 177,0 166,9 170,9
E30 183,7 184,2 196,9 175,9 174,9
E31 222,0 154,1 192,0 201,4 199,6
E32 190,2 180,9 186,9 204,7 196,8
E33 187,2 162,5 177,1 185,7 183,2
E34 160,6 181,1 174,6 168,8 167,5
E35 186,6 169,0 181,3 220,9 206,2
E36 208,6 173,5 187,3 216,3 216,5
E37 172,0 183,7 175,6 193,7 180,3
E38 174,3 183,6 182,5 187,1 177,2
E39 174,3 184,2 184,0 186,3 188,3
E40 163,7 181,9 170,6 168,6 157,1
E41 175,6 169,9 174,7 225,0 197,2
E42 182,5 177,6 174,3 220,6 190,6
E43 188,7 178,0 188,0 205,4 188,4
E44 170,0 167,8 180,8 178,9 167,9
E45 218,6 163,5 194,6 219,2 212,2
E46 185,3 178,0 187,9 182,2 179,7
E47 177,5 192,4 187,6 189,8 193,3
E48 182,0 182,8 187,7 203,1 187,5
E49 171,2 179,2 180,5 177,0 175,1
E50 226,3 179,6 214,9 210,0 211,6
E51 180,3 180,6 187,9 193,4 185,1
E52 182,3 176,4 186,0 196,9 188,7
E53 189,6 174,4 190,0 189,5 185,2
E54 173,6 177,1 183,4 181,7 179,0
E55 175,1 184,8 188,9 188,9 181,6
E56 213,3 163,2 186,4 185,1 184,4
E57 214,6 165,5 195,5 185,2 181,7
E58 206,1 177,5 202,4 185,5 181,9
E59 189,0 166,4 184,5 207,3 199,1
E60 202,4 173,8 193,0 191,2 203,9
S01 175,7 134,1 138,7 171,7 153,4
S02 173,1 175,1 173,4 163,8 166,1
S03 172,4 185,9 181,0 181,0 165,9
S04 175,0 172,1 180,1 193,4 174,0
S05 169,5 172,1 178,0 201,9 182,9
S06 183,4 166,3 166,8 187,6 173,7
S07 181,6 175,8 179,8 221,6 192,5
S08 188,3 174,8 181,3 186,8 171,0
S09 185,8 174,5 178,0 200,0 180,5
S10 230,6 158,1 185,5 190,9 181,7
S11 204,8 166,8 185,4 246,3 224,4
S12 217,5 160,1 185,0 248,1 232,2
S13 246,6 160,2 206,6 220,1 226,7
S14 229,1 162,5 216,5 207,5 226,6
S15 221,2 166,1 193,1 221,4 214,3
S16 204,5 164,6 174,6 251,2 220,2
S17 172,9 174,9 188,9 180,9 175,9
S18 161,0 175,0 184,8 184,7 169,5
S19 160,6 190,0 180,2 177,8 168,6
S20 164,6 188,5 204,3 185,0 174,5
S21 176,3 172,5 175,6 184,6 169,6
S22 171,9 172,0 181,6 207,3 189,6
S23 163,0 198,4 190,3 177,0 175,3
S24 176,2 182,3 174,3 149,5 176,7
S25 158,6 166,9 182,0 172,5 165,9
S26 151,9 189,4 150,4 132,3 156,8
S27 156,6 176,1 161,4 134,1 163,2
S28 153,4 198,1 151,9 145,8 163,0
S29 157,9 196,6 175,9 158,5 168,0
S30 161,3 171,6 141,7 168,3 156,6
S31 161,3 171,6 141,7 168,3 156,6
S32 187,8 168,6 159,2 201,4 192,2
S33 188,1 169,2 173,8 170,2 169,9
S34 165,3 175,0 185,7 212,6 178,7
S35 197,1 166,5 178,3 213,6 190,5
S36 152,9 197,1 177,2 180,6 180,6
S37 176,6 171,9 175,2 172,4 191,6
S38 163,8 194,2 193,0 186,2 185,7
S39 162,9 194,7 177,3 167,7 171,1
S40 172,1 180,1 184,9 202,0 196,2
S41 183,2 165,0 180,4 171,2 182,9
S42 164,1 163,9 174,8 178,0 163,3
S43 159,7 182,6 185,3 185,3 179,0
S44 193,3 169,0 157,4 236,6 205,3
S45 154,0 170,9 157,0 164,1 150,8
S46 164,0 169,1 172,0 178,2 165,9
S47 175,1 167,8 176,9 189,3 174,3
S48 163,3 190,7 180,2 175,6 167,7
S49 165,1 177,3 174,0 169,6 170,2
S50 186,6 165,1 182,2 219,1 194,1
S51 166,3 164,1 183,0 193,2 175,3
S52 190,5 178,7 183,2 199,5 182,8
S53 160,4 160,8 140,4 169,0 162,5
S54 163,9 158,7 146,5 131,6 162,4
S55 161,5 173,4 152,6 151,1 166,6
V01 172,7 184,1 188,1 188,3 179,8
V02 165,1 189,9 194,8 174,8 169,9
V03 171,1 172,0 141,5 113,8 147,0
V04 214,6 169,8 202,8 186,6 198,0
V05 171,3 186,0 189,1 175,6 174,3
V06 184,3 185,8 192,7 182,4 182,8
V07 175,5 181,8 199,8 176,3 171,5
V08 172,2 173,7 178,6 153,7 165,5
V09 180,2 171,6 173,3 188,7 175,7
V10 183,7 172,2 181,3 203,1 179,1
V11 168,5 178,9 178,8 164,7 154,9
V12 175,5 178,0 192,4 194,5 182,0
V13 183,1 177,1 189,3 174,5 171,4
V14 167,6 186,6 198,5 176,5 173,2
V15 173,4 178,1 190,9 190,7 179,2
V16 164,1 180,3 189,5 187,5 169,6
V17 177,3 185,1 187,3 171,8 174,6
V18 182,6 181,5 188,0 190,8 187,2
V19 179,1 168,9 180,2 180,0 170,2
V20 191,0 175,2 191,8 186,0 204,4
V21 194,9 163,7 178,2 171,3 175,1
V22 185,3 174,2 180,9 179,1 176,9
V23 165,3 178,9 181,5 192,3 168,7
V24 232,5 162,0 195,8 214,8 219,7
V25 192,8 163,2 184,1 194,0 180,9
V26 184,6 173,0 172,6 178,4 173,7
V27 196,9 172,5 186,6 189,5 182,4
V28 167,7 184,6 180,2 176,9 170,5
V29 189,0 177,9 187,6 185,4 178,3
V30 181,9 182,8 184,8 194,8 178,9

180,92 176,14 181,79 184,54 181,48

faac Nero LAME MPC Vorbis

Graphicaly and sorted by size:

 
I tried with older version of aacenc32.dll: identical - identical - identical... 
In short, -streaming 'high' encodings are bit-to-bit identical from aacenc32.dll version 2.9.9.999 (dated from 2004.12.22) to 3.2.0.20! The difference start to appear with 2.9.9.998 dated from 2004.11.27. In other words, no progress in this area since eight months. What happened with Nero Digital team? Have Ivan only worked on SBR and low bitrate stuff during this time?

examples:
files are not available anymore for download
Here is the bitrate table (based on 150 full tracks):


Code: [Select]
FILE	faac	Nero	LAME	MPC	Vorbis

A01 175,0 199,2 194,7 164,1 201,7
A02 241,6 156,8 210,8 192,2 206,2
A03 230,6 165,7 230,3 205,1 293,1
A04 203,5 176,1 217,8 188,2 207,3
A05 183,3 161,9 175,8 177,9 182,6
E01 151,3 183,6 135,2 121,8 154,7
E02 179,1 185,6 182,0 186,7 178,8
E03 190,9 179,4 187,0 188,8 184,3
E04 172,9 179,8 178,0 167,6 174,8
E05 215,8 159,5 188,6 167,2 194,4
E06 173,6 186,9 188,9 187,2 179,0
E07 163,6 189,9 180,8 178,2 168,7
E08 182,4 147,8 191,5 168,0 199,4
E09 176,3 187,2 181,4 188,3 178,0
E10 176,4 170,1 167,9 173,3 166,7
E11 211,3 175,8 201,6 188,3 188,3
E12 163,4 184,1 169,5 157,9 167,3
E13 169,6 176,8 172,6 193,3 184,5
E14 163,6 190,7 184,9 185,2 176,5
E15 159,9 198,5 180,5 190,8 176,5
E16 188,0 181,6 194,0 199,7 202,3
E17 188,2 180,7 196,2 178,2 185,7
E18 169,5 197,0 174,3 157,4 177,6
E19 167,9 180,4 167,0 155,2 168,3
E20 166,0 180,2 179,4 157,3 161,8
E21 166,8 185,7 181,3 162,6 170,4
E22 176,5 182,8 187,9 187,6 177,1
E23 158,5 181,8 182,4 172,8 163,0
E24 161,9 189,5 175,9 169,1 163,3
E25 191,3 178,6 194,4 188,6 182,6
E26 176,3 178,0 184,5 174,6 171,3
E27 178,8 175,5 184,9 180,8 168,4
E28 178,8 183,0 191,4 159,6 183,8
E29 189,4 166,1 177,0 166,9 170,9
E30 183,7 184,2 196,9 175,9 174,9
E31 222,0 154,1 192,0 201,4 199,6
E32 190,2 180,9 186,9 204,7 196,8
E33 187,2 162,5 177,1 185,7 183,2
E34 160,6 181,1 174,6 168,8 167,5
E35 186,6 169,0 181,3 220,9 206,2
E36 208,6 173,5 187,3 216,3 216,5
E37 172,0 183,7 175,6 193,7 180,3
E38 174,3 183,6 182,5 187,1 177,2
E39 174,3 184,2 184,0 186,3 188,3
E40 163,7 181,9 170,6 168,6 157,1
E41 175,6 169,9 174,7 225,0 197,2
E42 182,5 177,6 174,3 220,6 190,6
E43 188,7 178,0 188,0 205,4 188,4
E44 170,0 167,8 180,8 178,9 167,9
E45 218,6 163,5 194,6 219,2 212,2
E46 185,3 178,0 187,9 182,2 179,7
E47 177,5 192,4 187,6 189,8 193,3
E48 182,0 182,8 187,7 203,1 187,5
E49 171,2 179,2 180,5 177,0 175,1
E50 226,3 179,6 214,9 210,0 211,6
E51 180,3 180,6 187,9 193,4 185,1
E52 182,3 176,4 186,0 196,9 188,7
E53 189,6 174,4 190,0 189,5 185,2
E54 173,6 177,1 183,4 181,7 179,0
E55 175,1 184,8 188,9 188,9 181,6
E56 213,3 163,2 186,4 185,1 184,4
E57 214,6 165,5 195,5 185,2 181,7
E58 206,1 177,5 202,4 185,5 181,9
E59 189,0 166,4 184,5 207,3 199,1
E60 202,4 173,8 193,0 191,2 203,9
S01 175,7 134,1 138,7 171,7 153,4
S02 173,1 175,1 173,4 163,8 166,1
S03 172,4 185,9 181,0 181,0 165,9
S04 175,0 172,1 180,1 193,4 174,0
S05 169,5 172,1 178,0 201,9 182,9
S06 183,4 166,3 166,8 187,6 173,7
S07 181,6 175,8 179,8 221,6 192,5
S08 188,3 174,8 181,3 186,8 171,0
S09 185,8 174,5 178,0 200,0 180,5
S10 230,6 158,1 185,5 190,9 181,7
S11 204,8 166,8 185,4 246,3 224,4
S12 217,5 160,1 185,0 248,1 232,2
S13 246,6 160,2 206,6 220,1 226,7
S14 229,1 162,5 216,5 207,5 226,6
S15 221,2 166,1 193,1 221,4 214,3
S16 204,5 164,6 174,6 251,2 220,2
S17 172,9 174,9 188,9 180,9 175,9
S18 161,0 175,0 184,8 184,7 169,5
S19 160,6 190,0 180,2 177,8 168,6
S20 164,6 188,5 204,3 185,0 174,5
S21 176,3 172,5 175,6 184,6 169,6
S22 171,9 172,0 181,6 207,3 189,6
S23 163,0 198,4 190,3 177,0 175,3
S24 176,2 182,3 174,3 149,5 176,7
S25 158,6 166,9 182,0 172,5 165,9
S26 151,9 189,4 150,4 132,3 156,8
S27 156,6 176,1 161,4 134,1 163,2
S28 153,4 198,1 151,9 145,8 163,0
S29 157,9 196,6 175,9 158,5 168,0
S30 161,3 171,6 141,7 168,3 156,6
S31 161,3 171,6 141,7 168,3 156,6
S32 187,8 168,6 159,2 201,4 192,2
S33 188,1 169,2 173,8 170,2 169,9
S34 165,3 175,0 185,7 212,6 178,7
S35 197,1 166,5 178,3 213,6 190,5
S36 152,9 197,1 177,2 180,6 180,6
S37 176,6 171,9 175,2 172,4 191,6
S38 163,8 194,2 193,0 186,2 185,7
S39 162,9 194,7 177,3 167,7 171,1
S40 172,1 180,1 184,9 202,0 196,2
S41 183,2 165,0 180,4 171,2 182,9
S42 164,1 163,9 174,8 178,0 163,3
S43 159,7 182,6 185,3 185,3 179,0
S44 193,3 169,0 157,4 236,6 205,3
S45 154,0 170,9 157,0 164,1 150,8
S46 164,0 169,1 172,0 178,2 165,9
S47 175,1 167,8 176,9 189,3 174,3
S48 163,3 190,7 180,2 175,6 167,7
S49 165,1 177,3 174,0 169,6 170,2
S50 186,6 165,1 182,2 219,1 194,1
S51 166,3 164,1 183,0 193,2 175,3
S52 190,5 178,7 183,2 199,5 182,8
S53 160,4 160,8 140,4 169,0 162,5
S54 163,9 158,7 146,5 131,6 162,4
S55 161,5 173,4 152,6 151,1 166,6
V01 172,7 184,1 188,1 188,3 179,8
V02 165,1 189,9 194,8 174,8 169,9
V03 171,1 172,0 141,5 113,8 147,0
V04 214,6 169,8 202,8 186,6 198,0
V05 171,3 186,0 189,1 175,6 174,3
V06 184,3 185,8 192,7 182,4 182,8
V07 175,5 181,8 199,8 176,3 171,5
V08 172,2 173,7 178,6 153,7 165,5
V09 180,2 171,6 173,3 188,7 175,7
V10 183,7 172,2 181,3 203,1 179,1
V11 168,5 178,9 178,8 164,7 154,9
V12 175,5 178,0 192,4 194,5 182,0
V13 183,1 177,1 189,3 174,5 171,4
V14 167,6 186,6 198,5 176,5 173,2
V15 173,4 178,1 190,9 190,7 179,2
V16 164,1 180,3 189,5 187,5 169,6
V17 177,3 185,1 187,3 171,8 174,6
V18 182,6 181,5 188,0 190,8 187,2
V19 179,1 168,9 180,2 180,0 170,2
V20 191,0 175,2 191,8 186,0 204,4
V21 194,9 163,7 178,2 171,3 175,1
V22 185,3 174,2 180,9 179,1 176,9
V23 165,3 178,9 181,5 192,3 168,7
V24 232,5 162,0 195,8 214,8 219,7
V25 192,8 163,2 184,1 194,0 180,9
V26 184,6 173,0 172,6 178,4 173,7
V27 196,9 172,5 186,6 189,5 182,4
V28 167,7 184,6 180,2 176,9 170,5
V29 189,0 177,9 187,6 185,4 178,3
V30 181,9 182,8 184,8 194,8 178,9

180,92 176,14 181,79 184,54 181,48

faac Nero LAME MPC Vorbis

Graphicaly and sorted by size:

 
I tried with older version of aacenc32.dll: identical - identical - identical... 
In short, -streaming 'high' encodings are bit-to-bit identical from aacenc32.dll version 2.9.9.999 (dated from 2004.12.22) to 3.2.0.20! The difference start to appear with 2.9.9.998 dated from 2004.11.27. In other words, no progress in this area since eight months. What happened with Nero Digital team? Have Ivan only worked on SBR and low bitrate stuff during this time?

examples:
files are not available anymore for download

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #66
Wow!

Thank you, guruboolez!

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #67
Thank you for the test.
I hope it will encourage other people to perform tests at high bitrates, which have been long considered as out of reach.
I'd like to test some metal samples with the same encoders but I'm so short of time in order to do all what I'd like to do. In fact, I've got a lot of time, but too much interesting things to do !

To remarks about statistics : MPC and Vobris seem tied, but the results don't take into account the ABX results, which adds a lot of significance. Maybe if we could analyse all the data, some encoders that look tied wouldn't be so anymore.
And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #68
Quote
And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.
[a href="index.php?act=findpost&pid=324341"][{POST_SNAPBACK}][/a]

I believe that is correct. Friedmann can be used in the case of ties, but some modifications to the calculation of the F-value have to be made, if I remember it correctly.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #69
I feel great urge to thank Guruboolez and Aoyumi for their very different, yet so coherent work on Vorbis. The next big thing is aoTuV beta 5 (or will it be the official release, like 1.2 or something?), which I hope will reduce some ugly noise issues (I can upload a few samples which are ABX-able @ -q5 and, supposedly, -q6).

Guru, your tests will always be greatly appreciated by me and my friends. Thank you, again.

Quote
What I'm trying to say is that, unlike most other genres, classical samples can be very different and so when you have a thorough test like this one, you can get a pretty good overall picture of each codec (encoder).
[a href="index.php?act=findpost&pid=321772"][{POST_SNAPBACK}][/a]

It depends on what do you call a difference. Music can be very different even while being in the same genre, let alone “guitar” or “electronic” music…

Quote
In fact, I haven't visited HA for a while but once I saw a listening test done by guruboolez, I headed straight for this thread
[a href="index.php?act=findpost&pid=321901"][{POST_SNAPBACK}][/a]

Hey, me too!  I've just returned from my vacation, heh.
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #70
And in my opinion, the handling pre-echo mechanism which entered in beta3  are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)
Waiting for next great work of Aoyumi 

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #71
Quote
And in my opinion, the handling pre-echo mechanism which entered in beta3  are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)
Waiting for next great work of Aoyumi 
[a href="index.php?act=findpost&pid=324395"][{POST_SNAPBACK}][/a]


I came to the same conclusion with beta3 using castanets and hihat. Beta2 does a better job at handling pre-echo in these extreme samples.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #72
Quote
Quote
And I think that Friedemann's method can't be used here, because, if I'm not mistaken, it assumes that no encoder can be tied.
[a href="index.php?act=findpost&pid=324341"][{POST_SNAPBACK}][/a]

I believe that is correct. Friedmann can be used in the case of ties, but some modifications to the calculation of the F-value have to be made, if I remember it correctly.
[a href="index.php?act=findpost&pid=324350"][{POST_SNAPBACK}][/a]


I believe I made the required modifications (it's been a long while since I wrote the program) to take into account ties for a Friedman analysis

Consider 3 codecs, 2 of them tied for 1st, and the 3rd last.  The rankings would be valued at 2.5, 2.5, 1

BTW, Guru's graph shows that he used parametric analyses (Fisher's LSD and Tukey's HSD), not a Friedman anlysis.  The executable which runs all the analyses is named friedman.exe, though, because the friedman was the first one I wrote.

ff123

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #73
Quote
And in my opinion, the handling pre-echo mechanism which entered in beta3 are not excellent (this conclusion I made in last week after some tests with killer samples like castanets2)

Can you exhibit the log of concrete ABX and HR test?  It is more good if there is a detailed comment. In my test range(example - some castanets samples), a result worse than 1.1.0(1) hardly comes out, and, in many cases, the way of beta4 is accepted.

Quote
I came to the same conclusion with beta3 using castanets and hihat. Beta2 does a better job at handling pre-echo in these extreme samples.

Were you tested by beta4 in them?  hihat of -q4 and castanets2 of -q6 have improved from beta3.

EDIT:An expression is corrected.

MPC vs VORBIS vs MP3 vs AAC at 180 kbps

Reply #74
I tested castanets2 with beta4
Now I have no ABX log, but I remember result in castanets2 at q5

wav vs beta4 - 21/25

wav vs 1.1.1 - 15/25

I have the screenshots from my visual analyse encoded samples
I can mail it to you.

edit: my home acoustic is poor and q5 near transparent for me
but beta4 pre-echo is large...