Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Earguy\'s improved Digital Ear (Read 26297 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Earguy\'s improved Digital Ear

Reply #25 –
Quote
The results aren't shocking. They demonstrate that MPC is superior to LAME at equal bitrates...something we've known for a while. The constant harping about MPC is deserved.
No, in fact it's not. MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec. And these results show it's not. Bitrate to bitrate matchups aren't the issue. The highest possible quality of output, at any reasonable bitrate is the issue. The fact of the matter is that in this test, with this tool, the top LAME preset beat the top MPC preset. Period. Cry all you want, it doesn't change the result. And look at the midrange area in which api was clearly superior to MPC. That's in the audible range, even for those with less than stellar hearing. If you know of problem samples for api, name them. Don't just voice your assupmtions that there "must be some" as an argument that's anything but laughable.
Quote
I don't bother with --api because I don't care for the fixed 320kbps size. Even tweaked MPC xtreme files stay below 320kbps 99% of the time...
What do you care about file size? MPC isn't even available for portables. in which case file size would be a legitimate issue. Saving disk space should have no bearing on a discussion of which lossy codec has the highest quality output.

Earguy\'s improved Digital Ear

Reply #26 –
Quote
You ask about pre echo artifacts - has anyone EVER detected ANY artifacts on an alt insane LAME file???


Yes.  castanets.wav is an easy file to hear lame's pre-echo on using api.

ff123

Earguy\'s improved Digital Ear

Reply #27 –
Quote
Originally posted by layer3maniac
I don't know why everyone finds these results so shocking except as a result of the constant harping about MPC being the "best" encooder there is. You ask about pre echo artifacts - has anyone EVER detected ANY artifacts on an alt insane LAME file???

Yes.
Many people have.

Earguy\'s improved Digital Ear

Reply #29 –
Quote
Originally posted by layer3maniac
No, in fact it's not. MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec. And these results show it's not. Bitrate to bitrate matchups aren't the issue. The highest possible quality of output, at any reasonable bitrate is the issue. The fact of the matter is that in this test, with this tool, the top LAME preset beat the top MPC preset. Period.

Ah, but you missed one VERY big thing. The --api preset is the best possible setting for LAME. But --insane is NOT the best possible setting for MPC. Sure, it's the best preset, but we aren't talking about presets; we are talking about best possible quality at ANY conceivable setting. You can still tweak MPC further by using the --nmt and --tmn switches. You can't do that with LAME. 320kbps is the limit (to stay compliant with the MP3 spec).

Try the same test with --insane --nmt 16 --tmn 32 and let's see the results. LAME is maxed out with --api...you really can't improve it. However, MPC can use 1000kbps+ frames if you allow it: use --insane --nmt 99 --tmn 99. There's no way LAME has a chance against that.

So "MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec" is a true statement.
Quote
Originally posted by layer3maniac
And look at the midrange area in which api was clearly superior to MPC. That's in the audible range, even for those with less than stellar hearing. If you know of problem samples for api, name them. Don't just voice your assupmtions that there "must be some" as an argument that's anything but laughable.

If MPC is allegedly inferior in the midrange, then why is it typically ranked the highest in most blind encoder comparisons? Remember that VL is not the final word in human hearing perception. Also remember that EarGuy used MPC 1.7.x whereas most people here use 0.9x.

As for your samples, it's useless to dig them up. As long as a sole listener can hear them, then they exist. I could present a sample and you could say "I can't ABX it" and I could say "but I can." So does it exist or not? Yes, it does. You will always find somebody who can hear something wrong with some sample using some encoder. YOU may not find any artifacts with --api - nor, say, 95% of the users - but that doesn't mean it can be labelled artifact-free. If its lossy, it has artifacts. LAME, Ogg, MPC, whatever.
Quote
Originally posted by layer3maniac
Saving disk space should have no bearing on a discussion of which lossy codec has the highest quality output.

If you have two files that are of equally-perceived quality, then the file that is smaller is preferred. Look at the results. MPC performs better (smaller avg diff) than LAME at equal bitrates (interpolate the results). From the objective standpoint, MPC is the superior choice for this particular situation. What is so hard to understand about this?

Earguy\'s improved Digital Ear

Reply #30 –
This is an interesting thread, but could someone clarify a few points for a newbie first? TIA.

1. a silly question: isn't LAME api supposed to produce 320 kbps CBR files? Why does the graph say "323 k average bitrate"? Where did the 3 extra k come from?

2. EarGuy says VL works in the time domain, is it possible that it would therefore subject codecs working in the frequency domain (LAME and OGG?) to unfair disadvantages?

3. It sounds to me that VL is like another codec or psymodel, because it's also based on studies about what/how a human ear hear. If that's the case, the result would in large part hinge on how similar the evaluated target (LAME aps, e.g.) is to the VL's Ear Model. Let me put it another way, if today we have an encoder (Superlame) that is built on the same psymodel as the Ear Model, it would probably score the best, wouldn't it?

4. IMVHO, the effectiveness of VL as a general quality assessment tool, would in large part depend on how well the Ear Model in mimicking true human ears. It's not enough to simply say "treat it like just another ear", because if this "ear" is very different from an average human year, its value would be significantly diminished.

So my question is, how many of you have more in-depth knowledge about the dissertation (the Ear Model) VL is based on? Do we know about its reputation among academics? Has it been widely cited as a good model for human ear?


Sorry for all the questions. Please forgive me if I sound clueless. I'm after all from a social science backgroup, and I've been trying my best to understand these things.
tw101

Earguy\'s improved Digital Ear

Reply #31 –
Quote
Originally posted by ff123
It would also be nice to compare VL against the group listening tests, particularly dogies.wav and wayitis.wav.  If you are interested in doing this, Earguy, I can make the files available.
ff123

From your website I understand the two group listening tests you organized were comparing different encoders at 128 kbps, right? If that's the case, I wonder how much correlation their results have with VL's result. Isn't it a known fact that an encoder good at 128 k isn't necessary good at 200+k? Or am I getting it all wrong?
tw101

Earguy\'s improved Digital Ear

Reply #32 –
Quote
Yes.  castanets.wav is an easy file to hear lame's pre-echo on using api.
And where are your abx results posted? I searched the site and couldn't find them...

Earguy\'s improved Digital Ear

Reply #33 –
Quote
Originally posted by tangent
Yes.
Many people have.
Can you link me to the threads?

Earguy\'s improved Digital Ear

Reply #34 –
castanets is so easy to hear pre-echo with api that it's really not necessary to ABX

I did perform ABX before I wrote the following message on the Napigator forums, but I didn't document my scores (you'll have to trust me that they were at least 14/16)

Quote
On castanets.wav, I can't hear a difference between --alt-preset insane and -b320 -ms -h -k --lowpass 22.05.  But both are distinguishable from the original because of the transient smearing.  On the other hand, Fraunhofer's "Alternate" codec, as found in fastencc.exe -br 320 -hq, is indistinguishable to me from the original.

On fatboy.wav at 320 kbits/s, this Fraunhofer codec is the worst, and --alt preset insane is indistinguishable to me from the original.  Fewtch's command line is in the middle, with some audible artifacting.

On most music, I would think these kinds of differences are the finest of fine points.  I personally think 320 kbit/s mp3 files are overkill, so I stick with --alt-preset standard when I encode to mp3.

ff123

Edit:  I actually encoded the Fraunhofer with fastencc.exe.  I removed the reference to MMJB because at 320, I believe that at least one version of that program uses the fast codec (FastEnc), even when the "very high" setting is selected. 

Earguy\'s improved Digital Ear

Reply #35 –
Quote
From your website I understand the two group listening tests you organized were comparing different encoders at 128 kbps, right? If that's the case, I wonder how much correlation their results have with VL's result. Isn't it a known fact that an encoder good at 128 k isn't necessary good at 200+k? Or am I getting it all wrong?


I'm not trying to figure out which encoder VL thinks is best at 128 and correlate that with encoders at 200+.  I'm trying to figure out if VL hears like a real person.

ff123

Earguy\'s improved Digital Ear

Reply #36 –
Quote
I'm not trying to figure out which encoder VL thinks is best at 128 and correlate that with encoders at 200+.  I'm trying to figure out if VL hears like a real person.
But that is entirely subjective. For instance, I honestly can't hear the pre echo in castanets api. Yet I can undoubtedly abx files with missing or distorted high frequencies that you can't. Isn't our hearing human? Who's to say what artifact or missing sound is more or less acceptable to any given individual? By the way, castanets sounds crappy to me whether it's the original or encoded. Did they rip that off vinyl?

Earguy\'s improved Digital Ear

Reply #37 –
Quote
But that is entirely subjective. For instance, I honestly can't hear the pre echo in castanets api. Yet I can undoubtedly abx files with missing or distorted high frequencies that you can't. Isn't our hearing human? Who's to say what artifact or missing sound is more or less acceptable to any given individual?


Not entirely subjective, at least not in the way that I think you mean.  Of course I am not saying that any individual's preferences are wrong.

However, the listening tests showed that preferences can be grouped together in a statistically meaningful way.  Yes, individual preferences can and do vary a lot from person to person, but get a group of people together who are experienced with audio codecs and they can pretty much speak with one voice in saying that one codec sounds a lot better or a lot worse than another.

EAQUAL couldn't even say that Xing sounded much worse than the others on dogies.wav at 128!  Whereas it was patently obvious from the listening test results that it really was.

ff123

Earguy\'s improved Digital Ear

Reply #38 –
Quote
Originally posted by layer3maniac
I don't know why everyone finds these results so shocking except as a result of the constant harping about MPC being the "best" encooder there is.


Constant harping?  Eh.. not quite. It's one thing if you don't like a codec on principle, but another to disagree with the sentiments reached by a large portion of the community via blind listening tests and external verification, while offering nothing compelling to back up your challenge.

I consider MPC to be the best there is as far as lossy codecs because in my testing, and the testing of others, who have proven themselves multiple times, we continually find this to be the case.

So far I haven't seen sufficient evidence to refute this.  Instead of trying to prove the quality of some other codec as being superior to MPC I only see people trying to bring other points into the discussion such as the merit of open source.  This never has, and never will, have anything to do with the level of measured quality of the codec at a given point in time.  Just the same as whether you like a codec or not on principle has nothing to do with quality either.

Quote
You ask about pre echo artifacts - has anyone EVER detected ANY artifacts on an alt insane LAME file???


Yes, and I can verify this perhaps more than anyone else here.

Earguy\'s improved Digital Ear

Reply #39 –
Quote
Originally posted by layer3maniac
No, in fact it's not. MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec. And these results show it's not.
Another issue is, is the VL right. I can name many samples where --api sounds considerably worse than mpc --insane. Just few that come to my mind: castanets,death2, badvilbel, drone, florida_seq, them, etc.  These are just a short list that are very easy to ABX. Then there are numerous other samples which need more careful listening and show that --api is inferior to mpc --insane.
I'll just say that I personally disagree with these results, even though the old 1.7.9c mppenc is used.

Quote
Bitrate to bitrate matchups aren't the issue. The highest possible quality of output, at any reasonable bitrate is the issue. The fact of the matter is that in this test, with this tool, the top LAME preset beat the top MPC preset. Period. Cry all you want, it doesn't change the result.
--insane is not the highest possible quality of output by MPC. Although I claim it's considerably better than --api, I have long ago suggested to Andree that --insane produces pretty low bitrate.
Heh, this starts to remind the EAQUAL thread between Vorbis and MPC, I'm a bit worried. These results should not be treat as "the truth".
Quote
And look at the midrange area in which api was clearly superior to MPC. That's in the audible range, even for those with less than stellar hearing.
So there should be many samples that are non-transparent with mpc --insane and clearly be worse than --api. Can you name some of these samples, and a thread where people agree this is the case?
Quote
If you know of problem samples for api, name them. Don't just voice your assupmtions that there "must be some" as an argument that's anything but laughable.
I already named some. I think Dibrom, the main developer of --api will know even better..
Quote
What do you care about file size? MPC isn't even available for portables.
Ok, this starts to go totally off-topic again layer3maniac. I seriously hope we can avoid the EAQUAL "Ogg vs MPC" thread, layer3maniac...
Quote
in which case file size would be a legitimate issue. Saving disk space should have no bearing on a discussion of which lossy codec has the highest quality output
Again, mpc --insane is not the highest output that MPC can produce. Some people here use clearly higher settings. Even VL results with old mppenc encoder show that MPC is clearly superior to MP3 at similar average bitrate level. Also, I would say, that --api has still clearly inferior qulity compared to mpc --insane. There are severl samples that show this. There are very few samples which might sound better with --api, but the results are quite controversial.

EAQUAL didn't in practise hear pre-echo, we don't even know yet how good VL is regarding this, infact we have no proof yet that VL correlates with true listening results. I'm pretty sure VL is better than EAQUAL though, but still, I will trust my own and group listening test results much more.
Juha Laaksonheimo

Earguy\'s improved Digital Ear

Reply #40 –
Quote
Originally posted by layer3maniac
No, in fact it's not. MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec. And these results show it's not.


These results show that a theoretical model of a virtual ear, based on an imperfect psymodel (by principle), constructed from incomplete (by principle) information about human hearing, suggests that another psymodel may show more variation in a particular situation than another.

You are honestly telling me that you are going to base your whole argument on such an unsolid foundation?  This model hasn't even been proven to us to be effective at gauging quality at all.  How do you even know that it's not extremely buggy?  You don't, and you can't verify it, aside from community data, because you don't have the source either.  In fact, you know so little about this model that I don't possibly see how you can make the assumption that it is correct, "just because".  That's ridiculous.

Not only are you trusting something which is unproven, you are trusting it based on it's results in, what would be to us, entirely uncontrolled conditions.  We have no direct say over what files it encodes (aside from simple requests or suggestions), we have no idea what kind of problems may arise in those samples or anything like that.

Quote
Bitrate to bitrate matchups aren't the issue. The highest possible quality of output, at any reasonable bitrate is the issue. The fact of the matter is that in this test, with this tool, the top LAME preset beat the top MPC preset.


This is simply absurd.  So now you are trying to imply that one codec is inferior to another because of the top end of a preset?  Heh... ok.  I hope you do realize that it's an extraordinarily simple task to add a few switches to MPC's presets (far fewer than LAME I might add) which would eliminate this point entirely.

Quote
And look at the midrange area in which api was clearly superior to MPC. That's in the audible range, even for those with less than stellar hearing.


Let me ask you something.  Have you ever listened to both of these presets in detail?  I suspect not judging from your responses here.  I have, however.  I designed the --alt-preset insane mode also, and I can assure you that it does not compare to MPC -insane.

You imply that MPC must now be producing audible flaws according to what the graphs here show, but you are also completely disregarding all other known and proven data on this matter, instead promoting entirely unproven and unsubstantiated (at this point) data.

You seem to be forgetting here that MPC has had the advantage of having been tuned by some of the best ears in this community over a period of many months.  Has this ear model had the same luxury?  I don't think so.  This simple point alone should show that just because this model says there may be deviation, it might not be the case.  It may not be (and probably isn't) tuned to the same threshold as MPC is.  Furthermore, if you don't like that example you can use any other codec in it's place.  Improvements to the psymodels of both LAME and Vorbis have been fairly incredible over the last year or so, as a result of hand tuning.  Many of these codecs begin with psymodels based only on results from research papers, much as I assume this one has, yet they have found incredible improvements from community validation.

Quote
If you know of problem samples for api, name them. Don't just voice your assupmtions that there "must be some" as an argument that's anything but laughable.


death2
drone_clip
castanets
blips

there's many more if you really dig around.

--alt-preset insane suffers from pre-echo as a result of an insufficiency in the MP3 format.  This will never change as long as LAME continues to produce ISO compatible output.

Earguy\'s improved Digital Ear

Reply #41 –
Quote
Originally posted by tw101
3. It sounds to me that VL is like another codec or psymodel, because it's also based on studies about what/how a human ear hear. If that's the case, the result would in large part hinge on how similar the evaluated target (LAME aps, e.g.) is to the VL's Ear Model.

I think the difference between "traditional" psychoacoustic model and Virtual Listener is, that VL is more psycho-physiological model. VL tries more directly simulate how an ear physiologically works. Lame's psycoacoustic model just uses known data about characteristics of hearing, but it does not simulate actual human ear or physiological hearing process inside human ear like VL does.

According to Ivan, Baumgarte's ear model like Virtual Listener, is very complicated and it is mostly based on neural response simulations and is therefore very slow.
Juha Laaksonheimo

Earguy\'s improved Digital Ear

Reply #42 –
Quote
Originally posted by layer3maniac
No, in fact it's not. MPC is touted as being the best possible encoder for those seeking the highest quality output from a lossy codec.
And these results show it's not.


Hold on a minute here. You're missing something fundamental, and I'm surprised noone else corrected you on this: Who says that the 'difference' values this thing outputs are hearable?

To the best of our knowledge, --alt-preset standard does not artifact audibly on most samples. The data you are looking at is the average over 100 of those. Some of those may artifact, but the average won't.

At that point, the actual difference values are irrelevant; the point of a lossy codec is to artifact up to the point where it just isn't audible. The better it is in that, the better the codec. Inaudible improvements are useless and even bad if space is an issue (and it always is).

--
GCP

Earguy\'s improved Digital Ear

Reply #43 –
I know it seems to many of you that I am "out to get" MPC for some reason or other. In reality, I only have one personal objection to MPC. It's not the issue of open source, or that it's not going to have hardware support anytime in the near future, or that I have some sort of "personal problem" with Andree and just want to see it fail. Quite the contrary. I think Andree made a great encoder. Open source projects can be a royal pain as evidenced by the GPL issues raised to john33 and the friction which developed between Dibrom and some of the LAME people. And hardware support is not really a big issue for me personally either.

What really bothers me about MPC is this: the attitudes of people who, after seeing some digital ear test results, abx results, or even unsubstantiated individual listener feedback (how dare they), say something like (paraphrasing here) "these results MUST be flawed because everyone knows that MPC is the highest quality codec." That attitude disturbs me deeply. It reminds me of the early days of "cd quality" r3mix.

Earguy\'s improved Digital Ear

Reply #44 –
Quote
Originally posted by layer3maniac
What really bothers me about MPC is this: the attitudes of people who, after seeing some digital ear test results, abx results, or even unsubstantiated individual listener feedback (how dare they), say something like (paraphrasing here) "these results MUST be flawed because everyone knows that MPC is the highest quality codec." That attitude disturbs me deeply. It reminds me of the early days of "cd quality" r3mix.
Well, firstable VL results clearly generally show that MPC is better than Lame. But even that is not enough to prove that VL really functions properly.
VL is not yet tested against group listening test results or tested how it detects one of the biggest problem in lossy encoding - pre-echo.
I really don't see any correlation to r3mix "cd quality". I think one of the main issues hydogenaudio.org community tries to offer and demand is proper proofs. It goes like this: proper describtion of problem, ABX-results, offering of test samples for general testing and verification.

It's good to question things, but ask yourself, considering all the facts and what you know about the reliability of VL, is your method the right one? I really can't understand how can you be so non-sceptical towards artifical quality measurement tools even before there are any proof whether those work even half decently or not??? Or is the true reason, that you really don't hear much problems with lossy encodings in general thus you blindly rely on any artificial methods availalbe, no matter if it isn't proven to give even half decent results.
Juha Laaksonheimo

Earguy\'s improved Digital Ear

Reply #45 –
Quote
I really don't see any correlation to r3mix "cd quality". I think one of the main issues hydogenaudio.org community tries to offer and demand is proper proofs. It goes like this: proper describtion of problem, ABX-results, offering of test samples for general testing and verification... It's good to question things, but ask yourself, considering all the facts and what you know about the reliability of VL, is your method the right one?
I have no method. I know virtually nothing about VL, EAQUAL, or Joe Blow in New Jersey's abx results. But I do know a bandwagon when I see one. That's the correlation to r3mix. An opinion, by it's very nature, can not be proven. Blue is my favorite color and I can prove it!  Just look at all my pants! Lossy codecs and presets have come to remind me of automobiles. What is the highest quality automobile? What is the lowest? If someone is happy driving a Yugo, who am I to tell them that they ought to be driving a Lexus? I know people who are very happy using Blade, WMA, and Xing. Why should that bother me? A studio engineer like Beatles may prefer AAC or MPC to FhG or OGG. So what? He's not the one listening to my music. I am...

Earguy\'s improved Digital Ear

Reply #46 –
Quote
Originally posted by layer3maniac
I have no method. I know virtually nothing about VL, EAQUAL, or Joe Blow in New Jersey's abx results. But I do know a bandwagon when I see one. That's the correlation to r3mix.
There's one or few fundamendal differences: proofs (proper testing done by several different people) and the source of the proofs (available test samples).
Juha Laaksonheimo

Earguy\'s improved Digital Ear

Reply #47 –
Quote
I have no method. I know virtually nothing about VL, EAQUAL, or Joe Blow in New Jersey's abx results. But I do know a bandwagon when I see one. That's the correlation to r3mix. An opinion, by it's very nature, can not be proven. Blue is my favorite color and I can prove it!  Just look at all my pants! Lossy codecs and presets have come to remind me of automobiles. What is the highest quality automobile? What is the lowest? If someone is happy driving a Yugo, who am I to tell them that they ought to be driving a Lexus? I know people who are very happy using Blade, WMA, and Xing. Why should that bother me? A studio engineer like Beatles may prefer AAC or MPC to FhG or OGG. So what? He's not the one listening to my music. I am...


Did you completely miss my last post?  Lossy audio quality is not just completely a matter of individual preference.  Preferences vary from person to person, but they can also be grouped.  So if you've got a whole bunch of people with normal hearing and good equipment who know what artifacts sound like saying Blade and Xing sucks, guess what -- they're probably right in a general sense.

The fact that you think there might be some "group think" going on regarding MPC does not justify a non-skeptical position on EAQUAL or VL!  Judgment should be reserved until it can prove itself to at least some degree.

ff123

Earguy\'s improved Digital Ear

Reply #48 –
Anybody knows if the ear has more 'time resolution' or more 'frequency resolution'?

Earguy\'s improved Digital Ear

Reply #49 –
Quote
So if you've got a whole bunch of people with normal hearing and good equipment who know what artifacts sound like saying Blade and Xing sucks, guess what -- they're probably right in a general sense.
A hundred engineers and a thousand mechanics can agree that Yugos suck. It doesn't matter to my girlfriend - she LOVES hers...
Quote
The fact that you think there might be some "group think" going on regarding MPC does not justify a non-skeptical position on EAQUAL or VL!  Judgment should be reserved until it can prove itself to at least some degree.
I agree whole-heartedly. Did I leave you with the impression that I don't approve of skepticism? Skepticism is a "way of life" for someone like me.