Skip to main content ↑ ↓

Notice

If you are using a Hotmail or Outlook email address, please change it now, as Microsoft is rejecting all email from our service outright.
Topic: graph shows 320 kbps LAME 'never achieves transparency'? (Read 30525 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #25 –
Clearly this is someone who believes in the validity of whatever supports his own personal beliefs. If this graph had been drawn a little differently he would have argued vigorously how worthless it was, probably using many of the same points that have been made here.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #26 –
Quote from: LithosZA on
I think by full 'Quality' on the graph they mean that it will never be bit exact quality from the source file. You never will be guaranteed that it will be lossless even if you pump up the bitrate to 320Kbit/s.
Transparent might be the wrong word. Transparent means that is 'sounds' transparent to the user.
Even though 192Kbit/s or even 320Kbit/s Vorbis sounds 'transparent' to me doesn't mean that is is full 'Quality'.

The way you put it, it's wrong. Quality is always a subjective term that can or can not be determined by objective parameters. Only if such a determination has been established, one can talk of "quality" as an objective term. Now if you want to compare a signals perceived quality to that a reference signal, you can safely assume that, if the signal is identical to the reference signal, the quality is also potentially identical (given that you listen to it exactly the same way). Thus lossless encoding can be said to have a "quality" of 100%, or in different words, to be transparent. Now if we leave away parts of the original signal, its quality potentially degrades, but we can't say exactly in which way, because of the unknown (infinitely complex) relation between our perception and the data-reducing (psycho-acoustical) algorithmns involved.

So we have to test empirically. That's how those algorithms are established and that's how we compare their outcome. We can do an ABX test if we want to check for transparency, or simply an AB test if we want to compare for quality. If such a test has been properly conducted and the number of trials and individual participants is large enough, we may draw relations like those (improperly) shown on this graph. Such a graph could very well show a quality of 100% (or 10 in this case) for a lossy encode. This relation however, is as solid as is the emprical data behind it. Strictly speaking, it's never possible to empirically prove that it would be transparent for every individual under any circumstance, but if our data is good enough we can assume it with pretty much the same confidence we have in the sun to rise again tomorrow.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #27 –
Quote from: Notat on
What is the definition of "transparent"?

Assuming you do not intend this question to be rhetorical, I would suggest a definition of "demonstrable over the course of multiple double-blind listening tests to be indistinguishable from the uncompressed (or losslessly-compressed) source with a statistical signifcance of p<0.05".

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #28 –
Perhaps I'm completely ignorant about p values, but I thought they are used to demonstrate the probability that someone has had a successful outcome without guessing.  It is also my understanding that when someone cannot distinguish a difference and is left to guessing, the p value increases approaching 1.  Based on this, saying someone is guessing with p<0.05 doesn't make any sense.

Can someone enlighten me?

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #29 –
Yep, the p-value is the probability that the observed result could otherwise have been obtained by chance. A result over the chosen significance level (usually 0.05) leads one to (not accept, but) fail to reject the null hypothesis, which usually proposes no effect of whichever treatment is being investigated.

I too am not sure how one can demonstrate that there is not an audible difference. One cannot confirm the null hypothesis (i.e. prove absence of an effect), only fail to reject it. Thus, transparency could be argued if multiple users failed to achieve statistically significant p-values, but not the converse.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #30 –
Quote from: Zarggg on
Quote from: Notat on
What is the definition of "transparent"?

Assuming you do not intend this question to be rhetorical, I would suggest a definition of "demonstrable over the course of multiple double-blind listening tests to be indistinguishable from the uncompressed (or losslessly-compressed) source with a statistical signifcance of p<0.05".

It was not rhetorical. I have read the HA wiki and Wikipedia. Neither your proposed definition nor either of these articles tells me precisely what listener we're talking about. The HA article suggests it is fair to use killer samples to demonstrate non-transparency. The graph we're discussing reportedly used conventional material. The lack of error bars are a problem but, the graph does appear to make a case for a lack of "transparency" according the definitions I've read.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #31 –
Quote from: Notat on
I have read the HA wiki and Wikipedia. Neither your [Zarggg] proposed definition nor either of these articles tells me precisely what listener we're talking about.

Those discussing transparency usually either refer to the majority (a problematic concept in itself as it usually refers to fairly tech-savvy listeners, but we'll have to make do until Big Brother mandates listening tests) or to the specific listener being addressed in a given discussion. Since you seem to have been referring specifically to this by krabapple . . .
Quote from: krabapple on
[. . .] Apparently he thinks the graph shows mp3s can never be transparent.  You see the artificial gulf there? [. . .] [This] hinges on the strictest definition of 'transparent', and puts a burden on that graph that it wasn't meant to bear.  Misses the forest for the trees.
. . . I will agree that it seem to be implying some objective definition of transparency, which is obviously a non-starter, as is made plain on the Knowledgebase page:
Quote
Transparency, like sound quality, is subjective.

But I do not understand your question about the listener(s), and krabapple is correct to point out the nonsensicality of amirm saying that MP3 can never be transparent. It remains to be answered whether the latter is simply questioning the graph, which has already been discredited; or is lambasting the format overall as being insufficient for every possible use-case, in which event he is blatantly incorrect.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #32 –
I think the wiki article on "transparent" is pretty good. With all those caveats, something is transparent for you if you can't hear the difference (in a double blind test - but that goes without saying  ).

But more generally, the things we accept as transparent are things where reports of hearing a difference in a double-blind test are rare and uncorroborated.


I think far longer has now been spent discussing that graph than went into making it. It would probably have been better if the quality scale had no units. I'm not aware of any reliable units of subjective audio quality which have a globally understood and repeatable quantity in the same way as the units on a ruler.

Cheers,
David.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #33 –
Quote from: 2Bdecided on
I think far longer has now been spent discussing that graph than went into making it. It would probably have been better if the quality scale had no units. I'm not aware of any reliable units of subjective audio quality which have a globally understood and repeatable quantity in the same way as the units on a ruler.

That strikes me as throwing the baby out with the bathwater. The scale on a MUSHRA test is calibrated by its anchors. We're always going to have uncertainty in these results so the ruler is not a good model.

The original question here even before the graph entered the conversation was, "Is 320 kb LAME transparent?" So long as we can agree on a technical definition of "transparent" it seems like a simple experiment to conduct. The graph indicates that it has been attempted and the unverified results indicate that it is not transparent.

But, I'm getting the sense that there is no technical definition for transparent; That it's a "subjective" thing with context-dependent meaning. The HA article, for instance, describes how to demonstrate non-transparency then lists some transparent audio formats that I'm pretty sure have been shown to be non-transparent by these guidelines at least with specific program material and ears.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #34 –
Quote from: Notat on
The HA article, for instance, describes how to demonstrate non-transparency then lists some transparent audio formats that I'm pretty sure have been shown to be non-transparent by these guidelines at least with specific program material and ears.
It lists formats and bitrates at which they may be artifact free.... It does not state that they are transparent at those bitrates for all material.

A point about the article - the statement "use double-blind tests where the high anchor is the original uncompressed audio" infers that compressed audio should not be used. Why would the original audio compressed in FLAC be unsuitable? Surely either "uncompressed" should be removed or changed to "lossless" or "unprocessed" or something like that.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #35 –
Quote from: Notat on
"Is 320 kb LAME transparent?" So long as we can agree on a technical definition of "transparent" it seems like a simple experiment to conduct.


There are (just need to search hydrogenaudio) killer samples that can be quite easily ABXed between MP3 at 320kbps and original.

One could say that this is a proof that MP3 is not transparent at 320kbps.

Yet, that would be a pretty bad definition of transparent, especially since we are talking about lossy codecs, and I explain myself:


If I get a single file which is encoded with a lossless codec, and the audio portion is not bit-for-bit identical to the original, I can say that the encoder is buggy/is not working as expected/its quality is not 100%.

If I get a single file which is encoded with a lossy codec at its highest setting, and I am able to ABX it against the original, I can only say that the file is a killer sample for this encoder and version, which might or might not be a killer sample for the format itself, and I will have to test if other lossy codecs do a better job or also fail on it.

I cannot imply that the format or encoder is not transparent. I can only say that there is a sample which is not transparent, but transparency with a lossy codec implies what happens in the general case.

Said it in other words: there are many files where one would fail to ABX an MP3 at 320kbps versus its original. Since this number is empirically bigger than the cases where it can be ABXed, this setting is considered transparent for the general case.


At last, there is no way to demonstrate transparency, other than demonstrating that the file is bit-for-bit identical (In which case, the transparency is a logical consequence of the files being the same, not the opposite).
As said above, one can only demonstrate a difference using an ABX test. A non-difference cannot be demonstrated with this method.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #36 –
<sigh>

A link to the discussion was given regarding the origin of that graph.  Who bothered to read it?

Please allow me to repost (bold type added by me):
Quote from: Gabriel on
What about a perceived quality vs size graph?

Arbitrarily, I'd give the following quality levels:

--abr 56: 3
--abr 90: 5
-V5: 7
-V4: 8
-V3:8.5
-V2: 8.7
-V0: 9.1
-b 320: 9.2

This is purely informal, but if you trace a graph of perceived quality vs average size, you will probably obtain a nice curve with valuable indication regarding efficiency of the settings.

As for the -V1 datapoint:
Quote from: Synthetic Soul on
I added -V1 in there, to get the unsightly kink out of the first graph (qualily value is simply the average between -V2 and -V0).

IOW, these numbers are not based on scientific testing.  That we need to somehow go down the rabbit hole justifying the unjustifiable by attempting to (re?)define the word transparent is deserving of ridicule.

Throwing out the baby with the bath water?  There is no baby to throw.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #37 –
In regards to the OP, what is the motive for slamming LAME anyway? I didn't read this "expert's" thread because hearing FUD about a codec gets... lame, pun intended I guess. Is he/she pimping another MP3 encoder, or another lossy format? I don't get the motivation for using the graph as evidence when it is clearly a loose (a very, very, loose) representation of bitrate-to-quality tradeoff.

I really don't see a vast number of HA users- the ones that encode to MP3- using LAME because of its known inferior design of no-transparency, duh

As other users already suggested, he/she should do some ABX, or better yet participate in one of the public listening tests and just see how much (or little) a contribution they can give.
"Something bothering you, Mister Spock?"

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #38 –
Sorry about reader fatigue and sloppy research.

I find this topic interesting because the religion here at HA is that modern first-generation lossy encoding sounds the same as the original. "Transparent" is commonly used to describe this feature. And yet we have many audio lovers (including HA members) who insist on using lossless formats. And why is that? Because lossless gives them peace of mind? Because they think lossless sounds better? Or is it because perceptual coding, no matter the sophistication or bitrate, potentially introduces significant and audible degradation?

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #39 –
I think the main reason that many of us here at HA use lossless encoding is to avoid generational degradation. If I thought that I would never need to reencode to a different format EVER, then I might be perfectly happy with a high quality lossy encode for my collection.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #40 –
Ignore killer samples and golden ears, and instead think of transparency of referring to general usage and general users. Does the problem now magically evaporate?

Transparency is subjective in the sense that it is limited to certain user(s) by definition, but this is a much better kind of subjective than the things ToS8 exists to avoid. Is there any point in quibbling over semantics and lamenting the fact that one can never objectively describe the quality of representation provided by a lossy codec?

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #41 –
Quote from: Notat on
And yet we have many audio lovers (including HA members) who insist on using lossless formats. And why is that? Because lossless gives them peace of mind? Because they think lossless sounds better? Or is it because perceptual coding, no matter the sophistication or bitrate, potentially introduces significant and audible degradation?
Quote from: pdq on
I think the main reason that many of us here at HA use lossless encoding is to avoid generational degradation.

When dealing with master tracks lossy is a non-option. In that last regard MP3 could be considered ephemeral and for casual listening but never for archival.

Not to throw the thread out of orbit, I think there might a benefit to defining terms for "transparency" in regards to the degree of "lossy quality." Of course, users' ears will differ and just as much as users' music genres/preferences will differ: i.e. classical vs. pop rock will have different average bitrates at -V2. Artifacts are likely to surface in lossy but how annoying they are and what the listener considers annoying (pre-echo vs. warbling) is also another area of opinion.
"Something bothering you, Mister Spock?"

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #42 –
I have gone ahead and updated the caption for the graph in the wiki, and linked to the relevant discussions.

Old caption: Here a trial to see how the perceived listening quality improves with settings/averaged filesize

New caption: This informal graph shows how LAME's highest quality settings result in progressively larger file sizes, but yield relatively smaller gains in perceived listening quality.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #43 –
Quote from: Destroid on
I think there might a benefit to defining terms for "transparency" in regards to the degree of "lossy quality."

For any particular individual at any given time, something is either transparent or it isn't.  Within these constraints there are no shades of gray.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #44 –
Quote from: mjb2006 on
I have gone ahead and updated the caption for the graph in the wiki, and linked to the relevant discussions.

Do away with the scale and data points and I might buy into the idea of there being a graph, otherwise dump it altogether.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #45 –
"Transparent" is a word, and words have different kinds of usage.

I take it that, in a technically defined sense, "transparent" means that in a particular (series of) test(s), particular listener(s) were unable to tell the difference between specific lossily-coded sample(s) and their original(s). Even in this defined sense, transparency can't be proved, since individual listeners can have good days and bad days.

As a word in general conversation, it seems to often be used in a way that relates to, but is not the same as, the technical sense. So in answer to a question like, "At what setting does LAME get to be transparent?" (which in some contexts would be a sensible enough practical question) you might say "V 3," meaning "V 3 for most people with most material most of the time." On HA people are careful to include the qualifiers, being conscious of the technical sense; in a non-technical setting, in which there's a danger of good advice being disregarded because of the MEGO factor.

IME, we are all likely to slip into the trap of treating ordinary-language words as though they were explicitly defined technical terms.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #46 –
I don't ever recall there being a problem with the meaning of the word as it relates to how it is used on this forum.

transparent = perceptually indistinguishable

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #47 –
Quote from: MichaelW on
I take it that, in a technically defined sense, "transparent" means that in a particular (series of) test(s), particular listener(s) were unable to tell the difference between specific lossily-coded sample(s) and their original(s). Even in this defined sense, transparency can't be proved, since individual listeners can have good days and bad days.

I would be tempted to read a corollary from this: You will always be able to find someone, somewhere, somehow who will experience perceptible degradation with any lossy coding system.

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #48 –
So transparency depends on the listener's abilities including listening conditions and the sample(s) he listens to. It's personal.

A practical definition of a codec's transparency could mean that there was no listener so far who was able to successfully ABX a sample. Sure the quality of this kind of transparency depends heavily on the number of listeners (and their ABXing abilities) trying hard to find non-transparent (in the personal sense above) samples.

As for the graph simple words about the diminishing returns fact would be more honest, but people like graphics. It's a good exercise never to simply beleive in graphics. People can be easily fooled by graphics (for instance by using a suggestive scale) even when the graphics is formally correct.
In our case it's clear that the quality scale can't have a real meaning and is chosen arbitrarily at the author's will to demonstrate the diminishing returns phenomenon.
lame3995o -Q1.7
opus --bitrate 140

graph shows 320 kbps LAME 'never achieves transparency'?

Reply #49 –
Quote from: greynol on
I don't ever recall there being a problem with the meaning of the word as it relates to how it is used on this forum.

transparent = perceptually indistinguishable

I didn't say there was a problem here; in fact, if you were actually to read things, you would see I explicitly made an exception for here. There is a problem elsewhere.

BTW, perceptually indistinguishable to whom, under what circumstances, always, sometimes?
 
SimplePortal 1.0.0 RC1 © 2008-2021