I was wondering if the mp3 audio codec does leave behind artifacts within the file its encoded in?I am asking because i saw this video on youtube and want to know if this is true?
link> http://www.youtube.com/watch?v=u5gdwpPrv_8...feature=related (http://www.youtube.com/watch?v=u5gdwpPrv_8&feature=related)
His assertion is nonsense. He is assuming that the difference is artifacts added by the mp3 encoder, whereas in reality it's the parts of the signal discarded by the mp3 encoder, plus any artifacts added by the mp3 encoder. He has no way of telling which is which. Now, we know that the mp3 encoding process discards sounds that your ear can't hear because they are masked by other, more important parts of the signal. That's one of the ways the mp3 encoder gets the files down to such a small size, so it's perfectly normal and expected for there to be a difference between the original wav and the encoded mp3.
Further, you would expect to discard more when the overall signal is louder and the sound busier, because there will be more sound there to do the masking, hence more sound able to be masked and discarded, hence the difference will be greater - which is exactly what his experiment shows.
If you subtract them you just get whatever was lost when you encoded the MP3. Essentially that guy has proved to himself that MP3 isn't a lossless format like WAV. Why he felt the need to tell youtube, well, thats anyones guess
Subtraction gives you the sample-by-sample mathematical difference, but it does NOT give you the SOUND difference! MP3 does not retain the original timing & phase information, so subtracting the samples doesn't tell you anything about the sound. It does not tell you what was added or removed from the sound during encoding.
The experiment does prove that MP3 is lossy. That is, it proves that the bytes in the file have changed.
I'll give a couple of examples that you can "try at home" with your audio editor...
Starting with two identical WAV files for each of the following experiments -
- Just as a starting point, subtract the two files. You'll get silence proving that the files are identical, and proving that no mathematical difference means no sound/audio difference.
- Invert one file. Listen to both files and compare the sound (most people won't hear a difference). Now, subtract these files (or invert again & add/mix). The result is truly the mathematical difference, but since you've "subtracted a negative", you've mathematically added the two identical files and doubled the volume. You are hearing the mathematical difference between the two files, but you're not hearing the difference in the sound.
- Take one of the identical files and add 10 milliseconds of silence to the beginning. Subtract and listen. Again, what you're hearing does NOT represent sound difference between the files, which will sound identical.
- Take one of the files an speed it up by 1/2% (I can't hear a half-percent tempo/pitch change). Subtract the two files and listen. Again the mathematical difference doesn't represent the true the sound-difference.
Note that in every case above, no artifacts are added and no sound is removed from the 2nd file before subtracting the two files, yet in every case there is a clearly audible matematical difference when you subtract the two files.
Now if you can just squeeze all that into a little comment that will fit on that YouTube video
P.S. A couple more silly subtraction experiments -
- Record something twice. (Yourself or someone else reading something, singing a song, playing a song on an instrument, etc.) Subtract the two recordings and listen to the "difference".
- Subtract two completely different files (two different songs, or one song and a recording of someone speaking, etc.) and listen to the "difference".
If you subtract them you just get whatever was lost when you encoded the MP3. Essentially that guy has proved to himself that MP3 isn't a lossless format like WAV. Why he felt the need to tell youtube, well, thats anyones guess
Lots of people have gotten hung up on this factoid. One of them was the editor of a well known high end audio magazine. I had a chance to comment on the issue before publication and tried to disabuse him of the error, but he went ahead and published it anyway.
I recently subtracted a wav file that was made by decoding a MP3 from the .wav file that the MP3 was made from, and found that the difference file looked and sounded a lot like either wave file. This sugggests to me that a lot of the difference was due to latency (time delay) in the encode/decode cycle.
OK. I'm bumping this topic because in my attempt at reviving the subject in another thread, my post was binned. But it's all good, no hard feelings.To start off, it is my belief that
there is no actual flaw in this experiment. Signal theory is largely based on transform theory, and this is where i will find most of my arguments. I take it that some of you already know all this, but i would still like to iterate a few points that i feel are relevant to this subject. Just bear with me.
For more info on signals and systems, DSP, filtering, general theory, i recommend any book by Chi-Tsong Chen on the subject.
http://www.ece.sunysb.edu/index.php?option...8&Itemid=26 (http://www.ece.sunysb.edu/index.php?option=com_content&task=view&id=18&Itemid=26)
--
The very foundation of signal theory lies upon the presupposition that signals can be added and subtracted together. For example, periodic signals can be constructed by means of a Fourier series: a sum of discrete sinusoidal signals, each with their own phase and amplitude. (See: http://www.dspguide.com/ch13/4.htm) (http://www.dspguide.com/ch13/4.htm))
Real world signals are rarely periodic, no worries because we can extrapolate this theorem by proposing that a non-periodic can actually be viewed as a periodic signal of infinite period (or null period), this leads us to the Fourier series in continuous form: the Fourier Transform. I would like to review basic but key properties of the Fourier transform (FT): Linearity and thus, inherently, additivity. Let f(a+b) = f(a) + f(b); Hence, the FT can be applied
before or
after the addition of two signals and yields the same output signal. (For a more detailed explanation, visuals, see: http://www.dspguide.com/ch10/1.htm) (http://www.dspguide.com/ch10/1.htm))
The FT is also reversible, which indicates that both the time (amplitude) and frequency (frequency, phase) domains exist conjointly. It follows that any operation on one domain or the other affects its counterpart simultaneously. Encoding of the MP3 relies largely on operations applied in the frequency domain, ie. band filtering, noise shaping, etc. but these operations have the direct effect of altering the signal (time domain) by adding '
encoding noise', if you will. Adding, here, has its literal meaning-- addition (of amplitude values). This addition can be reversed by subtracting the encoding noise from the encoded signal, from there recovering the initial signal, unaltered.
(This is exactly how residual coding works. In lossless compression, the encoding noise, or prediction noise, rather, is deflated and kept, instead of being discarded like lossy compression does. Whether you add the amplitude values digitally or in the analog stage (playback), the result is the same.)
Example: You could try an experiment where you digitally add noise to a mono track, and compare it (at playback) with a stereo track with noise only in the right channel + clean signal in the left (the speakers would need to be close together and/or the listener far away to eliminate spacial distance between channels, or more simply you could render the sound through a mono device without stereo input, its all the same.) The output is your noisy track.
This is usually a very intuitive concept, so let me illustrate it with another
example: Two people talking simultaneously, recorded in mono. At the same time, record a stereo as control with each voice in individual channels. How do you recover a single voice from the mono recording? You simply
subtract the amplitude values of the other voice (single stereo channel) from the mono recording. In the same manner, you can losslessly downmix 5.1 Dolby digital to stereo by adding amplitude values of all the left and right channels together, the sub and center channels are equally spread to both left and right.
Therefore... you can determine exactly what encoding noise is applied to an mp3 by subtracting it from the original source. You can subsequently listen to it,
just as if it were another human voice (above example). At that point you may judge if the signal has a significant effect on your hearing and enjoyment. It is possible that you would not even hear it at all, conveying good lossy compression, poor hearing, a bad sound system, etc.
--
Finally, addressing some of your concerns:
- Just as a starting point, subtract the two files. You'll get silence proving that the files are identical, and proving that no mathematical difference means no sound/audio difference.
That is correct.
- Invert one file. Listen to both files and compare the sound (most people won't hear a difference). Now, subtract these files (or invert again & add/mix). The result is truly the mathematical difference, but since you've "subtracted a negative", you've mathematically added the two identical files and doubled the volume. You are hearing the mathematical difference between the two files, but you're not hearing the difference in the sound.
Like you said, inverting a signal, or negating all its samples, has the effect of adding the signal to itself when subtracting it to original signal ie. a - (-a) = a + a = 2a; Amplification by a gain of 2. In the same fashion, two recordings playing the exact same waveform exactly at the same time will create amplification by a factor of 2.
Shifting the phase of a signal is an operation intuitively done in the frequency domain, but this directly alters the signal by inverting it (if shifted by pi rad). Although it may still sound the same, in this instance, adding them both together results in amplification due to the nature of the shift operation.
- Take one of the identical files and add 10 milliseconds of silence to the beginning. Subtract and listen. Again, what you're hearing does NOT represent sound difference between the files, which will sound identical.
The signals may sound identical disregarding the time shift, but they are not the same signal. So adding them would create a similar effect similarly as delaying one channel in a stereo system.
- Take one of the files an speed it up by 1/2% (I can't hear a half-percent tempo/pitch change). Subtract the two files and listen. Again the mathematical difference doesn't represent the true the sound-difference.
Speeding up a signal is equivalent to modulation (frequency shift). Although it may sound like the original, a slight modulation, again, alters the signal. It must be demodulated before any operation can be applied.
- Record something twice. (Yourself or someone else reading something, singing a song, playing a song on an instrument, etc.)
Subtract the two recordings and listen to the "difference". laugh.gif
It would be difficult to record exactly the same thing twice. The signals are different.
- Subtract two completely different files (two different songs, or one song and a recording of someone speaking, etc.) and listen to the "difference".
This an interesting example because you present it the other way around. Subtracting two signals that have no relation would have the same effect as having one signal traveling through air in the opposite fashion as sound does, ie. compression of air becomes dilation and vice versa. This has no common sense and is difficult to imagine. Nonetheless you could probably still hear the non-inverted signal through the distortion.
If you want an output that makes sense you could trying adding them together instead of subtracting them. This would result in hearing both recordings at the same time.
Subtraction is the opposite of addition, but you need to do it right. In our initial case we subtract to calculate encoding noise, but we need to add this noise in order to retrieve the original signal. You can do it the other way around, you anticipate what kind of noise your encoder makes, then compute and subtract it; This way the noise will be the inverted with respect to the first case. That being said, subtracting for fun makes no sense in your example.
--
Note that all these situations you enumerate one after the other are mostly examples of slight modifications that might not necessarily manifest themselves in audition, however they fundamentally modify the signal. Usually, a slight offset error is more irritating and distraction than a larger one, eg. two slightly off guitar strings will produce high rate pulsation while a larger interval might produce harmony; A realistic humanoid might seem creepy while a robot like ASIMO (Honda) looks friendly.
Please revise your analogies and comparisons, maybe signal theory, or your own intuition, before rejecting my arguments or deleting my post.
Therefore... you can determine exactly what encoding noise is applied to an mp3 by subtracting it from the original source.
Correct, you can do this.
You can subsequently listen to it,
Correct.
At that point you may judge if the signal has a significant effect on your hearing and enjoyment. It is possible that you would not even hear it at all, conveying good lossy compression, poor hearing, a bad sound system, etc.
Nonsense. Listening to what's been discarded tells you nothing of any value. The whole design goal of the psychoacoustic model associated with lossy music compressors is to work out what can safely be discarded with no perceptual impact on the listener.
Here's an analogy for you. Cook a meat sauce (like a bolognese sauce) for a family of four, with 2 teaspoons of salt, then cook an identical one but with only 1.5 teaspoons of salt. Taste what's been left out (the half teaspoon of salt), I guarantee that it will be very noticeable and not very pleasant to taste. However, do a double blind tasting of the two sauces and you're unlikely to be able to taste the difference - tasting the difference has told you nothing about the difference between the two versions of the sauce. It's the same with lossy encoding - the only valid test is to compare the uncompressed file with the compressed one in a double blind test.
OK. I'm bumping this topic because in my attempt at reviving the subject in another thread, my post was binned. But it's all good, no hard feelings....
Truncated
While I'm inclined to agree with your post, theres a small factor that DVDdoug said:
MP3 does not retain the original timing & phase information, so subtracting the samples doesn't tell you anything about the sound.
If this is true, and the timing is different in an MP3 encoding from it's original, then the subtraction method used in the video doesn't truely show what was removed from the encoding process.
I tried to google what doug was talking about, but couldn't find any information, so I'm going to create a scenario. This is just hypothetical because I don't have a major in psychoacoustics and don't know exactly where the information that can be safely removed is, this is just my best guess at a scenario.
If you have a tom tom that is hit at exactly the same time as a snare at a similar volume, assume that there is information that can be removed from the waveform without a perceptible loss. If the information is removed, all is well and the listener (hopefully) hears no difference. But what if the tom tom was hit 10ms before the snare? An encoder would have to produce the early attack of both drums separately, it is possible that the encoder would benefit (as in smaller file size) from moving the attack of the tom tom up to match with the attack of the snare. In this hypothetical case changing the timing information would serve the file size while not appreciably altering the sound.
If (yes, another IF) the MP3 encoder does this, then simply subtracting the MP3 from the WAV will not show exactly what was removed, it will also produce an amount of distortion that is near impossible to determine.
Therefore... you can determine exactly what encoding noise is applied to an mp3 by subtracting it from the original source.
Correct, you can do this.
You can subsequently listen to it,
Correct.
At that point you may judge if the signal has a significant effect on your hearing and enjoyment. It is possible that you would not even hear it at all, conveying good lossy compression, poor hearing, a bad sound system, etc.
Nonsense. Listening to what's been discarded tells you nothing of any value. The whole design goal of the psychoacoustic model associated with lossy music compressors is to work out what can safely be discarded with no perceptual impact on the listener.
Here's an analogy for you. Cook a meat sauce (like a bolognese sauce) for a family of four, with 2 teaspoons of salt, then cook an identical one but with only 1.5 teaspoons of salt. Taste what's been left out (the half teaspoon of salt), I guarantee that it will be very noticeable and not very pleasant to taste. However, do a double blind tasting of the two sauces and you're unlikely to be able to taste the difference - tasting the difference has told you nothing about the difference between the two versions of the sauce. It's the same with lossy encoding - the only valid test is to compare the uncompressed file with the compressed one in a double blind test.
Yes i agree. The psychoacoustic model determines what kind of noise could be added to the mix for the effect to be negligible. Another analogy would be a yelling crowd, you can determine which people you can remove from the crowd to make sure the overall yelling sounds the same. However, my point is, that to a certain extent, it is reliable to hear the people you've discarded from the crowd to realize what kind of noise the overall crowd is missing.
The role of the psychoacoustic model is to ensure that the discarded sound is as least audible as possible
to achieve the target bitrate. Our analogies are good for visualization, but it is hard to compare on the same scale; for many acoustic phenomenons act upon the logarithmic and exponential scales.
On the other hand, if you eat meat sauce everyday for a whole year with the precise same recipe and exact quantities, perhaps at the end of the year you would be able to detect a slight change in taste, a little as it may be.
OK. I'm bumping this topic because in my attempt at reviving the subject in another thread, my post was binned. But it's all good, no hard feelings....
Truncated
While I'm inclined to agree with your post, theres a small factor that DVDdoug said:
MP3 does not retain the original timing & phase information, so subtracting the samples doesn't tell you anything about the sound.
If this is true, and the timing is different in an MP3 encoding from it's original, then the subtraction method used in the video doesn't truely show what was removed from the encoding process.
I don't have specific any references to falsify this quote, but i am fairly familiar with the mpeg-1 audio standard to believe that converting to mp3 does not change any timings deliberately (the quote from DVDdoug is pretty vague). In simple terms, it only filters out certain frequencies and introduces noise shaping according to the psychoacoustic model. One argument that could incline you to believe this is that the MPEG-1 layer 3 standard uses MDCT, which does not take into account explicit expression and modification of phase values, such as the FFT does.
But even so, modifying phase at certain frequencies (actually, just thinking about it now) wouldn't really matter because this would be translated into noise in the time domain. As for timing, there is really no interest in delaying a signal for compression purposes and the mp3 std does not delay the input signal unless specified afaik.
On the other hand, if you eat meat sauce everyday for a whole year with the precise same recipe and exact quantities, perhaps at the end of the year you would be able to detect a slight change in taste, a little as it may be.
Precisely, you detect the difference by tasting the meal you've become familiar with, NOT by eating the half a spoonful of salt that was removed.
C.
However, my point is, that to a certain extent, it is reliable to hear the people you've discarded from the crowd to realize what kind of noise the overall crowd is missing.
I'm not sure what you mean by reliable in this context, but your point about listening to the discarded signal is just wrong. The discarded signal has been discarded because it can't be heard underneath the retained signal. Listening to the discarded signal tells you nothing about how well the codec has identified what signal elements can be discarded, because you don't know what was going on with the retained signal at the same time.
Again, you can only assess the transparency of a lossy codec by listening to the original signal and the compressed signal in a DBT, not by examining or listening to the difference. The fact that you can see the difference, and that you can play it through a speaker system, doesn't mean that it's a useful measure of how your brain will perceive the compressed signal.
On the other hand, if you eat meat sauce everyday for a whole year with the precise same recipe and exact quantities, perhaps at the end of the year you would be able to detect a slight change in taste, a little as it may be.
Precisely, you detect the difference by tasting the meal you've become familiar with, NOT by eating the half a spoonful of salt that was removed.
C.
That's true. Read below.
However, my point is, that to a certain extent, it is reliable to hear the people you've discarded from the crowd to realize what kind of noise the overall crowd is missing.
I'm not sure what you mean by reliable in this context, but your point about listening to the discarded signal is just wrong. The discarded signal has been discarded because it can't be heard underneath the retained signal. Listening to the discarded signal tells you nothing about how well the codec has identified what signal elements can be discarded, because you don't know what was going on with the retained signal at the same time.
Again, you can only assess the transparency of a lossy codec by listening to the original signal and the compressed signal in a DBT, not by examining or listening to the difference. The fact that you can see the difference, and that you can play it through a speaker system, doesn't mean that it's a useful measure of how your brain will perceive the compressed signal.
Well, i'd rephrase it like this: The discarded signal is the
least likely to be heard when combined with the main signal; keep in mind that you still have to obtain a target bitrate, so performance varies. Listening to the discard tells you
exactly what is missing, and what is missing is the least likely signal to be heard with the main signal.
I am
not disputing this: To
assess or
compare lossy compression performance, you NEED to listen to the lossy files because it is difficult to determine what kind of noise may or may not be heard combined with the main track. I am only saying that listening to the discarded noise reveals the exact artifacts that are introduced by the encoder.
...your point about listening to the discarded signal is just wrong. The discarded signal has been discarded because it can't be heard underneath the retained signal.
Thats the
goal of psychoacoustics, the fact that many people claim to hear the difference between lossless and lossy says that they haven't quite achieved that goal yet - at least at 320Kbps and under.
Listening to the discarded signal tells you nothing about how well the codec has identified what signal elements can be discarded, because you don't know what was going on with the retained signal at the same time.
Again, you can only assess the transparency of a lossy codec by listening to the original signal and the compressed signal in a DBT, not by examining or listening to the difference. The fact that you can see the difference, and that you can play it through a speaker system, doesn't mean that it's a useful measure of how your brain will perceive the compressed signal.
I have to disagree with this.
When I started ripping/downloading all of my music in FLAC, I only did it because I had just a gotten bigger hard drive, and saw no reason to keep ripping in MP3. I ran ABX tests in foobar and would consistently fail to tell a difference between +192Kbps MP3s and FLAC. Over time I started to notice things I've never heard before in tracks I've had for a long time, but I would still fail ABX tests. At this point you would assume that the only difference is all in my head, but when I heard the difference file in the youtube link it sounds a lot like the sounds that I occasionally notice in FLAC files. Note I'm referring to the lower volume part of the clip, once it hits the louder part it just sounds like a mess.
To use the sauce analogy, it's like eating the sauce with 1.5tbsp of salt for a month, then eating the sauce with 2tbsp of salt for a month and tasting a difference that you're not even sure is really there, until someone gives you 0.5tbsp of salt and it hits you - "that was what changed".
I don't have specific any references to falsify this quote, but i am fairly familiar with the mpeg-1 audio standard to believe that converting to mp3 does not change any timings deliberately (the quote from DVDdoug is pretty vague). In simple terms, it only filters out certain frequencies and introduces noise shaping according to the psychoacoustic model. One argument that could incline you to believe this is that the MPEG-1 layer 3 standard uses MDCT, which does not take into account explicit expression and modification of phase values, such as the FFT does.
But even so, modifying phase at certain frequencies (actually, just thinking about it now) wouldn't really matter because this would be translated into noise in the time domain. As for timing, there is really no interest in delaying a signal for compression purposes and the mp3 std does not delay the input signal unless specified afaik.
Should I take this to mean that the noise in the time domain won't result in noise in the frequency (or rather, audible) domain? I'm having a hard time wrapping my head around time-related noise.
Well it seems a lot more intuitive in many ways to not alter the timing, so it's probably a good thing that they don't do it for the folks over at the lame development team
At that point you may judge if the signal has a significant effect on your hearing and enjoyment. It is possible that you would not even hear it at all, conveying good lossy compression, poor hearing, a bad sound system, etc.
To quote myself, i would like to clarify that not hearing the signal noise could indeed signify good lossy compression. But this is only a one-way implication, the opposite is not true (hearing it does not mean bad lossy). I will also rectify the "judging" part of that statement, it can only give you an
objective rendition of what is not being heard, no judging involved.
That being said, i don't know why anyone would ever settle for lossy, unless portability or filesize is an issue. Because, you know, that spoonful of salt may well be the one that triggers that fatal heart attack...
Should I take this to mean that the noise in the time domain won't result in noise in the frequency (or rather, audible) domain? I'm having a hard time wrapping my head around time-related noise.
Well it seems a lot more intuitive in many ways to not alter the timing, so it's probably a good thing that they don't do it for the folks over at the lame development team
Nope, both are simultaneously related by the transforms: Altering frequency adds noise, and adding noise alters frequency. The magic is in the psychoacoustic noise shaping; it shifts the noise to less audible frequency bands. The noise has the same power, shaped or not, and by listening to the discards only you may or may not be able to tell which is better than the other. Although you could make an educated guess, because noise shaping tends to produce higher pitched noise... But after matching them with the original audio, you will be in a better position to tell which one has better performance.
I am only saying that listening to the discarded noise reveals the exact artifacts that are introduced by the encoder.
...and this signal played by itself is completely meaningless in determining the sound quality of the lossy signal from which it was derived. The only proper way to determine this is through double-blind testing. There is no way to wiggle your way out of this; not on this forum.
Subtraction gives you the sample-by-sample mathematical difference, but it does NOT give you the SOUND difference! MP3 does not retain the original timing & phase information, so subtracting the samples doesn't tell you anything about the sound.
I disagree!
Mp3 encoding does preserve timing and phase information. Unlike parametric coders, Mp3 strives to preserve the
waveform, including its phase. The distortion introduced by mp3 — a quantization noise — is fully revealed when mp3 is subtracted from wav (provided that your decoder does not time-shift the decoded file). And this distortion gets quite small when the bit rate gets higher.
Of course, you cannot judge on audibility of this quantization noise w/o having the rest of the audio signal, due to masking.
I am not disputing this: To assess or compare lossy compression performance, you NEED to listen to the lossy files because it is difficult to determine what kind of noise may or may not be heard combined with the main track. I am only saying that listening to the discarded noise reveals the exact artifacts that are introduced by the encoder.
So what? It doesn't tell you whether they are audible in context.
Thats the goal of psychoacoustics, the fact that many people claim to hear the difference between lossless and lossy says that they haven't quite achieved that goal yet - at least at 320Kbps and under.
That's an incorrect generalisation - it's only true for a very few killer samples, and for a few people. In general, modern lossy codecs (MP3 / AAC / OGG) achieve perceptual transparency at much lower bit rates for the vast majority of people and for the vast majority of music. That's what all of the properly conducted tests have revealed, and that's why many people who do their own tests settle on LAME -V2 (around 220 kb/S) or even lower for their MP3 encoding.
When I started ripping/downloading all of my music in FLAC, I only did it because I had just a gotten bigger hard drive, and saw no reason to keep ripping in MP3. I ran ABX tests in foobar and would consistently fail to tell a difference between +192Kbps MP3s and FLAC. Over time I started to notice things I've never heard before in tracks I've had for a long time, but I would still fail ABX tests. At this point you would assume that the only difference is all in my head, but when I heard the difference file in the youtube link it sounds a lot like the sounds that I occasionally notice in FLAC files. Note I'm referring to the lower volume part of the clip, once it hits the louder part it just sounds like a mess.
It is illogical to claim that the difference file on Youtube sounds like the sounds you occasionally notice in FLAC files. If you can't ABX it then you clearly
can't notice the difference, so it's all in your head i.e. it's a placebo effect.
Again, the psychoacoustic models in lossy codecs exploit the way your ears and brain perceive sound, and ABX is the only generally available method that measures your perception of the sound. Looking at the difference file, or listening to it in isolation, tells you nothing.
I disagree!
Mp3 encoding does preserve timing and phase information. Unlike parametric coders, Mp3 strives to preserve the waveform, including its phase.
There is nothing in the mp3 standard that says that you have to do anything of the kind. If current implementations DO preserve timing and phase then that may be because it is easiest to implement that way.
If someone discovered a way to encode to mp3 such that the timing and phase were not preserved, but the result was peceptually transparent at lower bitrates, then everybody would switch to doing it that way.
In fact, the only thing that the mp3 standard specifies is how to decode an mp3 file to wav.
I'd like to see DVDdoug defend his comment regarding phase response.
There is nothing in the mp3 standard that says that you have to do anything of the kind. If current implementations DO preserve timing and phase then that may be because it is easiest to implement that way. If someone discovered a way to encode to mp3 such that the timing and phase were not preserved, but the result was peceptually transparent at lower bitrates, then everybody would switch to doing it that way.
May I assure you that there no practical way to encode an mp3 file better than with a standard phase-preserving quantization of coefficients of a phase-preserving filter bank.
I'd like to see DVDdoug defend his comment regarding phase response.
I'd like to hear some comments from people who have actually taken say a 44/16 400 Hz square wave .wav file, converted it into a 128 kb MP3, and then compared the reconstructed .wav file to the original wav file.
One the one hand, the reconstructed MP3 is not totally trashed and still looks something like a square wave. On the other hand, some fairly obvious violence has been done to it by the MP3 processing. Some of the violence involves seems to involve timing (IOW phase) and the other violence looks like noise.
If the square wave file is tone bursts which would be more like music, then the trashing is even more obvious.
Its fair to say that phase information is neither totally lost, nor accurately preserved.
May I assure you that there no practical way to encode an mp3 file better than with a standard phase-preserving quantization of coefficients of a phase-preserving filter bank.
So you are saying that the mp3 standard disallows the use of parametric encoding?
There is nothing in the mp3 standard that says that you have to do anything of the kind. If current implementations DO preserve timing and phase then that may be because it is easiest to implement that way. If someone discovered a way to encode to mp3 such that the timing and phase were not preserved, but the result was peceptually transparent at lower bitrates, then everybody would switch to doing it that way.
May I assure you that there no practical way to encode an mp3 file better than with a standard phase-preserving quantization of coefficients of a phase-preserving filter bank.
Providng a mathematical proof of that would probably be a pretty good master's thesis project... ;-)
I am not disputing this: To assess or compare lossy compression performance, you NEED to listen to the lossy files because it is difficult to determine what kind of noise may or may not be heard combined with the main track. I am only saying that listening to the discarded noise reveals the exact artifacts that are introduced by the encoder.
So what? It doesn't tell you whether they are audible in context.
Where's the line between audible and inaudible? ...in context? If I can't hear you a mile away, how about right in front of me... in the context of a jet engine running?
Lossy has the purpose of achieving a target bitrate and getting acceptable signal quality for the intended use. For the application of music, people have decided to go for 'perceived human hearing' as a model. Perhaps you'll be surprised to know that the MP3 standard doesn't specify any normative implementation of the psyacoustic model; It's all up to the coder to decide.
With music we go for high fidelity, namely the most exact rendition of a recording. Is it justified to retract noises that are not naturally prone to being perceived by the human ear? Yes, because this is the best compromise to achieve target bitrate and retain quality. Is this a hifi process? Not really (in my definition), because these noises are not given a chance, they're simply dulled out when normally they would sparkle (high pitch) through the audio. So if you want to know exactly what kind of sparkle is being taken away from your music, i suggest hearing them alone. Note that it might not give you insight on subjective performance, but it does reveal exactly what is left out.
I will post some audio samples so you can really get a feel for it.I disagree!
Mp3 encoding does preserve timing and phase information. Unlike parametric coders, Mp3 strives to preserve the waveform, including its phase.
There is nothing in the mp3 standard that says that you have to do anything of the kind. If current implementations DO preserve timing and phase then that may be because it is easiest to implement that way.
If someone discovered a way to encode to mp3 such that the timing and phase were not preserved, but the result was peceptually transparent at lower bitrates, then everybody would switch to doing it that way.
In fact, the only thing that the mp3 standard specifies is how to decode an mp3 file to wav.
Phase and timing aren't the same thing... Phase
cannot be preserved, because you add noise that's reflected in both time and freq domains. But timing doesn't change with respect to this noise.
About your last sentence... I have the ISO/IEC 11172-3 standard right in front of me, and... hmm... it's not true. Decoder architecture is only part of the standard, amongst other parts like Encoding and Storage.
Your jet engine analogy shows that you understand the concept of masking, but you then go back to your misconceptions...
....they're simply dulled out when normally they would sparkle (high pitch) through the audio.
No, they wouldn't normally sparkle. You can't hear them in the context of the rest of the music, because they would be masked. If you can hear them in the original but detect a loss of "sparkle" in the encoded version, then output from the lossy codec would be easily detectable in an ABX test. There's no point retaining something that your ear can't hear - that's the basis of psychoacoustic modelling in lossy codecs.
So if you want to know exactly what kind of sparkle is being taken away from your music, i suggest hearing them alone.
Great, but it doesn't tell you anything because you are ignoring the signal that is masking the "sparkle".
Note that it might not give you insight on subjective performance
Finally, you have got it. As has been said numerous times in this and other threads, listening to the difference gives you absolutely no insight into the subjective performance of the lossy codec. Only an ABX test can do that.
May I assure you that there no practical way to encode an mp3 file better than with a standard phase-preserving quantization of coefficients of a phase-preserving filter bank.
Exactly. Keeping phase info is not desirable.
I'd like to see DVDdoug defend his comment regarding phase response.
I'd like to hear some comments from people who have actually taken say a 44/16 400 Hz square wave .wav file, converted it into a 128 kb MP3, and then compared the reconstructed .wav file to the original wav file.
There is a fatal flaw to this experiment, because a square wave cannot be coded to PCM. The max possible freq in 44.1kHz is 22050 Hz, and a perfect square wave has infinite freq (impulse) at its edges. Therefore the signal is aliased.
So you can't sample a non-bandlimited signal (eg. square wave) without serious aliasing unless prefiltering it with a lowpass @ 22050 (smooth it). In this case the experiment would be more reliable, but your signal would be a smoothed out square wave and could still be a bit aliased depending on the nature of your prefilter and its performance. All mathematical operations from there would be valid.
To visualize what kind of waveform your seemingly perfect square wave ought to be, you can ideally interpolate it with the sinc function. You will inevitably notice a lot of ringing near the edges (equivalent to ghosting in imaging).
Thats the goal of psychoacoustics, the fact that many people claim to hear the difference between lossless and lossy says that they haven't quite achieved that goal yet - at least at 320Kbps and under.
That's an incorrect generalisation - it's only true for a very few killer samples, and for a few people. In general, modern lossy codecs (MP3 / AAC / OGG) achieve perceptual transparency at much lower bit rates for the vast majority of people and for the vast majority of music. That's what all of the properly conducted tests have revealed, and that's why many people who do their own tests settle on LAME -V2 (around 220 kb/S) or even lower for their MP3 encoding.
When I started ripping/downloading all of my music in FLAC, I only did it because I had just a gotten bigger hard drive, and saw no reason to keep ripping in MP3. I ran ABX tests in foobar and would consistently fail to tell a difference between +192Kbps MP3s and FLAC. Over time I started to notice things I've never heard before in tracks I've had for a long time, but I would still fail ABX tests. At this point you would assume that the only difference is all in my head, but when I heard the difference file in the youtube link it sounds a lot like the sounds that I occasionally notice in FLAC files. Note I'm referring to the lower volume part of the clip, once it hits the louder part it just sounds like a mess.
It is illogical to claim that the difference file on Youtube sounds like the sounds you occasionally notice in FLAC files. If you can't ABX it then you clearly can't notice the difference, so it's all in your head i.e. it's a placebo effect.
Again, the psychoacoustic models in lossy codecs exploit the way your ears and brain perceive sound, and ABX is the only generally available method that measures your perception of the sound. Looking at the difference file, or listening to it in isolation, tells you nothing.
Actually, it is a perfectly logical statement. If the goal of the MP3 encoder is to mask the sounds that you (hopefully) would never hear anyway, it is logical to assume that pin pointing those sounds in an ABX test would be nearly impossible.
Not being able to pin point something does NOT mean it doesn't exist.
This is starting to look like something that will break TOS#8, so I'll put this disclaimer here; my argument is that the mathematical difference between a WAV and MP3 file of the same song can help to reveal the differences between lossy and lossless. I am
not arguing that either one is better.
So you are saying that the mp3 standard disallows the use of parametric encoding?
Yes, to the best of my knowledge.
Phase and timing aren't the same thing... Phase cannot be preserved, because you add noise that's reflected in both time and freq domains. But timing doesn't change with respect to this noise.
Well,
phase is impacted by quantization noise just as much as
amplitude. However there's no preference in mp3 for preserving amplitude versus preserving phase. When bit rate is high, both amplitude and phase are well preserved.
Its fair to say that phase information is neither totally lost, nor accurately preserved.
I'd say that distortion to phase information is "comparable" to distortion of amplitude information (although it's truly like apples vs. oranges).
So if you want to know exactly what kind of sparkle is being taken away from your music, i suggest hearing them alone.
Great, but it doesn't tell you anything because you are ignoring the signal that is masking the "sparkle".
If by "masking" you mean dissimulating... OK.
Keep in mind that it's not a logical "AND" mask like in computing, eg. '11001100' AND '12345678' = '12005600'; The original signal (2nd) is irretrievable using only the mask and output. (You can apply such an operation in the frequency domain to implement bandpass filters and whatnot.)
But here we're talking about a
temporal masking eg. (lets take bigger values) '1 0 -2 1 3' + '100 50 75 53' = '101 50 73 56'; Here the signal
is retrievable using only the mask and output. Saying that the noise is completely masked temporally is quite a stretch... though it may be well dissimulated (input and output are very similar).
It's really not surprising that many prefer CD quality over MP3s. On the other hand, not everyone has the same auditory ability, or hifi stereo chain, so in the same way its not a surprise that many people can't distinct one from the other --MP3 is
designed to have well dissimulated noise!
Note that it might not give you insight on subjective performance
Finally, you have got it. As has been said numerous times in this and other threads, listening to the difference gives you absolutely no insight into the subjective performance of the lossy codec. Only an ABX test can do that.
I never disputed this claim. What i defend is the fact that subtracting MP3 from WAV objectively reveals encoding noise (artifacts).
The doesn't change the fact that the person who compiled the youtube video is completely clueless about how to assess the sound quality of a lossy encoding.
No one here is disputing that subtracting one file from another can leave something audible. The point is that what is audible has no meaning if it is masked by the sound that has been omitted though the unnatural manipulation of subtracting out the original lossless signal.
May I assure you that there no practical way to encode an mp3 file better than with a standard phase-preserving quantization of coefficients of a phase-preserving filter bank.
Ok, here's what I did. I took first 4096 samples from "testcase.wav" included with LAME distribution, encoded them with lame -b 128, decoded with lame --decode and did FFT on both original and decoded. The phases of resulting matrices are far from identical. So the phase spectrum is not preserved, or am I missing something?
Actually, it is a perfectly logical statement. If the goal of the MP3 encoder is to mask the sounds that you (hopefully) would never hear anyway, it is logical to assume that pin pointing those sounds in an ABX test would be nearly impossible.
No, your original statement is illogical. You said:
when I heard the difference file in the youtube link it sounds a lot like the sounds that I occasionally notice in FLAC files
If you noticed those sounds in the FLAC file but not in the MP3 file then the codec would "fail" an ABX test. If it "passes" the ABX test (i.e. you can't detect the difference in a double blind test") then by definition you CAN'T hear those sounds in the FLAC file. Hence your statement is illogical (unless you are deliberately using a lossy codec that creates audible artifacts).
Not being able to pin point something does NOT mean it doesn't exist.
In a perceptual listening test it means almost exactly that. If you can't hear the difference, it effectively doesn't exist. The fact that it exists mathematically (when you subtract A from B) is a red herring.
my argument is that the mathematical difference between a WAV and MP3 file of the same song can help to reveal the differences between lossy and lossless. I am not arguing that either one is better.
It can reveal the mathematical differences, but not the audible differences. You are completely ignoring the fact that the noise that is removed is removed precisely because it is inaudible
when played in the presence of the rest of the reconstructed signal. If it were audible as part of the original signal then the codec would "fail" an ABX test.
Perhaps I need to go back and re-read, but I don't believe anyone is disputing that neither amplitude nor phase in the frequency domain are perfectly preserved during lossy encoding. I just think DVDdoug's overly-simplistic* comment was merely off-the-cuff. I'm hoping someone with greater expertise will chime-in, though I doubt it since this thread was washed-up before it even began.
(*) to the point that it does more harm than good.
I took first 4096 samples from "testcase.wav" included with LAME distribution, encoded them with lame -b 128, decoded with lame --decode and did FFT on both original and decoded. The phases of resulting matrices are far from identical. So the phase spectrum is not preserved, or am I missing something?
1. Check if the signals are aligned in time.
2. Phase spectrum of FFT is not the same as "phase information". You have to understand what you are looking for and take care of several things: weighting windows; analysis of spectrum sparsity; only paying attention to phase of significant spectrum components; etc.
So you are saying that the mp3 standard disallows the use of parametric encoding?
Yes, to the best of my knowledge.
I thought that mp3's intensity stereo was a form of parametric encoding?
I thought that mp3's intensity stereo was a form of parametric encoding?
Oh, yes, you are right! I don't think that it's a widely used feature though: Lame doesn't have it and FhG only uses it for very low bit rates.
Actually, it is a perfectly logical statement. If the goal of the MP3 encoder is to mask the sounds that you (hopefully) would never hear anyway, it is logical to assume that pin pointing those sounds in an ABX test would be nearly impossible.
No, your original statement is illogical. You said:
when I heard the difference file in the youtube link it sounds a lot like the sounds that I occasionally notice in FLAC files
If you noticed those sounds in the FLAC file but not in the MP3 file then the codec would "fail" an ABX test. If it "passes" the ABX test (i.e. you can't detect the difference in a double blind test") then by definition you CAN'T hear those sounds in the FLAC file. Hence your statement is illogical (unless you are deliberately using a lossy codec that creates audible artifacts).
Not being able to pin point something does NOT mean it doesn't exist.
In a perceptual listening test it means almost exactly that. If you can't hear the difference, it effectively doesn't exist. The fact that it exists mathematically (when you subtract A from B) is a red herring.
my argument is that the mathematical difference between a WAV and MP3 file of the same song can help to reveal the differences between lossy and lossless. I am not arguing that either one is better.
It can reveal the mathematical differences, but not the audible differences. You are completely ignoring the fact that the noise that is removed is removed precisely because it is inaudible when played in the presence of the rest of the reconstructed signal. If it were audible as part of the original signal then the codec would "fail" an ABX test.
I think you are misunderstanding what I mean by small differences here. I'm not talking about a difference where you can just simply skip back 20 seconds in the song and hear it all over again. I'm talking about the kind of differences that are so small you can't be sure it's actually there.
If we defined reality by what can be easily understood, seen and measured, science would grind to a hault and very little progress would be made. Just because it is too small to be easily measured, does not mean it can't be heard. If a computer can 'hear' it with no problems, why is it so hard for you to accept that a person, on some level, can?
The sub-conscious's latent inhibition will disregard small stimuli at a given time, and at another time for no readily-apparent reason it will present them to your conscious mind. This is exactly the type of case where these tiny differences will always be hard to measure.
They are designed to be hard to measure.Also, you keep saying that the things that are removed are removed because they are inaudible. That is, again, the goal of applied psychoacoustics and can be proven that it has not been met millions of times over. As proof to that statement, I think it would be safe to say that anyone in the world with average hearing (and say, 5 years old or older) can hear the difference between a 32kBps MP3 and a 16/44.1 lossless file. To say that the information removed is inaudible, I assume you were trying to imply that it occurs at higher bit-rates. At what rate does this occur?
Anyway, this is kind of off topic anyways. I originally posted here (and signed up for the sole purpose of) because I was trying to find out if the mathematical difference was actually what was removed from the WAV and not a result of timing differences. We all got the same answer, which is: yes and no.
my argument is that the mathematical difference between a WAV and MP3 file of the same song can help to reveal the differences between lossy and lossless.
Are you sure about this? Has it helped you? If so, then can you provide a sample as an example to help those who have trouble ABXing lossy from lossless?
I'm not particularly interested in -V5 since I can often ABX -V5 from lossless without much difficulty. Perhaps you can help me do this with -V3 or -V2.
I think you are misunderstanding what I mean by small differences here. I'm not talking about a difference where you can just simply skip back 20 seconds in the song and hear it all over again. I'm talking about the kind of differences that are so small you can't be sure it's actually there.
Either they are inaudible or they aren't. If you can't ABX the original from the lossy file then the differences are inaudible in the context of the original file, nomatter what they sound like or look like when you examine the differences on their own.
If we defined reality by what can be easily understood, seen and measured, science would grind to a hault and very little progress would be made. Just because it is too small to be easily measured, does not mean it can't be heard. If a computer can 'hear' it with no problems, why is it so hard for you to accept that a person, on some level, can?
Computers don't hear. People hear. The purpose of lossy compression is to exploit features in the human auditory process, not to fool computers.
Also, you keep saying that the things that are removed are removed because they are inaudible. That is, again, the goal of applied psychoacoustics and can be proven that it has not been met millions of times over. As proof to that statement, I think it would be safe to say that anyone in the world with average hearing (and say, 5 years old or older) can hear the difference between a 32kBps MP3 and a 16/44.1 lossless file. To say that the information removed is inaudible, I assume you were trying to imply that it occurs at higher bit-rates.
Many thousands of properly conducted ABX tests prove that the bold statement is completely wrong with modern codecs configured properly, where people have been unable to tell the difference between the original WAV and the compressed MP3. The fact that old codecs, or very low bitrates, can be distinguished from the original, is just that - a fact. It contributes nothing to this argument, where the person who posted on Youtube was arguing that the differences he measured proved that even a well tuned MP3 encoder could be distinguished from the WAV, and his method was to measure the mathematical differences, not to do a blind listening test.
At what rate does this occur?
It depends on the sensitivity of the listener's auditory process. That's why, time and time again, people are told to perform their own ABX testing to decide what is transparent for them.
OK. I did the experiment suggested by Arnold B. Krueger, with the square wave, and i think it has been most revealing in terms of favorably validating the statement "Subtracting MP3 from WAV reveals artifacts". Judge for yourself:
http://www.multiupload.com/YDPTV9XPH8 (http://www.multiupload.com/YDPTV9XPH8)
Method: An "ideal" PCM square wave is generated with MATLAB @ 440Hz (although a mathematical continuous reconstruction of the signal bears a pseudo-square wave suffering from aliasing.) The wave is then converted to LAME CBR 320kbps (best quality), and V5 VBR 130kbps ('threshold' quality). Finally, both signals are subtracted from the original PCM source to (hypothetically) reveal their artifacts.
Observations:
1. MATLAB PCM square wave rings @ A440, sounding very digital.
2. LAME 320kbps MP3 sounds indiscernibly like the original PCM signal.
3. LAME V5 130kbps sound very similar to the original PCM signal, with a slight added 'hush'.
4. Error signal PCM-320kbps is inaudible, explaining the absence of audible discrepancies described in (2), thus supporting transparency in this instance.
5. Error signal PCM-V5 sounds exactly like the noisy 'hush' described in (5), supporting the fact that artifacts are indeed exposed by this method, in this instance.
Sorry, but artifacts are not "exposed" unless they are identified in an ABX test.
4. Error signal PCM-320kbps is inaudible, explaining the absence of audible discrepancies described in (2), thus supporting transparency in this instance.
5. Error signal PCM-V5 sounds exactly like the noisy 'hush' described in (5), supporting the fact that artifacts are indeed exposed by this method, in this instance.
Yeah this isn't valid reasoning. You can't conclude #5 (4 is pretty doggy too) because masking exists. Or to put it another way, lossy audio would be pretty much useless if #5 actually worked. But MP3s can sound pretty good, so obviously 5 must be wrong
Ok, so here is what I did, hopefully someone more knowledgeable than I will decide if this methodology is sound.
I took a lossless track, encoded it into an mp3 (128k) and opened both files up in audacity. After lining up the tracks (it appears the timing information isn't changed, when encoded, even on a sample by sample basis) I inverted one, then mixed the two tracks. I now have a single track consisting of the difference information, correct? Good. I saved this file as a WAV.
I then opened up the original 128k MP3 in audacity along with the new difference WAV and played them side by side, after lining up as best as possible. Is it correct in assuming that I should now be listening to the original lossless track + any distortion added by the encoder? If so, then I can now simply check and uncheck the mute button (on the difference track) during playback and have an instant transition between what is close to lossless and what is lossy.
I haven't yet tried this method at higher than 128k (believe me I will be soon enough), but if correct, this method very audibly reveals the differences at 128Kbps.
4. Error signal PCM-320kbps is inaudible, explaining the absence of audible discrepancies described in (2), thus supporting transparency in this instance.
5. Error signal PCM-V5 sounds exactly like the noisy 'hush' described in (5), supporting the fact that artifacts are indeed exposed by this method, in this instance.
Yeah this isn't valid reasoning. You can't conclude #5 (4 is pretty doggy too) because masking exists. Or to put it another way, lossy audio would be pretty much useless if #5 actually worked. But MP3s can sound pretty good, so obviously 5 must be wrong
I'm not even gonna answer this. Please look at my explanation on masking.
Ok, so here is what I did, hopefully someone more knowledgeable than I will decide if this methodology is sound.
I took a lossless track, encoded it into an mp3 (128k) and opened both files up in audacity. After lining up the tracks (it appears the timing information isn't changed, when encoded, even on a sample by sample basis) I inverted one, then mixed the two tracks. I now have a single track consisting of the difference information, correct? Good. I saved this file as a WAV.
I then opened up the original 128k MP3 in audacity along with the new difference WAV and played them side by side, after lining up as best as possible. Is it correct in assuming that I should now be listening to the original lossless track + any distortion added by the encoder? If so, then I can now simply check and uncheck the mute button (on the difference track) during playback and have an instant transition between what is close to lossless and what is lossy.
I haven't yet tried this method at higher than 128k (believe me I will be soon enough), but if correct, this method very audibly reveals the differences at 128Kbps.
This is a great idea. But
make sure you convert all files to WAV before importing them to Audacity; this way they will line up perfectly by themselves. So do WAV<->MP3 conversions with foobar. Play noise + MP3, mute noise as you wish, notice the difference.
Ok, so here is what I did, hopefully someone more knowledgeable than I will decide if this methodology is sound.
Sighted tests are absolutely not sound.
Foobar2000 + ABX plugin or something that performs the same function is what you need.
@JapanAudio, it is clear that you do not have the faintest idea what masking is.
The two of you seem so unwilling to discuss, let alone make use of double-blind testing. Why is this?
How many more rounds of cluelessness can this thread handle?
How many more rounds of cluelessness can this thread handle?
None. If this thread were a horse you'd shoot it immediately (if you had a heart).
C.
4. Error signal PCM-320kbps is inaudible, explaining the absence of audible discrepancies described in (2), thus supporting transparency in this instance.
5. Error signal PCM-V5 sounds exactly like the noisy 'hush' described in (5), supporting the fact that artifacts are indeed exposed by this method, in this instance.
Yeah this isn't valid reasoning. You can't conclude #5 (4 is pretty doggy too) because masking exists. Or to put it another way, lossy audio would be pretty much useless if #5 actually worked. But MP3s can sound pretty good, so obviously 5 must be wrong
I'm not even gonna answer this. Please look at my explanation on masking.
I didn't realize you'd posted about it. Rereading it I see that you're not really clear on what we mean by masking. We're not talking about it in the sense you are, but rather this:
http://en.wikipedia.org/wiki/Auditory_masking (http://en.wikipedia.org/wiki/Auditory_masking)
You may want to take a long read through that page until you understand what I meant by "masking" above. But essentially, because of masking, you can't linearly add sounds and expect to hear their superposition. Thus listening to the difference, and trying to draw conclusions about what the original would sound like is incorrect.
greynol, have you even listened to the square wave example? If you try his (HibyPrime's) method with the V5 square wave + V5 noise you obtain the original PCM square wave, and by muting the noise you will instantaneously notice the artifacts. This is not about ABX.
At the music conservatory, students don't take ear tests out of the blue, just like that, without any training and practice. In fact they train for years and their abilities grow enormously. Musical dictation, chord identification, interval comparison are very difficult without training. ABX tests with audio samples are the same, actually they're harder since the difference is so minimal. So don't expect people to be proficient at ABX tests without proper training, listening time, controlled conditions etc.
I believe listening to sound differences is a proper tool, one of many, that can teach or at least indicate what kind of noises and artifacts to look for in an ABX test. I'm not trying to replace your ABX testing method that you seem to praise so wildly, i'm giving you something to complement it.
greynol, have you even listened to the square wave example? If you try his method with the V5 square wave + V5 noise you obtain the original PCM square wave, and by muting the noise you will instantaneously notice the artifacts. This is not about ABX.
Sure and I've created quite a few difference files of real music as well. I would never be so daft as to suggest that this is how one should judge the quality of a perceptual encoding, however.
At the music conservatory, students don't take ear tests out of the blue, just like that, without any training and practice. In fact they train for years and their abilities grow enormously. Musical dictation, chord identification, interval comparison are very difficult without training.
People don't really know it but I am a musician and come from a family of musicians. I understand what you're talking about but as far as this discussion is concerned it's a non-sequitur.
So don't expect people to be proficient at ABX tests without proper training, listening time, controlled conditions etc.
I don't.
I believe listening to sound differences are a proper tool, one of many, that can teach or least indicate what kind of noises and artifacts to look for in an ABX test. I'm not trying to replace your ABX testing method that you seem to praise so wildly, i'm giving you something to complement it.
Do you have any proof that your method works? Even something anecdotal would be fine. Right now we have HibyPrime looking quite foolish covering his ass by asking how to perform tests that he says are helpful which he's never actually conducted. Perhaps you'll be able to tell him that he isn't wasting his time, though the same can't be said for those who have had to read what has transpired since you decided to resurrect this discussion.
>"praise so wildly"
Really?!? Do you know some alternative method of performing an objective evaluation of the sound quality of perceptual encoding that isn't double-blind which is deserving of any praise at all?
I believe listening to sound differences is a proper tool, one of many, that can teach or at least indicate what kind of noises and artifacts to look for in an ABX test. I'm not trying to replace your ABX testing method that you seem to praise so wildly, i'm giving you something to complement it.
His point is that you are basing this whole "tool" on one absurdly artificial example (square waves aren't music!) and trying to generalize it to all music based on sighted tests and a suspect understanding of what an encoder actually does. Thats not a good start.
I really don't know what happens when you use LAME to encode a square wave, and I don't really care because its a silly example of something the encoder was never meant to do and should not be expected to work correctly. I suspect that if you took greynol's advice and tried this with a sample thats actually very hard to ABX you would find it doesn't work nearly so well in a proper test. Of course if it does then you're welcome to "give" it all to us, but one thing at a time
Heaven forbid we try this with something encoded at -V2.
What happens with a difference sample where the RMS levels can be as high as -25dB, if not higher?
Such an encoding can't possibly be transparent, now can it?
I've taken too much of my time trying to explain this very simple concept, by providing countless examples founded on mathematical theorems that have been known for centuries, and yet you seem not to have the slightest grasp of what so many people understand instinctively. What do i get? A reference to a Wikipedia article (A+), and the same bottomless counterarguments over and over.
Well if someone else wants to take up the torch, be my very welcome guest. Now, I'll leave you two to your metaphysical world in which nothing can be falsified and where everything has a shade of greynol.
I've taken too much of my time trying to explain this very simple concept, by providing countless examples founded on mathematical theorems that have been known for centuries,
Since I think you're misunderstanding everyone else here, we are not saying superposition is wrong, just saying its not a universal property of systems. This is one of those things you learn in an introductory signals course. You can apply superposition only to linear systems. Try applying to a system in which linearity is violated (such as one in which masking effects can occur) and you'll see that you get the wrong answer!
This should be obvious. If masking occurs, then superposition cannot hold. Think about it.
What do i get? A reference to a Wikipedia article (A+), and the same bottomless counterarguments over and over.
Did you take a look at the wikipedia article? I really do think it would help you. I'm not interested in these arguments, but if you have questions I'm willing to take the time to explain these things to you. They're quite easy once you get your head around them.
Well if someone else wants to take up the torch, be my very welcome guest. Now, I'll leave you two to your metaphysical world in which nothing can be falsified and where everything has a shade of greynol.
I'm afraid I don't follow you here. I'm just suggesting that you need to apply these " mathematical theorems that have been known for centuries" correctly, not that they don't exist or something like that.
This is a great idea. But make sure you convert all files to WAV before importing them to Audacity; this way they will line up perfectly by themselves. So do WAV<->MP3 conversions with foobar. Play noise + MP3, mute noise as you wish, notice the difference.
Converting the mp3 back to wav doesn't seem to remove the line-up problem. I must be doing it wrong - is there any way to convert to MP3 and back without exporting?
Ok, so here is what I did, hopefully someone more knowledgeable than I will decide if this methodology is sound.
Sighted tests are absolutely not sound.
Foobar2000 + ABX plugin or something that performs the same function is what you need.
Hmm, I prefer using both methods personally. I wouldn't want to limit myself to only one source of information, that would be just bad practice
The two of you seem so unwilling to discuss, let alone make use of double-blind testing. Why is this?
Thats a two way street, saying that we're unwilling to discuss.
I'm new here, I'm not used to everyone expecting double blind tests to be done before you are taken seriously - I'm not saying thats a bad thing, just that it's a bit foreign to me. I do use double-blind tests, I just don't consider them the end-all be-all of what is true. Also blind testing myself solves one problem and introduces another; many people feel 'pressure' to make sure they get the right answers in a blind situation, and since I have anxiety problems to begin with, ABX testing only makes that worse.
Anyway, BACK ON TOPIC: Audacity is giving me some weird issues, I'll save a file then open it a minute later and the track is silent when it shouldn't be, so it's making it difficult to try out multiple songs... Aside from that, I still managed to get Joe Satriani's
Always with me, Always with you into the method I described earlier at 256Kbps. The difference is small, but still clear enough for me to hear 'something' different, not enough to say it's any better or worse - but enough to say it is there. The difference is most clear in the maracas throughout the song. After noticing this, I fired up foobar and ABX'ed away. Due to reasons described above, I failed miserably.
I also added the original flac track into the same audacity project as the difference and MP3 tracks. I accidentally discovered that if you solo and mute a track, it still plays that track as if it was solo'd, not sure why this is. So I solo + muted the FLAC track, and checked/unchecked the solo button, and could not hear any difference between the FLAC track and the difference+MP3 tracks combined.
Also blind testing myself solves one problem and introduces another; many people feel 'pressure' to make sure they get the right answers in a blind situation, and since I have anxiety problems to begin with, ABX testing only makes that worse.
Just repeat the test until its routine enough that you don't feel anxious.
Aside from that, I still managed to get Joe Satriani's Always with me, Always with you into the method I described earlier at 256Kbps. The difference is small, but still clear enough for me to hear 'something' different, not enough to say it's any better or worse - but enough to say it is there. The difference is most clear in the maracas throughout the song. After noticing this, I fired up foobar and ABX'ed away. Due to reasons described above, I failed miserably.
If you can't ABX it I would not assume you can hear a difference. It sounds like its in your head.
I also added the original flac track into the same audacity project as the difference and MP3 tracks. I accidentally discovered that if you solo and mute a track, it still plays that track as if it was solo'd, not sure why this is. So I solo + muted the FLAC track, and checked/unchecked the solo button, and could not hear any difference between the FLAC track and the difference+MP3 tracks combined.
Not sure what you're asking here? Do you mean to ask if adding the difference to the MP3 gives you the lossless copy back? It does, at least to within rounding error. FWIW this is actually a good part of how quite a few lossless codecs work. FWIW I'd probably use something a little easier to compare samples in though. IMO matlab works great for this if you have access to it and are familiar with its syntax.
Thats a two way street, saying that we're unwilling to discuss.
Perhaps if you brought a new argument to the table which is actually valid, then I'm sure it would be discussed. So far I haven't seen a new and valid argument, at least not to my satisfaction. I recommend using the forum's search function. The fact of the matter is that many of us aren't interested in hashing over the same tired subject.
I'm new here, I'm not used to everyone expecting double blind tests to be done before you are taken seriously
That's the rule to which you agreed in order to participate on this forum.
Also blind testing myself solves one problem and introduces another; many people feel 'pressure' to make sure they get the right answers in a blind situation, and since I have anxiety problems to begin with, ABX testing only makes that worse.
It would be much easier if you simply accept that the differences you think you hear during a sighted test are imagined rather than conjure excuses. For those who insist that the differences are night and day, I believe these excuses should be summarily dismissed without a second-thought.
Anyway, BACK ON TOPIC: Audacity is giving me some weird issues
Actually this is off-topic.
Due to reasons described above, I failed miserably.
I think that placebo effect isn't allowed to work in an ABX test is the most likely reason here.
oy.. Started with logical fallacies, now we're at name calling?
I'm stepping out here, it was fun while it lasted (not really).
Name calling???
FWIW, I did edit the first paragraph in my previous post but I can assure you that I didn't call you any names. If anything the edit resulted in a tougher stance than what was originally there.
Right now we have HibyPrime looking quite foolish covering his ass by asking how to perform tests that he says are helpful which he's never actually conducted. Perhaps you'll be able to tell him that he isn't wasting his time
Name calling???
FWIW, I did edit the first paragraph in my previous post but I can assure you that I didn't call you any names. If anything the edit resulted in a tougher stance than what was originally there.
Telling you that you're looking foolish and calling you a fool are two different things. Perhaps you don't like my language, but I can't think of a more fitting description of your behavior, especially considering your evasion of the questions I asked you earlier:
http://www.hydrogenaudio.org/forums/index....st&p=730222 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=82292&view=findpost&p=730222)
FWIW, I look foolish more often than I'd like. The trick is to learn from your mistakes.
Greynol, please never give up your tireless dedication to logic and reason on this board. And I'm not talking about sequencers. The whole battle is reminiscent of humanities struggle between superstition and science except for a change, here, science is winning. Bravo.
I've taken too much of my time trying to explain this very simple concept, by providing countless examples founded on mathematical theorems that have been known for centuries, and yet you seem not to have the slightest grasp of what so many people understand instinctively. What do i get? A reference to a Wikipedia article (A+), and the same bottomless counterarguments over and over.
You should have spent this time to learn. Basic DSP theory (you are so proud of) has little in common with actual problems present in audio quality assessment. Once you don't know what the problem is and don't respect people trying to explain you the situation the conversation has no sense.
1. Check if the signals are aligned in time.
LAME takes care of this.
2. Phase spectrum of FFT is not the same as "phase information".
Well then, I'd like to know what
exactly do you mean by "phase-preserving quantization", and how it can be proven mathematically.
You have to understand what you are looking for and take care of several things: weighting windows; analysis of spectrum sparsity; only paying attention to phase of significant spectrum components; etc.
I'm looking for a difference, and I see difference. You are right that I probably had to take amplitude spectrum into consideration as well (if amplitude component is close to zero, phase component is practically meaningless).
At least I believe it is obvious that mp3 does not preserve the phase spectrum in exact sense. The question is how significant the introduced difference is, in the context of this thread.
For what I know, phase spectrum distortion is almost universally inaudible, yet it can result in significant change of the waveform shape. So if mp3 encoding can, at least potentially, indroduce phase distortion resulting in such changes, then it would invalidate the "test" being discussed even further. That's why I'd like to clarify this issue.
OK. I did the experiment suggested by Arnold B. Krueger, with the square wave, and i think it has been most revealing in terms of favorably validating the statement "Subtracting MP3 from WAV reveals artifacts". Judge for yourself:
http://www.multiupload.com/YDPTV9XPH8 (http://www.multiupload.com/YDPTV9XPH8)
Can't download it. I keep getting offered a download of "whitesmokewirter.exe".
Method: An "ideal" PCM square wave is generated with MATLAB @ 440Hz (although a mathematical continuous reconstruction of the signal bears a pseudo-square wave suffering from aliasing.) The wave is then converted to LAME CBR 320kbps (best quality), and V5 VBR 130kbps ('threshold' quality). Finally, both signals are subtracted from the original PCM source to (hypothetically) reveal their artifacts.
I don't know if the Lame parameters are optimal, but at least this is some kind of practical example.
Observations:
1. MATLAB PCM square wave rings @ A440, sounding very digital.
Likely a false claim. If you do this experiment properly there will be ringing on the 440 Hz square wave, but the ringing will be at the Nyquist frequency which in this case is 22050 Hz. There is a considerable history of people comparing square waves coded 44/16 and comparing them to square waves generated in the analog domain with accurate sources. Results of any reasonable subjective comparison has always been random guessing.
2. LAME 320kbps MP3 sounds indiscernibly like the original PCM signal.
OK.
3. LAME V5 130kbps sound very similar to the original PCM signal, with a slight added 'hush'.
If memory serves, 128 kbps is not generally considered to be a sufficient bitrate for reliable transparent coding in general.
4. Error signal PCM-320kbps is inaudible, explaining the absence of audible discrepancies described in (2), thus supporting transparency in this instance.
My perspective on this is that we know that if *all* kinds of error are small enough, then audible differences for sure won't be heard.
However, some errors are far more audible than others, so this kind of a criteria will turn out to be overkill in many cases.
For example, putting *all* errors 100 dB down is always sufficient to *guarantee* audible transparency, but putting them *only* 80 dB down generally gets the job done. OTOH, errors that are 60 dB down will often be audible. But, if I get to pick the kind of error, I can put it only 60 dB down and everybody will be fooled all of the time. An example of an error that is 60 dB down is a broadband 0.1 dB level shift. Nobody ever hears it. Period.
5. Error signal PCM-V5 sounds exactly like the noisy 'hush' described in (5), supporting the fact that artifacts are indeed exposed by this method, in this instance.
This statement may be true, but it is a very narrow statement.
Because of the extreme variation in the audibility of various common kinds of errors, simple signal subtraction is not a very efficient way to rate audio equipment and software. I think that it is even a far worse criteria than THD, certainly no better.
Well then, I'd like to know what exactly do you mean by "phase-preserving quantization", and how it can be proven mathematically.
Consider quantization of waveform samples of a signal. This operation is considered phase-preserving because there's nothing that can flip or rotate phase in it. This quantization can be considered as "result = original + quantization noise", where quantization noise is typically small.
Mp3 is similar: it splits the signal into a few hundred subband signals and quantizes them. There's nothing that can flip or rotate phases in this operation.
I'm looking for a difference, and I see difference. You are right that I probably had to take amplitude spectrum into consideration as well (if amplitude component is close to zero, phase component is practically meaningless).
Yes, that's what I meant.
At least I believe it is obvious that mp3 does not preserve the phase spectrum in exact sense.
Of course it doesn't. But even adding a tiny amount of noise alters the phase spectrum of the signal. By "preserving phase information" I mean "preserving phase information of significant spectral components, down to a quantization step". There's no strict definition of this concept.
For what I know, phase spectrum distortion is almost universally inaudible, yet it can result in significant change of the waveform shape. So if mp3 encoding can, at least potentially, indroduce phase distortion resulting in such changes, then it would invalidate the "test" being discussed even further. That's why I'd like to clarify this issue.
No, mp3 does not introduce phase distortion that could invalidate the subtraction test (unless you are using
Intensity stereo mode, which is absent in Lame and many other encoders).
Why even bother with the square wave example? It is totally inadequate for the given context and the poster is too full of himself to even notice. Instead take two such tightly spaced impulses, that the first masks the second psycho-acoustically. For humans it will be 100% indiscernible from a version where a capable lossy encoder, such as LAME, has removed the second impulse to save space. The difference file of both samples will naturally include the second but not the first impulse at full amplitude. The difference file will indicate clear audibility of the removed impulse, while the actual lossy file would be 100% indiscernible by ears to the original. A very simple but clear example of why difference files are totally inadequate to evaluate psycho-acoustic compression.
A very simple but clear example of why difference files are totally inadequate to evaluate psycho-acoustic compression.
An apparently serious attempt is being made by Soundexpert:
http://soundexpert.org/ (http://soundexpert.org/)
It would be interesting to check if their results (codec ratings (http://soundexpert.org/encoders)) are much different from those found by HA.
http://www.multiupload.com/YDPTV9XPH8 (http://www.multiupload.com/YDPTV9XPH8)
Can't download it. I keep getting offered a download of "whitesmokewirter.exe".
Observations:
1. MATLAB PCM square wave rings @ A440, sounding very digital.
Likely a false claim. If you do this experiment properly there will be ringing on the 440 Hz square wave, but the ringing will be at the Nyquist frequency which in this case is 22050 Hz. There is a considerable history of people comparing square waves coded 44/16 and comparing them to square waves generated in the analog domain with accurate sources. Results of any reasonable subjective comparison has always been random guessing.
(Technical corrections: About your download, i can't really help you as it works on my side, you can try this mirror: http://hotfile.com/dl/81114913/4dd8de4/square_wave.zip.html (http://hotfile.com/dl/81114913/4dd8de4/square_wave.zip.html) ; Wrong choice of wording, by "rings" i just mean "sounds", or is periodic at that freq, rather. Not talking about aliasing.)
Perhaps you'll be surprised to know that the MP3 standard doesn't specify any normative implementation of the psyacoustic model; It's all up to the coder to decide.
"Perhaps you'll be surprised to know..."?!?
In retrospect, having browsed the next two pages, I think someone should have binned the guy or girl right there.
Coming to a board where lossy codecs are invented, designed, developed and tuned, and throwing in a comment like that, is either ignorance or trolling.
Combined with the arrogance that he or her is going to teach us, and not learn, and this could only end one way...!
Cheers,
David.
How many more rounds of cluelessness can this thread handle?
I think HA needs a "like" button!
Cheers,
David.
The very foundation of signal theory lies upon the presupposition that signals can be added and subtracted together.
Only in linear systems.
I look forward to your proof that the ear is a linear system, and hence that your subsequent analysis is relevant.
Hopefully you know this is nonsense. You don't really think the human ear is a linear device, do you?
Good. So the ear is non-linear, and the superposition principle doesn't hold. That might just give you a clue as to why your undergraduate mathematics don't say anything useful about the errors introduced by lossy coding (or anything else!) in as much as they relate to a human
listening to the resulting audio.
If you have some use for mp3 other than "humans listening to audio", maybe you should have shared that at the start.
Cheers,
David.
A very simple but clear example of why difference files are totally inadequate to evaluate psycho-acoustic compression.
An apparently serious attempt is being made by Soundexpert:
http://soundexpert.org/ (http://soundexpert.org/)
It would be interesting to check if their results (codec ratings (http://soundexpert.org/encoders)) are much different from those found by HA.
The last paper of the methodology "Difference level" dictates exactly how you can judge lossy quality by taking the difference signal and amplify it until it becomes audible. Higher the amplification, higher the quality. This is a direct application of how the difference signal can be used as a tool for quality assessment.
http://soundexpert.org/testing-methodology (http://soundexpert.org/testing-methodology)
The last paper of the methodology "Difference level" dictates exactly how you can judge lossy quality by taking the difference signal and amplify it until it becomes audible. Higher the amplification, higher the quality. This is a direct application of how the difference signal can be used as a tool for quality assessment.
http://soundexpert.org/testing-methodology (http://soundexpert.org/testing-methodology)
Excerpts from the actual paper about difference level you have cited here:
Nevertheless, today we all are clever enough not to measure perceptual audio quality with any simple instrumental parameter or even a set of them.
Non-linear nature of human hearing and masking thresholds makes perception of gradually unmasked artifacts uneven.
Since the exploitation of masking is a fundamental aspect of modern psy-models, I wonder how you still not understand the ineptness of your proposal.
Twice now (two different people) a direct challenge to the fundamentals of your argument have been posted, JapanAudio, pointing out that hearing is a non-linear system.
How about you refrain from picking minutia to respond to for a while and answer this challenge to the foundation of your argument.
EDIT: punctuation.
A very simple but clear example of why difference files are totally inadequate to evaluate psycho-acoustic compression.
An apparently serious attempt is being made by Soundexpert:
http://soundexpert.org/ (http://soundexpert.org/)
It would be interesting to check if their results (codec ratings (http://soundexpert.org/encoders)) are much different from those found by HA.
I read the paper and I think I see its point. The basic weakness of the paper is that it is somewhat based on a study of correlation. There's an old saying "Correlation is not the same as causality". In this case we may find that if we examine contemporary, competitive encoders, we may find that the ones with smaller errors are also the ones that get high scores in good subjective testing. What should be inferred from this?
Should we infer that the encoder with the lower amount of error is always the one that will sound best? Should we infer that given two coders that have similar amounts of error, they can reasonably be expected to sound equally good? Should we infer that the coder with higher error is always one that sounds worse? Should we infer that since all coders produce finite error signals, their coded files all must be scenically distinguishable from the source .wav file?
We've seen some or all of these assertions made in recent times.
My innate skepticism says that the evidence presented is not sufficient to promote any of the above assertions into global truths, even though they may be true for a reasonable-looking collection of good modern coders. We can think of enough counter-examples. We can think of enough possible reasons why counter examples may always exist.
The last paper of the methodology "Difference level" dictates exactly how you can judge lossy quality by taking the difference signal and amplify it until it becomes audible. Higher the amplification, higher the quality. This is a direct application of how the difference signal can be used as a tool for quality assessment.
http://soundexpert.org/testing-methodology (http://soundexpert.org/testing-methodology)
Excerpts from the actual paper about difference level you have cited here:
Nevertheless, today we all are clever enough not to measure perceptual audio quality with any simple instrumental parameter or even a set of them.
Non-linear nature of human hearing and masking thresholds makes perception of gradually unmasked artifacts uneven.
Since the exploitation of masking is a fundamental aspect of modern psy-models, I wonder how you still not understand the ineptness of your proposal.
I think taking quotes out of context isn't really fair, anyway i am well aware that the author states that this method of "quality assessment" has to be further researched. Thankfully, my point doesn't have anything to do with quality assessment; I was just trying to emphasize on the title of the thread, precisely, that difference signals are the artifacts. Whether they be audible or not, good or bad sounding, and comparing them to judge quality is none of my business. Note that the author never counters the fact that difference signals reveal artifacts (however beautiful or inaudible they might be).
I was just trying to emphasize on the title of the thread, precisely, that difference signals are the artifacts.
The point of lossy encoding is to save space by exploiting properties of the human auditory system, which allow us to hide the actual data reduction. Inaudible artifacts are a
necessity in this endeavour. So what is the point of "emphasizing" anything about artifacts regardless of their audibility?
Don't make the backpedaling look even more pathetic than the failed argument.
A very simple but clear example of why difference files are totally inadequate to evaluate psycho-acoustic compression.
An apparently serious attempt is being made by Soundexpert:
http://soundexpert.org/ (http://soundexpert.org/)
It would be interesting to check if their results (codec ratings (http://soundexpert.org/encoders)) are much different from those found by HA.
I read the paper and I think I see its point. The basic weakness of the paper is that it is somewhat based on a study of correlation. There's an old saying "Correlation is not the same as causality". In this case we may find that if we examine contemporary, competitive encoders, we may find that the ones with smaller errors are also the ones that get high scores in good subjective testing. What should be inferred from this?
Should we infer that the encoder with the lower amount of error is always the one that will sound best? Should we infer that given two coders that have similar amounts of error, they can reasonably be expected to sound equally good? Should we infer that the coder with higher error is always one that sounds worse? Should we infer that since all coders produce finite error signals, their coded files all must be scenically distinguishable from the source .wav file?
We've seen some or all of these assertions made in recent times.
My innate skepticism says that the evidence presented is not sufficient to promote any of the above assertions into global truths, even though they may be true for a reasonable-looking collection of good modern coders. We can think of enough counter-examples. We can think of enough possible reasons why counter examples may always exist.
You're absolutely right. From the beginning i was never trying to talk about quality issues, because its a touchy subject that isn't fully regulated, and probably can't be.
Thankfully, my point doesn't have anything to do with quality assessment; I was just trying to emphasize on the title of the thread, precisely, that difference signals are the artifacts.
So all you're trying to say is that if
c = a + b
then
b = c - a
?
Cheers,
David.
An apparently serious attempt is being made by Soundexpert:
http://soundexpert.org/ (http://soundexpert.org/)
It would be interesting to check if their results (codec ratings (http://soundexpert.org/encoders)) are much different from those found by HA.
This has been discussed on the forum on more than one occasion. While Serge may take his method seriously, HA does not.
Ok, I believe now we can really close this thread.
To JapanAudio:
You've been told already the correct use of the term "masking".
Now I would like to add the correct use of the term "artifact".
An artifact is a difference, but not all differences are artifacts. Concretely, an artifact is an unwanted sound which is heard in the context of the decoded signal, that was not present in the original signal.
This is why there has been so much emphasis in this thread in disprooving your claim. The difference signal may be helpful for codec developers to detect a problem in their codecs, but never as a way to determine its quality (except if the codec in question is a lossless codec, in which case we know the difference file has to be digital silence).
Edit: Also, there's another problem with the difference signal if we are going to take it as "what can be heard". The difference signal can have both, things that the lossy encoder added as well as things that it missed (the hush in your square wave example is an example of added sound, while the lowpass filter would be an example of removed sound).
This, coupled with masking makes the difference signal a lot less useful to know what artifacts can be heard on the decoded signal.
I was waiting for you to chime in, JAZ. Thanks, and thanks to everyone else.
Ok, I believe now we can really close this thread.
Agreed.