HydrogenAudio

Lossy Audio Compression => MP3 => MP3 - Tech => Topic started by: paracent on 2013-08-14 20:42:37

Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-14 20:42:37
Hello,

For encoder(s) testing purpose I created a 19.3kHz >>tone<< , 48kHz, 10 sec sample wav file., using 'sox'.
After a number of different tests for quality/sound reproduction ( doing on aac he/h2 & ogg ) I decided to test it on mp3 using lame 3.98 & 3.99.5
All samples were imported into 'baudline' to build a spectrogram. ( total about 50 )
when I got to lame, I set the low-pass filter to 19.5, 20, 21, and 22kHz, after importing the 20 and lower low-pass samples into baudline, my first words were: WTF?

here is the original WAV: ( all just 19.3khz tone )
(http://s21.postimg.org/r6ut0ubxj/sox_19_3k_Hz_48k_Hz.png)

this is a 48kHz / 21kHz low-pass MP3: ( lame )
(http://s8.postimg.org/mvdvobhxx/sample_19_3khz_48khz_lp21_mp3.png)

this is the, same sample, but 19.5kHz low-pass ( lame ) ( this is when I asked WTF? )
(http://s9.postimg.org/blrlciqb3/sample_19_3khz_48khz_lp19_5_mp3.png)


in all test cases for  OGG / AAC / MP3 I encoded to the specific format, and back to WAV - to make sure the results are constant, using ffmpeg to reencode )
even if I do not re-encode back to wav, the results are the same on LAME for low-pass lower, or equal to 20kHz.

now, this is just one tone 19.3kHz... Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower.

The tests I am doing are about the encoder accuracy, and of course, if the accuracy is higher, quality of the encoded file will be also better.
So far, the results I got, put WAV, FLAC and OGG as the most accurate, closest to the original sample formats. ( various samples, including single, high freq. tone(s) )

None of other formats I have been working on shows such "strange" behavior...

If anybody has an explanation, to my "WTF" question, I would really appreciate....

Mike

edit:
used the following LAME opts:

lame -V0 --lowpass 19.5 output.wav output_v0_lp19.5_99lame.mp3
and
lame -V0 --lowpass 21 output.wav output_v0_lp21_99lame.mp3
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: saratoga on 2013-08-14 21:26:21
Clipping?  If you use a tone at -6dB does it work normally?
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-14 21:35:32
well, this is not about clipping...

look at the last screenshot - you should see only 19.3kHz there, what about the rest of tones that "appeared from nowhere" after using LAME, lo-pass =<20kHz
this is the issue I am trying to find an explanation...

Thanks!



Clipping?  If you use a tone at -6dB does it work normally?

Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: saratoga on 2013-08-14 21:41:33
look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes.  Does it clip? 
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: mixminus1 on 2013-08-14 21:50:52
Quote
Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower

I'm not imagining very many music collections that have a substantial amount of material at a 48 kHz sampling rate (to say nothing of 19 kHz content @ 0 dBFS) - try your experiment at 44.1 kHz...

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...and as I implied above, the artifacts are greatly reduced at 44.1 kHz @ 0 dBFS.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-14 21:52:21
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:

n_samples: 480431
mean_volume: -3.1 dB
max_volume: 0.0 dB
histogram_0db: 135966

>no clipping< under no assumptions - I do not do all these tests assuming anything...



look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes.  Does it clip?

Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: DonP on 2013-08-14 21:53:31
Clipping?  If you use a tone at -6dB does it work normally?


Shouldn't clipping just produce harmonics of the fundamental tone?  Still a useful question.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-14 22:49:34
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:
Quote
>no clipping< under no assumptions - I do not do all these tests assuming anything...
But you should. Lossy encoding can introduce clipping via the creation of inter-sample overs, which does not require clipping in the original file but is especially favoured by the latter having a lot of samples close to 0 dBFS. You should assume that.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: saratoga on 2013-08-14 23:06:08
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped.  You need to lower the level. 

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right.  Good guess. 
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-14 23:49:56
Yet another example of someone not understanding lossy compression and attempting to second-guess the encoder by adding additional unnecessary switches.

For encoder(s) testing purpose...

Any results derived from these tests will be completely useless unless the entire purpose is to ABX these specific tones, in which case you should be using ABX tools, not frequency analysis.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 00:05:38
Did more tests..
Well, without lowering by -3dB,
with 20.1kHz lo-pass there is no change... still looking bad, same result as 20kHz
with 20.2kHz lo-pass it looks much better, the "foreign" frequencies go up to  max -95 ~ -100dB

- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely? ( or to -95dB max )?
this is what happened, and this is also my original "wtf" question.




no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped.  You need to lower the level. 

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right.  Good guess.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 00:11:57
looks

This is the problem, full stop.

We listen with our ears, not our eyes.  How the signal "looks" is irrelevant.

Brush up on TOS #8 (http://www.hydrogenaudio.org/forums/index.php?showtopic=3974) before posting on this topic again.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 00:30:33
I guess a better place to ask this question would be the lame mailing list, sorry for skipping the TOS MOD.
on the other hand, this information will prevent me from trusting my ears today, and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late... I guess I am not the only one that found himself in such position.
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)



looks

This is the problem, full stop.

We listen with our ears, not our eyes.  How the signal "looks" is irrelevant.

Brush up on TOS #8 (http://www.hydrogenaudio.org/forums/index.php?showtopic=3974) before posting on this topic again.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 00:33:39
TOS, either agree or gtfo
Pretty much.

Quote
regardless on what you got to say
If it isn't compliant with our terms we aren't interested.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 00:36:26
on the other hand, this information will prevent me from trusting my ears today
Why? Do you think your eyes know how to hear better than your ears do? Can you hear a difference? If not, what you see is irrelevant.

Quote
and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late...
What do you mean?

Quote
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)
Reminds you of what? Come out and say it. Being passive-aggressive by implying through ellipsis isn’t going to get you anywhere.

[edit]

To be clear, I at least am not saying you’re not allowed to talk about observations that aren’t audible, or that this is desirable behaviour for an encoder, but those things are very different from concluding that the effect is audibly problematic without actually performing a valid test for audibility. And because we’re overwhelmingly interested in things that are audible, we tend not to spend much time worrying about the visual effects of lossy compression, which have no correlation with its effects upon the ears.

Personally, I’d be most interested in a discussion of whether this is unavoidable in MP3/LAME, rather than worrying about its implications for normal music; I’m sure we’d have heard all about those if they existed.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 00:40:54
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Propheticus on 2013-08-15 00:42:08
If you actually read tos 8 you'd see it's there for a good reason. This forum is full of discussions about this subject too.

As to this 'problem': Just lower your input a bit, as suggested to you but you stubbornly refused to do. Going from sources at 0dB to mp3 can add clipping. Just lower to -3dB and check again.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 00:47:18
FWIW, I don't think this is resulting from clipping.  Feel free to generate the samples and have a look for yourself.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Makaki on 2013-08-15 00:57:38
This discussion would be a lot more useful if someone added, at least a quick ABX of said samples. It would be an interesting Visual observation if it is followed by the hearing observation.
Being a single tone, I don't think the ABX would take that long.

PS: Personally, I can't hear a 19.3kHz tone.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 01:14:17
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 05:39:12
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse? so you telling me now what am I allowed to listen to?

Quote
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance"


is this a quote that you just apply to anything? is using a 20/21 kHz lo-pass a 'misuse' of switches? I see... are you really using 16.5kHz lo pass for all your music needs? - this would explain...

I re-encoded a couple of samples to about 300 result files, used 5 codecs, different options for ogg, aac... and the only mess, that is not only visible, but sounds terrible, is when lame is used in a certain condition...

It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all   

Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: saratoga on 2013-08-15 07:32:13
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse?


MP3 isn't made to compress clipped synthetic signals.  You're supposed to give it some headroom, or else use replaygain.  If you don't, the results will distort. 

I'm surprised they're so badly distorted here, but yes, you're doing it wrong

so you telling me now what am I allowed to listen to?


No one has said anything like that.

It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age.


If you're not willing to accept the answers to questions, you should probably not ask them in the first place.  Its just a waste of everyone's time to argue with you in that case.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: [JAZ] on 2013-08-15 12:00:39
- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely?


If you don't pay attention to what you do, and don't listen to people that know how things work, then, you're expected to get invalid results.
Now, do the test again and read what LAME is telling you in the command line about the exact lowpass frequency it is using, and not what you tell it to use.

Next, I did the same test with LAME 3.99.5 (windows, 32bit rarewares build) -V0, (stereo, 48Khz 19.3Khz 0dBFS, filtered with --lowpass 19.5, with a 16bit and a 24bit file), and in all cases, the noise was below 90dBs, much below than your case.


If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".


Sure he's changing the defaults, and who knows what more is he doing since I can't replicate it, but the change is not that far from defaults. I.e. Using -V1 instead of -V0 already uses a filter in that range.




It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all 


If you had, for a moment, the thought that Hydrogenaudio is a place where we simply sit down and accept whatever "dogma" is established, you should learn a bit more about us before continue.

Said that, what we have learned in these years is to not blindly believe in what a spectogram shows when talking about lossy compression, and to avoid focusing on artificial samples when testing the reliability of these codecs.

What you found might or might not be a misfunction of the filter, but so far, my results don't show the same than yours.




Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 12:55:17
Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).
Fantastic. Now we can start getting somewhere. Thanks for the explanation!
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 13:17:10
perfect, "start getting somewhere" sounds great ( even if it is distorted ). a bumpy road, maybe a couple of wrong expressions I used, but I am glad, that there is an explanation other than "-3dB". Problems are found on "per case basis", they often can be "hidden" similar to this case, otherwise Windoze should never show a BSOD. Suggesting workarounds in some cases is acceptable, and often an only way, but I did not start the topis to hear workarounds... If there is a confirmed bug/issue with Lame, regardless from the comments on how "improper" my encoding attempts are, I will be more than happy that this topic was started. Don't you agree? Thanks again!
Mike

[edit]

this is the source wav and two mp3 files, at 20.1 and 20.2kHz lo-pass

original wav (http://www.mediafire.com/download/m0c6kjk7svkth7d/sample.sox.48khz.19.3khz.wav)
20.1kHz lo pass mp3 (http://www.mediafire.com/download/r8ksklm74css9sy/out_v0_lp20.1_99lame.mp3)
20.2kHz lo pass mp3 (http://www.mediafire.com/download/dyanulb8gm9zdq0/out_v0_lp20.2_99lame.mp3)

thanks!
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: 2Bdecided on 2013-08-15 13:43:48
looks

This is the problem, full stop.

We listen with our ears, not our eyes.  How the signal "looks" is irrelevant.

Brush up on TOS #8 (http://www.hydrogenaudio.org/forums/index.php?showtopic=3974) before posting on this topic again.
Generally, especially when you don't know what you're doing, graphs of any kind are a terrible way to judge lossy codecs.

However, in this case the spectrogram shows silence below 19kHz in the original, and a signal hitting -40dB right across the audible band after encoding+decoding. I don't need to listen to know that I can hear crap at -40dB in the audible range when there's no other signal to mask it. And [JAZ] has used a spectrogram to diagnose the problem.

It would be kind of easy to listen though.  That said, I'd be careful listening to artefacts added to a 19kHz sine wave - depending on the level of the artefacts and your hearing, you might inadvertently raise the volume level too high and fry your tweeters.


btw, the point the OP seems still to have missed, and it's been said several times in this thread, is that because lossy codecs intentionally change the waveform (though hopefully not the sound), you need to keep the signal away from 0dB to avoid clipping. Usually the clipping is caused by the adaptive lossiness of the encoder rather than it's "fixed" filter, and usually the clipping is inaudible, but it's still good practice and not news.

Cheers,
David.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 14:02:13
I agree with everything David said. And yes, paracent, if your test has revealed a problem with LAME that can be fixed (which I presume it can), then it will end up being very useful after all.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 14:41:30
are you really using 16.5kHz lo pass for all your music needs? - this would explain...

How I encode my music is determined through listening, not through looking.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 14:45:43
"Generally, especially when you don't know what you're doing, graphs of any kind are a terrible way to judge lossy codecs."

In general, this might be right. Working on a few hundred samples in last a few days, and seeing something completely unexpected in the "Lame", again, made me open this topic. Different codecs, different features, different profiles, different approaches to reproduce sound in a lossy environment can make people with a bit less knowledge completely lost when looking at the spectrograms. For example look at AAC_PLUS / AAC_HE, what it does with the frequencies above ~8-10kHz, especially in low bit rates... it is done on purpose, to provide a person with a feel of a Clean, decent quality sound - fine & great! But in the Lame "case", I do not think that this was done on purpose, and if it was, there might be a need to re-visit it. Assuming "not knowing what you are doing", might be a sign of proper judgement, but at the same time it might be a simple sign of ignorance.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 14:51:33
"How I encode my music is determined through listening, not through looking. "

I agree with you, to a point. When using a lossy codec, you probably want to use a codec, setting, profile, feature, whatever, that is >>less<< lossy than the other one, while not being as space hungry as FLAC or WAV. At least this is my goal, and beside my ears, I also need a proof - we live in digital world, and a proof in this case easy to get, rather than just relying on what I "hear". In the longer run, If I do the job right today, ( at least I hope ) I will not cry tomorrow, in a year or two...

Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 15:04:06
When using a lossy codec, you probably want to use a codec, setting, profile, feature, whatever, that is >>less<< lossy than the other one

How is this being judged?

You found that a full-scale sinusoid gives problems with a low-pass setting that is 1% above it. Despite what David said, I don't see this a problem with Lame, rather I see this as another example of how to break a codec.

Besides your eyes, what has given you the idea that -V0 without a user-configured low-pass doesn't provide enough cow bell?
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 15:15:23
When using a lossy codec, you probably want to use a codec, setting, profile, feature, whatever, that is >>less<< lossy than the other one

How is this being judged?

You found that a full-scale sinusoid gives problems with a low-pass setting that is 1% above it. Despite what David said, I don't see this a problem with Lame, rather I see this as another example of how to break a codec.

Besides your eyes, what has given you the idea that -V0 without a user-configured low-pass doesn't provide enough cow bell?



V0 was used only to compare, as an example, along with 50 other tests I did on Lame, which I am not planning to use anyway.

here the answers to your questions:

1> V0 without lo-pass above ~20.2 kHz introduces whatever it is it introduces, which is not acceptable.
2> testing different quality profiles ( all V ) with forcing low-pass in 18-22kHz range I was comparing the results and ended up with the V0

Gargamel, what else do you want to tall me , to convince me, that whatever I do makes no sense?
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 15:26:55
You didn't answer either of my questions.

Do I think you are judging codecs in incorrectly? Without any hint of doubt, yes, absolutely!

Don't use Lame. I couldn't care less.  If you want to call mosquito tones music that is your business.  I don't think this is how the developers see it.

So humor me, what does Lame -V0 (no --lowpass) introduce?
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: [JAZ] on 2013-08-15 15:41:41
On the technical side of the problem:

I believe this might not get fixed, not because of being a special case, but because of the way it is implemented.

This is not an IIR or FIR lowpass filter working in the time domain, but instead, the filter is applied over the calculated band energies in the mdct tool.
In some way, it could be considered as a filter that acts in the frequency domain, and the mirroring in question is similar to the artifact of a resampler with a soft filter. (with the difference that in this case it is mirrored above the cutoff, not below).


I would like to mention that the problem happens in the whole frequency range.
For example, a 15.5Khz sine sampled at 48Khz and using lame -V0 --lowpass 16 (which means "Using polyphase lowpass filter, transition band: 15677 Hz - 16258 Hz") also shows the mirroring. (In this case, at 16Khz).

Usually, we should expect signals at these frequencies to be down -30dBFS or more, and the aliasing intensity to decrease fast the further from the frequency cutoff point.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 15:42:20
Gargamel: "Humor me" - This would be off-topic. Find your amusement somewhere else.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 15:44:56
"Humor me" - This would be off-topic. Find your amusement somewhere else.

Really? Is this really how you wriggle out of a request to confirm that the default settings do not introduce the problem you reported? By taking issue with a metaphorical item of language? In case you are unaware, to humour someone in this context means to put up with their request, even if you are not personally keen on it, in order to remove uncertainty from the situation.

I agree with you that the problem with the non-default lowpass is worth investigating. I do not, however, agree with you continuing to passive-aggressively create diversions to avoid actually replying to other members.

And his username is greynol. I suspect you are leveraging his avatar to make more unstated implications about his character. You should probably stop that. If that amuses you, perhaps you are the one who should go elsewhere for humour.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 15:50:08

On the technical side of the problem:

I believe this might not get fixed, not because of being a special case, but because of the way it is implemented.

This is not an IIR or FIR lowpass filter working in the time domain, but instead, the filter is applied over the calculated band energies in the mdct tool.
In some way, it could be considered as a filter that acts in the frequency domain, and the mirroring in question is similar to the artifact of a resampler with a soft filter. (with the difference that in this case it is mirrored above the cutoff, not below).


I would like to mention that the problem happens in the whole frequency range.
For example, a 15.5Khz sine sampled at 48Khz and using lame -V0 --lowpass 16 (which means "Using polyphase lowpass filter, transition band: 15677 Hz - 16258 Hz") also shows the mirroring. (In this case, at 16Khz).

Usually, we should expect signals at these frequencies to be down -30dBFS or more, and the aliasing intensity to decrease fast the further from the frequency cutoff point.


JAZ - thanks a lot for the reply - is the mirroring at a higher, not "exepected" dBFS a property / design of Lame? since I have not seen similar effects on AAC or Vorbis for example.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: paracent on 2013-08-15 15:53:55
db1989 - I am no here to find amusement, an not here to humor anybody, especially on request... My response was not right, neither his/her request.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 15:58:09
What was wrong with the request to confirm that the default settings do not produce the problem, and the question about why you feel the need to use non-default settings? I understand why you might want to use customised settings for testing purposes, but greynol is making the point that the default settings are default for a good reason. All you were asked was to answer those questions; to my mind, if you dance around them and avoid doing so, people are justified in repeating them. That does not mean anyone is making fun of you, victimising you, or anything else.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: [JAZ] on 2013-08-15 16:01:52
It is a property of the implementation that is used to do the filtering, except that the increase in amplitude (in time domain) of the original signal is not a property per-se, but a consequence of mixing several signals (in this case, the sine, and the mirrored aliasing, which is the real property).

Other codecs probably use a FIR filter on the input, or other means.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: dhromed on 2013-08-15 16:08:13
Unless I'm way off base, the pictures look like regular aliasing to me, though, with some added stripes due to the spectrograph's window function. Wouldn't be surprising given a 19.5kHz low-pass and a 19.3KHz tone.

not here to humor anybody,


I sense a language barrier (superfluous commas and quotes being a strong hint!). 'humor me' has nothing to do with entertainment or laughter. It just means, "Why not? Won't hurt to let me know." or something to that effect.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: 2Bdecided on 2013-08-15 16:17:40
beside my ears, I also need a proof
An ideal psychoacoustic-based codec will give the maximum possible perceived quality at a given bitrate, or deliver a given quality level (e.g. no audible difference from the original) at the lowest possible bitrate.

Given that some things visible on graphs are completely inaudible, and other easily audible things are invisible on certain graphs, how exactly do you propose to use a graph to judge which lossy codec is working best? The final arbiter of lossy coding is the human ear, because it is the human ear that the lossy codec is trying to trick.


The graphs you posted earlier in this thread are useful because they demonstrate a particular problem which, from a graph, we can clearly say is audible. The audibility of most of the changes caused by a lossy codecs cannot be predicted from their appearance on graphs.

This has all been discussed many times here. Please go and look through the FAQ. Top right, near search (another useful tool  ).

Cheers,
David.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Rescator on 2013-08-15 16:52:15
the point the OP seems still to have missed, and it's been said several times in this thread, is that because lossy codecs intentionally change the waveform (though hopefully not the sound), you need to keep the signal away from 0dB to avoid clipping. Usually the clipping is caused by the adaptive lossiness of the encoder rather than it's "fixed" filter, and usually the clipping is inaudible, but it's still good practice and not news.



I can further attest to clipping being very important to eliminate. Having developed a shoutcast streaming player which actually shows the peak after decoding. (unclipped/floating point decoding so peaks above 0dBFS can be shown), I have seen max peaks reach as high as over +6dBFS.
Mind you that was with the source not using ReplayGain, and 48kbit HA-AAC, and FAAD2 based decoding. These are probably intersample peaks that manifest themselves during the encoding and then decoding process.
So not hitting 0dBFS does not guarantee this won't happen with lossy formats. Using -3dBFS or even as low as -6dBFS might be needed.
It is also worth nothing that it is highly compressed (as in it's dynamics) music that exhibit the most extreme overs.
Also note that a impulse can do really odd things. I managed to create sample with a impulse that causes a, what was it +11dBFS intersample peak, and that was in a lossless format.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 16:52:36
FWIW, I don't think this is resulting from clipping.  Feel free to generate the samples and have a look for yourself.

Like [JAZ] had done initially, I went back and decoded to floating point.  The resulting signal is the original tone with the imaged tone.  Alternately, I could have used foobar2000 or MP3Gain to reduce the global gain before decoding to fixed point.

David will not claim there is sufficient evidence of audibility.

EDIT:
Regarding the importance of eliminating clipping, I'm still waiting for another example of a published title of music that causes an audible problem when a lossy-encoded version is decompressed to fixed point.

This is a non-issue.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Rescator on 2013-08-15 17:45:05
Regarding the importance of eliminating clipping, I'm still waiting for another example of a published title of music that causes an audible problem when a lossy-encoded version is decompressed to fixed point.

Lana Del Rey's "Born To Die" or "Paradise" are two possible candidates. I'm not sure if it was just one particular iTunes release or not (which was said to have been fixed later). But several have reported massive clipping/crackling (audible) on her tracks. A victim of the Loudness War. Daft Punks Tron Legacy soundtrack too though in their case it's hard to know if they did it on purpose or not, but the CD release seem to suffer from the Loudness War a little too. I seem to recall Paradise was said to be a little worse than Born To Die.

Quote
This is a non-issue.

Though one could guess clipping makes the job of a lossy encoder a tad more difficult bit allocation-wise (then again, any modern encoder and decoder should have code to handle that as well).
I tried to search the forum but the only thing I found was a question here http://www.hydrogenaudio.org/forums/index....amp;#entry24453 (http://www.hydrogenaudio.org/forums/index.php?showtopic=2501&st=0&p=24453&#entry24453) though I have to admit I have not done a exhaustive forum search.

If you mean just the odd clipping here or there then yeah, folks will most likely never notice (too brief) and it should not impact a encoder/decoder either. (again, too brief)
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 17:50:26
Clipping in the mastering stage, which necessarily means prior to lossy compression, is completely irrelevant here.

The challenge was to provide a track that is properly mastered and therefore does not clip, but that is forced to exhibit audible clipping by subsequent lossy compression.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Rescator on 2013-08-15 18:43:00
Clipping in the mastering stage, which necessarily means prior to lossy compression, is completely irrelevant here.

The challenge was to provide a track that is properly mastered and therefore does not clip, but that is forced to exhibit audible clipping by subsequent lossy compression.


Oh right! Yeah, I highly doubt one will find that outside artificially created ones.
Then again properly mastered is not that descriptive, didn't Iggy Pop on purpose master one of his albums so hot that well, it was the sound that he wanted (or so he said).

But since I'm partly to blame for veering off topic here.
To try and get back on topic. What is seen in that image is an impulse, and those can do weird things.
But a impulse is normally not found that often in normal music (normal being a tad relative here though).


Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: db1989 on 2013-08-15 18:48:04
Nope, according to the OP, it was a sustained tone lasting for ten seconds, not an impulse.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-15 18:52:50
It's a full-scale sinusoid, not an impulse, but you're right it won't be found in normal content.

If I ever needed to compress a 19.3kHz FS test tone I would not use a lossy encoder.  I don't see this as a legitimate reason to choose one codec over another for music, however.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: BFG on 2013-08-16 04:43:29
How's it sound in lame 3.100a2?

(I'd also be curious what the -Y switch does with it, but I suspect that'd be straying off-topic.)
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: Gecko on 2013-08-16 11:58:53
Using the foobar ABX component, the provided mp3 files sound very different. One is silent to my ears, the other is buzzing. See log below.

However, using regular foobar playback, they are both silent. After some tinkering with/without DSP, WASAPI vs. DirectSound, RG on/off, the reason -- apparently -- is that I apply -3dB for non-replaygained files by default, which of course is ignored by the ABX component. When going closer to 0dB gain, the buzzing is added.

I opened the files in Audacity and even after applying a gain of -12dB, one file is still buzzing and the other is silent (when played back via Audacity). I assume the clipping manifests itself when Audacity initially decodes the file to fixed-point (default sample format 16bit) and is therefore unaffected by the following gain. When I set the default sample format to float32 and re-open the file, the buzzing can be heard at FS but disappears with increasingly negative gain.

This should not be news/surprising to any of the regulars but maybe helps the OP to understand what is going on: On this type of synthetic signal, LAME's processing leads to signals which exceed FS which causes audible clipping when decoded to fixed-point PCM. The clipping can be avoided by reducing the gain prior to decoding (e.g. using replaygain) or prior to encoding. As far as I understood [JAZ], the lowpass filter in LAME might actually be the one causing the larger-than-FS signal but can not be easily fixed due to the implementation.

The question remains whether this is in any way relevant for real-world signals. Previous experience suggests it is not. But it seems like good engineering practice to add as little content above FS as possible during the processing.

Code: [Select]
foo_abx 1.3.4 report
foobar2000 v1.2.6
2013/08/16 11:34:25

File A: F:\incoming\music\lame lowpass\out_v0_lp20.1_99lame.mp3
File B: F:\incoming\music\lame lowpass\out_v0_lp20.2_99lame.mp3

11:34:25 : Test started.
11:34:33 : 01/01  50.0%
11:34:36 : 02/02  25.0%
11:34:38 : 03/03  12.5%
11:34:41 : 04/04  6.3%
11:34:43 : 05/05  3.1%
11:34:45 : 06/06  1.6%
11:34:47 : 07/07  0.8%
11:34:49 : 08/08  0.4%
11:34:52 : Test finished.

 ----------
Total: 8/8 (0.4%)
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: greynol on 2013-08-16 14:47:38
How's it sound in lame 3.100a2?

(I'd also be curious what the -Y switch does with it, but I suspect that'd be straying off-topic.)

You tell us.

-Y is not a low-pass.  If you want to create an output of two tones you will likely need to use a low-pass. I doubt -Y would do much of anything as this sample likely isn't complex enough to starve the encoder of precision in order to meet the quality threshold of -V0, even with the ill-advised, overly-tight corner.

Remember, samples beyond FS are a decoding artifact, not an encoding artifact.
Title: strange results using low-pass filter ( lame 3.98/3.99.5 )
Post by: BFG on 2013-08-17 01:04:53
How's it sound in lame 3.100a2?

(I'd also be curious what the -Y switch does with it, but I suspect that'd be straying off-topic.)

You tell us.

-Y is not a low-pass.  If you want to create an output of two tones you will likely need to use a low-pass. I doubt -Y would do much of anything as this sample likely isn't complex enough to starve the encoder of precision in order to meet the quality threshold of -V0, even with the ill-advised, overly-tight corner.

Remember, samples beyond FS are a decoding artifact, not an encoding artifact.

I'll admit that I didn't hear a difference at all.  But I have far from discerning ears - I cannot hear anything in the top band (15.7kHz or higher), and frequently cannot ABx what others can.
I typically use -Y instead of a lowpass filter; I'd rather preserve some semblance of the top band, even if I cannot hear it, in case friends or family listen in.  It didn't sound, or look, like -Y did much in this case...probably because, as you said, it was such a simple sample, and the upper frequencies weren't causing bitrate bloat.