Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: strange results using low-pass filter ( lame 3.98/3.99.5 ) (Read 29624 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Hello,

For encoder(s) testing purpose I created a 19.3kHz >>tone<< , 48kHz, 10 sec sample wav file., using 'sox'.
After a number of different tests for quality/sound reproduction ( doing on aac he/h2 & ogg ) I decided to test it on mp3 using lame 3.98 & 3.99.5
All samples were imported into 'baudline' to build a spectrogram. ( total about 50 )
when I got to lame, I set the low-pass filter to 19.5, 20, 21, and 22kHz, after importing the 20 and lower low-pass samples into baudline, my first words were: WTF?

here is the original WAV: ( all just 19.3khz tone )


this is a 48kHz / 21kHz low-pass MP3: ( lame )


this is the, same sample, but 19.5kHz low-pass ( lame ) ( this is when I asked WTF? )



in all test cases for  OGG / AAC / MP3 I encoded to the specific format, and back to WAV - to make sure the results are constant, using ffmpeg to reencode )
even if I do not re-encode back to wav, the results are the same on LAME for low-pass lower, or equal to 20kHz.

now, this is just one tone 19.3kHz... Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower.

The tests I am doing are about the encoder accuracy, and of course, if the accuracy is higher, quality of the encoded file will be also better.
So far, the results I got, put WAV, FLAC and OGG as the most accurate, closest to the original sample formats. ( various samples, including single, high freq. tone(s) )

None of other formats I have been working on shows such "strange" behavior...

If anybody has an explanation, to my "WTF" question, I would really appreciate....

Mike

edit:
used the following LAME opts:

lame -V0 --lowpass 19.5 output.wav output_v0_lp19.5_99lame.mp3
and
lame -V0 --lowpass 21 output.wav output_v0_lp21_99lame.mp3

 

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #1
Clipping?  If you use a tone at -6dB does it work normally?

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #2
well, this is not about clipping...

look at the last screenshot - you should see only 19.3kHz there, what about the rest of tones that "appeared from nowhere" after using LAME, lo-pass =<20kHz
this is the issue I am trying to find an explanation...

Thanks!



Clipping?  If you use a tone at -6dB does it work normally?


strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #3
look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes.  Does it clip? 

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #4
Quote
Imagine, all the music collections converted to MP3... using lo-pass 20kHz or lower

I'm not imagining very many music collections that have a substantial amount of material at a 48 kHz sampling rate (to say nothing of 19 kHz content @ 0 dBFS) - try your experiment at 44.1 kHz...

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...and as I implied above, the artifacts are greatly reduced at 44.1 kHz @ 0 dBFS.
"Not sure what the question is, but the answer is probably no."

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #5
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:

n_samples: 480431
mean_volume: -3.1 dB
max_volume: 0.0 dB
histogram_0db: 135966

>no clipping< under no assumptions - I do not do all these tests assuming anything...



look at the last screenshot - you should see only 19.3kHz there,


Under the assumption that there is no clipping, yes.  Does it clip?


strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #6
Clipping?  If you use a tone at -6dB does it work normally?


Shouldn't clipping just produce harmonics of the fundamental tone?  Still a useful question.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #7
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:
Quote
>no clipping< under no assumptions - I do not do all these tests assuming anything...
But you should. Lossy encoding can introduce clipping via the creation of inter-sample overs, which does not require clipping in the original file but is especially favoured by the latter having a lot of samples close to 0 dBFS. You should assume that.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #8
no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped.  You need to lower the level. 

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right.  Good guess. 

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #9
Yet another example of someone not understanding lossy compression and attempting to second-guess the encoder by adding additional unnecessary switches.

For encoder(s) testing purpose...

Any results derived from these tests will be completely useless unless the entire purpose is to ABX these specific tones, in which case you should be using ABX tools, not frequency analysis.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #10
Did more tests..
Well, without lowering by -3dB,
with 20.1kHz lo-pass there is no change... still looking bad, same result as 20kHz
with 20.2kHz lo-pass it looks much better, the "foreign" frequencies go up to  max -95 ~ -100dB

- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely? ( or to -95dB max )?
this is what happened, and this is also my original "wtf" question.




no, there is no clipping, and even assumption is not possible - just look at the sine at the bottom, it is reaching 0db:


If it reaches 0dB (that is, full scale) then its certainly clipped.  You need to lower the level. 

Also, what saratoga said - I can replicate your results with a 0 dBFS tone @ 48 kHz, but dropping the level to -3 dBFS completely eliminates the artifacts...


Ha, i was right.  Good guess.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #11
looks

This is the problem, full stop.

We listen with our ears, not our eyes.  How the signal "looks" is irrelevant.

Brush up on TOS #8 before posting on this topic again.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #12
I guess a better place to ask this question would be the lame mailing list, sorry for skipping the TOS MOD.
on the other hand, this information will prevent me from trusting my ears today, and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late... I guess I am not the only one that found himself in such position.
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)



looks

This is the problem, full stop.

We listen with our ears, not our eyes.  How the signal "looks" is irrelevant.

Brush up on TOS #8 before posting on this topic again.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #13
TOS, either agree or gtfo
Pretty much.

Quote
regardless on what you got to say
If it isn't compliant with our terms we aren't interested.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #14
on the other hand, this information will prevent me from trusting my ears today
Why? Do you think your eyes know how to hear better than your ears do? Can you hear a difference? If not, what you see is irrelevant.

Quote
and tomorrow, if I decide to use a different codec, I will find out what mistake I made in the first place, but this might be already too late...
What do you mean?

Quote
Of course, TOS, either agree or gtfo... regardless on what you got to say - reminds me on something (...)
Reminds you of what? Come out and say it. Being passive-aggressive by implying through ellipsis isn’t going to get you anywhere.

[edit]

To be clear, I at least am not saying you’re not allowed to talk about observations that aren’t audible, or that this is desirable behaviour for an encoder, but those things are very different from concluding that the effect is audibly problematic without actually performing a valid test for audibility. And because we’re overwhelmingly interested in things that are audible, we tend not to spend much time worrying about the visual effects of lossy compression, which have no correlation with its effects upon the ears.

Personally, I’d be most interested in a discussion of whether this is unavoidable in MP3/LAME, rather than worrying about its implications for normal music; I’m sure we’d have heard all about those if they existed.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #15
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #16
If you actually read tos 8 you'd see it's there for a good reason. This forum is full of discussions about this subject too.

As to this 'problem': Just lower your input a bit, as suggested to you but you stubbornly refused to do. Going from sources at 0dB to mp3 can add clipping. Just lower to -3dB and check again.


strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #18
This discussion would be a lot more useful if someone added, at least a quick ABX of said samples. It would be an interesting Visual observation if it is followed by the hearing observation.
Being a single tone, I don't think the ABX would take that long.

PS: Personally, I can't hear a 19.3kHz tone.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #19
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #20
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse? so you telling me now what am I allowed to listen to?

Quote
If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance"


is this a quote that you just apply to anything? is using a 20/21 kHz lo-pass a 'misuse' of switches? I see... are you really using 16.5kHz lo pass for all your music needs? - this would explain...

I re-encoded a couple of samples to about 300 result files, used 5 codecs, different options for ogg, aac... and the only mess, that is not only visible, but sounds terrible, is when lame is used in a certain condition...

It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all   


strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #21
implying through ellipsis isn’t going to get you anywhere.

He's gotten nowhere so far, except to show what happens when you misuse a codec with signals that no one will ever want to encode for normal usage.


misuse?


MP3 isn't made to compress clipped synthetic signals.  You're supposed to give it some headroom, or else use replaygain.  If you don't, the results will distort. 

I'm surprised they're so badly distorted here, but yes, you're doing it wrong

so you telling me now what am I allowed to listen to?


No one has said anything like that.

It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age.


If you're not willing to accept the answers to questions, you should probably not ask them in the first place.  Its just a waste of everyone's time to argue with you in that case.

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #22
- without dropping to -3dB, would a lo-pass filter of 0.1kHz higher eliminate the artifacts completely?


If you don't pay attention to what you do, and don't listen to people that know how things work, then, you're expected to get invalid results.
Now, do the test again and read what LAME is telling you in the command line about the exact lowpass frequency it is using, and not what you tell it to use.

Next, I did the same test with LAME 3.99.5 (windows, 32bit rarewares build) -V0, (stereo, 48Khz 19.3Khz 0dBFS, filtered with --lowpass 19.5, with a 16bit and a 24bit file), and in all cases, the noise was below 90dBs, much below than your case.


If it hurts when you "lame.exe --useless --additional --switches --that --cripple --performance" then don't "lame.exe --useless --additional --switches --that --cripple --performance".


Sure he's changing the defaults, and who knows what more is he doing since I can't replicate it, but the change is not that far from defaults. I.e. Using -V1 instead of -V0 already uses a filter in that range.




It is not just about to accept, what is served on the table, but improve the taste, and the quality of service. If none of human would do this, we would be still in stone age. That is why I came here, and started this topic, and not to fight with Gargamel. In any case, your avatar says it all 


If you had, for a moment, the thought that Hydrogenaudio is a place where we simply sit down and accept whatever "dogma" is established, you should learn a bit more about us before continue.

Said that, what we have learned in these years is to not blindly believe in what a spectogram shows when talking about lossy compression, and to avoid focusing on artificial samples when testing the reliability of these codecs.

What you found might or might not be a misfunction of the filter, but so far, my results don't show the same than yours.




Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #23
Edit: Ok, I found why my results were so different, and definitely verifies what saratoga said from the beginning.
I opened the MP3s directly in Audacity, and they opened in floating precision. The sines go above full scale but since I had it in float, they didn't clip and the noise was maintained below 90dBFS.

Why is there such a difference? Well, there's a small problem with the filter, because it generates some aliasing/mirroring (i.e in the spectogram, there's a line at 19.3 and another at 19.7, around 24dBs softer).
Fantastic. Now we can start getting somewhere. Thanks for the explanation!

strange results using low-pass filter ( lame 3.98/3.99.5 )

Reply #24
perfect, "start getting somewhere" sounds great ( even if it is distorted ). a bumpy road, maybe a couple of wrong expressions I used, but I am glad, that there is an explanation other than "-3dB". Problems are found on "per case basis", they often can be "hidden" similar to this case, otherwise Windoze should never show a BSOD. Suggesting workarounds in some cases is acceptable, and often an only way, but I did not start the topis to hear workarounds... If there is a confirmed bug/issue with Lame, regardless from the comments on how "improper" my encoding attempts are, I will be more than happy that this topic was started. Don't you agree? Thanks again!
Mike

[edit]

this is the source wav and two mp3 files, at 20.1 and 20.2kHz lo-pass

original wav
20.1kHz lo pass mp3
20.2kHz lo pass mp3

thanks!