Skip to main content

Topic: Advanced Limiter (Read 2781 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Advanced Limiter
I'm just trying to understand how the Advanced Limiter works. Is there a point at which it never limits? I noticed when converting to flac, having the Advanced Limiter in the conversion chain always changes the md5, so I ran a few little tests to see if it was changing the volume or limiting. Here's the result according to ReplayGain (4x oversampling) after creating two flac files with a peak of around -26dB.

Without Advanced Limiter
Track Gain +31.06dB Track Peak 0.050017

With Advanced Limiter
Track Gain +31.06dB Track Peak 0.050012

I know it's not much but it seems odd that it'd limit even at such a low volume. Maybe that's just how it works, but the Hard -6dB Limiter doesn't behave the same way.

Thanks.

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #1
Ew, that's surprising.  I always wished there was official documentation on what the heck Advanced Limiter actually does.

  • kode54
  • [*][*][*][*][*]
Re: Advanced Limiter
Reply #2
ReplayGain correction is applied before DSPs, not after.

Re: Advanced Limiter
Reply #3
That's not a ReplayGain correction. There was no Replaygain involved when creating those files. They're the scans after they were encoded. I think I was messing around with some downmixing at the time and used the Matrix Mixer DSP to reduce the volume, which I followed with and without the Advanced Limiter.

I'd also be keen to learn how the Advanced and Hard -6dB limiters handle multiple channels. Even stereo. I'll do some testing later and maybe work out if they limit channels independently, or whether stereo channels are always reduced in volume by the same amount...  if they're linked, so to speak.

Thanks.

  • kode54
  • [*][*][*][*][*]
Re: Advanced Limiter
Reply #4
It limits, but only if there's ever a sample exceeding ±0.9999, with a logarithmic history based on the magnitude of the samples. When the history's oldest and highest value drops below ±0.9999, it deactivates its scanning again.

Whether it is active or not, an output sample is pulled from the oldest slot of a 256 sample circular FIFO buffer, and the current input sample is fed in in its place, then the counter is incremented. Thus, this DSP always has a 256 sample latency.

When active, the minimum gain is 0.9999 / maximum historical sample, otherwise it's 0.9999. A low-pass filtered version of this gain is maintained by calculating 0.9 * itself plus 0.1 times the minimum gain each iteration.

The running gain starts out as 1.0. Then each iteration, it is multiplied by 0.999 and a fixed 0.001 is added to it.

Then if the running low-pass filtered gain is less than this gain value, the gain becomes the low pass filtered value. BUG: When not actually gaining anything, the fact that the limiter maximum is 0.9999 means that minimum gain becomes 0.9999, and the running gain also becomes this, so all non-clipping audio is forcibly scaled by 0.9999 when this filter is enabled.

A final check is run against the current output sample, comparing its absolute value times the current gain value against 0.9999. If it exceeds this, the gain is immediately reset to 0.9999 divided by the absolute value of the output sample.

The output sample is then scaled by the gain value and returned.

In order to make this filter lossless, it is probably a good idea to examine what happens if limiter_max is set to 1.0f. Or, if that is a bad idea, replacing some of the comparison values with 1.0f, including setting minGain to 1.0f when the filter is deemed "inactive" by the lack of recent clipping.
  • Last Edit: 17 January, 2017, 10:25:27 PM by kode54

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #5
I love when documentation finds bugs. :)

Thank you for the analysis.  As you suggested, it would be nice to learn more about the magic numbers, including the 256-sample buffer.  For example, why isn't this size a percentage of the output buffer?

  • Peter
  • [*][*][*][*][*]
  • Administrator
Re: Advanced Limiter
Reply #6
Bug fixed for the next foobar2000 update, thanks for reporting.

  • kode54
  • [*][*][*][*][*]
Re: Advanced Limiter
Reply #7
For example, why isn't this size a percentage of the output buffer?
It's an even power of two, designed to be just enough to catch clipping ahead of time, without incurring noticeable latency. Or at least the last two parts are my take. I just know it's a power of two to speed up wrapping within the buffer's size, and also to deal with the exponential aspect.

It's not logarithmic, it steps back one sample, then doubles the step every time until it reaches the end of the buffer. The source code of the original version still exists if you can find any copies of the 0.8.3 SDK or older which included what was then foo_dsp_extra's source. Or, "Monkee Limiter".

The hard limiter is just:

Code: [Select]
static const float limiter_max = (float)0.9999;

static const float limiter_max_05 = (float)(limiter_max - 0.5);
if (val < -0.5)
val = (tanh((val + 0.5) / limiter_max_05) * limiter_max_05 - 0.5);
else if (val > 0.5)
val = (tanh((val - 0.5) / limiter_max_05) * limiter_max_05 + 0.5);

So, for every sample greater than ±0.5, scaling it down so 0.0 is 0.5 and 1.0 is clipping, feeding the result into hyperbolic tangent, which produces a slope that approaches but never reaches 1.0.

Definitely has more potential for distortion than the Advanced Limiter, but has no effect if the signal doesn't exceed ±0.5.

  • naturfreak
  • [*][*][*]
Re: Advanced Limiter
Reply #8
Curious question about using clipping limiters.

Is limiting needed for audio that is compressed with lossless formats? All lossless formats have a hard limit of signal by definition. Audio stored by using lossy compression can cause clipping during decoding after converting them back to PCM.

Would it be a good idea using the limiter only for lossy audio formats but not for lossless audio formats?

Re: Advanced Limiter
Reply #9
naturfreak,
The hard limit of lossless formats you refer to is exactly why limiters are needed. For lossless formats with a limit of 0dB, any peaks above that are simply flattened out, ie clipped. The idea of a limiter is to reduce the peaks so they stay below 0dB and aren't clipped.

It's probably a good idea to always ensure the audio never exceeds 0dB no matter how it's encoded, because sooner or later it's going to end up 16 or 24 bit PCM again after it's decoded, so any peaks above 0dB will be clipped. Because lossy encoders can alter the peaks a little they can increase them when the audio is decoded (compared do the original input), so some people recommend always keeping peaks to a maximum of -3dB or even -6dB, although I don't get that carried away myself.

Re: Advanced Limiter
Reply #10
kode54,
Thanks for the detailed explanation, although I'll confess I've not got my head around it yet.

I assume there's no channel linking or anything of that nature? ie if stereo audio has peaks above 0dB in the right channel, but the left channel is much lower in volume, only the right channel would be limited, or have it's volume reduced?

Cheers.

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Re: Advanced Limiter
Reply #11
Is limiting needed for audio that is compressed with lossless formats? All lossless formats have a hard limit of signal by definition.
Most lossless formats support only integer PCM, but some (Wavpack, OptimFROG) support also floating point samples.

Would it be a good idea using the limiter only for lossy audio formats but not for lossless audio formats?
Very often limiter is not the only DSP in a chain. Previous DSPs can result in peaks in a signal that exceed 0 dBFS.

  • naturfreak
  • [*][*][*]
Re: Advanced Limiter
Reply #12
naturfreak,
The hard limit of lossless formats you refer to is exactly why limiters are needed. For lossless formats with a limit of 0dB, any peaks above that are simply flattened out, ie clipped. The idea of a limiter is to reduce the peaks so they stay below 0dB and aren't clipped.
If the clipping is already in the source file, then you need declipping processing, not simply level limiting.
Limiting of such audio does not restore sample values. The original sample values are lost.

It's probably a good idea to always ensure the audio never exceeds 0dB no matter how it's encoded, because sooner or later it's going to end up 16 or 24 bit PCM again after it's decoded, so any peaks above 0dB will be clipped.
I wonder if DAC does some processing steps to conceal clipped samples.
Even expensive Audio-CD-players do not apply additional software-based limiting. How do they process PCM with clipped samples?


Is limiting needed for audio that is compressed with lossless formats? All lossless formats have a hard limit of signal by definition.
Most lossless formats support only integer PCM, but some (Wavpack, OptimFROG) support also floating point samples.
Yes, you are right. But a guess only a few people use audio stored in floating point format in daily life.
Since ADC and DAC operates with integer values I don't see the benefit of using them except for music production.


Would it be a good idea using the limiter only for lossy audio formats but not for lossless audio formats?
Very often limiter is not the only DSP in a chain. Previous DSPs can result in peaks in a signal that exceed 0 dBFS.
Additional DSP does alter the level of the audio signal anyway. Does it really matter that much if a limiter decrease the signal level by a very small amount?
  • Last Edit: 19 January, 2017, 12:02:10 PM by naturfreak

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #13
Even expensive Audio-CD-players do not apply additional software-based limiting. How do they process PCM with clipped samples?
PCM contains no way to distinguish whether a full-scale sample is clipped.  From the perspective of PCM, clipped samples don't exist.  The only thing that we can objectively control is how samples are converted to a waveform:  http://www.audioholics.com/audio-technologies/issues-with-0dbfs-levels-on-digital-audio-playback-systems

High quality DAC components should have enough headroom based on how they choose to reconstruct the waveform from the samples.  There's nothing inherently wrong with full-scale samples.
  • Last Edit: 19 January, 2017, 05:34:23 PM by SmartOne

Re: Advanced Limiter
Reply #14
naturfreak,
The hard limit of lossless formats you refer to is exactly why limiters are needed. For lossless formats with a limit of 0dB, any peaks above that are simply flattened out, ie clipped. The idea of a limiter is to reduce the peaks so they stay below 0dB and aren't clipped.
If the clipping is already in the source file, then you need declipping processing, not simply level limiting.
Limiting of such audio does not restore sample values. The original sample values are lost.

True , but I assumed your original question referred to clean,  unclipped audio, not audio in need of repair.

I wonder if DAC does some processing steps to conceal clipped samples.
Even expensive Audio-CD-players do not apply additional software-based limiting. How do they process PCM with clipped samples?

The samples shouldn't be clipped, but the peaks of the reconstructed analogue wave form may be clipped if the limit is still 0dB after conversion to analogue, but I assume DACs would have a little headroom.

Foe the record, you can store 32 bit float in a wave file too, It doesn't have to be integer.

Additional DSP does alter the level of the audio signal anyway. Does it really matter that much if a limiter decrease the signal level by a very small amount?

I guess from an audio perspective it doesn't matter if a limiter reduces the volume by a tiny amount (I assume you're referring to the reduction described in the OP?), but from an "expected result" perspective it'd be better if it didn't when it's not limiting. Changing the volume changes the md5 calculated for the encoded audio, which should remain the same if the DSP isn't doing any processing.
  • Last Edit: 20 January, 2017, 03:51:45 AM by yetanotherid

Re: Advanced Limiter
Reply #15
Post deleted. I quoted my previous post instead of editing it, creating a duplicate.
  • Last Edit: 20 January, 2017, 03:40:10 AM by yetanotherid

  • naturfreak
  • [*][*][*]
Re: Advanced Limiter
Reply #16
PCM contains no way to distinguish whether a full-scale sample is clipped.  From the perspective of PCM, clipped samples don't exist.
For real audio signals there should no contingous samples at max. or min. value, only single ones.
The original recommendation for mastering audio CD was that the PCM values should not reach limits at any time.

The only thing that we can objectively control is how samples are converted to a waveform:  http://www.audioholics.com/audio-technologies/issues-with-0dbfs-levels-on-digital-audio-playback-systems

High quality DAC components should have enough headroom based on how they choose to reconstruct the waveform from the samples.  There's nothing inherently wrong with full-scale samples.
Ok, then. Is a additional limiter as a postprocessing step needed or can we rely on the DAC to handle that?

  • naturfreak
  • [*][*][*]
Re: Advanced Limiter
Reply #17
If the clipping is already in the source file, then you need declipping processing, not simply level limiting.
Limiting of such audio does not restore sample values. The original sample values are lost.

True , but I assumed your original question referred to clean,  unclipped audio, not audio in need of repair.
No, sorry, perhaps I was not clear enough.
For clean, unclipped audio you need a limiter only in case you use additional DSP or preamplifying in the processing chain.
I'm no big fan of limiting as a postprocessing step. Limiting should be done in music production.

I wonder if DAC does some processing steps to conceal clipped samples.
Even expensive Audio-CD-players do not apply additional software-based limiting. How do they process PCM with clipped samples?

The samples shouldn't be clipped, but the peaks of the reconstructed analogue wave form may be clipped if the limit is still 0dB after conversion to analogue, but I assume DACs would have a little headroom.
I think so too.

Foe the record, you can store 32 bit float in a wave file too, It doesn't have to be integer.
I know that, but what is the advantage of that for music listeners? I can only see a benefit in the process of music production.

Additional DSP does alter the level of the audio signal anyway. Does it really matter that much if a limiter decrease the signal level by a very small amount?

I guess from an audio perspective it doesn't matter if a limiter reduces the volume by a tiny amount (I assume you're referring to the reduction described in the OP?), but from an "expected result" perspective it'd be better if it didn't when it's not limiting. Changing the volume changes the md5 calculated for the encoded audio, which should remain the same if the DSP isn't doing any processing.
That sounds like you use the limiter for converting audio files, not just for music playback.
Wouldn't it be a better solution to remove the advanced limiter from converter DSP chain?
foobar2000 has a seperate DSP setup for the converter.
If you don't want any change of your files remove all DSP from converter setup.
Where is the point using a DSP if you don't need it?

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #18
For real audio signals there should no contingous samples at max. or min. value, only single ones.
Define a "real" signal (you can't).  What about a square wave at maximum amplitude.

The original recommendation for mastering audio CD was that the PCM values should not reach limits at any time.
Recommendation != requirement, though people always equate the two.  A mastering person would be stupid not to use the full, objective dynamic range available to him for the sake of "fixing by preemption" bad downstream component implementation, which is absolutely subjective.

Is a additional limiter as a postprocessing step needed or can we rely on the DAC to handle that?
I would prefer leaving the PCM unaltered and buying a high-quality DAC.  The nice thing about subjectivity is you get to arbitrarily choose an approach.

  • naturfreak
  • [*][*][*]
Re: Advanced Limiter
Reply #19
Define a "real" signal (you can't).
A real audio signal means a signal that is band-limited by filtering to prevent aliasing.

What about a square wave at maximum amplitude.
There is no mathematical perfect square wave as long as frequency limits matter.
https://en.wikipedia.org/wiki/Gibbs_phenomenon

I guess that leads to a semi off topic discussion that is only partly related to the question of the thread opener.

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #20
A real audio signal means a signal that is band-limited by filtering to prevent aliasing.
Even if PCM contained band-limiting requirements, it would have to contain a specification of the filter itself to obtain objectively consistent results, because the perfect filter requires infinite time.  There are infinitely many practical filters that can achieve a loose definition of "band-limiting."  I guess the best one would be numerically transparent given the resolution of the samples.

This is like saying, "All images must have quality x."  What's x?

Back to whether to use the Advanced Limiter: Since I prefer to not have to adjust the volume constantly, I use ReplayGain with limit according to peak.  Hopefully that reduces the necessity of volume changes.  A nice side effect is it seems like most track levels are reduced.  When I have to resample, I rely on this observation, hope there's enough headroom, and monitor the Peak Meter.  If I notice peaks over 0 dB,  I reduce the input Preamp on the Playback pane during that track.  Then I don't have to worry about a dynamic limiter doing dynamic things.  If I wanted a "set it and forget it" solution, I'd use the Advanced Limiter.
  • Last Edit: 21 January, 2017, 06:53:20 PM by SmartOne

  • tedsmith
  • [*][*][*]
Re: Advanced Limiter
Reply #21
A real audio signal means a signal that is band-limited by filtering to prevent aliasing.
Even if PCM contained band-limiting requirements, it would have to contain a specification of the filter itself to obtain objectively consistent results, because the perfect filter requires infinite time.  There are infinitely many practical filters that can achieve a loose definition of "band-limiting."  I guess the best one would be numerically transparent given the resolution of the samples.

This is like saying, "All images must have quality x."  What's x?
No, you keep moving the goal posts.  The point is that your square wave example is fallacious:  any max level bandlimited square will overflow and if you lower it's level until it doesn't overflow it's no longer a max level square wave.  E.g. the Gibbs phenomenon.

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #22
Yes, what I'm saying is perfectly objective.  It's up to the DAC to handle overflow, AKA headroom, AKA remember this link I posted?  It still applies:
The only thing that we can objectively control is how samples are converted to a waveform:  http://www.audioholics.com/audio-technologies/issues-with-0dbfs-levels-on-digital-audio-playback-systems

We're arguing whether contiguous full-scale samples are wrong.  Just because they seem unnatural doesn't make them wrong.  You know me, I love a good argument, though.  Keep it coming.

  • tedsmith
  • [*][*][*]
Re: Advanced Limiter
Reply #23
We're arguing whether contiguous full-scale samples are wrong.  Just because they seem unnatural doesn't make them wrong.  You know me, I love a good argument, though.  Keep it coming.
See my previous post - if two contiguous sample are at the max and that doesn't represent overflow then the source signal must have been flat between them.  If the signal is bandlimited then it has to be flat forever (otherwise there's a discontinuity in slope somewhere and Gibbs rears it's ugly head.)  Even when taking into account filters that don't span all time all practical filters span more than a few samples.  (If you argue that you can avoid discontinuities in the slope then the slope of the slope must have a discontinuity and Gibbs is still there...)  Do the math.

  • SmartOne
  • [*][*]
Re: Advanced Limiter
Reply #24
Yes, two contiguous full-scale samples tend to represent overflow.  PCM doesn't specify any band-limiting requirements; it's up to the ADC to choose whether and how to band-limit, and the DAC gets to make the same choice, implementing a filter that's probably inconsistent with the ADC filter (or a lack thereof).

There's nothing inherent to PCM that says you can't do a zero-order hold.  You can do whatever the heck you want; the quality of the output is up to you.

Addressing the "do the math" jab: lol

(If I'm wrong [and I have been wrong, believe me], I'd love to see a proof.)