Poll
Question:
FIR or IIR? Linear phase or minimum phase?
Option 1: Minimum phase IIR
votes: 5
Option 2: Minimum phase FIR
votes: 0
Option 3: Linear phase FIR
votes: 5
Option 4: Other (please specify in reply)
votes: 1
An audio company that I have some influence over is designing a new digital audio player and programming the digital EQ section. Their engineer told me that they're implementing it as an FIR filter. They're planning to play back hi-res music files up to 192k/24b on a somewhat dated SoC. I see two potential problems:
1. 192000Hz audio requires a LOT of taps in an FIR filter to get at the low frequencies. Frequency resolution is calculated as (Fs/N(taps)), so even a 2048 tap FIR filter can only get down to 100Hz or so. Not much of a bass boost filter, more like a midbass bloat effect.
2. The company director was also complaining of "phase shift" with digital filters. Now as you may know, corrective EQ filters are best implemented as minimum phase filters to cancel out the phase shift introduced by the headphones themselves, but FIR filters tend to be implemented as linear phase. However linear phase looks better on paper and sounds better to... somebody?
I suggested that they change the design to a minimum phase IIR filter, but the engineer said "isn't FIR supposed to be higher fidelity?" Where I have yet to get his word on what this "higher fidelity" entails.
I searched and found this:
http://www.dspguru.com/dsp/faqs/iir/basics (http://www.dspguru.com/dsp/faqs/iir/basics)
1.4 What are the advantages of IIR filters (compared to FIR filters)?
IIR filters can achieve a given filtering characteristic using less memory and calculations than a similar FIR filter.
1.5 What are the disadvantages of IIR filters (compared to FIR filters)?
They are more susceptable to problems of finite-length arithmetic, such as noise generated by calculations, and limit cycles. (This is a direct consequence of feedback: when the output isn't computed perfectly and is fed back, the imperfection can compound.)
They are harder (slower) to implement using fixed-point arithmetic.
They don't offer the computational advantages of FIR filters for multirate (decimation and interpolation) applications.
Now, I suspect that neither of the first two disadvantages apply, since they told me they're implementing their existing filter using 64-bit arithmetic. That sounds like plenty enough accuracy, and 64-bit has got to be floating point, I guess?
But what does the last point mean?
And can someone offer me some examples of how much more noise / distortion an IIR filter would generate compared to an FIR filter given the same bit precision in calculations?
The last point refers to sampling rate conversion, so it is not applicable to your case. For a basic EQ, IIR filters should not have any problems with precision.
This is what we based the EQ in rockbox on: http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt (http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt)
FWIW, this is an IIR filter as they are much faster on low power, fixed point hardware. We would not have had the CPU available to implement an FIR system.
For audio, linear phase would be ideal. You can't do this with IIR of course, but there are designs that get reasonably close. I think minimum phase designs are often used, but I'm not an expert on this.
I would not say that linear-phase EQs are better, in terms of quality, than nonlinear-phase ones. Preferences differ among sound engineers.
I would not say that linear-phase EQs are better, in terms of quality, than nonlinear-phase ones. Preferences differ among sound engineers.
In an ideal world why would you want anything other than linear phase?
I would not say that linear-phase EQs are better, in terms of quality, than nonlinear-phase ones. Preferences differ among sound engineers.
In an ideal world why would you want anything other than linear phase?
Because
1. linear phase EQ leads to pre-ringing and post-ringing in equal proportions, when human hearing is much more sensitive to the former (people here of all places should know that!
2. as I pointed out in the OP, sound transducers (especially headphones) are often near minimum-phase systems such that compensating their response at the output stage with a minimum-phase EQ leads to improvement in phase response as well as magnitude response.
But I don't know what would happen if someone decides to take Beats headphones and apply another 15dB bass boost instead...
I know a little about DSP (I have a couple of DSP books), but I'm not a DSP expert...
I'd leave the details to your DSP programmer. It's not hard to make a filter that's better than human hearing (where any artifacts are inaudible).
But, there are no perfect filters in hardware or software. There are trade-offs. People tend the get the idea that a digital (software) filter can be perfect. The big difference between hardware & software filter design is that it costs more money to make a more-complex hardware filter, but in software (or firmware) additional complexity is free. And in hardware, additional filter stages usually introduce additional noise, and you can run into voltage-headroom limitations, etc.
There are sometimes reasons for making a minimum-phase design, and there may be advantages to optimizing something else... You can't
hear phase-shift, unless you phase-shift the left channel without phase-shifting the right channel, or perhaps if the phase-shift is extreme. There have been experiments with all-pass filters (filters that introduce different phase shifts at different frequencies without changing frequency response). As far as I know,
only "audiophiles" can hear a difference and they can
only hear a difference in non-blind listening tests.
If you are doing real-time DSP, processing-power could be a limitation. And for example, you may have to make certain compomises if you don't have a floating-point processor, etc.
...since they told me they're implementing their existing filter using 64-bit arithmetic. That sounds like plenty enough accuracy, and 64-bit has got to be floating point, I guess?
Floating-point is prefered because there is a lot of summation in DSP and you need the ability to handle big numbers. But it hat may depend on your hardware, so I'm not sure what you are using.... Maybe your hardware works better with 64-bit integers.
In a hardware-player, you may have a floating-point processor. That doesn't mean you can't use floating point, but there is more processing required to handle floating-point with an integer processor.
32-bit floating point is more than enough. But, 64-bit floating might be used because you have a 64-bit processor and a 64-bit operating system and "Why not?", or for marketing and "keeping up with the competition".
There have been experiments with all-pass filters (filters that introduce different phase shifts at different frequencies without changing frequency response). As far as I know, only "audiophiles" can hear a difference and they can only hear a difference in non-blind listening tests.
Phase shifts between linear-phase and nonlinear-phase EQs can be easily heard when the EQ curve is steep and the shift happens in the audible range. With steep frequency responses, like notch filtering of hum, ringing becomes a significant concern.
A 32-bit FP format does not always have sufficient precision for IIR filtering at low frequencies. When dealing with low frequencies, a 64-bit double internal precision is preferred.
I would not say that linear-phase EQs are better, in terms of quality, than nonlinear-phase ones. Preferences differ among sound engineers.
In an ideal world why would you want anything other than linear phase?
Because
1. linear phase EQ leads to pre-ringing and post-ringing in equal proportions, when human hearing is much more sensitive to the former (people here of all places should know that!
2. as I pointed out in the OP, sound transducers (especially headphones) are often near minimum-phase systems such that compensating their response at the output stage with a minimum-phase EQ leads to improvement in phase response as well as magnitude response.
I don't see how either of these follow that you'd want minimum phase. For 1, the degree of ringing is a design choice anyway, and for 2 you're assuming that you know the dispersion in speakers. Just choosing minimum phase does not imply that you've chosen the correct response to linearize the speaker's response, or even that you necessarily want to. Generally my take is that speakers are probably designed without an EQ, and trying to tweak their phase response to some abstractly defined ideal is not necessarily a benefit.
I think for a given scenario, its probably a better idea to minimize dispersion. Or at least its a safer idea unless the user has a very specific goal in mind.
I've once used the vst plugin analyser by christian w budde, and I noticed that the "group delay" jump to big values when eq-ing lows in minimum phase mode. I'm unsure if that the "group delay" value is more representative than " phase shift", regarding audible differences (if you have some information regarding that matter , you are welcome). Otherwise I've done more subjective experiments, and I preferred "linear phase" mode , when eq-ing lows with fast-paced music , such like psytrance.
So far I've decided to use eqs this way: ( I combine two eq if I want ).
- linear phase for the lows (let's say below 200 hz)
- minimum phase everywhere else.
Or I just avoid to eq below 200 hz, and just use minimum phase (perhaps the best solution ?) .
Otherwise the "popular" opinion I've noticed at kvraudio, is to avoid to eq in "linear phase" mode . What I've understood, is that linear phase mode somehow degrade the transients , because it adds "pre echos" at the begin of them, while "post echos" are not noticed (or are perceived as more "natural").
Oh and also an advantage I've noticed with "linear phase" eq, is that you can use them to split sound into two separate frequency bands easily.
And perhaps "linear phase" is better, if you wish to eq mid & side or left & right separately (since there's no delay introduced between channels).
I've once used the vst plugin analyser by christian w budde, and I noticed that the "group delay" jump to big values when eq-ing lows in minimum phase mode. I'm unsure if that the "group delay" value is more representative than " phase shift",
Group delay == delay. If the speed of light in your audio cables is .66*c, then 1 uS of group delay == 200 meters of extra (lossless) cable.
A group delay corresponds to a linear increase in phase shift with frequency, hence a linear phase filter has only a group delay and no phase distortion.
So the advice for the engineer, if he has 64 bit float arithmetic at his disposal but otherwise limited CPU grunt, is to do minimum phase IIR?
What if it's 64bit integer instead of float?
They just intend to implement a 10 band graphic EQ btw.
Part of me is actually hoping they'd go for broke with FIR filtering as I could then suggest to them an option to replace the 10-band EQ with a custom convolution impulse
1. linear phase EQ leads to pre-ringing and post-ringing in equal proportions, when human hearing is much more sensitive to the former (people here of all places should know that!
Exactly.
2. as I pointed out in the OP, sound transducers (especially headphones) are often near minimum-phase systems such that compensating their response at the output stage with a minimum-phase EQ leads to improvement in phase response as well as magnitude response.
That too.
To me it's a no-brainer to
never use a linear phase filter. As far as I can tell, linear phase equalizers were developed in response to the "phase shift is damaging" nonsense spewed endlessly by misinformed audio magazine writers. The myth that phase shift is a problem was repeated so many times over the years that audio gear makers came up with a way to avoid phase shift, just to use as a marketing claim. The problem is LP filters are audibly worse than minimum phase! You can't always hear the pre-ringing, but often you can depending on the frequency and other settings.
--Ethan
A group delay corresponds to a linear increase in phase shift with frequency, hence a linear phase filter has only a group delay and no phase distortion.
Well I used an analyzer (the one from christian w budde) on different "linear phase" eq.
For all these linear phase eq, the analyzer indicated that the group delay was set to a fixed value across all frequencies , or nul (so basically it would just mean there's just a fixed delay) . However, for some of these eq, the phase shift increase regularly with frequency, while for other it remains nul ; I don't know if this have any particular meaning .
Anyway my question was , if it wasn't more pertinent to talk of "group delays" instead of "phase shift" regarding audible problems.
@Ethan Winer
The problem is LP filters are audibly worse than minimum phase! You can't always hear the pre-ringing, but often you can depending on the frequency and other settings.
I thought that in the particular case of fast-paced electronic , that I prefer to boost the bass with LP. I've posted once a precise example at kvraudio, and it just seemed to make some of these people angry. They explained me that I should boost the bass to much lesser amount , and with a much smoother slope (which actually is an other way to reduce the audibility of these "phase shift").
2048 FIR taps is not a lot.
I am using 6-channel 131.072 taps FIR filters generated by Audiolense.
Theres a killer test I stumbled upon that involves nothching out a pulsing sine wave. Between the IIR-mininum FIR-mininum and FIR-linear modes avaliable in ElectriQ vst plugin IIR-mininum produced the smoothest result, FIR-linear had insane amounts of both pre-and post echo.
An audio company
...needs to hire someone who understands digital filtering before they can call themselves that (unless they only do analogue equipment!).
The audibility of filter ringing is discussed in this thread...
http://www.hydrogenaudio.org/forums/index....mp;#entry604471 (http://www.hydrogenaudio.org/forums/index.php?showtopic=67619&st=25&p=604471&#entry604471)
...and the samples are still available.
Cheers,
David.
FIR-linear had insane amounts of both pre-and post echo.
Insane ? I don't know which eq you are using (try fabfilter pro or psp neon hr) , but for me it require careful listening, to notice any "pre-echo" (well for a "regular" eq use) , and actually I don't hear it that way, more like a (tiny) smearing. And by the way, FIR has less "post echo" than IIR, it's just that the "echo" is distributed equally before and after transients, which ensure that from "the average", there's no phase shift. (While for IIR , there's only post-echo, it's asymmetric, hence the phase shift).
Insane ? I don't know which eq you are using (try fabfilter pro or psp neon hr) , but for me it require careful listening, to notice any "pre-echo" (well for a "regular" eq use) , and actually I don't hear it that way, more like a (tiny) smearing. And by the way, FIR has less "post echo" than IIR, it's just that the "echo" is distributed equally before and after transients, which ensure that from "the average", there's no phase shift. (While for IIR , there's only post-echo, it's asymmetric, hence the phase shift).
Yes, insane amounts. The plugin I tested is revealed in the same post you are replying to. Your reponce begs the question, why question the implementation when you can easily generate the test signal and put your esteemed linear phase EQs through their paces? I'm sure many would like to know if this turns out to be a bad implementation of linear phase in a pretty highly regarded EQ plugin.
Insane ? I don't know which eq you are using (try fabfilter pro or psp neon hr) , but for me it require careful listening, to notice any "pre-echo" (well for a "regular" eq use) , and actually I don't hear it that way, more like a (tiny) smearing. And by the way, FIR has less "post echo" than IIR, it's just that the "echo" is distributed equally before and after transients, which ensure that from "the average", there's no phase shift. (While for IIR , there's only post-echo, it's asymmetric, hence the phase shift).
Yes, insane amounts. The plugin I tested is revealed in the same post you are replying to. Your reponce begs the question, why question the implementation when you can easily generate the test signal and put your esteemed linear phase EQs through their paces? I'm sure many would like to know if this turns out to be a bad implementation of linear phase in a pretty highly regarded EQ plugin.
Ooops I've overlooked you mentioned Electri-Q. Well, I'll study this with a particular example later.
Another case for minimum phase; Ask any pro mixing engineer what he uses to notch out unwanted resonances in live tracks. They will always mention an IIR minimum phase EQ like Waves Q10.
An audio company
...needs to hire someone who understands digital filtering before they can call themselves that (unless they only do analogue equipment!).
The audibility of filter ringing is discussed in this thread...
http://www.hydrogenaudio.org/forums/index....mp;#entry604471 (http://www.hydrogenaudio.org/forums/index.php?showtopic=67619&st=25&p=604471&#entry604471)
...and the samples are still available.
Thats an awesome link that I somehow missed. I admit, my experience with filters is not for audio mostly, so I could be wrong, but here is what I think I understand:
1) Ringing is a problem if you have very steep filter transition bands
2) A linear phase system will tend to have more ringing because of the requirement that the impulse response be symmetric.
3) EQ systems tend to use relatively gradual filters. I know for Rockbox we default to a Q of 1 and a smooth roll off. This is fine because spectral leakage is acceptable/desirable.
Given 3, is ringing actually a problem in a linear (or at least not-minimum) phase EQ? I would think the actual tradeoff in practice would be between efficiency and frequency dispersion, with minimum phase maximizing the former, and linear phase the latter. Or are people really building 5 Hz wide, -30dB EQ filters (maybe 50/60hz notch??) and I'm just not aware of it?
I would be sceptical about an "audio engineer" that claimed that linear phase was the best with no disclaimers.
1. What is your model of the "error" that you are trying to correct
2. What is your model of human perception
3. What are the practical details in your application (delay requirements, computation structure etc)
I believe that IIR filters tend to be disproportionally inefficient on things like x86: the number of mult/adds are less relevant than the number of N-element SIMD operations that can be done simultaneously.
If you are correcting minimum phase errors, then a minimum phase equalizer will correct phase distortion "for free".
If you are doing creative spectral modifications, then a linear-phase EQ will do them without messing with phase.
Humans tends to be sensitive to pre-ringing, less so to post-ringing.
Perhaps the "optimal" shape depends on if you are doing very sharp & deep "spectral removal" (phase response close to the stop band might not matter?), while moderate boosts/shelving filters might be more sensitive to phase behaviour.
I am not an EQ designer. If asked to place a bet, I would bet on some (unknown) intermediate form in-between linear phase and minimum phase for general EQ tasks. I.e. not the symmetric, very long delay linear-phase shape, and not the typical "left-heavy" minimum phase response, but some moderately asymmetric response.
-k
1) Ringing is a problem if you have very steep filter transition bands
Yes. Steep filtering is not uncommon, think of notch filters for hum removal.
2) A linear phase system will tend to have more ringing because of the requirement that the impulse response be symmetric.
Linear-phase systems have less
measured ringing, but more
audible ringing, because a significant part of their ringing is
pre-ringing.
3) EQ systems tend to use relatively gradual filters.
Yes, unless you need steep cuts for restoration purposes (like hum removal) or steep resonances for special effects.
I believe that IIR filters tend to be disproportionally inefficient on things like x86: the number of mult/adds are less relevant than the number of N-element SIMD operations that can be done simultaneously.
This is not true. Both IIR and FIR filters can be computed with SIMD operations, but FIR filters typically require much more operations to achieve the specified frequency response. To give you an example, most commercial IIR EQs are using one or two 2-nd order IIR filters per control node, which adds to a dozen ops per sample. To achieve the same frequency response with a FIR filter, you'd need an order of hundreds or even thousands (esp. at low frequencies), which would lead to hundreds operations per sample.
Well , markanini was previously discussing about how he got "insanely bad" pre-echo artifacts using the eq electri-Q in linear phase mode.
I just don't hear this, but for those that hear these "insane" artifacts , I've elaborated a comparison with Fabfilter pro Q here:
http://www.hydrogenaudio.org/forums/index....howtopic=103339 (http://www.hydrogenaudio.org/forums/index.php?showtopic=103339)
And by the way it seem that Electri-Q was discontinued this year. I've looked at the kvraudio forum , and users were complaining before of the lack of support / updates . In my case, Electri-Q refused to load inside soundforge; and it made foobar crash after generating a file with convert command.
Well , markanini was previously discussing about how he got "insanely bad" pre-echo artifacts using the eq electri-Q in linear phase mode.
I just don't hear this, but for those that hear these "insane" artifacts , I've elaborated a comparison with Fabfilter pro Q here:
http://www.hydrogenaudio.org/forums/index....howtopic=103339 (http://www.hydrogenaudio.org/forums/index.php?showtopic=103339)
And by the way it seem that Electri-Q was discontinued this year. I've looked at the kvraudio forum , and users were complaining before of the lack of support / updates . In my case, Electri-Q refused to load inside soundforge; and it made foobar crash after generating a file with convert command.
Except I was talking about narrow notch filters on a sine wave, here's you're applying a bass boost to a 90's pop song.
Except I was talking about narrow notch filters on a sine wave, here's you're applying a bass boost to a 90's pop song.
hmm ok... I've never used notch filters. I'd think it would be better to use some noise reduction , with a "noise print" restricted to a particular band.
Can we have a show of hands among the experts on the best kind of EQ for a mobile device? From looking at the thread there should be much more than 1 supporter of minimum phase IIR and where did so many supporters of linear phase FIR come from Please explain your position
There isn't a best kind. The product's design, including filters, either meets its specifications (and budgets), or it doesn't.
If its mobile iir is absolutely the right way to go. People won't like your 5x200 tap linear phase filter bank destroying their batteries.
Did RMAA remove phase/impulse response testing? I just tried the current version and I cannot get the impulse response to display. Was going to do some tests of my own.
RMAA never displayed an impulse response, only a frequency response. If I recall correctly, you can save an impulse response to a WAV file by selecting a sine-sweep test for frequency response.
RMAA never displayed an impulse response, only a frequency response. If I recall correctly, you can save an impulse response to a WAV file by selecting a sine-sweep test for frequency response.
What is the "Impulse/Phase response" test it lists? Actually I might be thinking of your graphics on http://src.infinitewave.ca/ (http://src.infinitewave.ca/), which appear to be done with RMAA, but might just be different software using the same GUI toolkit.
Edit: Heres our EQ for reference:
(http://i.imgur.com/87qxDTB.png)
Units are radians and dB. For reference, that uses about 2.5MHz per band for realtime decone on ARMv5 @ 44.1khz.
Ah, yes, that test displays phase response and saves the impulse in a WAV file, but does not display it.
The impulses from InifiniteWave are plotted with iZotope RX.
Is there a way to get it to display the phase response without having to hilbert transform first in matlab? I didn't see it in RMAA.
I don't get the question. There's no Hilbert transform in RMAA.
I believe that IIR filters tend to be disproportionally inefficient on things like x86: the number of mult/adds are less relevant than the number of N-element SIMD operations that can be done simultaneously.
This is not true. Both IIR and FIR filters can be computed with SIMD operations, but FIR filters typically require much more operations to achieve the specified frequency response. To give you an example, most commercial IIR EQs are using one or two 2-nd order IIR filters per control node, which adds to a dozen ops per sample. To achieve the same frequency response with a FIR filter, you'd need an order of hundreds or even thousands (esp. at low frequencies), which would lead to hundreds operations per sample.
The difference in #mults/adds for a given magnitude response is well known and taught in any dsp text-book. The complexity can (at least for certain cases) be improved by using multi-rate techniques (but then you are not LTI anymore, not even a nice floating-point approximation to LTI).
I don't understand how e.g. a single 1-pole biquad working on continous data can (efficiently) fit into the 128-bit or 256-bit SIMDs of Intel (or similar ARM) instructions, where you (as I recall it) typically have access to parallell float adds, mults, etc but very little in terms of intra-register arithmetics? How do you exploit all of those 8 shiny single-cycle (?) floating-point multipliers if every single output sample depends on the previous one?
-k
I don't get the question. There's no Hilbert transform in RMAA.
No there isn't. I had to use matlab's because I couldn't figure out how to get RMAA to show me any phase information at all.
I don't understand how e.g. a single 1-pole biquad working on continous data can (efficiently) fit into the 128-bit or 256-bit SIMDs of Intel (or similar ARM) instructions, where you (as I recall it) typically have access to parallell float adds, mults, etc but very little in terms of intra-register arithmetics? How do you exploit all of those 8 shiny single-cycle (?) floating-point multipliers if every single output sample depends on the previous one?
IIUC, you'd have 3 independent parallel multiply-adds per channel in that case: x[n], x[n-1], y[n-1]. That means you can fill 6 32 bit MAC operations per sample. A typical ARM A9 system has 64 bit wide multiply/accum, or on the newer Qualcomm systems, 128. So for mobile applications you would be fine.
However, I think that filter is a bit unrealistic. With plane scalar ARMv4 instructions, you'd be looking at about 1 Mhz per output channel at 44.1khz. Most likely you would either use a higher order filter, or else simply not bother optimizing because the CPU time is too low to matter.
I don't understand how e.g. a single 1-pole biquad working on continous data can (efficiently) fit into the 128-bit or 256-bit SIMDs of Intel (or similar ARM) instructions
1. A single biquad in a direct form 1 requires 5 parallel multiplications, and 5 additions.
2. Biquads are rarely used alone. When an EQ has several control nodes, biquads can be parallelized, if needed.
No there isn't. I had to use matlab's because I couldn't figure out how to get RMAA to show me any phase information at all.
Why would you need a Hilbert transform to measure phase? You just need an FFT of the impulse response for that. If I recall correctly, RMAA does it for you: it displays phase and group delay graphs when the “Impulse/Phase” test finishes. Is it not the case for you?
I don't understand how e.g. a single 1-pole biquad working on continous data can (efficiently) fit into the 128-bit or 256-bit SIMDs of Intel (or similar ARM) instructions, where you (as I recall it) typically have access to parallell float adds, mults, etc but very little in terms of intra-register arithmetics? How do you exploit all of those 8 shiny single-cycle (?) floating-point multipliers if every single output sample depends on the previous one?
IIUC, you'd have 3 independent parallel multiply-adds per channel in that case: x[n], x[n-1], y[n-1]. That means you can fill 6 32 bit MAC operations per sample. A typical ARM A9 system has 64 bit wide multiply/accum, or on the newer Qualcomm systems, 128. So for mobile applications you would be fine.
However, I think that filter is a bit unrealistic. With plane scalar ARMv4 instructions, you'd be looking at about 1 Mhz per output channel at 44.1khz. Most likely you would either use a higher order filter, or else simply not bother optimizing because the CPU time is too low to matter.
I am not disputing your claims as much as updating myself on IIR filters (I've been mostly into FIR filtering).
If we can normalise b(1), assuming that gain will be carried out later on anyways and there is no numerical penalty:
N=1000;
x = rand(N,1);
L = 2;
[b,a] = butter(L,0.42);
%% calculate reference
y_ref = filter(b,a,x);
% normalize against b(1) (assume that gain is to be carried out later on anyways)
b2 = b./b(1);
gain = b(1);
x2 = [zeros(L+0,1); x];
y2 = zeros(size(x2));
c = [b2(2:L+1)'; a(2:L+1)'];
for n=(L+1):N+L
buf = [x2(n-(1:L)); y2(n-(1:L))];
buf = buf .* c;
y2(n) = sum(buf(1:L)) - sum(buf(L+1:2*L));
y2(n) = y2(n) + x2(n);
end
y2 = y2(L+1:end);
y2 = gain*y2;
%% pass/fail
if(max(abs(y_ref-y2))<=1.5*eps)
disp('test passed')
else
disp('test failed')
end
I figure that my pseudo-implementation will have to:
1. Read 2 inputs and 2 previous outputs into one 4-element register
2. Do a 4-element dot product
3. Do 3 intra-register add/sub into scalar
4. Read 1 input
5. Do 1 add
6. Store scalar output
I don't see any potential to roll out the loop as there is dependency between iterations.
While this may well be "fast enough" for many audio applications, it seems to be far from having the 8-fold speedup that one might hope for in a 256-bit AVX cpu compared to a similarly clocked scalar cpu? Also, there are some cumbersome load operations, intra-register operations, unlike a streamlined FIR filter that can load neat "chunks" from memory, do nice multiply-accumulate, and store output.
-k
I don't understand how e.g. a single 1-pole biquad working on continous data can (efficiently) fit into the 128-bit or 256-bit SIMDs of Intel (or similar ARM) instructions, where you (as I recall it) typically have access to parallell float adds, mults, etc but very little in terms of intra-register arithmetics? How do you exploit all of those 8 shiny single-cycle (?) floating-point multipliers if every single output sample depends on the previous one?
IIUC, you'd have 3 independent parallel multiply-adds per channel in that case: x[n], x[n-1], y[n-1]. That means you can fill 6 32 bit MAC operations per sample. A typical ARM A9 system has 64 bit wide multiply/accum, or on the newer Qualcomm systems, 128. So for mobile applications you would be fine.
However, I think that filter is a bit unrealistic. With plane scalar ARMv4 instructions, you'd be looking at about 1 Mhz per output channel at 44.1khz. Most likely you would either use a higher order filter, or else simply not bother optimizing because the CPU time is too low to matter.
I am not disputing your claims as much as updating myself on IIR filters (I've been mostly into FIR filtering).
If we can normalise b(1), assuming that gain will be carried out later on anyways and there is no numerical penalty:
N=1000;
x = rand(N,1);
L = 2;
[b,a] = butter(L,0.42);
%% calculate reference
y_ref = filter(b,a,x);
% normalize against b(1) (assume that gain is to be carried out later on anyways)
b2 = b./b(1);
gain = b(1);
x2 = [zeros(L+0,1); x];
y2 = zeros(size(x2));
c = [b2(2:L+1)'; a(2:L+1)'];
for n=(L+1):N+L
buf = [x2(n-(1:L)); y2(n-(1:L))];
buf = buf .* c;
y2(n) = sum(buf(1:L)) - sum(buf(L+1:2*L));
y2(n) = y2(n) + x2(n);
end
y2 = y2(L+1:end);
y2 = gain*y2;
%% pass/fail
if(max(abs(y_ref-y2))<=1.5*eps)
disp('test passed')
else
disp('test failed')
end
I figure that my pseudo-implementation will have to:
1. Read 2 inputs and 2 previous outputs into one 4-element register
2. Do a 4-element dot product
3. Do 3 intra-register add/sub into scalar
4. Read 1 input
5. Do 1 add
6. Store scalar output
I don't see any potential to roll out the loop as there is dependency between iterations.
While this may well be "fast enough" for many audio applications, it seems to be far from having the 8-fold speedup that one might hope for in a 256-bit AVX cpu compared to a similarly clocked scalar cpu? Also, there are some cumbersome load operations, intra-register operations, unlike a streamlined FIR filter that can load neat "chunks" from memory, do nice multiply-accumulate, and store output.
-k
Well, for real-time audio application, it just has to run faster than realtime, right? And the rest of the CPU that you "can't" utilize would be free to do important stuff like decoding the rest of the audio, responding to user input, etc. etc. right?
I don't see any potential to roll out the loop as there is dependency between iterations.
If loading 5 out of 8 SIMD registers is not enough for you, there are several methods to speed it up:
1. run several filters in parallel,
2. remove the dependence between iterations,
3. run Intel IPP.
And remember that float may not be enough for IIR filtering at low frequencies, you may need double.
Linear-phase systems have less measured ringing, but more audible ringing, because a significant part of their ringing is pre-ringing.
They actually have longer ringing because the main energy is NOT concentrated as close as possible after an impulse (causality! first there's lightning then you hear thunder, not the other way around or a mixture). Instead it has to be
spread symmetrically before and after an impulse.
As such, if you compare a simple 2nd order IIR with an equivalent FIR the FIR will be less efficient and can be
much more inefficient (in case of narrow bandwidth, low center frequency for example) ... unless you can live with inaccurate frequency response and/or ripple.
There may be a point however where combining numerous IIR filters into one FIR could yield computational advantages.
Another thing to note: Min phase filters are easily reversible. The post-ringing cancels beautifully if you cascade two identical filters with opposing gain. That is not the case with linear-phase filters.
Linear-phase systems have less measured ringing, but more audible ringing, because a significant part of their ringing is pre-ringing.
They actually have longer ringing because the main energy is NOT concentrated as close as possible after an impulse (causality! first there's lightning then you hear thunder, not the other way around or a mixture). Instead it has to be spread symmetrically before and after an impulse.
Yes, linear-phase filters have to be somewhat
longer than minimum-phase filters to achieve the same specification for the frequency response. However the amplitude of ringing is
lower in linear-phase filters. See this example:
(http://audio.rightmark.org/lukin/temp/rmm/LinMinPhase.gif)
No there isn't. I had to use matlab's because I couldn't figure out how to get RMAA to show me any phase information at all.
Why would you need a Hilbert transform to measure phase? You just need an FFT of the impulse response for that. If I recall correctly, RMAA does it for you: it displays phase and group delay graphs when the “Impulse/Phase” test finishes. Is it not the case for you?
After a lot of looking I could not find that in the current version. Instead I unwrapped the phase of the linear frequency sweep. Am I just being stupid and missing the option in rmaa? I checked the phase response check but but still couldn't find it.
Hmm, it could have been pulled from the recent version, not sure why. Try this version, it does work for me: rmaa6.zip (http://audio.rightmark.org/lukin/temp/exchange/rmaa6.zip).
Yes, linear-phase filters have to be somewhat longer than minimum-phase filters to achieve the same specification for the frequency response. However the amplitude of ringing is lower in linear-phase filters.
It's not only somewhat, but
much longer if you want a clean filter. The decay of a min phase filter is faster, and the IIR will decay down to zero essentially while with a FIR you have to truncate at some point (else the filter would be extremely long) and apply a window function, which changes the frequency response.
2nd order min phase IIR, -30 dB at 60 Hz, Q=6, Fs=192k vs linear phase FIR. The plot shows 20*log10(abs(filter_output)):
(http://xserv.mooo.com/xnor/audio/images/minvslinear.png)
Higher amplitude ringing that has the main energy concentrated near an impulse is imo quite preferable to long pre- and post-ringing in the audible range of frequencies because of masking effects.
Can you post these impulses as WAV files? Looking at the pics, it's hard for me to believe that these filters have the same frequency response.
I don't see any potential to roll out the loop as there is dependency between iterations.
If loading 5 out of 8 SIMD registers is not enough for you, there are several methods to speed it up:
1. run several filters in parallel,
2. remove the dependence between iterations,
3. run Intel IPP.
And remember that float may not be enough for IIR filtering at low frequencies, you may need double.
1. Sure
2. Would it still be a biquad?
3. Are you saying that Intel IPP does a single biquad employing more than 5/8? How?
-k
2. Yes, it would be numerically identical, albeit somewhat less stable.
3. Perhaps, I haven't measured, but IPP is probably much faster than what you can hand-write.
Hmm, it could have been pulled from the recent version, not sure why. Try this version, it does work for me: rmaa6.zip (http://audio.rightmark.org/lukin/temp/exchange/rmaa6.zip).
This version works great. Thanks!
1. Sure
2. Would it still be a biquad?
3. Are you saying that Intel IPP does a single biquad employing more than 5/8? How?
The point I was trying to make before is that one would never bother to SIMD such a low order filter for audio. The sample rate would have to be MHz before it would be worthwhile.
Usually the filters people SIMD are of larger order and thus much easier to SIMD on very wide vector systems like SSE4+.
Can you post these impulses as WAV files? Looking at the pics, it's hard for me to believe that these filters have the same frequency response.
minvslinear.7z (http://xserv.mooo.com/xnor/audio/stuff/minvslinear.7z)
The FIR doesn't match the IIR exactly but it's pretty close.
Thanks. I'm wondering if this is universal or happens because you are trying to approximate IIR filter with FIR. I know that when you go the other way — convert lin-phase FIR to a min-phase FIR, — the filter length does not change (i.e. the duration of ringing remains the same).
The point I was trying to make before is that one would never bother to SIMD such a low order filter for audio. The sample rate would have to be MHz before it would be worthwhile.
Usually the filters people SIMD are of larger order and thus much easier to SIMD on very wide vector systems like SSE4+.
I guess that is application-dependant. If you do some offline audio processing (e.g. lossy compression), you generally want it done as fast (and low power consumption) as possible. If some (granted small) sub-component can be optimised by 2:1, this might be worth it. If you do realtime processing, the penalty for high load can be high (missed deadline or increased latency to account for possible missed deadlines). In both cases, choosing filter structure based on the #mult can be a non-ideal proxy for choosing based on (realistic) complexity estimates for the given hardware/libraries.
The point that I tried to make was that the classical dsp measures of complexity (#multiplications per sample or per second) motivated by the (for the time significant) silicon area needed to implement a multiplication unit are becoming less relevant as modern architectures offers multiple cores, multiple-element vector units that does N multiplies in the same time as 1 multiply (or N additions). I believe that Intel (and even ARM) tend to spend a small fraction of the area/cost/power on floating-point multiplicators. The probably _could_ increase that count, but various issues (typical application load, typical compiler smartness etc) makes that a bad business proposition.
Thus the question is not "what filter do I choose that minimise the number of mults for a given filtering task", but "what filter do I choose that minimise total system load/cycle count/power draw/whatever performance metric for a given filtering task". Given that you have the unusual luxury of being able to write optimal code for some platform. In practice, it may boil down to "what can IPP/equivalents do for me with the least possible effort"
-k
... I know that when you go the other way — convert lin-phase FIR to a min-phase FIR, — the filter length does not change (i.e. the duration of ringing remains the same).
What is it that you convert, and how do you define "duration of ringing"?
I did a simple MATLAB test, designing some linear phase filter, inverting maximum-phase zeros to have a minimum-phase FIR filter of the same magnitude response. Total impulse response length remains equal, but how do you quantify "duration of ringing" in a way that gives a number <inf for IIR filters? Some "effective" number ala RT60?
When you "render" a filter spec into an actual filter, you loose information about the design objectives. I would assume that designing the final filter directly would come closer to the real goals of some application, rather than design a linear filter, then transform it into some other form?
http://www.dspguru.com/dsp/howtos/how-to-d...ase-fir-filters (http://www.dspguru.com/dsp/howtos/how-to-design-minimum-phase-fir-filters)
N=11;
F = [0 0.4 0.6 1];
A = [1 1 0 0];
b = firpm(N, F, A);
a = 1;
[b,a] = eqtflength(b,a);
[z,p,k] = tf2zp(b,a);
z(abs(z)>1) = 1./z(abs(z)>1);
[b2,a2] = zp2tf(z,p,k);
b2 = b2.*(sum(b)/sum(b2));%normalize DC gain
[H,W] = freqz(b,a,512);
[H2,W2] = freqz(b2,a2,512);
figure(1)
zplane(b)
figure(2)
stem(b)
figure(3)
semilogx(W,20*log10(abs(H)))
axis tight
grid on
figure(4)
zplane(b2)
figure(5)
stem(b2)
figure(6)
semilogx(W,20*log10(abs(H2)))
axis tight
grid on
Thanks. I'm wondering if this is universal or happens because you are trying to approximate IIR filter with FIR. I know that when you go the other way — convert lin-phase FIR to a min-phase FIR, — the filter length does not change (i.e. the duration of ringing remains the same).
No that's impossible since more energy is stored right after the impulse with min phase. Maybe you're just looking at the number of taps being output by some algorithm, but that has nothing to do with the filter's performance.
I don't see a difference if going into the other direction. Well, there is one: if using the 60 Hz filter from above as prototype a simple 2nd order IIR matches the FIR better than the other way around (due to truncation... 262k taps doesn't seem enough ...).
No that's impossible since more energy is stored right after the impulse with min phase. Maybe you're just looking at the number of taps being output by some algorithm...
Of course not! I'm looking at the decay rate of actual ringing, not at the number of taps. And I have already given an example of a filter that I'm talking about, see post #48. I'm attaching a pair of filters, in case you want to analyze: linminphase.zip (http://audio.rightmark.org/lukin/temp/rmm/linminphase.zip).
What is it that you convert, and how do you define "duration of ringing"?
I'm looking at the decay rate, it is easy to see in graphs posted by
xnor.
Of course not! I'm looking at the decay rate of actual ringing, not at the number of taps. And I have already given an example of a filter that I'm talking about, see post #48. I'm attaching a pair of filters, in case you want to analyze: linminphase.zip (http://audio.rightmark.org/lukin/temp/rmm/linminphase.zip).
Regardless of ringing, in the case of such a lowpass in the audio range you should choose min phase over linear phase (or at list a mix towards min) due to potentially audible pre-ringing except if you have very special requirements.
How did you design this filter? What IIR order is it, if applicable?
The point I was trying to make before is that one would never bother to SIMD such a low order filter for audio. The sample rate would have to be MHz before it would be worthwhile.
Usually the filters people SIMD are of larger order and thus much easier to SIMD on very wide vector systems like SSE4+.
I guess that is application-dependant. If you do some offline audio processing (e.g. lossy compression), you generally want it done as fast (and low power consumption) as possible. If some (granted small) sub-component can be optimised by 2:1, this might be worth it. If you do realtime processing, the penalty for high load can be high (missed deadline or increased latency to account for possible missed deadlines).
I think you're not understanding how trivial the filter you proposed is. You're looking at << 1MHz/mbit/s output. For a mid range Core i5 system, to run it at as fast as possible using plain scalar ops would require reading in PCM data at something like 5-10 Gbit/s. You're need an SSD RAID array just to feed that filter! You could process an entire CD in a fraction of a second. On each of your cores. SIMD would be pointless, you wouldn't be able to feed the filter fast enough to keep up
The point that I tried to make was that the classical dsp measures of complexity (#multiplications per sample or per second)
I posted actual benchmarks above for a 2 pole/zero filter on ARM above. I'm not assuming, I'm telling you how fast these things actually run!
I did a simple MATLAB test, designing some linear phase filter, inverting maximum-phase zeros to have a minimum-phase FIR filter of the same magnitude response. Total impulse response length remains equal, but how do you quantify "duration of ringing" in a way that gives a number <inf for IIR filters? Some "effective" number ala RT60?
exp(-1) is very widely used for quantifying the decay of filters. I'd use that unless there was a compelling reason to use something else.
How did you design this filter? What IIR order is it, if applicable?
The lin-phase FIR filter was designed from a frequency response of some analog filter with 48 dB/oct decay. If I recall correctly, it's been a Butterworth filter of order 6. The min-phase FIR filter has been designed by finding a min-phase response for the same frequency response using a Hilbert transform.
To me it's a no-brainer to never use a linear phase filter. As far as I can tell, linear phase equalizers were developed in response to the "phase shift is damaging" nonsense spewed endlessly by misinformed audio magazine writers. The myth that phase shift is a problem was repeated so many times over the years that audio gear makers came up with a way to avoid phase shift, just to use as a marketing claim. The problem is LP filters are audibly worse than minimum phase! You can't always hear the pre-ringing, but often you can depending on the frequency and other settings.
There is one real world example I occasionally encounter where a linear phase eq is useful. An all pass filter isn't audible by itself, but if you use an all pass filter on one track and layer that track against another, then the difference is extremely audible. A good example would be to layer kick drums ... add an all pass filter to just one kick drum and you can affect the combined sound in a very audible way. Likewise, eq-ing the low frequencies of a single kick drum can occasionally introduce enough phase shift to be audible not in isolation, but when combined with other kick drums. So if your 808 kick was already in phase with your other kick/bass tracks perfectly and you wanted to take out some 60 hz, a minimum phase eq would introduce phase shift and maybe some odd cancellation against another track, and you can use a linear phase eq to avoid the problem.
This is admittedly a very rare case and I've never used it in a song, but I have used all-pass filters to get kick drums to sit against each other better.
IPP is probably much faster than what you can hand-write.
IPP is (in some ways) like a compiler: it needs to solve a (set of) generic problems that is sufficiently "wide" so as to be useful for many users, yet sufficiently "narrow" so as to allow the low-level gurus to do their thing. And AFAIK, they solve that generic problem really, really well, across a range of intel platforms. If what you need is to solve that generic problem, you probably don't want to compete with Intel.
If, however, your problem deviate from the general case in any way, there is a window of opportunity to beat IPP. For a IIR filter, this might be things like precision requirements, knowledge about coefficients, knowledge about frame sizes, targeting a single cpu architecture etc. Solving specific problems is a lot easier than solving general problems.
While software developers often fall in the trap of "over-generalization" (writing code to handle every possible future requirement), I am talking about "degeneralization" (solving what needs to be solved now and nothing else).
-k
(http://audio.rightmark.org/lukin/temp/rmm/LinMinPhase.gif)
Wheee! A chance to finally ask a question I've been wondering since forever but never knew how to phrase.
There are two functions depicted in this image (perhaps only approximately, but whatever, context). The first is sinc, which extends to infinity in both directions. The second is what? It extends to infinity only in one direction.
I don't know if any closed-form expression exists for this function. Strictly speaking, a perfect sinc does not have a min-phase variant — only an approximation. In my example, it is a minimum-phase filter which can be obtained from a linear-phase filter by either spectral factorization or complex cepstrum folding.