Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Nyquist was wrong?! (Read 60057 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Nyquist was wrong?!

I read something rather interesting today.  I browsed through the
headphone amp designs available at headwize.com and found this article.

Look at figure 1.  The guy explains why 44.1kHz isn't fast enough to
sample a 21kHz signal.  Nyquist was completely wrong!

Now.. if you look more closely at the second picture of figure 1, you
will notice a flaw in the "21kHz image-signal" picture.  He simply
performs a linear interpolation between every point. (!)

How could such an intelligent guy makes such a mistake?  I mean, read
the article.. you will see this guy is far from being stupid.

Am I missing something?!?!

Nyquist was wrong?!

Reply #1
I am in no means qualified to really discuss this... it seemed logical to me though. In what sense do you feel he was incorrect though? How would you draw it differently?

Nyquist was wrong?!

Reply #2
I've never read such a crock of lard in my life. Nyquist would be scandalized. Clearly this character doesn't know the first thing about oversampling, sigma delta modulation, and decimation filtering. Instead, he advocates using a less-than-brickwall analog filter to "warm up" the sound in the passband. Hey - let's add some phase distortion (among other types) to the passband and completely ignore the fact that converters these days make it unnecessary to filter out the narrow gap between the edge of the passband and 22.05kHz. This fellow's project was a museum piece back in the mid eighties when the first 4X oversampling machine hit the stores.

Why can't people who think they know what they're talking about take a bit of effort to read up on DSP fundamentals...

:x

Nyquist was wrong?!

Reply #3
edit: I'm backing out of this conversation while I still can

Nyquist was wrong?!

Reply #4
Quote
I am in no means qualified to really discuss this... it seemed logical to me though. In what sense do you feel he was incorrect though? How would you draw it differently?

If I didn't know I was dealing with sines, I could use a quadratic, cubic, Nth order polynomial function to approximate what really was the function (the data between the two points).  A linear approximation is really crude.

Knowing I am dealing with sines, well... that's even easier.  There is mathematically one and only one way to represent a sine given two points (with only two points you can represent it perfectly.. frequency wise and amplitude wise).  I am not inventing anything here.  This has been discussed amply.

So why does he say you cannot represent a 21kHz sine (well.. he says anything beyond 15kHz I think) correctly when sampling at _only_ 44.1kHz?  His linear interpolation is debatable to say the least.

I am not saying 16/44.1 is perfect.  Don't want to go there either.  I was just wondering if I was missing his point or something...

Nyquist was wrong?!

Reply #5
Exactly. The shortest distance between two points is not a straight line when dealing with a band-limited signal. By drawing straight lines, he completely misses the point and introduces spanking brand new harmonics to the original sampled signal outside the passband. I think we learned not to draw straight lines on the first or second day of my 4th year Discrete Time Systems course...

:x

Nyquist was wrong?!

Reply #6
Phony. The guy takes the sampling of a 41.5 kHz wave, and since it's like a 2.5kHz wave, he extrapolates backwards and says that's a problem at 21kHz, which it's not.

Nyquist was wrong?!

Reply #7
I see. I would really like to learn about this stuff, starting from a pretty ignorant background - any tips as to a good website or book? I thought his description was pretty easy to follow, too bad it is apparently bulls**t

Nyquist was wrong?!

Reply #8
I don't think Meier adequately (or correctly) described why the ringing occurs in the first place, and therefore I can see why you all are missing the point. It is definitely not because DSP algorithms use linear approximations. His picture is misleading. Series (Fourier, Laplace, etc) are often used to represent analog signals (mathmatically sound by the way).

Some sort of low-pass filtering must be used for digital conversion; you notice CDs cut off around 22kHz (44.1kHz /2, according to Nyquist's formula). Ringing occurs when an infinite series is truncated (discontinuous), and occurs with any practical digital filter. This is known as Gibbs phenomenon and is mentioned in the article. According to Meier this ringing is most pronounced in the case of a hard cut-off (or brick-wall filter).  Meier's filter apparently uses a method to ease this hard cut off, and thus lessen the effect of Gibbs phenomena (according to Meier). I have to admit I haven't taken enough signal processing classes to say if it would work or not, or even if common modern digital filters are inadequate in this regard.

Because this ringing is due to the filter, it doesn't make sense to say Nyquist's formula is wrong.


Nyquist was wrong?!

Reply #10
Maybe he just put the linear-approximation diagram on the page to encourage people to buy his $260 Analoguer. The rest of the post might at first seem reasonable, though (to an undergraduate EE student like myself): the more gradual the low-pass, the less ringing present. For people with poor high-frequency hearing such as Meier, high-frequency attenuation won't hurt, so you get rid of the ringing "for free".

But from what I've read in my digital audio books, modern low-pass filters don't suffer from significant ringing, even with relatively sharp filtering. And even then, if the brick-wall filter introduces high-frequency ringing (I think that was what Joe Bloggs briefly mentioned at the beginning of this thread), Meier won't notice it, because he has already admitted that he can't hear high frequencies. What's the point of filtering out something you can't hear? If there is aliasing present in the recording, a high-pass filter won't fix it without removing the frequencies that should be present. Perhaps I'm committing a serious fallacy here, but to me, the Analoguer appears to be pretty useless.

And also, some of this was discussed in an ancient thread back in 2001...

@MTRH: this isn't an FB2K discussion (that was appalling). Try not to post such useless comments.

Nyquist was wrong?!

Reply #11
It should also be noted that this ringing occurs in the frequency domain, not the time domain (Meier's article makes it look the other way). That is, it creates a ripple on the amplitude of the frequency response close to the point of discontinuity, and graduately evens out as frequency decreases. Check out the animation on this page for an illustration of this effect.

Jebus: Meier's explanation for why there is ripple (and what kind of ripple) seems to be incorrect, and does not give credibility to his design.

SometimeWarrior: (off-topic) You go to UCSB? I'm at Cal Poly, computer engineering...

Nyquist was wrong?!

Reply #12
Quote
SometimeWarrior: (off-topic) You go to UCSB? I'm at Cal Poly, computer engineering...

Yes, I'm a computer engineering student at UCSB. They've somehow avoided teaching me what a Fourier series is, after two years. Maybe I should have attended SLO instead? 

I'll dig up my math book and look again at your link in a couple of days, thanks. I'll probably end up eating my words...

Nyquist was wrong?!

Reply #13
Don't torture yourself man! We didn't get to Fourier analysis until the end of our second year, and it was not the most exciting thing in the world. I've been avoiding my next signal analysis class like the plague. I avoided looking at the equations on that page; the animation was the part I found interesting.

But hey, I'd have to say that SB just may have more hotter women - but it's a close call. Can't wait 'till it summer weather  B)

Nyquist was wrong?!

Reply #14
Quote
Phony. The guy takes the sampling of a 41.5 kHz wave, and since it's like a 2.5kHz wave, he extrapolates backwards and says that's a problem at 21kHz, which it's not.

No no, I don't think it's phony. He sampled a 21kHz wave @ 44kHz (which is okay), but then he interpolated linearly (which is the problem).

Except for that incorrect example though, I think this paper is (mostly) correct.

Question 1: would that 21kHz wave be accurately reconstructed after the 44kHz sampling, if correct interpolation were used (ie: sinc) ? Nyquist says yes.

Question 2: for transients, what is more important: the shape or the curve, or the precise harmonic content ?

Nyquist was wrong?!

Reply #15
Quote
A little question: would the wave be accurately reconstructed after the 44kHz sampling, if correct interpolation were used (ie: sinc) ?


The problem occurs when a low-pass filter is introduced, and this is the issue Meier is trying to address. A sine wave at a fixed frequency of 21kHz should be able to be reproduced accurately, and so would a sine wave at a fixed frequency of 19KHz. The real problem is that they may have inaccurate amplitudes. The problem is in the frequency response, and it's not clear if Meier even realizes this. It would be nice if he labeled the axis of his graphs. He describes a picture of a sine wave at a fixed frequency of 21KHz wobbling; this is not correct as I understand it.

So the answer to your question: depends on the quality of the filter. But there is an ongoing debate of whether 44.1kHz is high enough to begin with - so there may be no easy answer.

Nyquist was wrong?!

Reply #16
freakngoat:

I can understand that the ringing problem makes sense, if the ear is more sentitive to the graphical shape of transients than to their harmonic content (which is very possible...).

About sampling, though, I had always thought that any [0Hz .. 22kHz[ tone could be reproduced exactly when using 44kHz sampling and no quantization.  Well, it seems that any such tone can be reproduced, but not always accurately ? Or, is his interpolation solely at fault ? Would "sinc" solve that problem ? I don't understand that..

Nyquist was wrong?!

Reply #17
Right: here are the facts.

A brick wall low pass filter should ring. You see his first criticism - that the 21kHz sine wave is amplitude modulated? The ringing removes the amplitude modulation. It's the ringing that fills in the gaps between the sample points, and also maintains the 21kHz wave at the correct amplitude in the regions where the samples appear to be low in amplitude. Without the ringing, you'll get the amplitude modulation. Pick your poison!


This begs an obvious question: if the "correct" filter rings on and on like this, surely that causes problems when the 21kHz sine wave stops. Surely, in this system, it can't just "stop dead"? Correct!

The thing is, if a 21kHz sine wave just "stops", it will generate a click, which has frequencies WAY above 22kHz (the nyquist limit). You can't have a sine wave that stops dead, but has no higher frequency components at the stopping point. It's a fundamental of audio (and other areas too!) - you can even try it practically in Cool Edit! So, nyquist implies that a 21kHz tone can't stop dead, because this would mean it contained frequencies way above 21kHz - i.e. above the nyquist limit.

This means that, the nearer the frequency is to the nyquist limit, the poorer the time resolution is (in terms of cycles taken for the sine wave to decay to some fraction of it's former amplitude) - but this isn't a fault - it's just the physics of bandlimitting the signal.


All he's doing in the article is making a gentle filter. There's no magic. A gentle filter won't ring as much. This means that either:
a) you allow frequencies higher than the nyquist limit through - which will give you apparent amplitude modulation of the 21kHZ tone (he didn't show you that his box still does that, but it may!), or
B) remove frequencies below the nyquist limit. This will give a dull sound. Also, because there's almost no 21kHz signals left after you've killed them with the gentle filter, you're less likely to notice any ringing that may remain.

He's doing both. In a way, this is ideal. With higher sampling rates, you can use a gentler filter, and there will be no ringing or amplitude modulation to speek of. BUT at 44.1kHz, it's a difficult compromise. But it's not magic. You can either:
a) keep all the frequencies up to nyquist, kill all the ones above, and have lots of ringing at the nyquist frequency, or
B) compromise this, reducing ringing, but letting more frequencies above nyquist through, or reducing the amplitude of some frequencies below nyquist that may be audible.


You shouldn't be able to hear ringing at 22kHz - but that's with perfect equipment and old ears. There's a theory that when the energy of an impulse is stretched in time, even though the stretching is only at inaudible frequencies, that it make an audible difference. This is because real audio equipment (especially speakers) produces distortion with all sounds, but not with silence. So, you're adding distortion before and after all sounds. Thinking about how sensitive we are to pre-echo (think about mp3!), and how long a true brick-wall filter will ring for (infinity!) - if the equipment makes this inaudible ringing become audible, then it could be a real problem.

In practice, it's all very subtle stuff, creating very subtle effects. But it's one argument for higher sampling rates - you can use gentle filters and avoid any possible compromise. The best thing at 44.1kHz seems to be to filter gently, starting at around 18kHz, making sure it's nearly dead by 22kHz. Because of the compromise, it makes designing good "sounding" nyquist digital filters as much of an art, as it is a science.


Coming back to the original article, in his box, one of the filters is 2dB down at 10kHz and 4dB down at 15kHz - it's hardly surprising that it makes things sound "less harsh" - try this filtering in Cool Edit and things will sound less harsh too!

Cheers,
David.

Nyquist was wrong?!

Reply #18
NumLOCK: I've just been reviewing my books, and yes, mathematically it can be proven that a 44.1kHz sampling rate is able to recover the exact original waveform with a bandwidth of 22kHz (Nyquist's theorem) so you are correct.

Perhaps the basis for the debate on whether 44.1kHz is high enough has to do with the effect of ringing from the lowpass filters being used.  (48kHz would allow a bandwidth of 24kHz and more diminished ringing in the audible range). Or maybe it's because various alterations to the waveform typically being applied these days (effects, equalizers) diminish the original signal less at higher sampling rates. Unfortunately I don't know the answer.

Strictly speaking, all approximation by the analog to digital process is done in the quantization step as long as Nyquist's theorem is satisfied. This is where more bits help.

Nyquist was wrong?!

Reply #19
Quote
A brick wall low pass filter should ring. You see his first criticism - that the 21kHz sine wave is amplitude modulated? The ringing removes the amplitude modulation. It's the ringing that fills in the gaps between the sample points, and also maintains the 21kHz wave at the correct amplitude in the regions where the samples appear to be low in amplitude. Without the ringing, you'll get the amplitude modulation. Pick your poison!


I don't see any reason why the sampling process would add amplitude modulation to the original waveform. The whole idea is to reproduce the original waveform, and this can be done precisely since 44KHz > 2*21kHz. Certainly the sampling process is not used as a demodulation step. Therefore I can see no reason why ringing would ever be desirable.

Also, you talk about how "long" a filter will ring for. I'm not sure what you mean by this. Do you mean the bandwidth of frequencies which produce noticable ringing? It is my understanding that this ringing is in the frequency domain, not the time domain - however I am going to research this further.

As far as the click at a sudden cutoff, you may be right, but it is not discussed in any of the papers, books, or webpages I have read so far. Since a true brick-wall filter would have a slope of infinity and therefore be non-differentiable at the cut-off point, practical ones do not exist and therefore my be a non-issue. All practical filters have some slope. From what I've read the biggest problem with a very steep filter is the ringing effect. You are right in that if you can't increase the sampling rate some sort of softer filter is needed to reduce ringing. I don't think the answer is to have a filter transisition beyond the Nyquist limit, since this will introduce aliasing errors. My guess is that some sort of gentle filter starting around 20kHz would be good and would not audibly dull the sound.

My question is, what type of filters are typically being used these days for audio? Maybe we're talking about something so slight that it doesn't even make an audible difference. It would be nice to hear some samples.

Just a side note: This is not just a problem in audio, this is also a problem in digital imaging as well, and there are well known filters (Lanczos, cubic) to reduce the same ringing we've been talking about due to Gibbs Phenomenon.

Nyquist was wrong?!

Reply #20
Quote
I don't see any reason why the sampling process would add amplitude modulation to the original waveform.

Sampling does exactly this to frequencies near the nyquist limit - if you look at the sample points themselves, rather than the reconstructed waveform. It's what he's trying to show in his first figure.

At the DAC, before the low pass filter, the spectrum extends to infinity (theoretically), each sample having infinite height, infinitely short duration, with an area equal to the amplitude value. That's the theory. In practice, the samples are often reproduced as a staircase waveform - this still has a near-infinite spectrum (just as a "real" square wave has an infinite spectrum). Most importantly, from 22-44kHz, this spectrum is a mirror image of the real spectrum from 0-22kHz.

That's what causes the "amplitude modulation" if you just look at the sample points themselves. For a 21kHz tone, there's also a 23.1kHz alias - without a filter, the two beat - that's what you see in his first figure.

Quote
Also, you talk about how "long" a filter will ring for. I'm not sure what you mean by this. Do you mean the bandwidth of frequencies which produce noticable ringing? It is my understanding that this ringing is in the frequency domain, not the time domain - however I am going to research this further.


I read this earlier up the thread, but forget to comment - the ringing is in the time domain. Frequency domain wobbles in the filter response have little to do with this discussion. So, by length, I mean duration.

Quote
As far as the click at a sudden cutoff, you may be right, but it is not discussed in any of the papers, books, or webpages I have read so far.


That's because I haven't written any yet  Seriously, this is advanced stuff. The ideas come from some backround research which I did for a post doctoral research project which never happened! They're not all my ideas - I spoke to a lot of clever people.

Quote
Since a true brick-wall filter would have a slope of infinity and therefore be non-differentiable at the cut-off point, practical ones do not exist and therefore my be a non-issue.


Yes - infinite slope = infinite duration. Impossible. Rather than thinking about making a finite slope, and seeing what the result will be in the time domain, it's easier to take the infinite length time domain filter (a sinc function) - chop it (or window it) down to a sensible length (e.g. a few thousand samples), and see what the result is in the frequency domain. It's the practically the same either way, but I find it easier to think about it this way. You can do it all in Cool Edit with an impulse (drag one sample up), the FFT filter, and the frequency analysis window.

Quote
All practical filters have some slope. From what I've read the biggest problem with a very steep filter is the ringing effect. You are right in that if you can't increase the sampling rate some sort of softer filter is needed to reduce ringing. I don't think the answer is to have a filter transisition beyond the Nyquist limit, since this will introduce aliasing errors. My guess is that some sort of gentle filter starting around 20kHz would be good and would not audibly dull the sound.


Yes. Though, in the analogue world (I know we're not talking about the analogue world, but anyway...) a filter which drops from 0 to -90dB within 1/10th of an octave isn't called "gentle"!

Quote
My question is, what type of filters are typically being used these days for audio? Maybe we're talking about something so slight that it doesn't even make an audible difference. It would be nice to hear some samples.


I can't help you there (how would you listen to them anyway?!) - but you can read...
http://www.dcsltd.co.uk/papers.htm
- the second paper addresses the issue of energy smearing, which is caused by (and, in effect, is) ringing.

"typical" filters are a compromise, and are usually realsed as cheaply as possible. You can download specifications from several companies e.g.
http://www.analog.com/UploadedFiles/Datash...892AD1855_b.pdf

Quote
Just a side note: This is not just a problem in audio, this is also a problem in digital imaging as well, and there are well known filters (Lanczos, cubic) to reduce the same ringing we've been talking about due to Gibbs Phenomenon.


Though the best compromise is vastly different in image processing. Images look OK with the kind of aliasing that would sound terrible in audio. Conversely, audio sounds OK with the kind of ringing that would look awful in images!

Nyquist was wrong?!

Reply #21
We discussed about all this some time ago: http://www.hydrogenaudio.org/forums/index....2957#entry29125

Quoting myself, here are my views:

Quote
Ringing in filters happens whenever there is an abrupt discontinuity in the frequency response of the filter. If there is sonic "content" in the original signal at the frequency this discontinuity exists, pre or post ringing will appear. The more abrupt the discontinuity (~ higher slope, or steeper filter), the greater the time-domain ringing.

In CD brickwall filters, the discontinuity happens near 22 KHz. So, if there is any ringing, it will appear at these frequencies. Also, for this ringing to appear, the signal must have content at these frequencies.

Thinking a little bit more about this, in a properly recorded cd signal, there should be no content at these frequencies, because it must have been filtered at the AD stage in order to avoid aliasing. However, this AD pre-filtering can also produce ringing, if there was any content at the filtering frequencies in the original signal. So, if there is any ringing, it is already present in the recorded signal, and (most likely) not produced by the cd player brickwall filter. I say most likely because if the CD player filters at a lower frequency than the AD filter, it will eliminate the AD filter ringing, but will introduce its own. However, I don't think this is likely to be the case. Also, in the case of same synthetic generated signals, there can be no AD stage, then the ringing would happen due to the player filter, but I also think this is not very common, or likely to happen in commercial CD's.

So, there can be ringing in the cd signal, but this ringing will be near 22 KHz, which is inaudible. Also, using a proper not ultra-steep AD filtering, this ringing time duration can be minimized.

Although 22 KHz is not audible, if there are nonlinearities in the playback path at this frequencies, this 22 KHz ringing might intermodulate with other signals and produce intermodulation products which fall into the audible band of frequencies, and in fact "became" audible. However, and as recapitulation, for this intermodulation products to happen and be audible (which is our "final" main concern now), some conditions have to be satisfied:

- There must be signal content in the original signal at 22 KHz. For the ringing to have audible significance, this 22 KHz content must be of relatively high amplitude, and of "transient" type, that is, of very short attack or decay times.

- The AD stage pre-filtering must be quite steep. I believe that with a relatively "soft-edge" filter, the ringing duration and amplitude can be quite minimized. However, I don't know how steep are actually the filters commonly used.

- The playback chain (mostly speakers or headphones) must be able to reproduce this 22 KHz signals, and also be quite nonlinear a this frequencies. I think that good speakers of headphones capable of reproducing such high frequencies are not likely to be very nonlinear. However, I could be wrong.


So, I think that this ringing in CD audio is very, very difficult to be noticeable, and even if it was a real problem, could be effectively adressed using proper AD pre-filtering.


I think that usual FIR filters at players have quite short pre-ringing, due to the limited number of taps used.

Edit: corrected wrong link.

Nyquist was wrong?!

Reply #22
Quote
Question 1: would that 21kHz wave be accurately reconstructed after the 44kHz sampling, if correct interpolation were used (ie: sinc) ? Nyquist says yes.

I'm not an expert in signal processing (yet), but i think the problem is cardinal (sinc) interpolation is practically impossible to do in real-time, since you should know the amplitude of samples which are after the ones you're processing (in the time domain). Someone correct me if I'm saying bulls**t.

Norman

Nyquist was wrong?!

Reply #23
WTF are you guys talking about!!!!!

I thought Chemistry was difficult!!

auldyin

Nyquist was wrong?!

Reply #24
Quote
I'm not an expert in signal processing (yet), but i think the problem is cardinal (sinc) interpolation is practically impossible to do in real-time, since you should know the amplitude of samples which are after the ones you're processing (in the time domain).

Yeah, that's why FIR filters cause pre-ringing: they begin to output filtered signal *before* the actual signal has come out of the filter. They work realtime by delaying the signal a few ms., they need to take data in advance, in order to work. I don't know if I've explained myself very well...