HydrogenAudio

Hydrogenaudio Forum => General Audio => Topic started by: KMD on 2012-03-10 10:58:57

Title: Quantization Grid
Post by: KMD on 2012-03-10 10:58:57
I have thought that this phenomana may be real for years . I call it the "quantiztion grid". An  analog waveform can change direction or cross the zero line at an infinite number of locations. Sampling and Quantization create a grid to which features of  the waveform are  statistically constrained.  Increasing the sample rate at a fixed bandwidth inreases the quantization grid density. Whether it is perceivable is what is in question.  This experiment may have proved it. I will give some thought to a simillar experiment that forum members can do for themselves  and start a new topic on it.
Title: Quantization Grid
Post by: AstralStorm on 2012-03-10 16:28:50
There's no "quantization grid", it's a myth. PCM can and will reproduce audio with sub-sample delay, assuming the signal is band-limited to your sample rate.

Dead horse flogging alert: http://www.stevehoffman.tv/forums/showthread.php?t=85436 (http://www.stevehoffman.tv/forums/showthread.php?t=85436)
http://www.hydrogenaudio.org/forums/index....49043&st=25 (http://www.hydrogenaudio.org/forums/index.php?showtopic=49043&st=25)

Before you ask, it is trivial to band-limit the signal, either using right analog microphones or with a brickwall filter.
Title: Quantization Grid
Post by: KMD on 2012-03-10 20:30:57
The quantization grid is not caused by sampling it is caused by the interaction of quantization levels  with  sampling points. It may not be perceptable but there is no doubt that a waveform created from  a digital  file must be formed from a selection of points that are selected from a finite number of pre- determined regularly spaced co-ordinates. The digital file is formed from regularly spaced sampling points and reguarly spaced quantization levels therefore the waveform derived from it must have a coresponding regularity.
Title: Quantization Grid
Post by: lvqcl on 2012-03-10 20:43:38
The digital file is formed from regularly spaced sampling points

This means that the reconstructed signal doesn't have frequencies >=  Fs/2.

and reguarly spaced quantization levels.

This adds constant white noise to the signal (simplest case: TPDF dither, no noise shaping).


IOW: there is no interaction between sampling and quantisation if the signal is properly dithered.
Title: Quantization Grid
Post by: KMD on 2012-03-10 21:05:26
The output of the D to A is derived from a regularly spaced set of coordinates so it therefore has that reqularity.
Title: Quantization Grid
Post by: Speedskater on 2012-03-10 21:16:04
About a year or so ago, j.j. wrote:

In fact, we can resolve, at 20kHz, for a "Redbook CD", a time resolution proportional to, and very near  1/(2*pi*20000*216) in a mechanical fashion.

That is a very, very large number of time points per second.
It's about 27143360.
Title: Quantization Grid
Post by: lvqcl on 2012-03-10 21:21:20
The output of the D to A is derived from a regularly spaced set of coordinates so it therefore has that reqularity.


Here is the graph of the CD noise floor with a standard TPDF dither:
Title: Quantization Grid
Post by: KMD on 2012-03-10 21:21:57
Just for clarify when I say co-ordinate I mean x and y. The cordinates would be more than just 2^16 *44100 per second. I'm thinking twice that or more. Any Idea how  jj got that figure.
Title: Quantization Grid
Post by: KMD on 2012-03-10 21:24:24
ivqci - I am talking about regularity in the shape of the waveform
Title: Quantization Grid
Post by: lvqcl on 2012-03-10 21:31:27
But we cannot hear the shape.
Title: Quantization Grid
Post by: KMD on 2012-03-10 21:39:36
lvqcl-  But whatever we  do hear must be derived from the regularised data set wouuld you concur
Title: Quantization Grid
Post by: googlebot on 2012-03-11 11:50:16
lvqcl-  But whatever we  do hear must be derived from the regularised data set wouuld you concur


This regularity is effectively removed by dithering.
Title: Quantization Grid
Post by: NullC on 2012-03-12 00:30:07
This regularity is effectively removed by dithering.


Not just "wave your arms"/good enough "effectively".  But it's removed  completely and replaced with your-choice-shaped additive noise by proper dithering&noise shaping.

Like the sampling theorem itself this isn't particular intuitive though it becomes clear if you actually step through the mathematics.  The first couple of sections of this thesis (http://uwspace.uwaterloo.ca/handle/10012/3867?mode=full) explain it pretty well.
Title: Quantization Grid
Post by: KMD on 2012-03-12 14:58:45
The movement of the quantization levels by dithering averages to zero over a number of points. Statistically the dithered quantization levels still fall on the original quantization points plus a deviation ether side with the statistical distribution  shape determined by the noise used. I think it would be useful to be able to statistically analyse the shape of a piece of audio to asertain what process recorded it.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-12 16:08:35
ivqci - I am talking about regularity in the shape of the waveform
Yes, and what you say is true - easily visible in the waveform, and sometimes visible on a scope (depends on the DAC).

However, there's no mechanism in the ear to pick out that regularity. Once you do any kind of filtering (and the ear contains a continuum of bandpass filters), it disappears. You're just left with what looks like random noise.


Also, the error signal (original minus quantised = dither + quantisation error) contains remnants of the original signal. The classic Lipschitz and Vanderkooy dither paper from 1984 (I think!) showed graphs of error signals for various orders of dither. rectangular = first order, triangular = second order. More orders add more noise, but make the error even less correlated with the original. As the ear seems insensitive to higher orders of correlation, there's no need to go further.

This is all ignoring the real noise and signal levels in most recordings which make this (and even dither itself) academic.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-12 16:23:26
when you say can be seen on a scope I guess you mean before the reconstruction filter.  If so the reconstruction filter forms a waveform from a  product of those samples and quantization points  and  so there is literally trillions of points in just a second to define the smoothed  output
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-12 17:23:24
when you say can be seen on a scope I guess you mean before the reconstruction filter.  If so the reconstruction filter forms a waveform from a  product of those samples and quantization points  and  so there is literally trillions of points in just a second to define the smoothed  output
No, with the right test waveform the effect can be quite visible after the reconstruction filter. Mathematically you'll get different values after the oversampling/reconstruction filter, but visually you can see remnants of the original quantisation steps.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-19 10:14:37
Thanks for the discussion.

My conclusion on this is. - There is no quantization grid. And no regimentation effect in amplitude due to quantization when dither is used.
As dither scans the input up and down over the quantization steps, it makes quantization benign like sampling. In fact the word Quantization is innapropriate for the finished process as the resulting waveform is not a quantized waveform!  It has infinite resolution because the dither noise fills the space between quantization steps, so there is no room for anything to go undetected. This is another reason why dither is important.

Cheers

Owen.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-19 12:39:48
Thanks for the discussion.

My conclusion on this is. - There is no quantization grid. And no regimentation effect in amplitude due to quantization when dither is used.
As dither scans the input up and down over the quantization steps, it makes quantization benign like sampling. In fact the word Quantization is innapropriate for the finished process as the resulting waveform is not a quantized waveform!  It has infinite resolution because the dither noise fills the space between quantization steps, so there is no room for anything to go undetected. This is another reason why dither is important.


I question that *any* real world process whether analog or digital, has infinite resolution.  A process with infinite resolution would output a signal that is exactly the input signal. The error that would be calculated by subtracting the input and output signals would always be exactly zero.

In a proper digital process with randomized quantization, the resolution is not infinite and the timing and level errors inherent in quantization are randomized by dither. There are no other errors unless they are intentionally introduced.

In an analog process the resolution of storage, processing and transmission is also not infinite. There is practically speaking no such thing as infinite resolution in the analog domain. The timing and level errors inherent in real world analog audio are produced by natural effects such as thermal noise other noise, as well as transmission and processing losses (or gains). These noises and errors come from many sources. Even reasonably short transmission lines and their line drivers and receivers cause errors. All known forms of analog signal storage introduce relatively massive errors.


Every resistor, capacitor, transducer, transformer, inductor and active or passive device in the analog domain adds noise and amplitude and timing errors unless it is perfect and is operating at absolute zero
. That pretty well excludes everything in every real world audio experiment from the realm of things with infinite resolution.

So, you are comparing errors and non-infinite bandpass and resolution in the analog domain with errors and non-infinite bandpass and resolution in the digital domain.

Nothing in the real world is perfect or perfectible or even close to perfect except relatively abstract things like digital data storage and transmission.

At this time the only things that are even close to be being perfect at performing audio tasks in the analog domain are relatively short transmission lines, a small but enlarging group of operational amplifiers, and digital<-> analog converters. At this time it appears that what drives the development of low noise, low distortion operational amplifiers is the development of digital<-> analog converters.

It has been found by many experimenters and observers that relatively inexpensive digital recorders and players can be free of audible flaws in normal use. This was never achieved and is probably unachievable with purely analog devices.
Title: Quantization Grid
Post by: KMD on 2012-03-19 13:28:48
Hello Arnold


Signal to noise ratio, and resolution, are not the same thing.




Owen.
Title: Quantization Grid
Post by: icstm on 2012-03-19 14:14:12
Is Arnold not saying that the output signal might replicate the input information perfectly, but will also introduce noise. So actually the output signal is not the same as the input even if all the orginial information is there.

Quote
A process with infinite resolution would output a signal that is exactly the input signal. The error that would be calculated by subtracting the input and output signals would always be exactly zero.
this I agree with
Title: Quantization Grid
Post by: KMD on 2012-03-19 14:19:53
Hi icstm

An analogue recording system has infinite resolution but not infinite signal to noise ratio

A dithered digital recording system has infinite resolution but not infinite signal to noise ratio


Owen.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-19 14:24:34
Until you define the meaning of the term "resolution", such statements are meaningless.

If they include the word "infinite", they're probably wrong, too.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-19 14:29:05
Resolution is the smallest detail a system can record. In an undithered digital system that is a detail with height greater than one quantisation step. In a dithered system it is infinitely small.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-19 14:48:30
Resolution is the smallest detail a system can record. In an undithered digital system that is a detail with height greater than one quantisation step. In a dithered system it is infinitely small.
It's only infinitely small if it persists for infinitely long, the system is DC coupled, and I'm allowed to average the result over an infinite number of samples.

Try again

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-19 15:00:53
Sampling has nothing to do with it . Sampling is totally benign. The thing under scrutiny is quantisation. details small in duration can also get lost
in an analogue recording if the noise happens to be its mirror image for its duration. Resolution refers to the peak to peak level of siganal that that can be recorded hence resolved
Title: Quantization Grid
Post by: icstm on 2012-03-19 15:07:49
yes, but what makes you say a analogue signal can be resolved to infinite precision with 100% accuracy?
Title: Quantization Grid
Post by: KMD on 2012-03-19 15:11:28
I did not mention accuracy and precision. The subject matter is resolution, the smallest  peak to peak signal that can be recorded hence resolved. The resolution in a dithered digital system is infinite because the dither noise takes up the space between quantisation steps and so there is no room for a small peak to peak signal to go undetected.
Title: Quantization Grid
Post by: icstm on 2012-03-19 16:14:21
resolution is comparible with precision it is the "how many decimal places can I measure this to".
Accuracy is the "and are the significant digits right"
To say somthing has infinite resolution is only useful it that measurement is correct.
In the anologue world this is impacted by the sensitivity of the system. Are you saying you have an infinitely sensitive system?
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-19 18:03:53
I did not mention accuracy and precision. The subject matter is resolution, the smallest  peak to peak signal that can be recorded hence resolved. The resolution in a dithered digital system is infinite because the dither noise takes up the space between quantisation steps and so there is no room for a small peak to peak signal to go undetected.
Detected by who? Using what?!


btw, useful measures are noise, distortion, and monotonicity.

No one doubts that digital resolution can be infinite, in theory (see above). In practice, the same limits impact both analogue and digital. Pesky electronics. this is The weak point of digital storage. The weak point of analogue storage is elsewhere (the medium, the transduction, etc).

Cheers,
David.
Title: Quantization Grid
Post by: knutinh on 2012-03-19 22:32:07
Being able to recreate infinitely small signals would indicate that there are an infinite number of different, resolvable signals between "1" and "1/inf". I guess that would imply that the system could encode an infinite amount of information on a storage medium of finite capacity?

-k
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 09:38:40
Being able to recreate infinitely small signals would indicate that there are an infinite number of different, resolvable signals between "1" and "1/inf". I guess that would imply that the system could encode an infinite amount of information on a storage medium of finite capacity?
...which is tantamount to inventing a perpetual motion machine.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-20 09:57:06
- icstm  ,  you are confusing resolution with signal to noise ratio.
Title: Quantization Grid
Post by: icstm on 2012-03-20 10:28:15
no, I am saying that resolution is only relevant in the context that of SNR.
For an instrument to have high resolution and for that to be of use we are saying it offers more significant digits in its reading than something else. I would suggest that can only be the case if what it is measuring is the real signal.
Title: Quantization Grid
Post by: KMD on 2012-03-20 10:57:50
No, the small variations in the waveform are there in the lower significant bits with noise added
Title: Quantization Grid
Post by: greynol on 2012-03-20 14:09:44
Why do you keep insisting that dither must be present in order for there to be voltages between quantization steps in a reconstructed digital signal?

Tell me KMD, do you have any formal training or is this just from the seat of your pants?
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 16:33:21
Why do you keep insisting that dither must be present in order for there to be voltages between quantization steps in a reconstructed digital signal?
For a low-ish frequency input signal, ignoring noise etc, most of the instantaneous output voltages will be essentially "on" the quantization steps. You can see this very easily at low amplitudes.

Cheers,
David.
Title: Quantization Grid
Post by: greynol on 2012-03-20 17:11:01
I'm more than a bit skeptical of that.
Title: Quantization Grid
Post by: pdq on 2012-03-20 17:36:23
I'm more than a bit skeptical of that.

I'm surprised that that surprises you. It seems rather obvious to me. Perhaps someone will generate a low-ish frequency sine wave in 8 bits without dither and show us the reconstructed waveform?
Title: Quantization Grid
Post by: greynol on 2012-03-20 17:46:30
Are you trying to tell me inter-sample points don't transition smoothly as if the reconstruction filter doesn't do its job?

And why are we stipulating that the bandwidth be somewhere other than Nyquist?!? It's not as if the OP had also included such a stipulation.
Title: Quantization Grid
Post by: pdq on 2012-03-20 17:53:11
The reconstruction filter only filters out frequencies above fs/2. The steps in the DAC output for a low frequency will contain much lower frequencies than fs/2, which aren't filtered out, unless you have proper dither.

Picture the case of a sine wave that is above a step threshold half the time, and below it the other half. The DAC produces a square wave, which gets low-pass filtered. All of those odd harmonics above fs/2 will be filtered out, leaving a square wave with rounded edges.

Edit: Am I responding to the wrong thing? I though you were skeptical of what David said.
Title: Quantization Grid
Post by: greynol on 2012-03-20 18:06:37
leaving a square wave with rounded edges.

The part I emphasized is at the very crux of my argument, unless you are claiming that dither is required for this rounding to occur(?).

The unnecessary stipulation of lowish frequency only muddies the waters.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 18:23:44
Signal to noise ratio, and resolution, are not the same thing.


Not exactly the same, but dynamic range and resolution are related by a simple monotonic algebraic equation according to Shannon (Information Theory).

In most systems dynamic range and SNR differ only due to the presence of nonlinear distortion, which becomes moot if it is small enough. In modern systems they can be very close.

So a first approximation is that in high quality modern systems  SNR approximately equals dynamic range which pretty well defines resolution.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 18:30:43
Hi icstm

An analogue recording system has infinite resolution but not infinite signal to noise ratio

A dithered digital recording system has infinite resolution but not infinite signal to noise ratio


Both statements are false.

No real world system has infinite resolution.

Shannon's information theory provides the relationship between noise+distortion  and resolution.

Shannon Information Channel Capacity link (http://en.wikipedia.org/wiki/Channel_capacity)
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 18:33:39
- icstm  ,  you are confusing resolution with signal to noise ratio.


They are often close enough. Many of us often interchange the two knowing full well the details about the differences.  SNR and Dynamic range of high quality practical digital audio systems are often very similar.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 18:36:47
I'm more than a bit skeptical of that.

I'm surprised that that surprises you. It seems rather obvious to me. Perhaps someone will generate a low-ish frequency sine wave in 8 bits without dither and show us the reconstructed waveform?


That seems like unecessary work given all the times I've seen this played out with real world equipment.

The output of a DAC contains all of the intermediate voltages even if the signal is undithered. The steps get lost in the reconstruction filter. If the digital signal is undithered there is correlated quantization error but it is smoothed out by the reconstruction filter's low pass effects.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 18:41:13
Are you trying to tell me inter-sample points don't transition smoothly as if the reconstruction filter doesn't do its job?


Bingo!

The reconstruction filter does do its job and the output voltage changes smoothly.

What puzzles people is what happens when the digital data is not dithered. Yes, the in-band quantization error is there, but it does not take the form of steps. It gets smoothed, too.

I base this on looking at 100s of DACs.  I've actually seen old, old audio interfaces that put out stairsteps, reason being they left out the reconstruction filter. Very, very early Soundblasters, for example.
Title: Quantization Grid
Post by: greynol on 2012-03-20 18:42:41
The output of a DAC contains all of the intermediate voltages even if the signal is undithered.

Thank you!
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 18:43:36
I'm surprised that that surprises you. It seems rather obvious to me. Perhaps someone will generate a low-ish frequency sine wave in 8 bits without dither and show us the reconstructed waveform?
You can use CoolEdit/Audition/whatever to take 44.1kHz 8-bit to 44.1kHz 16-bit then to 352.8kHz 16-bit (with the best oversampling filter available = simulated near-ideal reconstruction). Those original 8-bit quantisation steps are still easily visible...
[attachment=6974:nodither.jpg]

You can even see their remnants if the original signal is dithered...
[attachment=6975:withdither.jpg]


In the first instance, the reconstruction filter is only removing the aliased harmonics of the truncation distortion - turning a square wave into a slightly ringy square wave.

In the second instance, the reconstruction filter is also removing the aliases of the dither, hence the more obvious change in shape.

Cheers,
David.

P.S. EDIT: note amplitude and time scales on those plots. The full vertical scale is about 8 LSBs in 8-bit audio. The frequency of the sweep at that point is about 50Hz. It's still pretty similar at 500Hz though.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 18:52:56
The output of a DAC contains all of the intermediate voltages even if the signal is undithered.

Thank you!

You thanked him a bit early.

I'm not sure why anyone thinks a filter at 22kHz is going to change a square-ish low frequency waveform that much. The example I posted was a low amplitude sine wave at 50Hz; at 8-bits it ends up with square-wave-like transitions at ~250Hz due to quantisation. In this example, you can comfortably fit the first 40 harmonics within the transition band. That's more than you need to make a square wave look something like a square wave.

Cheers,
David.
Title: Quantization Grid
Post by: greynol on 2012-03-20 18:54:05
Those original 8-bit quantisation steps are still easily visible...
It wasn't my intention to put this in dispute.

You thanked him a bit early
Not really; rather I think the issue is that not everyone is following the claims put forth by the OP (could only be me, but I somehow doubt it).

Thanks for the detour, nonetheless.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 18:59:06
Those original 8-bit quantisation steps are still easily visible...
It wasn't my intention to put this in dispute.

Thanks for the detour, nonetheless.
Then what is your point?

The effect of quantisation is visible beyond the reconstruction filter. That is my point. The "grid" survives, though not perfectly.

btw, I don't think there's any audible relevance to this.

Cheers,
David.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-20 19:02:23
The output of a DAC contains all of the intermediate voltages even if the signal is undithered.

Thank you!

You thanked him a bit early.

I'm not sure why anyone thinks a filter at 22kHz is going to change a square-ish low frequency waveform that much. The example I posted was a low amplitude sine wave at 50Hz; at 8-bits it ends up with square-wave-like transitions at ~250Hz due to quantisation. In this example, you can comfortably fit the first 40 harmonics within the transition band. That's more than you need to make a square wave look something like a square wave.


The quantization filter only changes a low frequency wave enough so that the transition isn't perfectly square.  As soon as the wave stops being perfectly square, all of the intermediate voltages are there.
Title: Quantization Grid
Post by: greynol on 2012-03-20 19:02:31
I edited my previous response to include your follow-up post.  It should help to present my point, though I think I made it pretty clear earlier (http://www.hydrogenaudio.org/forums/index.php?showtopic=94113&view=findpost&p=790033) by suggesting that people are assuming facts not in evidence about the OPs claims.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 19:03:21
not everyone is following the claims put forth by the OP
I think the OP has changed their mind throughout this thread.

They've gone from digital = an evil fixed grid that wrecks audio, to digital = infinite resolution. (I'm paraphrasing!). From one extreme to the other. Neither is correct IMO, but the ideas are too vague to critique properly, and I haven't seen a good answer for amplitude "resolution". I know there's a formula for temporal "resolution" in another thread somewhere.

Cheers,
David.

Title: Quantization Grid
Post by: greynol on 2012-03-20 19:08:28
From one extreme to the other.

With such a professorial tone, no less.

Thanks for the images.  Hopefully people won't come away from this with the wrong idea this time around.  I fear that they would without them.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-20 19:09:28
Ah, maybe we're at cross purposes, and don't even disagree. I said "most of the instantaneous output voltages will be essentially 'on' the quantization steps (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94113&view=findpost&p=790025)". Of course there's some smoothing. It just doesn't remove the original "quantisation grid" completely (or much at all, for some waveforms).

Cheers,
David.
Title: Quantization Grid
Post by: googlebot on 2012-03-20 19:30:20
Aren't those pictures just showing that not enough dither has been applied? Not in the sense of audible necessity, but in the sense of optical waveform hygiene.
Title: Quantization Grid
Post by: drewfx on 2012-03-20 20:13:52
The effect of quantisation is visible beyond the reconstruction filter. That is my point. The "grid" survives, though not perfectly.


Isn't this just sort of saying, "there's quantization error/distortion/noise present"?
Title: Quantization Grid
Post by: pdq on 2012-03-20 20:30:40
It's always present. Dither simply replaces much of it with a different kind of noise.
Title: Quantization Grid
Post by: drewfx on 2012-03-20 20:39:01
Understood. The point I'm making is that the "surviving grid" is in fact just the quantization error. Which shouldn't be controversial.
Title: Quantization Grid
Post by: greynol on 2012-03-20 20:40:57
The image of the dithered 50Hz wave is pretty much how I remember the digital signal generators looking when set to minimum amplitude back when I used those things.

...or was it the digital oscilloscopes, I can't remember, though I usually used analog scopes. 
Title: Quantization Grid
Post by: greynol on 2012-03-20 20:43:37
The point I'm making is that the "surviving grid" is in fact just the quantization error.

I would say the quantization itself, but maybe this is a distinction without a difference.
Title: Quantization Grid
Post by: googlebot on 2012-03-20 21:07:24
The point I'm making is that the "surviving grid" is in fact just the quantization error.

I would say the quantization itself, but maybe this is a distinction without a difference.


The "surviving grid" is just correlated quantization error, that has not been decorrelated properly with an amount of noise suited for the input bit depth. It's neither quantization 'itself' nor any other voodoo. 2Bdecided has already explained it all very [a href='index.php?act=findpost&pid=789226']early[/a] in this thread. From there it went down, as if there really was some kind of unavoidable 'grid' residue, different from classical quantization error.
Title: Quantization Grid
Post by: greynol on 2012-03-20 23:00:50
Yet the "error" plainly indicates the available discrete levels.

How is this voodoo?

I used scary quotes around the word error since the actual error is an analog signal that is limited between -1/2 and +1/2 when dither is not applied.
Title: Quantization Grid
Post by: Speedskater on 2012-03-20 23:34:02
For a low-ish frequency input signal, ignoring noise etc, most of the instantaneous output voltages will be essentially "on" the quantization steps. You can see this very easily at low amplitudes.
Cheers,
David.

So you look at this instantaneous output voltage and if it is NOT "on" a quantization step - you say "ignore that, it's noise".
And if this instantaneous output voltage is "on" a quantization step - you say "see I told you so".
Title: Quantization Grid
Post by: greynol on 2012-03-21 00:03:05
Yes, noise (thermal, shot, interference, etc.) that causes the signal to deviate from one of the discrete quantization levels as opposed to quantization error which is the reason that the signal is clamping to one of the discrete quantization levels.

Other than that the operative is "most" and how much is dependent on the highest frequency present in the signal.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 10:02:36
For a low-ish frequency input signal, ignoring noise etc, most of the instantaneous output voltages will be essentially "on" the quantization steps. You can see this very easily at low amplitudes.
Cheers,
David.

So you look at this instantaneous output voltage and if it is NOT "on" a quantization step - you say "ignore that, it's noise".
And if this instantaneous output voltage is "on" a quantization step - you say "see I told you so".
It's hardly cherry picking.

Here are two graphs showing the distribution of sample values.

This is from the 8-bit without dither data, 8x oversampled (i.e. resampled to 352.8kHz) at 16-bits...
[attachment=6976:distribu...nodither.gif]

This is from the 8-bit with dither data, 8x oversampled (i.e. resampled to 352.8kHz) at 16-bits...
[attachment=6977:distribu...thdither.gif]

In both cases, the dominance of the original 8-bit quantisation steps is clearly visible: those huge peaks match the original 8-bit quantisation steps. The skirts around them are due to the effect of the resampling. In a real system, noise would broaden the skirts and reduce the peaks - though not noticeably for 8-bits! More importantly, an AC-coupled system will see the values drifting around with any DC in the sampled waveform.

EDIT: These distribution graphs are for full 20s 20Hz-1kHz log sweep -40dB signals - the same ones from which the waveform images in this post (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94113&view=findpost&p=790042) were taken.

Cheers,
David.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 10:03:29
2Bdecided has already explained it all very [a href='index.php?act=findpost&pid=789226']early[/a] in this thread.
Thank you - that's the bit where I explain that it doesn't matter. Worth repeating!

Cheers,
David.

Title: Quantization Grid
Post by: KMD on 2012-03-21 11:37:53
At what stage did you add the dither noise  David.




Owen.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 12:10:55
At what stage did you add the dither noise  David.
Steps:

1. Create new project: 44.1kHz 16bit.
2. Generate sine wave, 20s, 20Hz-1kHz, log sweep, 0dB
3. amplitude change: -40dB
4. convert to 8-bit with or without dither <--------------- here. dither shape = TPDF dither amplitude = 1-LSB RMS (2-LSBs peak-to-peak)
5. convert to 16-bit
6. convert to 352800Hz 16-bit

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-21 12:25:50
The fact that you are sweeping the frequency adds a layer of complexity. We don't quite know what  artifacts may allready be in a sweep synthesised by cooledit.  For simplicity can you do those steps , and the statistical analysis on a static 50Hz sine wave, as in the original cooledit images.


Cheers


Owen
Title: Quantization Grid
Post by: pdq on 2012-03-21 13:50:56
What would be the point? We have established that low level signals can be severely distorted by quantization, and that a nominal application of dither removes much, but not all, of that distortion. What more do you want to know?

Edit: typo
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 14:07:30
We don't quite know what  artifacts may allready be in a sweep synthesised by cooledit.
I can assure you, there are none with the sine wave sweep. (The square wave is atrocious, but I didn't use that).

I'd turned off the default dithering of all functions, so there is truncation quantisation distortion in the 16-bit representation, even though CEP is able to generate 32-bit float and dither to 16-bit normally by default when performing operations.

Just using 50Hz makes little difference...
[attachment=6978:distribu...nodither.gif]

[attachment=6979:distribu...thdither.gif]

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-21 14:08:52
pdq -look at the 2nd  image in 2bdecided post68  above . On first looking the dither seems to have not removed the distortions at all. I think it needs to be assertained what cool edit is actually doing.

David just seen your last post,  will read it now
Title: Quantization Grid
Post by: KMD on 2012-03-21 14:35:18

David -  What does the  frequncey spectrum analysis of that  50 HZ dithered look like


Owen
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 14:40:31
What would be the point?
None. I don't think KMD fully grasps the implications of the questions that they are asking.

Quote
We have established that low level signals can be severely distorted by quantization, and that a nominal application of dither removes much, but not all, of that distortion.
It removes all the audible distortion.

True, there's still a visible relationship between the original signal, and the dither+quantisation - i.e. if you subtract the 8-bit version from the 16-bit version, you can see the difference isn't purely uncorrelated noise...
[attachment=6980:difference.gif]

...but it certainly sounds like uncorrelated noise...
[attachment=6981:differen...on_noise.flac]

I also attach the original 16-bit sweep and the 8-bit quantised version for reference...
[attachment=6982:16bit_version.flac]
[attachment=6983:8bit_version.flac]

Cheers,
David.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 14:45:31
David -  What does the  frequncey spectrum analysis of that  50 HZ dithered look like
At 44.1kHz or 352.8kHz? Using what parameters?

Whichever, it looks like you'd expect - white noise 0-22kHz, a peak at 50Hz about 40dB above it, and (in the 352.8kHz version) more white noise 22kHz-176kHz about 65dB below it.

No other features.

Cheers,
David.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 14:48:46
pdq -look at the 2nd  image in 2bdecided post68  above . On first looking the dither seems to have not removed the distortions at all.
That post shows you that quantisation "quantises" - and that the resulting discrete levels remain fairly intact even when oversampling at higher resolution.

It says nothing about whether dither removes distortion or not. It's not intended to.

Correct dither removes audible distortion perfectly.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-21 15:07:47
Use the FFT to check that the harmonics are not there and the dither is doing what is expected
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 15:12:09
Use the FFT to check that the harmonics are not there and the dither is doing what is expected
http://en.wikipedia.org/wiki/Teaching_gran...er_to_suck_eggs (http://en.wikipedia.org/wiki/Teaching_grandmother_to_suck_eggs)
Title: Quantization Grid
Post by: KMD on 2012-03-21 15:16:03
do the fft to confirm that the harmonics are not there. Anyone looking at the statisticle analysis would suspect that they are.


Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 15:34:50
do the fft to confirm that the harmonics are not there. Anyone looking at the statisticle analysis would suspect that they are.
Yes boss!

with dither...
[attachment=6984:fft_dither.jpg]

and without dither...
[attachment=6985:fft_nodither.jpg]


Who is this person following this thread that suspects dither doesn't work because they've misinterpreted a graph showing that quantisation quantises?

If you want a simple example of dither working, without worrying about subsequent oversampling, look at this old picture...
[attachment=6986:dither_explanation.gif]

Cheers,
David.
Title: Quantization Grid
Post by: Rotareneg on 2012-03-21 15:40:20
Delta-sigma modulation (http://en.wikipedia.org/wiki/Delta-sigma_modulation) must be a hoax.
Title: Quantization Grid
Post by: KMD on 2012-03-21 16:11:54
" yes Boss" LOL


I'm glad we can have a  light hearted chat , some of the comments on this forum are  really chippy and uptight. not helpfull to getting anywhere at all.

any way so where were we


So the harmonics are not there but the statistical analys shows the quantization levels.

The last step to finally bake the cake is to do step 1, 2, 3, 4 and the statistical analysis with a music waveform.
Title: Quantization Grid
Post by: KMD on 2012-03-21 16:36:14
The image  in post 77 is way to square. A reconstructed dithered waveform should look like a fuzzy sine wave .



Title: Quantization Grid
Post by: 2Bdecided on 2012-03-21 16:46:36
The image  in post 77 is way to square. A reconstructed dithered waveform should look like a fuzzy sine wave .
The image in post 77 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94113&view=findpost&p=790136) isn't "reconstructed", or oversampled - it's the actual 44.1kHz data. The second one in post 49 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94113&view=findpost&p=790042) was 8x oversampled to 352.8kHz to simulate reconstruction.

Examining 8-bit quantisation with and without dither using typical pop music won't show much. If you're lucky, you'll be able to see+hear the added noise, but it'll be hard to spot any distortion on a graph.

Suitable classical recordings, or anything with quiet passages, will reveal the same as you've seen with the sine wave.

You'd hear the same too.
e.g. original: http://mp3decoders.mp3-tech.org/audio/brief_original.mp3 (http://mp3decoders.mp3-tech.org/audio/brief_original.mp3)
6-bit no dither: http://mp3decoders.mp3-tech.org/audio/brief_rounded.mp3 (http://mp3decoders.mp3-tech.org/audio/brief_rounded.mp3)
6-bit with dither: http://mp3decoders.mp3-tech.org/audio/brief_dithered.mp3 (http://mp3decoders.mp3-tech.org/audio/brief_dithered.mp3)
6-bit with noise shaped dither: http://mp3decoders.mp3-tech.org/audio/brief_nsdithered.mp3 (http://mp3decoders.mp3-tech.org/audio/brief_nsdithered.mp3)

Not much use for waveform analysis as these are mp3s, but they illustrate the different sound very well.

Cheers,
David.
Title: Quantization Grid
Post by: KMD on 2012-03-21 17:02:01
When you say 8x oversampled to simulate reconstruction can you confirm that that includes simulation the 22Khz low pass filter.

I would not expect anything to be immediately audible in the pop music either. But to see that statistical analysis done on music would prove that digital audio is fundamentally constrained in amplitude variation, with a statistical variance, around the quantization levels, which could effect the listening experience, and is a genuinely original revelation in digital audio engineering. I am a member of the Audio Engineering Society by the way.


-
Owen.
Title: Quantization Grid
Post by: lvqcl on 2012-03-21 17:49:36
which could effect the listening experience

omg.
Title: Quantization Grid
Post by: greynol on 2012-03-21 19:40:06
is a genuinely original revelation in digital audio engineering

To anyone who has had sophomore-level college (or equivalent) exposure in digital signals and has some knowledge in audiology, this would not be a revelation by any stretch of the imagination.

Before we get all caught up in the constraint of digital audio in amplitude variation, you might want to consider whether the human ear and its listening environment isn't also constrained in amplitude variation.

That you are a member of AES does little to demonstrate any competence in this discussion, by the way.  IEEE would have gotten you farther, but we'd then have reason to wonder why you seem to be so confused.
Title: Quantization Grid
Post by: Arnold B. Krueger on 2012-03-21 20:25:58
But to see that statistical analysis done on music would prove that digital audio is fundamentally constrained in amplitude variation, with a statistical variance, around the quantization levels, which could effect the listening experience, and is a genuinely original revelation in digital audio engineering.


You are aware that the grid for 16 bits is composed of lines that are  0.00026 dB apart at FS, more or less, right? 60 dB below FS they are about 0.1 dB apart. None of those level differences will ever be heard! Below -60 dB  even hearing the peak signal above the ambient noise, let alone the microscopic difference, will be a challenge.
Title: Quantization Grid
Post by: drewfx on 2012-03-22 00:18:17
But to see that statistical analysis done on music would prove that digital audio is fundamentally constrained in amplitude variation, with a statistical variance, around the quantization levels, which could effect the listening experience, and is a genuinely original revelation in digital audio engineering.


The fact that the reconstructed output waveform of a dithered digital signal might tend to follow the quantization levels is irrelevant.

The quantization error is what makes the reconstructed signal fall closer to the quantization levels. When you have no dither, the quantization error is correlated to the signal, which if audible, is undesirable.

When you add dither (while you are still in the digital domain), you end up with a quantized version of your signal plus dither. Since the dither is random and of an appropriate level compared to the quantization step size, you end up with your original signal plus uncorrelated noise.

It's still quantization error that makes the reconstructed dithered signal tend to follow the quantization level, but now the quantization error is based on the signal plus dither instead of just the signal. And if it's audible, it can indeed affect the listening experience - it sounds like noise. But there's nothing new, original or revelatory about it.
Title: Quantization Grid
Post by: KMD on 2012-03-22 09:00:46
I'm not sure why anyone thinks a filter at 22kHz is going to change a square-ish low frequency waveform that much. The example I posted was a low amplitude sine wave at 50Hz; at 8-bits it ends up with square-wave-like transitions at ~250Hz due to quantisation. In this example, you can comfortably fit the first 40 harmonics within the transition band. That's more than you need to make a square wave look something like a square wave.

Cheers,
David.


Because that is how it is taught in textbooks. Typically a stream of spiky samples is shown going into a reconstruction filter and out the other side comes a smooth wave. As the cooledit images shows the reconstruction filter and dither do neither. As your plot shows the output is a stream of jumps between quantization levels, which illustrates my original point in the listening test topic about a quantization grid except rather than a grid it is more of a vertically spaced grating.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-22 10:03:00
Because that is how it is taught in textbooks. Typically a stream of spiky samples is shown going into a reconstruction filter and out the other side comes a smooth wave. As the cooledit images shows the reconstruction filter and dither do neither. As your plot shows the output is a stream of jumps between quantization levels, which illustrates my original point in the listening test topic about a quantization grid accept rather than a grid it is more of a vertically spaced grating.
KMD, I think that you may be guilty of a little knowledge being a dangerous thing. And maybe misunderstanding the graphs.

I posted these graphs in response to a claim that the quantisation levels were never visible after reconstruction. The graphs show that, where the original signal is low frequency, and contains only a few levels, it's easy to see that the quantisation levels are visible after reconstruction.

However, this doesn't show a "fault" in reconstruction or digital audio. It shows that I know how to choose a signal that is largely unaffected by reconstruction  i.e. effectively a very low frequency square-ish wave, which really is a square-ish wave, and so after reconstructions still looks like a square-ish wave.

All the graphs you have seen in textbooks are true.

I can do them do

20kHz sine wave, sampled at 44.1kHz...
[attachment=6987:20k_44100Hz.gif]

...44.1kHz version 8x oversampled...
[attachment=6988:20k_352800Hz.gif]

...44.1kHz version 100x oversampled...
[attachment=6989:20k_4410000Hz.gif]

Cheers,
David.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-22 10:13:06
When you say 8x oversampled to simulate reconstruction can you confirm that that includes simulation the 22Khz low pass filter.
Yes, of course.

Quote
But to see that statistical analysis done on music would prove that digital audio is fundamentally constrained in amplitude variation, with a statistical variance, around the quantization levels
Of course it is. At the sample points, without noise, with an ideal reconstruction filter, it's absolute constrained. That's what quantisation does. Between the sample points, the reconstructed signal could go anywhere - though for a given input signal there's only one "correct" place for it go (defined by the sinc function) and that may or more likely may not be on an original quantisation step.

Quote
which could effect the listening experience
The ear has no conceivable mechanism to detect (i.e. hear) this, even in signals where the effect is visible.

Quote
I am a member of the Audio Engineering Society by the way.
My membership has lapsed, but I attend lectures when I can.

I wish I could get to tonight's...
http://www.aes-uk.org/event/loss-of-our-mu...ster-cambridge/ (http://www.aes-uk.org/event/loss-of-our-musical-heritage-%E2%80%93-the-rise-of-the-digital-remaster-cambridge/)
...though they would be preaching to the choir if I was there  . I suspect they are anyway.

Cheers,
David.
Title: Quantization Grid
Post by: greynol on 2012-03-22 12:55:56
Between the sample points, the reconstructed signal could go anywhere - though for a given input signal there's only one "correct" place for it go (defined by the sinc function) and that may or more likely may not be on an original quantisation step.

Since you're talking about time being filled between points that are sampled you would expect the waveform to fall between quantization levels. The amplitude in between may still be in error just as it may be at the sampled points and likely will be if those sampled points are in error, but this is hardly ground breaking.

My text books on the subject speak clearly about quantization error, so I reject KMD's claim to the contrary. Maybe the problem has to do with glancing at pictures instead of reading the text and equations?
Title: Quantization Grid
Post by: pdq on 2012-03-22 13:24:48
Since you're talking about time being filled between points that are sampled you would expect the waveform to fall between quantization levels.

The sampled points on either side of a maximum are both below the maximum of the reconstructed waveform. Similarly for a minimum.
Title: Quantization Grid
Post by: greynol on 2012-03-22 13:41:08
What about ringing?
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-22 13:57:30
My text books on the subject speak clearly about quantization error, so I reject KMD's claim to the contrary. Maybe the problem has to do with glancing at pictures instead of reading the text and equations?
I think the fact that people believe they can look at something for five minutes and make a genuine ground breaking discovery in a field that's been well understood for decades tells you a lot about 21st century culture.

Or something. I'm probably trying to be philosophical, and I know nothing about that field myself.

Cheers,
David.
Title: Quantization Grid
Post by: WernerO on 2012-03-22 13:59:25
What about ringing?


You mean the ringing that is seen in the reconstruction filter's impulse response?

There would be none, provided the original signal was indeed correctly bandlimited
prior to sampling.

If you meant something else then pray unread the above.

Title: Quantization Grid
Post by: greynol on 2012-03-22 14:12:49
There could be some if the signal was sufficiently low, synthesized or dithered.
Title: Quantization Grid
Post by: saratoga on 2012-03-22 14:23:54
My text books on the subject speak clearly about quantization error, so I reject KMD's claim to the contrary. Maybe the problem has to do with glancing at pictures instead of reading the text and equations?

KMD has posted this claim before.  When I asked him which textbook exactly he was referring to, he declined to answer.  I think he should either cite his source, or stop arguing about it.
Title: Quantization Grid
Post by: Notat on 2012-03-22 14:44:46
with dither...
[attachment=6984:fft_dither.jpg]

David, is this the spectrum of the 8-bit version (signal) or the difference (quantization error)?
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-22 15:03:44
The 8-bit signal (which include quantisation error), but analysed at the end of this process...
http://www.hydrogenaudio.org/forums/index....st&p=790120 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94113&view=findpost&p=790120)
...which was to show that individual quantisation steps can kind-of survive reconstruction filtering. I guess it also shows that dither "works", though there is no need for those extra steps if that's what you want to prove.

Cheers,
David.
Title: Quantization Grid
Post by: Glenn Gundlach on 2012-03-23 03:37:37
The quantization grid is not caused by sampling it is caused by the interaction of quantization levels  with  sampling points. It may not be perceptable but there is no doubt that a waveform created from  a digital  file must be formed from a selection of points that are selected from a finite number of pre- determined regularly spaced co-ordinates. The digital file is formed from regularly spaced sampling points and reguarly spaced quantization levels therefore the waveform derived from it must have a coresponding regularity.


I work more in commercial TV where video has been digitized since the mid '70s. If what you're describing existed you would not be able to to display diagonal lines, particularly nearly vertical ones. I assure that is not the case. The time resolution is infinitely variable. OK, I can only measure reliably to a nanosecond but for all practical purposes....


Title: Quantization Grid
Post by: 2Bdecided on 2012-03-23 09:22:45
I work more in commercial TV where video has been digitized since the mid '70s. If what you're describing existed you would not be able to to display diagonal lines, particularly nearly vertical ones. I assure that is not the case.
Actually, images and video resist the use of "ideal" filters, so aliasing is quite common. Those diagonal lines are often quite "steppy".

Cheers,
David.
Title: Quantization Grid
Post by: dhromed on 2012-03-23 13:24:57
Actually, images and video resist the use of "ideal" filters, so aliasing is quite common. Those diagonal lines are often quite "steppy".


Surely that is because most visual output devices have the same or lower resolution than/as the input signal?
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-23 13:38:22
Actually, images and video resist the use of "ideal" filters, so aliasing is quite common. Those diagonal lines are often quite "steppy".


Surely that is because most visual output devices have the same or lower resolution than/as the input signal?
No, it's because you can't use a sinc filter with a picture because you'll introduce visible ringing.

Full HD TV, 1:1 pixel mapped LCD - two examples of matched resolution to source - none have any filter on the output.

Similar problems with image capture for HD video.

Lesser problems these days for still images, because the lens itself often acts as a low pass filter for the x-mega-pixel sensor.

But you can't cleanly get "close" to the nyquist limit with images - not as close as you can with audio.

Cheers,
David.
Title: Quantization Grid
Post by: icstm on 2012-03-23 13:59:07
Lesser problems these days for still images, because the lens itself often acts as a low pass filter for the x-mega-pixel sensor.
you mean the lens on the sensor within the chip or the main lens? I think there are filters within the sensor. I have a vague recollection in my head, but I cannot remember what does filters and lenses do on the sensor.
Title: Quantization Grid
Post by: 2Bdecided on 2012-03-23 14:04:29
Lesser problems these days for still images, because the lens itself often acts as a low pass filter for the x-mega-pixel sensor.
you mean the lens on the sensor within the chip or the main lens? I think there are filters within the sensor. I have a vague recollection in my head, but I cannot remember what does filters and lenses do on the sensor.
There is usually a filter in front of the sensor. But you can't do the "flat to within a few percent of nyquist / kill everything above nyquist" response we're used to in audio.

Some people think they can do better than is usually achieved these days...
http://www.imaging.org/ist/publications/re...1_MA_7876_4.pdf (http://www.imaging.org/ist/publications/reporter/articles/REP26_2_EI2011_MA_7876_4.pdf)

Cheers,
David.
Title: Quantization Grid
Post by: saratoga on 2012-03-23 15:47:10
Lesser problems these days for still images, because the lens itself often acts as a low pass filter for the x-mega-pixel sensor.
you mean the lens on the sensor within the chip or the main lens? I think there are filters within the sensor. I have a vague recollection in my head, but I cannot remember what does filters and lenses do on the sensor.


A lens is a low pass filter, so in practice it serves as the anti-aliasing filter if nothing else.  Likewise, the pixels on a camera tend to be quite wide relative to their pitch so they'll integrate over a finite width and thus further lowpass the signal. 

Often though some degree of aliasing is tolerated in imaging systems because its difficult to build optical filters with steep drop offs.  Its quite common to design imaging systems with the spot size of the optics matched to 2x the pixel size, and then to just let the finite pixel width low pass out much of the frequencies that would alias.