HydrogenAudio

Hydrogenaudio Forum => Scientific Discussion => Topic started by: KMD on 2012-03-07 12:22:30

Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-07 12:22:30
To help with the understanding of dither, I am posting  an explanation, of how signal details smaller than the quantization level are retained, after quantization, when Dither Noise is added to the source material.
  A small signal detail  with height less than the quantization level, and centred half way between two quantization steps, is too small to cross a quantization step. It is lost when the waveform is quantized. With a Dither Noise waveform added, the height of the two waveforms summed is occasionally greater than the quantization  level. At these occasional points the waveform crosses a quantization step. So now when this waveform is quantized, although the result is noisy, the shape of the original signal detail is retained.
  Discussion invited.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: icstm on 2012-03-07 12:29:15
To help with the understanding of dither, I am posting  an explanation, of how signal details smaller than the quantization level are retained, after quantization, when Dither Noise is added to the source material.
  A small signal detail  with height less than the quantization level, and centred half way between two quantization steps, is too small to cross a quantization step. It is lost when the waveform is quantized. With a Dither Noise waveform added, the height of the two waveforms summed is occasionally greater than the quantization  level. At these occasional points the waveform crosses a quantization step. So now when this waveform is quantized, although the result is noisy, the shape of the original signal detail is retained.
  Discussion invited.


Yes I think this is good.
When I once used dither personally, it was for a video feed, so I could use both spactial and temporal dither. I think dither in time is only possible for audio, though I will think about it a litter more.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: pdq on 2012-03-07 14:07:47
You might add that even when that small signal DOES cross quantization steps, it will be highly distorted, e.g. converted into a square wave. Dither will then reduce the distortion.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-07 15:10:44
icstm  - Thanks

pdq - true, that is another function of dither. Frequently flipping the LSB by the sum of  the small signal and the dither noise retaines the correct shape.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: icstm on 2012-03-07 16:43:14
Spatial dither:
In a video feed spatial dither can be implemented by setting adjacent pixels that cannot be perfectly represented to different values and then altering these values in time depending on which is closest to the orginial value (temporal dither)
In audio we use this temporal dither.

How weird would be to use the multiple speaker channels for spatial dither?
Title: Dither—my explanation of how it keeps details below quantization level
Post by: mjb2006 on 2012-03-07 20:43:52
Please do what you can to improve our wiki article on dither (http://wiki.hydrogenaudio.org/index.php?title=Dither).
Title: Dither—my explanation of how it keeps details below quantization level
Post by: NullC on 2012-03-08 16:39:06

I highly recommend the first few sections of this thesis (http://uwspace.uwaterloo.ca/handle/10012/3867?mode=full) for a careful technical explanation of how dither works and why it's important.  It starts with the sampling theorem, shows how quantization breaks it, then shows how dither (of the right kind) fixes the breakage.  It's well worth the read, even though the latter parts are dealing with image processing rather than audio.

Title: Dither—my explanation of how it keeps details below quantization level
Post by: Paulhoff on 2012-03-08 18:03:36
You might add that even when that small signal DOES cross quantization steps, it will be highly distorted, e.g. converted into a square wave. Dither will then reduce the distortion.


This is sooooo true. I had try to record something where the signal was in the lowest digits, and when I added dither to it, it went from being very harsh sounding to noisy and natural sounding


Paul

   
Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-08 18:12:54
Paulhoff    - You were probably getting the benefit of both aspects. The retenion of the sub quantazation details and the removal  of  step crossing  distortion
Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-08 18:15:49
NullC  -  If you put  fig 2.4, 2.5 ,2.11, 4.6  in four browser tabs and switch between it is impressive
Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-08 19:05:36
Reference for this subject.      Paper titled, 
                            -    Resolution Below the Least Significant Bit in Digital Systems with Dither.  John Vanderkooy and Stanley Lipshitz. 1984.  -




               









Title: Dither—my explanation of how it keeps details below quantization level
Post by: KMD on 2012-03-09 14:24:34
mjb2006  - Will do
Title: Dither—my explanation of how it keeps details below quantization level
Post by: icstm on 2012-03-09 15:14:33
I highly recommend the first few sections of this thesis (http://uwspace.uwaterloo.ca/handle/10012/3867?mode=full) for a careful technical explanation of how dither works and why it's important.  It starts with the sampling theorem, shows how quantization breaks it, then shows how dither (of the right kind) fixes the breakage.  It's well worth the read, even though the latter parts are dealing with image processing rather than audio.
read sections 1 and 2. Good find.
I just wish it was written by an engineer rather than a mathematician
Title: Dither—my explanation of how it keeps details below quantization level
Post by: greynol on 2012-03-21 20:14:56
In light of the recent conversation over the quantization grid, I humbly request that knowledgeable people review our wiki article on dither:

http://wiki.hydrogenaudio.org/index.php?title=Dither (http://wiki.hydrogenaudio.org/index.php?title=Dither)

After a quick review I really must question the emphasis that is placed on how dither affects the visual appearance of waveforms.  This is clearly not in line with one of the most important tenets of our forum
Title: Dither—my explanation of how it keeps details below quantization level
Post by: benski on 2012-03-21 20:42:18
I remember the first lab in University Physics class.  We were given a rectangular block and asked to determine its length given only a ruler with centimeter markings.

For the sake of discussion, let's assume the object is 57mm long.  If you measure from the start of the ruler, you will get 50mm (5 cm) every time, as the ruler is only that precise.  If you measure from the opposite end of the ruler, you will get 60mm every time.  However, if you randomly place the ruler against the block, you will sometimes measure 50mm and sometimes measure 60mm.  If you repeat with enough trials and average the results, you will converge at the appropriate value: 57mm.

This is an almost perfect analogy for how dither works.  By adding in noise, you are essentially "randomly placing the ruler". 

If you were to imagine a DC signal at 0.6 that is being decimated to integer values.  By randomly adding noise, each sample value will be either 1 or 0, and their appearance should occur in a 6:4 ratio, and the RMS energy of the decimated, dithered signal will be 0.6 - the same as the original signal.

Error shaping works by "feeding forward" the prior error to increase the likelihood that the next value will offset the previous error.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: mjb2006 on 2012-03-22 00:57:55
After a quick review I really must question the emphasis that is placed on how dither affects the visual appearance of waveforms.  This is clearly not in line with one of the most important tenets of our forum


The article I had started last year was just edited rather heavily (http://wiki.hydrogenaudio.org/index.php?title=Dither&action=historysubmit&diff=23137&oldid=22523) about a week ago by KMD, as per his post above. I do appreciate that someone besides me put some effort into it, but aside from the addition of the questionable content greynol mentioned, several details that I don't think should've been removed were cut out.

For example, my original text for the first paragraph:
Quote
Dither is random or semi-random noise added to a signal in order to mask quantization noise and/or extend dynamic range. The simplest dither is quiet white noise, but more complicated forms of dither are possible using [[noise shaping]], and they can even be completely inaudible.

...is now this:
Quote
Dither is random noise added to a signal in order to increase the degree to which the quantization and sampling process accurately reproduces the image of the input signal. The simplest dither is quiet white noise, but more complicated forms of dither are possible using [[noise shaping]].


...and most of the text (the bold part) adapted from a post by Andy H-ha was completely removed:
Quote
Going from 24-bit to 16-bit, the quantization error is very small and the distortion/noise is extremely unlikely to be heard in any real music. Since quantization error isn’t audible in any real music at useable listening levels, whether dither must be used is more a matter of doctrine than functionality.


Other changes were made as well. It's possible some of this was done to give the appearance of neutrality, requiring claims about dither's inaudibility to be backed up, but I wanted that stuff in there because it's important to know while it's easy and common for dither to amount to "audible white noise", it need not be white/totally-random nor audible (especially for the common 24-to-16-bit conversion scenario). Besides, my understanding is that the onus is on the person claiming that something is audible, i.e. that there's a difference, not on the person who says there's not. I'm not saying what I wrote was perfect, but surely there's some middle ground.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: saratoga on 2012-03-22 01:05:43
Obviously someone who doesn't understand sampling should not be editing the wiki entry on dithering, so I've reverted those changes.  Still, some editing of the page would be nice.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: 2Bdecided on 2012-03-22 10:38:35
Important points:

Dither removes the potentially audible correlation between the quantisation error and the original signal.

Without dither, the quantisation error becomes highly correlated with the original signal under certain circumstances, especially where the original signal is low in level and/or is free from noise. In these cases, quantising the signal introduces harmonic distortion.

Dither does not mask this distortion, it prevents it from occurring at all.

Some types of noise in the original signal can act like dither.


A good explanation should show how, on average, correct dither linearises the quantiser (1st order effect), and present the noise level from varying with signal level (2nd order effect). It's all in that 1984 paper mentioned earlier in this thread.

I'm not touching the wiki, but I doubt KMD should be changing it so much either.

Cheers,
David.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: alanofoz on 2012-03-22 11:05:31
Warning: very shallow & not very scientific:

If you're not looking for an in-depth explanation, it might be helpful to just think of the signal as modulating the noise.
Title: Dither—my explanation of how it keeps details below quantization level
Post by: Notat on 2012-03-22 14:38:39
Please do what you can to improve our wiki article on dither (http://wiki.hydrogenaudio.org/index.php?title=Dither).

The Wikipedia article (http://en.wikipedia.org/wiki/Dither) could use some love too.

These online references are a powerful means for objectivists to make an impact. When someone Googles dither (http://lmgtfy.com/?q=dither), let's give them something correct and understandable.