0 Members and 1 Guest are viewing this topic.

##### 2011-12-27 10:05:38

diff = ((signed)nibble + 0.5) * step / 4,
^^

= (signed)( nibble[2] * step + nibble[1]*step/2 + nibble[0]*step/4 + step/8)
^^^^
the term step/8 is not exist in the quatization process.

isn't the dequantization the inverse of quatization?

what's the reason for adding step/8

and what will happen if I ignore the step/8 in dequantization step?

##### Reply #1 – 2011-12-27 13:27:47
the term step/8 is not exist in the quatization process.

isn't the dequantization the inverse of quatization?

That page does look a little odd in the way it talks about sign/magnitude and then works with its bias value (which would still make sense but for the parenthesisation around the cast).

In general, bias would be expected in only one of the two processes.  If the quantiser (supposing a round-to-nearest-int quantiser in this case) rounds to -inf, then the set of values which would be rounded to x are [x,x+1), and adding 0.5 to the dequantised value puts it in the middle of that range for the minimum worst-case error.

If the quantiser added 0.5 then it would be rounding to nearest, and any result x must have been from the range [x-0.5,x+0.5), and x would already be in the middle of that range and have the minimum worst-case error.

A reason to round to -inf in the quantiser is to make best use of the range of possible values in a 4-bit number.  Since you normally end up with extra range on the negative side, or you end up with two zero values in sign/magnitude, adding 0.5 to all of those values balances things (at the cost of having no way to express 0).  In a more complicated codec you'd simply use a range of values that was not a power of two, and you would use an entropy coder to take up the slack of un-used bit patterns.