Skip to main content

Topic: Resampling - Rounding Errors and Bit Depth. (Read 17596 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Okay so i recently got knowledge about the use of Bit Depth for Resampling.
I noticed that when i resampled old SNES music (which is pretty much, midi sound) i heard noise appearing on occasions.
At first i thought it was there originally, but then i rechecked, and well it wasn´t.

I then asked around about it, and got knowledge about Rounding Errors.
When resampling is done, there will be errors, and at 16bit (which was what i was working at) the errors are noticeable as there isn´t much do work with.
But when working at 32bit Float, the errors are so small that they pretty much don´t exist i guess, at least it´s not audible from my understanding.

But now to the thing that crossed my mind.

If 32bit float is great when working with resampling and such, how is 64bit float?
Is it much better, or isn't there any improvement?

  • AndyH-ha
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #1
Better for what? Quantization errors are obviously much smaller but 32 bit float errors are so far below audible that the difference is purely spiritual. If you have faith, you will be saved, but if you have not, you never will notice the slightest difference. Besides, when you convert back to 16 bit, it won't matter anyway.

If you think you hear noise from resampling at 16 bit, I think there is something strange going on. I don't play with midi, are you sure it was originally created at 16 bit? The quantization noise from converting from one sample rate to another sample rate, starting and ending with 16 bit, is down more than 90dBfs. It could not be heard at any normal listening level if there is other signal present (i.e. some music). However, if the midi was created at 8 bit, there would be quite a difference.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #2
Better for what? Quantization errors are obviously much smaller but 32 bit float errors are so far below audible that the difference is purely spiritual. If you have faith, you will be saved, but if you have not, you never will notice the slightest difference. Besides, when you convert back to 16 bit, it won't matter anyway.

If you think you hear noise from resampling at 16 bit, I think there is something strange going on. I don't play with midi, are you sure it was originally created at 16 bit? The quantization noise from converting from one sample rate to another sample rate, starting and ending with 16 bit, is down more than 90dBfs. It could not be heard at any normal listening level if there is other signal present (i.e. some music). However, if the midi was created at 8 bit, there would be quite a difference.



What happens when errors are in 32 bit and converted to 16 bit?
Are the noise removed as it´s below 16 bit or what?

I think it´s 8 bit originally, but it´s played at 16bit cause else it would be noisy.
So i guess that´s what makes the noise, cause it´s very audible.


  • jensend
  • [*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #3
Edit: I wrote the below before seeing Andy-ha's comment and your reply. Your reply shows that you are quite confused. This seems to be a pattern in many of your posts. You need to try to get a grasp of the basics of digital audio. The sources I list can help. Hint: if a recording is 8-bit, "playing it back at 16-bit" will improve nothing. You can't make up information out of nowhere.

--
Well, you've been around here long enough that I would have expected you to have taken the opportunity to educate yourself about these topics rather than needing to ask this kind of question. Try using the HA wiki or Wikipedia or watching the Xiph.org videos.

The short answer: even at 16 bit integer if you're hearing the noise there's something wrong with your setup. Maybe you have a bad resampler or are using poor options.

Yes, increased bit depth allows for lower noise energy. But even with 16-bit truncation the quantization noise will rarely be a problem. For perspective, the NES had 7-bit PCM (usu. stored as DPCM), so the noise floor of 16-bit PCM is basically 54dB lower. That's a lot.

Dither doesn't reduce the noise energy, but it allows you make sure the noise is spread across the spectrum rather than concentrated in harmonic distortions. This is an absolute necessity if you're quantizing to 12 bits or less; at 16 bits you can take it or leave it; there's no reason to bother at higher bit depths.

20 bits of precision is basically as much as you can get out of any real-world ADCs or DACs due to thermal noise. 24-bit is useful helps people have plenty of headroom in recording and processing/mastering.

The reason to use floats is not really precision but convenience and simplicity. Using 32-bit float only gives you one extra bit of significand precision over 24-bit integer; the exponent does give quiet samples more precision than they have in fixed-point, but again you aren't getting enough precision from ADCs and DACs to make that matter. 32-bit float is already kinda overkill for audio; there's no point even considering 64-bit.
  • Last Edit: 19 May, 2013, 06:30:14 PM by jensend

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #4
Probably, i have tried reading, but i guess i should look into it more than just ask away.

And this isn´t just 8-bit to 16bit.
It´s coming from emulation, so i am not sure how that works, but i guess the produced audio is coming from 8 bit, but it´played at 16bit, before even storing it as 8bit.
Something like that i guess, which only makes difference when resampling, as the information working with is "8-Bit".
It´s unique situation i guess.

I will look up more about how converting up and down in Bit Depth changes stuff etc, thanks for the help:)
  • Last Edit: 20 May, 2013, 07:25:56 AM by db1989

  • saratoga
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #5
If its 8 bit then playing it at 16 bit is no different than playing at 8 bit.

  • AndyH-ha
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #6
Quantization errors are differences between the floating point calculations of the target  (a different sample rate, in your case) and the discrete possible values in any format, integer or floating point. Only so many possible values can be stored because there are only so many bits.

If this isn't clear, think of doing long division on some value like Pi. No matter to how many digits you carry out the answer, there are still more because you never get an exact answer. When the audio data calculations are done, most answers are not a value which fits withing 16 bit, or 32 bit, or whatever number of bits you use. The answers must be truncated at the limit of the possible precision determined by the data format. The difference between that possible answer (determined by the number of bits available) and the "actual" calculated value which would run to more bits, is the quantization error. It is experienced as noise and distortion in the audio.

There are also errors of the same kind when converting from floating point to 16 bit (from any greater number to any lesser number of bits). There is no way to avoid quantization errors, thus more noise and distortion are added when converting down to 16 bits.

The reason for working in floating point instead of 16 bits is there will, in general, be many calculations done on each audio sample to accomplish the goal (in this case, resampling). Each of those calculations adds another error. In floating point format, the errors are too small to matter, whereas when working in 16 bit, each error is much larger.

When converting to 16 bit from 32 bit as the last step, you acquire only one of those larger errors. This is essentially impossible to hear under any normal circumstances.

The normal procedure when reducing bit depth is to dither with noise shaping. This accomplishes two things. One, the dither eliminates the quantization distortion -- not the error itself, only the distortion caused by the error. Two, the noise shaping moves most of the added noise to the highest parts of the audio spectrum. Since human hearing is much less sensitive at these high frequencies, most of the additional noise is inaudible.

In reality, although it is essentially the universal practice to dither, the noise and distortion acquired by converting from 32 bit to 16 bit probably is never audible unless you want to listen to the silence between tracks -- at a very high volume level. All this is quite different at 8 bit, as you have experienced, The noise and distortion are almost always audible at 8 bit.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #7
When converting to 16 bit from 32 bit as the last step, you acquire only one of those larger errors. This is essentially impossible to hear under any normal circumstances.

The normal procedure when reducing bit depth is to dither with noise shaping. This accomplishes two things. One, the dither eliminates the quantization distortion -- not the error itself, only the distortion caused by the error. Two, the noise shaping moves most of the added noise to the highest parts of the audio spectrum. Since human hearing is much less sensitive at these high frequencies, most of the additional noise is inaudible.

In reality, although it is essentially the universal practice to dither, the noise and distortion acquired by converting from 32 bit to 16 bit probably is never audible unless you want to listen to the silence between tracks -- at a very high volume level. All this is quite different at 8 bit, as you have experienced, The noise and distortion are almost always audible at 8 bit.


What do you mean that one 1 large error remain?

Yeah read about it a little while ago, Noise Shaping seems to be something like "debanding".

But it seems like there is no real disadvantage of converting to float when resampling (some resampler seems to do this automaticly) as it get´s bigger space to work with, and generate better results (less noise, but it matters of course).

And to the 8 bit as 16 bit.
It´s hard to explain as i don´t know myself, i have uploaded an example.

http://www.sendspace.com/file/eq568b

Try to resample it without ever changing the bit depth. It will introduce noise.

  • saratoga
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #8
I doubt you can even find a 16 bit resampler. Anything on PC will almost run at 32 bit or greater. Its actually much harder to make a 16 bit resampler than a floating point one, hence they were mostly just used on old 16 bit DSPs.

  • bennetng
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #9
http://www.sendspace.com/file/eq568b

Try to resample it without ever changing the bit depth. It will introduce noise.

The volume of your file is too low. Fixed-point audio will clip above 0dB and lose quality at low volume. Normalize to about -3dBFS before resampling can improve sound quality.

  • AndyH-ha
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #10
Quote
What do you mean that one 1 large error remain?
Quantization errors in 16 bit integer are much larger than quantization in floating point. Work in floating point and each error is "small". The same work, done in 16 bit integer, produces the same number of errors, each larger than those of floating point by more that two orders of magnitude.

So, work in floating point, then convert back to 16 bit integer. The conversion-to-16-bit error is larger than the errors while working in floating point, but there is only one such error for each audio sample. Dither the conversion and eliminate the distortion.

Still, the point that has been made several times. if you have genuine 16 bit data, and all you do is change the sample rate once, the error noise is not going to be audible under normal circumstances.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #11
http://www.sendspace.com/file/eq568b

Try to resample it without ever changing the bit depth. It will introduce noise.

The volume of your file is too low. Fixed-point audio will clip above 0dB and lose quality at low volume. Normalize to about -3dBFS before resampling can improve sound quality.



It doesn´t matter. If i normalize it to that, it will sound fine, but after resampling, there will be noise.
But as said, it´s only in these cases, not any problem really, just thought it was more common than it was.

And to saratoga and AndyH.

Well that´s good to hear, then there isn´t any reason to not work in 32bit float for resampling, other than saving resources or RAM maybe, which isn´t any problem today.

Quote
The conversion-to-16-bit error is larger than the errors while working in floating point, but there is only one such error for each audio sample. Dither the conversion and eliminate the distortion.


So if i resample in float, then convert to 16 bit, there will be a big error, but if dithered (which seems to be done by all resampler?) that big error will go away, or atleast it will be remove from being audible. Is that correct?

Thanks:)

  • bennetng
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #12
http://www.sendspace.com/file/eq568b

Try to resample it without ever changing the bit depth. It will introduce noise.

The volume of your file is too low. Fixed-point audio will clip above 0dB and lose quality at low volume. Normalize to about -3dBFS before resampling can improve sound quality.



It doesn´t matter. If i normalize it to that, it will sound fine, but after resampling, there will be noise.
But as said, it´s only in these cases, not any problem really, just thought it was more common than it was.


Which OS, resampler and soundcard are you using?
  • Last Edit: 20 May, 2013, 03:09:16 AM by bennetng

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #13
Windows 8, don´t know the resampler itself, but it´s in Avisynth, ResampleAudio(), it works in the bit depth you give it. Sound card it, Creative ZXR Sound Blaster.
  • Last Edit: 20 May, 2013, 07:27:00 AM by db1989

  • AndyH-ha
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #14
Quantization errors never "go away". They are impossible to avoid.
Dithering eliminates the distortion caused by the errors but not the errors.
Dither is noise. Dithering adds still more noise to fix the distortion.
Noise shaping moves the noise to less audible frequencies.
Therefore, you end up with more noise, no distortion, and cleaner sound.
Applying dither may be automatic in some programs but not in any I use.
One must choose to dither, choose the kind of dither, choose to use noise shaping and choose what kind of noise shaping.

  • bennetng
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #15
Windows 8, don´t know the resampler itself, but it´s in Avisynth, ResampleAudio(), it works in the bit depth you give it. Sound card it, Creative ZXR Sound Blaster.

http://www.hydrogenaudio.org/forums/index....ost&id=7530
Please download the files and play them directly, using normal volume as if you are listening to a normal song.

54321.wav: You should hear nothing. If you can hear anything then your playback environment is not optimized.

13579.wav: You should ONLY hear a single tone rising up. If you hear any other thing, such as something going up and down constantly then your playback environment is not optimized.

If you passed the tests above, use the resampler to convert the files to 44.1 and 48k and listen to them to see if they still pass the test or not. If not, then the resampler is a low quality one.
  • Last Edit: 20 May, 2013, 05:27:39 AM by bennetng

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #16
I don´t know how high volume i should have on 54321. Cause "normal" volume for me changes, as many stuff i listen to have different dynamics.
But i don´t hear anything if i have low volume on the player, but if i increase it, i hear noise.

13579 sound like you are saying, a tone going upp, and i hear this "noise" like when it´s increasing, but it think it´s supposed to sound like that, cause it´s not normal noise, it´s like when someone throw a grenade in a movie, and you get that tinnitus sound.
  • Last Edit: 21 May, 2013, 06:52:51 AM by db1989

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #17
Quantization errors never "go away". They are impossible to avoid.
Dithering eliminates the distortion caused by the errors but not the errors.
Dither is noise. Dithering adds still more noise to fix the distortion.
Noise shaping moves the noise to less audible frequencies.
Therefore, you end up with more noise, no distortion, and cleaner sound.
Applying dither may be automatic in some programs but not in any I use.
One must choose to dither, choose the kind of dither, choose to use noise shaping and choose what kind of noise shaping.



Yeah i understand that.

It´s pretty much like video dithering, it tried to replicate higher bit by adding noise that makes an illusion that it´s better than it´s acctually is.

I will have to look up about the dithering and noise shaping in the resampler i am using (and bit converter).

Is there any special shape that is good?

  • AndyH-ha
  • [*][*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #18
Some companies have made a big advertising thing about how their dithering-noise shaping is better than others but I think the reality is pretty much the same as with audiophile interconnects. Frequently, programs that provide different options will suggest various ones for differing kinds of audio material.

It is possible to make choices that will add audible noise to your music although that will depend partly on the music. Loud compressed stuff probably won't leave any room for the most blatant dither noise. Simple acoustical material with extended low level sections might show up some noise from some kinds of dither-noise shaping.

I just experimented with the options I have and selected what worked without adding anything I could hear for itself. I've found 0.5 bits of shaped triangular dither sufficient for all of my normal work. I suspect the label on the noise shaping depends on the program supplying it.

Created 10 seconds or so of silence in floating point or 24 bit integer. Convert to 16 bit with various dither and noise shaping options. You should see that some options add much more noise than others. If you can hear anything at less than an extremely high volume level, it is too much dither and/or poor noise shaping.

Generate a pure mid frequency tone in floating point or 24 bit at around -12dBfs. Convert to 16 bit without dither. You may be able to hear the distortion. You should be able to easily detect it in a spectral view and/or frequency analysis. Try a sweep tone that fades into silence.

Next repeat with your choice of dither & noise shaping. If the distortion isn't gone, you did not add enough dither. If you can hear anything added to what you hear in the higher resolution original, without turning up the volume above what is comfortable for the louder parts of your tone, the noise shaping is inadequate.

  • jensend
  • [*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #19
It´s pretty much like video dithering, it tried to replicate higher bit by adding noise that makes an illusion that it´s better than it´s acctually is.
Great heavens, no. This is not an illusion. It's not "faking" being "better than it actually is." It's substituting other noise for the input-dependent noise you get for truncation to make it even less perceptible.

Did you watch the Xiph video I linked which explains this stuff? If not, why not? Are you unwilling to take such a basic step to educate yourself?

AndyH-ha, you are doing a great job of helping this fellow become even more confused than he already is.

There's no reason on earth you should have to be concerned the subtle differences between different shaped dithers here. DITHER IS NOT YOUR PROBLEM. NOR IS BIT DEPTH. The sample you uploaded sounds just fine at 16 bits after being resampled with any decent resampler, regardless of dither options.

If you're having trouble you need to figure out what's wrong with your toolchain. A couple questions to start off- what's your target sample rate? Are all the clips starting out as 16-bit 48kHz like the one you uploaded?

I haven't tested it, but perhaps the avisynth resampler isn't too good. Some details about it in this post. 64 taps is not a lot esp with linear interpolation.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #20
By changing the noise and moving it so it doesn´t become audible , or a least less, that´s what i meant with it being like video dithering.
Sound is not like video, so i am not making a side comparison there, sorry for making it sound like that.

And i tried to watch the video, but it never loaded, tried many times with 480p and 720p etc, even downloading it, nothing worked:(

And i am not really having an real problem.
I just wanted to know about it, from the start i just wanted to know if 64bit float is better than 32bit float, which it seems like it is, but in such a degree that it doesn´t even matter.

I am grateful to AndyH, he is helping me out alot, as well as others, if you don´t want to answer or think that i shouldn´t bother by even asking, then i will won´t demand anything (and i don´t demand anything to start with though:) )


EDIT:

The video seems to work now, atleast the downloading works, not the flash player, so will watch it:)
  • Last Edit: 21 May, 2013, 06:53:25 AM by db1989

  • bennetng
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #21
13579 sound like you are saying, a tone going upp, and i hear this "noise" like when it´s increasing, but it think it´s supposed to sound like that, cause it´s not normal noise, it´s like when someone throw a grenade in a movie, and you get that tinnitus sound.

Then it is possible that the resampler is not good enough (if not bad). Is the noise
[1] audible when playing the file directly, or
[2] only audible after resampling, or
[3] always audible?

If it is [2], please upload the file.

Floating point is useful in resampling and many other processings, but floating point alone won't make a good resampler. 32-bit float vs 64-bit float is like giving a person 10 litres of drinking water vs 100 litres per day, if there is no food that person won't survive anyway.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #22
I only hear it after, so 2.

I did upload it here: http://www.sendspace.com/file/eq568b

Oh ah see, i thought More bit for Resampler was making all the difference.
I guess it´s not the best resampler then, what is difference between bad and good resampling?
I know more about Noise and Dither now, but that´s not done while resampling, but afterwards right?
  • Last Edit: 21 May, 2013, 06:53:41 AM by db1989

  • bennetng
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #23
I did upload it here: http://www.sendspace.com/file/eq568b

I mean the resampled 13579.wav, not this one.

  • zerowalker
  • [*][*][*][*]
Resampling - Rounding Errors and Bit Depth.
Reply #24
I did upload it here: http://www.sendspace.com/file/eq568b

I mean the resampled 13579.wav, not this one.


Oh, here: http://www.sendspace.com/file/iqpnf4

I hear noise as well on this one after resampling.
Not much, but you hear it pretty clearly at the end.