Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM (Read 6651 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Hello!

Recently I wanted to compress some of my DSD (.dsf) collection with WavPack. The encoded files play well in foobar2000 with foo_input_sacd as DSD stream, but when playing as 24bit 352.8kHz PCM, there were clicks/pops at the beginning of almost every track. I'm quite sure that my .dsf files are good as they came directly from an online music store, and they work well with my DAP. So I did the following test:

 • Play and output the .dsf files directly to a DAC - no clicks and pops;

 • Unpack the .wv files and force output to WAV format using WvUnpack, then play the .wav files with foobar2000 - clicks/pops occurred;

 • Apply a 30kHz lowpass filter to the .wav files using SoX, then play the processed files with foobar2000 - clicks/pops persist;

 • Trim the .wav files using SoX to keep only the first 0.125 seconds and inspect the spectrogram with Spek - found a weird vertical "blue bar" (please see attached screenshot as an example, all the tracks I tested have this "blue bar");

 • Apply a fade-in effect to the first 8 samples of the 24bit 352.8kHz PCM files (using SoX), then play with foobar2000 - the clicks/pops disappeared!

So I wonder if this is due to WavPack did not handle the DSD to PCM conversion correctly, or if I did something wrong in the process? And how to fix this?

Thank you  :)

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #1 –
Are there clicks if you don't use WavPack at all and simply play your unpacked .dsf files as 24/352 PCM via foo_input_sacd?

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #2 –
Quote from: bennetng on
Are there clicks if you don't use WavPack at all and simply play your unpacked .dsf files as 24/352 PCM via foo_input_sacd?

Hi bennetng - I just tested, and the result is No. I tried Multistage mode and Direct mode, as well as different sample rates and gain settings. None of the scenarios has audible clicks.

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #3 –
Thanks for posting this! I should have posted earlier because I know what’s going on, but I was hoping I’d get a chance to experiment with a fix (I haven’t yet).

Unfortunately, using WavPack’s built-in DSD to PCM conversion is not ideal because it uses a filter with a length of 56 DSD samples, which is 7 DSD “bytes”. Each “byte” of DSD data generates one 24-bit output PCM sample, which means that when converting a file there are 6 samples in the beginning that cannot be accurately converted because we have no history from the previous file. This means that gapless playback is never going to be perfect using this facility (I believe that foo_input_sacd does the PCM conversion itself, outside the file boundaries).

One solution to this would be to simply discard those 6 samples and have the PCM version slightly shorter than it should be. I didn’t really like that because it would mean, for example, that the length of a PCM conversion would change with different filters, and perhaps different versions of the decoder. Not ideal.

So I decided to pre-fill the filter with DSD silence and generate those 6 samples anyway, figuring that at least the silent gap case would be glitchless and the correct length. The problem is that unlike PCM, with DSD there are different flavors of digital silence. I used a single byte pattern of 0x55, but I have seen others. For example the DSD file in my test suite uses a 5-byte pattern of 0xD3, 0x2C, 0xD2, 0xD2, 0xD2.

What I didn’t realize until now was that if you transition from one flavor of DSD silence to another, it can generate a glitch that extends down into the audible band. Here’s what you get transitioning from the 0x55 “silence” to the 5-byte version:



In your spectral view of the audio, the “blue bar” is simply that 6-sample glitch (it would have actually been easier to see in waveform mode).

In any event, I’ll be experimenting a little with solutions that aren’t too ugly. But in the meantime you can rest assured that there’s nothing wrong with your WavPack files (especially if you created them with the -v option!) and the clicks are just an artifact of the files being independently converted to PCM before being stitched together. Obviously if you have the ability it’s better to play them natively.

If you wouldn’t mind, I’d appreciate getting the very beginning of those DSD files to find out what “silence” they are using. You are the first person to report hearing this, so there may be something additional going on. Probably the easiest way would be to use the “--until” option of wvunpack to do that, something like this:

wvunpack --dff --until=0:1.0 dsdfile.wv -o dsdfile-1sec.dff

(and of course you can WavPack that again to reduce the size).

Thanks!


Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #4 –
Quote from: bryant on
One solution to this would be to simply discard those 6 samples and have the PCM version slightly shorter than it should be. I didn’t really like that because it would mean, for example, that the length of a PCM conversion would change with different filters, and perhaps different versions of the decoder. Not ideal.

You can discard these samples and then pre-extrapolate the signal (see e.g. https://datatracker.ietf.org/doc/html/rfc7845#section-7 )

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #5 –
Quote from: bryant on
Unfortunately, using WavPack’s built-in DSD to PCM conversion is not ideal because it uses a filter with a length of 56 DSD samples, which is 7 DSD “bytes”. Each “byte” of DSD data generates one 24-bit output PCM sample, which means that when converting a file there are 6 samples in the beginning that cannot be accurately converted because we have no history from the previous file. This means that gapless playback is never going to be perfect using this facility (I believe that foo_input_sacd does the PCM conversion itself, outside the file boundaries).

When converting a 2.8 MHz DSD to a 24/352.8 PCM, I guess the number of samples in the output file should be 1/8 as the number of samples in the input file. I tried some conversion previously with foo_input_sacd, WavPack, and FFmpeg. Looks like WavPack is the only one among the three whose output has the "correct" sample count. So I was under the impression that foo_input_sacd and FFmpeg's conversion is not as "accurate" as WavPack :D


Quote from: bryant on
One solution to this would be to simply discard those 6 samples and have the PCM version slightly shorter than it should be. I didn’t really like that because it would mean, for example, that the length of a PCM conversion would change with different filters, and perhaps different versions of the decoder. Not ideal.

One thing on top of my head is that some tracks do not start with silence, so discarding 6 samples may (or may not) cause additional glitches.


Quote from: bryant on
If you wouldn’t mind, I’d appreciate getting the very beginning of those DSD files to find out what “silence” they are using. You are the first person to report hearing this, so there may be something additional going on.

The attached zip file has the first 4 seconds and the last 4 seconds from 10 tracks, 20 files in total, including a 3-movement piece (Mozart's KV 181) where there's no pause between movements. I'm happy to provide more files if there's a need.

Update: Not sure why I can't attach the zip file so I put it on Google Drive (test_files.zip)

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #6 –
Quote from: lvqcl on
Quote from: bryant on
One solution to this would be to simply discard those 6 samples and have the PCM version slightly shorter than it should be. I didn’t really like that because it would mean, for example, that the length of a PCM conversion would change with different filters, and perhaps different versions of the decoder. Not ideal.

You can discard these samples and then pre-extrapolate the signal (see e.g. https://datatracker.ietf.org/doc/html/rfc7845#section-7 )
Yes, this is what I was planning. Thanks for the link! I'm hoping to get away with something simpler than LPC for generating just 6 samples, but we'll see...  :)

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #7 –
Quote from: S1 on
Quote from: bryant on
One solution to this would be to simply discard those 6 samples and have the PCM version slightly shorter than it should be. I didn’t really like that because it would mean, for example, that the length of a PCM conversion would change with different filters, and perhaps different versions of the decoder. Not ideal.

One thing on top of my head is that some tracks do not start with silence, so discarding 6 samples may (or may not) cause additional glitches.
No, you're absolutely right. Throwing away samples only fixes the silent gap case. There definitely will be a glitch if the transition is not silence and samples are removed, so I won't be doing that.

Quote from: S1 on
Quote from: bryant on
If you wouldn’t mind, I’d appreciate getting the very beginning of those DSD files to find out what “silence” they are using. You are the first person to report hearing this, so there may be something additional going on.

The attached zip file has the first 4 seconds and the last 4 seconds from 10 tracks, 20 files in total, including a 3-movement piece (Mozart's KV 181) where there's no pause between movements. I'm happy to provide more files if there's a need.
Thanks so much for creating these samples; it's way more than I expected!

I am able to hear the glitches in the silent transitions, but I could not hear them in the non-silent cases (although it was not an ideal listening environment). However, I can see them when I zoom in so I should be able to tell if I've fixed them sufficiently.
 

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #8 –
Thanks again for finding and reporting this issue. I have fixed the glitches, or at least reduced their magnitude enough that I doubt they'd ever be audible.

WavPack 5.5.0 Release Candidate

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #9 –
Do you activated the low pass filter? DSD has too lots of ultrasonic noise (really noise, nor harmonic distortion or harmonics), when do you don't use low pass filter clicks, pops and other noises can apear. Your native DSD Digital to Analog Converter filters these noises but also add phase shifting, for learning I used this software during trial.

Unfortunately, the bast majority DSDs come from PCM files converted. And I don't know any converter that can trans-code without preringing and postringing. Only when the DSD came from classical music, jazz or chorus; maybe can be native DSD file because these simply are recorded without almost edition from native DSD Analog to Digital Converter.

Also unfortunately, almost doesn't exist native editors for DSD with the same features for PCM.

Sincerely.

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #10 –
But doesn't that noise work as intrinsic dithering?

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #11 –
Quote from: rutra80 on
But doesn't that noise work as intrinsic dithering?

The converters that I investigated these add no shaped dither because by dynamic range is far enough the hearing threshold since DSD 64.

The unique advantage for DSD is when you have a native DSD record from native DSD Analog to Digital Converter. The advantage is you don't have preringing and postringing, but you pay phase shift from your DAC by IIR filter.

I read that DSD is similar to analog records, and this is true because IIR filter or analog low-pass filter adds similar phase shift and high frequency attenuation at DSD  64. DSD 128, 256 and so until 1024 each one move the phase shift and high frequency attenuation to ultrasonic sound, therefore these flattening the audible frequency response for better performance and you could choose less aggressive filter if you're DAC can offer this.

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #12 –
Quote from: rutra80 on
But doesn't that noise work as intrinsic dithering?
Yes. but the process of converting DSD to PCM almost always involves sample rate decimation, if the decimation filter is too weak, aliasing will creep into the audible band, and therefore a bad thing.

Since the previous post linked to some Archimago's posts with foo_input_sacd and all of those plots were done using the "Multistage" decimation method, in one of his recent blog entry I attached an installable filter for him to try. Here is my comment:

https://archimago.blogspot.com/2022/06/notes-on-dac-dsd-1-bit-pdm-measurements.html?showComment=1655906640637#c3453242716296098960

Scroll up to the main blog content to see the differences between Multistage and my filter, notice the differences are not only on the ultrasonic noise, but also below 20kHz.

That's because DSD being a 1-bit format, it is very tricky to completely remove correlated distortion under all conditions. To see things more clearly, the website below shows some plots:
https://pcmdsd.com/Software/PCM-DSD_Converter_en.html

Notice there are some irregularities in the ultrasonic noise, if too much of these stuff are allowed to creep into the passband during PCM decimation, the overall signal quality below 20kHz after decimation will also be decreased.

Keep in mind that it is still a very basic 1kHz sine wave, so could be unfair to use these plots alone to judge the quality of different encoders. In the case of PCM-DSD_Converter, the strongest noise shaping among other encoders while has an advantage to maintain low noise floor below 20kHz, it also has a higher risk of overloading the modulator and completely trash the encoded DSD file.

In the case of DSD to analog conversion, while there is a general recommendation of 50kHz analog filter, different hardware devices (DAC chips and or other additional circuits / components) actually perform quite differently, the details could be found on the datasheets of different DAC chips for example. After converting to analog, if the ultrasonic noise level is still too high, the noise can also results in analog domain distortion (e.g. IMD).

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #13 –
To stay somewhat on-topic (foo_input_sacd and wavpack) yet still somewhat off-topic (about the audible click), here are some illustrations. The DSD test signal used was directly modulated from 2822400Hz, not upsampled from a lower sample rate. This signal is then decimated to 352.8kHz via different converters. When foo_input_sacd was used, all settings are using the 64-bit float version.

The -1dB@25kHz.txt filter I made can be download here:
https://archimago.blogspot.com/2022/06/notes-on-dac-dsd-1-bit-pdm-measurements.html?showComment=1655906640637#c3453242716296098960

These are the ones with more dramatic differences:
X
X
X


The last three screenshots are pretty close to the inherent noise floor of the modulator itself. It is kind of hard to visually differentiate some of the minor differences in some graphs even though one of them is using 24-bit output. Better download the pictures and use a photo viewer to rapidly switch back and forth to compare them. Of course, I've never claim audible differences, so don't ask for ABX log and such.
XXX




Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #14 –
I tend to think about PCM vs DSD, having two things on my mind:

1. PCM dynamic range is linear across whole frequency range, while in DSD it is not (low frequencies have far more dynamic range than ultrasonic ones which fall down to 1bit equivalent for 1411.2kHz@DSD64).

2. Both signals have bitrate, for stereo DSD64 it's what, 5644.8 kbps?

Converting DSD to PCM, kind of averages dynamic range.
So while keeping that 5644.8kbps, we could convert DSD64 to either:

- 16bit 176.4kHz PCM (that would keep quite a lot of loud noise at ultrasonic frequencies)

or

- 64bit 44,1kHz PCM (which maybe would be better - overshot dynamic range and almost no ultrasonics to deal with)


DSD would be interesting if it could stay like that in the whole audio chain - from recording to speakers. But it often gets converted to PCM sooner than later, making it quite useless IMO. Thats why I'd rather convert it to lower sample rate while keeping higher bit depth.

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #15 –
Quote from: rutra80 on
So while keeping that 5644.8kbps, we could convert DSD64 to either:

- 16bit 176.4kHz PCM (that would keep quite a lot of loud noise at ultrasonic frequencies)

or

- 64bit 44,1kHz PCM (which maybe would be better - overshot dynamic range and almost no ultrasonics to deal with)
For DSD64 to PCM, a target format of 24-bit 88.2kHz is enough. While there is some seemingly rise of noise above 20kHz, don't forget the noise floor of 16-bit is also 48dB higher than 24-bit, yet some CDDA releases still have noise shaping to pump up the noise floor beyond 16kHz or so. This means, despite the rise of noise, the spectrum of DSD64 can still be used to encode audio signal up to 40kHz or so.

64-bit is just used to minimize rounding error and prevent clipping during processing, it doesn't mean the DSD modulators themselves are capable of 64-bit dynamic range below 20kHz.

I used 352.8kHz in the illustrations simply because wvunpack uses 8x decimation. I have to use the same format in other converters for apple to apple comparisons.

In general, DSD is simply a 30+ years old concept. At that time audio ICs were capable of using a 1-bit format to achieve good results with low cost. DSD doens't have the same efficiency of PCM at the same bitrate.

Regarding "ringing", there are also digital IIR filters and minimum phase FIR filters with no pre-ringing like what analog filters do (analog filters still have post-ringing). But anyway, for non-synthetic and properly mastered music these things are non-issue.
https://archimago.blogspot.com/2018/01/audiophile-myth-260-detestable-digital.html

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #16 –
bennetng.

Really nice explanation, I can't resume these in that way. ;)

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #17 –
Hi everyone... I am an SACD enthusiast myself and I am still encountering those ticks or clicks at the beginning of DSF or ISO tracks. I am currently using the sacd_extract-WINDOWS-v0.3.9@setmind_git_2019-03-21 and I have also tried the sacd_extract-0.3.9.3-107-win64. I simply load the disc into my sony blu-ray player and connect by wifi. When I extract the ISO and play it in foobar, if I do hear anything, it is a very soft tic at the beginning of the first track.

I thought that the latest version of sacd_extract.exe was to have eliminated the pops/clicks? Or is this only a "partial" fix? I know that a certain amount of frames have been removed as mentioned in the log.

I am just wondering if there is something here I am missing, perhaps and any help or suggestions are greatly appreciated. I am not on these forums all the time and I hope someone can help.

Thanks,

Re: Clicks/pops occur at the beginning of DSD wv tracks when playing as PCM

Reply #18 –
Hello sacdman,

It has been some years since I did conversions on my pc from SACD ISOs to WavPack, but what I do remember from extracting an ISO into separate DSF files, is that in order to prevent audible clicks, you need an intermediate step to the DFF format. The problem with that is that the DFF format has no good metadata support, and that's why DSF is preferred in the end. But you can't go directly from ISO to DSF because then you get clicks.

It's entirely possible that this has been fixed now in updated softwares, but on the other hand it could still be the reason for your troubles. I haven't been following this topic because I did all my conversions some years ago.

I have attached the tool I used for my conversions without clicks. I originally got it from the website computeraudiophile I believe. This tool, called iso2dsf, automatically does the two-step conversion for you, while copying the metadata from the ISO to the final DSF file. A pdf manual is included, although it speaks of an installer which I don't have. It's just a zip file with the tools. You can use iso2dsf via the command line, or you can drag-and-drop ISO files onto the iso2dsf executable. iso2dsf then uses the other included executables (first sacdextract.exe and then dff2dsf.exe) to perform the two-step process while preservering the metadata, so keep those executables in the same folder.

Running executables you got from a stranger on the internet is very sketchy, I know, so maybe try it on an old isolated/spare computer or in a virtual machine, whatever. Or now that you know of this solution you can maybe find updated versions of these tools somewhere else.

I'm sure there's more than one way to skin a cat, but this iso2dsf tool has always worked flawlessly for me. After you get the click-free DSF file, you can then use WavPack to compress it.