Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: HDCD filter added to FFmpeg (Read 7067 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

HDCD filter added to FFmpeg

An HDCD filter was added in FFmpeg 3.1. It is based on the Foobar2000 HDCD component code. The documentation is... not existing.

If the conversion is done with wav, something like:
ffmpeg -i input16.wav -af hdcd output.wav
... the output will still be 16-bit, because FFmpeg will resample to 16-bit for wav by default. So if you want 24-bit output you must specify:
ffmpeg -i input16.wav -af hdcd -acodec pcm_s24le output24.wav
I don't know if it is still resampled or not, as the filter only outputs pcm_s32le.

If you use flac, it will output 24-bit flac:
fmpeg -i input16.flac -af hdcd output24.flac
Also a bit confusing, because the filter outputs pcm_s32le. I don't know if it is resampled or just stuffed into 24-bits, which should work anyway (and kindof what I expected) because only 20 of the 32 bits are actually used. Isn't that what the filter is doing when it converts the 16 to 20? just copying the 16-bits and then using more that 16 when peak-extend is used? I don't think it is resampling, but maybe. Can anyone explain?

BTW, I've started a page at HAK: High Definition Compatible Digital

Re: HDCD filter added to FFmpeg

Reply #1
Please, do not use the word "resample" when talking exclusively about bit depth.

The format "s32le" means 32bit signed integer in little endian bit order.  Since it is integer, and since there is no resampling, nor conversion to/from float, it is to be expected that it simply fills the lowest bits with zeroes.

ffmpeg works internally in s32le because that way it can maintain the precision between filters.  (actually, i think it has an additional name for the internal format used, but broadly speaking, it is like s32le).

So everything is logical.

About the article:  Even though you used the same description used in the wikipedia article, describing HDCD as a "Microsoft propietary codec" greatly exaggerates it:  As said in that article: "In 2000, Pacific Microsonics folded and Microsoft acquired the company and all of its intellectual property assets.".  So I would rather describe it as a Pacific Microsonics codec, now property of Microsoft.

Re: HDCD filter added to FFmpeg

Reply #2
I wasn't using "resample" incorrectly. Using wav without specifying pcm_s32le, ffmpeg does actually resample back to 16-bit. A resampling filter is added automatically. This can be seen by using -v 100.
I was asking if it is resampled, or if the bits are just extended/truncated. That was the question. After looking at the code a bit, I think you(we) are right, and it is not resampled, which I think is good.

I just quoted the Wikipedia article there so that some brief definition was included.

Re: HDCD filter added to FFmpeg

Reply #3
It's not resampling, it's truncation.

Re: HDCD filter added to FFmpeg

Reply #4
Yeah, thanks.

Re: HDCD filter added to FFmpeg

Reply #5
I updated the wiki as well.

Re: HDCD filter added to FFmpeg

Reply #6
I updated the wiki as well.
Ah I see that, but I really did mean resample in that case.
Try this:
ffmpeg -v 100 -i HDCD16.flac -af hdcd OUT.wav

You will see this line in the verbose output:
auto-inserting filter 'auto-inserted resampler 0' between the filter 'Parsed_hdcd_0' and the filter 'audio format for output stream 0:0'

So when the output is to be wav (defaults to pcm_s16le), it does seem to actually resample it. When the output is flac, it just truncates it to 24-bit flac. That line does not appear.

Edit:
Additionally, you can see that the loudness is the same for both the 16-bit wav output and the 24-bit flac output. If it had been truncated to 16 instead of resampled, then the 16-bit wav would be much louder that the 24-bit flac.

./ffmpeg -i HDCD16.flac -af hdcd OUT16.wav
./ffmpeg -i HDCD16.flac -af hdcd OUT24.flac

loudness tag HDCD16.flac OUT16.flac OUT24.flac
Album gain, Track gain, Album peak, Track peak                                 
  -1.85 dB,   -4.86 dB,   0.783112,   0.783112, HDCD16.flac
  -1.85 dB,    1.16 dB,   0.783112,   0.391571, OUT16.wav
  -1.85 dB,    1.16 dB,   0.783112,   0.391556, OUT24.flac

Both are quieter than the original 16-bit because those were just copied/extended from 16-bit to the 32-bit intermediate.

Re: HDCD filter added to FFmpeg

Reply #7
Again, do not mix things. The effect of changing between 16bit and 32bits is nonexistant if we talk about loudness/amplitude. What you describe is the effect of applying the HDCD decoder.
The reason why both "OUT" files are quieter than the HDCD original is because of the HDCD process ( If i'm not wrong, the HDCD original is somewhat (DSP) compressed, and the HDCD information is used to uncompress it, increasing the SNR. Since usually the HDCD original is already normalized near to full scale, the decoded part requires to lower the amplitude so as to not go above full scale).


As for the information related to what ffmpeg is doing:
- I was unaware about the auto-insertion text (In fact, i haven't used it much with output to wav). I am only assuming that when using a codec, the codec supports the ffmpeg internal format, so ffmpeg itself doesn't need to change the format (the codec implementation itself deals with it).
In case of output to wav, the codec does not do this automatically, so ffmpeg may detect this and add the resampler as a mechanism to do the bitdepth change.

Re: HDCD filter added to FFmpeg

Reply #8
Resampling refers to changing the sampling rate. As the name implies it literally means 'sampling again'.  Converting the numerical representation of samples is not resampling because a second sampling is not generated.

Re: HDCD filter added to FFmpeg

Reply #9
Resampling refers to changing the sampling rate. As the name implies it literally means 'sampling again'.  Converting the numerical representation of samples is not resampling because a second sampling is not generated.

I see, well you are right that the sampling rate is not changed, so not "resample", but some extra processing is done to the samples if the output sample format is to be less 16-bit, and that work is done by ffmpeg swr resampler, and only when the resampler is added.

So,
When moving through the hdcd filter, the samples are copied/sign-extended from 16 to 32-bits for any sample that is not to be processed, while some processed samples may be extended into the 4-bits above.

But after hdcd is done with it...
If the output sample format is to be 32-bit, the samples are left as is  (no resampler is added by ffmpeg)
if the output sample format is to be 24-bit, the samples are truncated  (no resampler is added by ffmpeg)
but if the output sample format is to be 16-bit or less, the samples are "rescaled" then? (by the resampler) Is that an alright term, "rescaled"? It's not just truncation in this case.

Try and compare, if you like, using ffmpeg git master:

./ffmpeg -i HDCD16.flac -af hdcd -acodec pcm_s32le WOUT32.wav
./ffmpeg -i HDCD16.flac -af hdcd -acodec pcm_s24le WOUT24.wav
./ffmpeg -i HDCD16.flac -af hdcd -acodec pcm_s16le WOUT16.wav     # resampler added
./ffmpeg -i HDCD16.flac -af hdcd -acodec pcm_u8 WOUT8.wav         # resampler added


BTW, I'm using this HDCD16.flac, as it was the ffmpeg sample of HDCD.

Re: HDCD filter added to FFmpeg

Reply #10
Changing the numerical format of a sample is a trivial operation, so I don't think it has a general name.  In the case of reducing precision without dithering, 'truncation' would be the literal operation applied.  No rescaling is required.

Re: HDCD filter added to FFmpeg

Reply #11
I see, it is truncating the other end. I was only thinking of truncating the msb end. Of course, it could reduce from 24 to 16 by just dropping the 8 least significant bits. Now, I feel foolish.

Thanks.

Re: HDCD filter added to FFmpeg

Reply #12
For anyone interested, the FFmpeg HDCD filter is updated.
Low-level gain adjustment is fixed (it was broken before), docs are slightly more useful, and there is now some feature information reported when using the filter.

 

Re: HDCD filter added to FFmpeg

Reply #13
Does anyone know if there is a way with FFmpeg to test if a file (WAVE or lossless) actually has HDCD data ?

I tried :
Code: [Select]
$ ffprobe -show_streams -show_format INPUT_FILE
which returns nothing useful

And :
Code: [Select]
$ ffmpeg -loglevel debug -i INPUT_FILE -af hdcd -f flac /tmp/out.flac
always succeeds with the same console output for HDCD or non HDCD input.

I'm using FFmpeg 3.1.1 from Arch Linux.

EDIT :
As suggested in the wiki, I also tried :
Code: [Select]
$ ffmpeg -hide_banner -nostats -y -v verbose -i INPUT_FILE -t 30 -af hdcd -f s24le /tmp/a.out 2>&1 | grep _hdcd_
but I do not get the HDCD detected: line, even with the samble HDCD file from the FFmpeg ticket.
Opus 96 kb/s (Android) / Vorbis -q5 (PC) / WavPack -hhx6m (Archive)

Re: HDCD filter added to FFmpeg

Reply #14
...
As suggested in the wiki, I also tried :
Code: [Select]
$ ffmpeg -hide_banner -nostats -y -v verbose -i INPUT_FILE -t 30 -af hdcd -f s24le /tmp/a.out 2>&1 | grep _hdcd_
but I do not get the HDCD detected: line, even with the samble HDCD file from the FFmpeg ticket.

You'll need to build ffmpeg from git, the patches for reporting HDCD stats have not been in a release yet. There is also a bug fix in the HDCD decoding that you'll want that isn't in a release yet.
Code: [Select]
git clone git://source.ffmpeg.org/ffmpeg
cd ffmpeg
./configure && make
./ffmpeg ...

Re: HDCD filter added to FFmpeg

Reply #15
Thanks @bp0 I have it working now.

I wrote a small script to scan my archive using threads to process files in parallel and posted the results in the other thread.
Opus 96 kb/s (Android) / Vorbis -q5 (PC) / WavPack -hhx6m (Archive)


Re: HDCD filter added to FFmpeg

Reply #17
@bp0 I see FFmpeg 3.1.2 has just been released, does it contain all the HDCD goodness, including your recent fixes, or is it still recommended to use the head of git master branch to use the HDCD filter?
Opus 96 kb/s (Android) / Vorbis -q5 (PC) / WavPack -hhx6m (Archive)

Re: HDCD filter added to FFmpeg

Reply #18
It does not. It is a bugfix release for 3.1 that is not allowed to add any features. There was one small bug fixed in the HDCD filter, so it is preferable to 3.1 or 3.1.1, but it doesn't have the stats reporting, or stereo processing mode, or analyze mode.

Re: HDCD filter added to FFmpeg

Reply #19
Thanks, I'll stick to git master until 3.2 is released then.
Opus 96 kb/s (Android) / Vorbis -q5 (PC) / WavPack -hhx6m (Archive)