Skip to main content ↑ ↓
Topic: HDCD Spotted - Post any rarities here (Read 5280 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: HDCD Spotted - Post any rarities here

Reply #25 –
Detected HDCD in my lossless archive, using recent FFmpeg:
  • Behemoth Demigod
  • Behemoth Slaves Shall Serve
  • Cannibal Corpse Gore Obsessed
  • In Flames Trigger
  • In Flames Soundtrack to Your Escape
  • John Williams Star Wars Episode III - Revenge of the Sith OST
  • Machine Head The More Things Change
  • Machine Head The Blackening
  • Melvin Taylor Rendezvous With the Blues
  • Meshuggah Catch Thirtythree
  • Nine Inch Nails The Fragile
  • Opeth Ghost Reveries
  • Sebastian Bach Bring 'Em Bach Alive!
  • Sonata Arctica Unia
  • Within Temptation The Silent Force

Re: HDCD Spotted - Post any rarities here

Reply #26 –
Quote from: dutch109 on
Detected HDCD in my lossless archive, using recent FFmpeg:
...
Very cool. If you still have the output from FFmpeg could you add rows for the missing items at List of HDCD-encoded Compact Discs? If you don't want to edit the wiki, you could post the ffmpeg log on the talk page or here and I will add the items.

Re: HDCD Spotted - Post any rarities here

Reply #28 –
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.

"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"

Re: HDCD Spotted - Post any rarities here

Reply #29 –
Quote from: dutch109 on
I'm not sure how to interpret the detailed FFmpeg ouput, so here is the raw log : http://pastebin.com/raw/UUt7wcby.
The main lines I use to check are the last one for the HDCD features enabled and the lines with A, B and C counters to confirm it has a high number of valid packets, otherwise the cd could be a false positive. However, I just noticed two strange things on your logs:

[Parsed_hdcd_0 @ 0x391e300] The HDCD filter is unlikely to produce a desirable result in this context.

[Parsed_hdcd_0 @ 0x391e300] HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0 (bad_config)

It is the first time I see them, so I am not sure why those warnings appeared.

Re: HDCD Spotted - Post any rarities here

Reply #30 –
Quote from: dutch109 on
I'm not sure how to interpret the detailed FFmpeg ouput, so here is the raw log : http://pastebin.com/raw/UUt7wcby.
I've added them to the table. Mostly they are HDCD-encoded discs that do not use any HDCD features. The Star Wars Soundtrack is a very strange case where there are a handful of completely valid HDCD packets in each track, but it is not fully HDCD, which is interesting, I will look into that. Thanks for the data.

Quote from: KozmoNaut on
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.

"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"
Indeed. There are many such discs. That is one of the reasons for making this detailed list. You can see which discs will light up the HDCD indicator without actually using any HDCD features.

Quote from: unfinished.hide on
[Parsed_hdcd_0 @ 0x391e300] The HDCD filter is unlikely to produce a desirable result in this context.
It is the first time I see them, so I am not sure why those warnings appeared.
This is a bug. It is supposed to warn you if there is some resampling or truncation in the stream which might destroy the HDCD code before the filter gets it (or undo the work after), but the warning is false in this case. s16-planar is being converted to s16, which is fine.

Re: HDCD Spotted - Post any rarities here

Reply #31 –
Quote from: bp0 on
I've added them to the table. Mostly they are HDCD-encoded discs that do not use any HDCD features. The Star Wars Soundtrack is a very strange case where there are a handful of completely valid HDCD packets in each track, but it is not fully HDCD, which is interesting, I will look into that. Thanks for the data.
You're welcome. If you update the FFMpeg filter and want me to scan again, let me know, I have fully automated the process.

Quote from: bp0 on
Quote from: unfinished.hide on
[Parsed_hdcd_0 @ 0x391e300] The HDCD filter is unlikely to produce a desirable result in this context.
It is the first time I see them, so I am not sure why those warnings appeared.
This is a bug. It is supposed to warn you if there is some resampling or truncation in the stream which might destroy the HDCD code before the filter gets it (or undo the work after), but the warning is false in this case. s16-planar is being converted to s16, which is fine.


If that helps I'm using the following command line:
Code: [Select]
ffmpeg -hide_banner -nostats -loglevel debug -threads 1 -y -i FILEPATH -af hdcd -f s24le /dev/null

And my custom built FFmpeg is:
Code: [Select]
ffmpeg version N-81284-g82e53b3 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --prefix=/tmp/ffmpeg.T1kOLSLRjF/FFmpeg --pkg-config-flags=--static --extra-cflags=-I/tmp/ffmpeg.T1kOLSLRjF/FFmpeg/include --extra-ldflags=-L/tmp/ffmpeg.T1kOLSLRjF/FFmpeg/lib --bindir=/opt/ffmpeg/bin --enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-encoder=vorbis --disable-encoder=aac --enable-x11grab --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --disable-debug --cpu=corei7
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 51.100 / 57. 51.100
  libavformat    57. 46.100 / 57. 46.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 49.100 /  6. 49.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100

Re: HDCD Spotted - Post any rarities here

Reply #32 –
Well, the bug is only that the warning is issued when it need not be. I only tested with FLAC, and didn't think of WavePack using a planar sample format. Nothing will change about the decoding behavior. The patch that fixes this turned out to be unexpectedly controversial, so I don't know what will happen.

Re: HDCD Spotted - Post any rarities here

Reply #33 –
Very happy to report that two HDCDs that used to have a problem with foo_hdcd, "Pleasant Smell" CD Single by 12 Rounds and "For The Masses - A Tribute to Depeche Mode" by various artists, are now being decoded properly with the latest 1.17 update ("Reinstated support for arbitrary channel counts"). The problem was that the left channel was encoded louder than the right. It's all good now. And both support peak extension.

I found a short, but very useful list of which HDCDs actually have features like peak extension, and which are "fake HDCDs": http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs

Re: HDCD Spotted - Post any rarities here

Reply #34 –
"For the Masses" actually has -4.0dB gain adjust encoded in one channel on the disc. It seems like a massive error. Anyway, ffmpeg:master and the newest foo_hdcd will ignore the mismatched gain values, but still use PE, as described in this issue.

For those interested, "analyze mode" is now in ffmpeg:master (commit), and is described at Analyze Mode.

Attached is a screenshot of Audacity showing the analyze mode ouput for "For the Masses". First track: decoded HDCD, Second track: lle (process_stereo=false); third track: pe. You can see there is one track without the problem.

Re: HDCD Spotted - Post any rarities here

Reply #35 –
Quote from: KozmoNaut on
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.

"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"

I do not remember the source off the top of my head, but I read that one of the Pacific Microsonics A/D converters included HDCD capability and could mistakenly trigger the flag on everything passed through it, regardless of whether it leveraged the peak extend and other features. I'll try and hunt that down.

Re: HDCD Spotted - Post any rarities here

Reply #36 –
I was reviewing the wiki articles on HDCD again and saw the term "reversible."  Is there any information out there indicating conclusively that this is the equivalent to "mathematically lossless" or is this just one more reason to be annoyed with this needlessly gimmicky format?

Also, is this system tied in any way to earlier generation ADCs that may exhibit audible aliasing due to far less than ideal filter design?

PS: I like that someone used the term scam. :)
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #37 –
Quote from: greynol on
"reversible."

(Wikipedia or HA wiki?)

Quote from: greynol on
"reversible."  Is there any information out there indicating conclusively that this is the equivalent to "mathematically lossless"

I sense a "42" here: I am not sure if I understand the question. Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?

An example: What will HDCD do to white noise at full volume?


Quote from: greynol on
PS: I like that someone used the term scam. :)

Funny. But should not be in the wiki I think.
Memento: this is Hydrogenaudio. Do not assume good faith.

Re: HDCD Spotted - Post any rarities here

Reply #38 –
I didn't think about HDCD for a while but when i remember right some decoded ones are simply a bit shift that equals -6dB. Shifting the decoded file one bit louder gives you the lossless original back.
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

Re: HDCD Spotted - Post any rarities here

Reply #39 –
Quote from: Porcus on
(Wikipedia or HA wiki?)
Either/both.  I'd hope my post would have motivated people to check.  The HA wiki article points to the Wikipedia article.  There shouldn't be an accessibility issue.

Quote from: Porcus on
Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?
Yes, but why only 16, when it is 20-bits that's being touted?  I am not talking about the non-decoded signal, so there is no reason to make any exceptions for control data that no longer exists in the decoded data stream.

Quote from: Porcus on
An example: What will HDCD do to white noise at full volume?
Sure, but instead of "what will HDCD do," let's be specific.  What will the HDCD encoding and decoding process do?  Take any commercial release that actually makes use of the entire bag of tricks (including peak extension).  We've already seen plenty or releases that are crushed without decoding.  When they are encoded how close are they to the original signal prior to encoding?

Quote from: Porcus on
Funny. But should not be in the wiki I think.
Perhaps, perhaps not.  The skepticism is very much valid and casual readers deserve to know that it exists.

I'm still curious about my other question regarding hardware and aliasing, as well.
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #40 –
Quote from: Wombat on
I didn't think about HDCD for a while but when i remember right some decoded ones are simply a bit shift that equals -6dB. Shifting the decoded file one bit louder gives you the lossless original back.
Yes, anything mastered on the PM A/D converter would insert HDCD packets even when all the features were off. So even if every packet contained a control code that signaled {PE:off, LLE:0.0dB}. In this case, the HDCD decoder has shifted the audio 1-bit to make room for peak extend that is never used, and if you shift it back, you'll get the original. Discs like this have a yellow background in the HDCD detected column on List of HDCD-encoded Compact Discs.

Quote from: Porcus on
Quote from: greynol on
"reversible."
(Wikipedia or HA wiki?)
In Wikipedia, and what it is referring to is the "soft-limiting" or scaling of the top -9dB of the 20-bit PCM into the top -3dB of the 16-bit PCM. The HDCD decoder will then "reverse" the soft-limiting. I don't think it is perfectly reversible, and I've seen some comments that the curve used in the original reverse-engineered software decoder is "approximate" based on trying to match a curve printed in the Keith Johnson paper. The open source decoder is based on that work, so it is not likely perfect either, but only good enough.

Quote from: Porcus on
Quote from: greynol on
PS: I like that someone used the term scam. :)
Funny. But should not be in the wiki I think.
It really was though. Even if it was a good idea to begin with, the execution was so poor in so many ways. HDCD's existence took more than it gave by a huge margin. But there is so much music out there from the last 20 years that have been touched by this...

Re: HDCD Spotted - Post any rarities here

Reply #41 –
Quote from: bp0 on
In this case, the HDCD decoder has shifted the audio 1-bit to make room for peak extend that is never used, and if you shift it back, you'll get the original.
Minus whatever data was discarded in the LSB in order to make room for the control data, not that the data in the LSB is likely to have any audible impact under all but extreme listening conditions and only for a minority or releases.

Quote
In Wikipedia, and what it is referring to is the "soft-limiting" or scaling of the top -9dB of the 20-bit PCM into the top -3dB of the 16-bit PCM. The HDCD decoder will then "reverse" the soft-limiting. I don't think it is perfectly reversible, and I've seen some comments that the curve used in the original reverse-engineered software decoder is "approximate" based on trying to match a curve printed in the Keith Johnson paper. The open source decoder is based on that work, so it is not likely perfect either, but only good enough.
And by that token, 16-bits without the gimmicks is also good enough. ;)  But thanks for answering. :)
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #42 –
Quote from: greynol on
Quote from: Porcus on
Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?
Yes, but why only 16, when it is 20-bits that's being touted?  I am not talking about the non-decoded signal, so there is no reason to make any exceptions for control data that no longer exists in the decoded data stream.

Concerning what the implementations deliver, bp0 gave the answer in that Christopher Key's hdcd.exe was reverse-engineered (from https://hydrogenaud.io/index.php/topic,30999.25.html and the links to doom9 I take it that he checked WMP, not only the curve in the HDCD paper) - without the chip you cannot expect it as bit-exact. And foo_hdcd does not (did not?) perfectly reproduce hdcd_exe either: https://hydrogenaud.io/index.php/topic,79427.400.html
So if the ffmpeg code emulates (?) foo_hdcd which not-completely-precisely emulates hdcd.exe which is a reverse engineering in the analogue domain, you have more than one lossy step. In practice.

But if the question is whether it could be lossless (if you knew the process exactly): well if you apply a strictly increasing transformation to a 24-bit signal before you convert to 16 bits, you could end up with roundoff errors - but you will also in practice apply dither in the process: a 16-bit signal is not bit-exact to the 16 MSBs of the 24 in any case. Scrutinizing whether it is "lossier than ordinary dithering" is surely beyond me (the Johnson/Pflaumer AES preprint claims the HDCD code is encrypted to appear as noise i.e. as part of the dither, so apart from the magic numbers Christopher Key mentions he used to reverse engineer, it need not any loss).

Which brings me to those "20" bits: there is a "noise shaping" part of this (details unknown to me; forum member SebastianG disputes the use of term "noise shaping", see the thread). It isn't lossless - and it isn't unique to HDCD, so one could very well claim that the "resolution" gained this way is part of the HDCD "format", only what you get from using Pacific's HDCD gear or anything else that does noise shaping equally well or better.

And by the way, the AES preprint writes "approximately 19 bit precision", and I have just conjectured that "20" is because anything above 19 will require a 20th bit if one were to store in a fixed-point PCM format. (By that argument, mp3 would be a great format, wouldn't it?)
Memento: this is Hydrogenaudio. Do not assume good faith.

Re: HDCD Spotted - Post any rarities here

Reply #43 –
I've read 20 bits long before your post so I wasn't attributing it to you.  Thanks for the summary, though.

I'm basically wondering if any of these HDCD releases would be preferable to non-HDCD masters or should one not bother to seek them out. I know I would avoid HDCD releases of the Van Halen catalog, for example (though only because they are squashed and do not use peak extension).
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #44 –
Quote from: greynol on
I'm basically wondering if any of these HDCD releases would be preferable to non-HDCD masters or should one not bother to seek them out.

Is there anything released both as HDCD and as non-HDCD, for the same "mastering"?
Memento: this is Hydrogenaudio. Do not assume good faith.

Re: HDCD Spotted - Post any rarities here

Reply #45 –
I don't know.  It doesn't really* matter to me either way.

If there are any particularly outstanding HDCD releases, especially in relation to non-HDCD alternatives (when they exist) then I'd be interested in hearing them.

Even though opinions about versions are subjective, reading about them here would elevate my level of confidence far above anything I might read from other communities which have no sanity controls.

(*) Of course if they are from the same dynamic master and there is a 16-bit non-HDCD version that makes proper use of the dynamic range available, then sure, I'd pick the 16-bit version over HDCD if I can't be assured proper HDCD decoding within the error afforded by 16-bit with allowances provided for processes that add noise.  Ideally there should be little if any difference between the two assuming proper HDCD decoding, correct? (meant as a rhetorical question and absolutely subject to TOS #8).
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #46 –
I have some HDCDs and dithered all of the ones with peak extension back to normalized 16 bit. I don't remember anything special that deserved more as this. The only exception may be Knopfler's Golden Heart that indeed had an extreme low noisefloor when i remember right. Not that i heard a problem but this is where you may use noiseshaped dither when keeping it at 16 bit to be safe.
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

Re: HDCD Spotted - Post any rarities here

Reply #47 –
I would decode HDCD prior to encoding to lossy and keep a lossless copy that was still HDCD-encoded.

"Safe" is not really ever a concern of mine.  If I think I hear a problem then I ABX and refine my practices as necessary, if only on a case by case basis.
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

Re: HDCD Spotted - Post any rarities here

Reply #48 –
Then there is nothing to write home about. My HDCDs are as loud as other CDs and normal playback will never come close enough for the noisefloor of 16 bit to be a problem. I have no ultra silent classic in HDCD.
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

Re: HDCD Spotted - Post any rarities here

Reply #49 –
I would definitely choose a dynamic HDCD release over a crushed non-HDCD release unless the HDCD release was from a grossly inferior source or had some other serious problem. 
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?
 
SimplePortal 1.0.0 RC1 © 2008-2019