I spotted my first HDCD. Well, it came as a FLAC of a single track for a bug report, but I ended up buying it from the Discogs marketplace anyway, and it should be here within a week or less.
Pleymo - Doctor Tank's Medicine Cake: French nu metal, or something. Not too bad from what I've heard of this track. I figure I'll listen to it occasionally.
Anyway, I'm not sure if it applies to any of the other tracks on this pressing, but the second track, Tank Club, uses Peak Extension, and I got an apparent bug report that foo_hdcd triggered on it, which it did correctly, since the track does make use of HDCD signaling and production. PE was the only feature used, and it managed to completely negate what otherwise looked like brickwall mastering of the original track.
I'll wait until my CD arrives and post scan results for the entire thing.
Feel free to post any other stories of unexpected HDCD findings.
I could try to look up some, but if I may then reiterate a feature request: the "HDCD Scan Result" window, could there be a way to sort the output and drag the content to a playlist?
and it managed to completely negate what otherwise looked like brickwall mastering of the original track.
Sounds plausible even without any ABX log :-o
Here is a strange one: https://www.discogs.com/Clawfinger-Clawfinger/release/5533875
According to my logs, I did unfortunately convert it upon ripping using HDCD.exe (one of those mistakes of my youth). But that makes this even stranger.
- It verifies AccurateRip with a different offset (with 1 to 5 different offsets, actually!), indicating there are HDCD-decoded pressings around.
- The HDCD code is not gone, at least not from the beginning: the tracks show up as HDCDs. And if I enable halving volume, it will boost volume after ten seconds.
... edit: is it perchance so that foo_hdcd has some 10 seconds window before it turns off? And that the explanation for the AccurateRip behaviour is just the beginning of the track?
Cannibal Corpse - Gore Obsessed. It is very unexpected to find brutal death metal released in HDCD.
(http://oi68.tinypic.com/2lj569w.jpg)
It is very unexpected to find brutal death metal released in HDCD.
Need not be, as a particular piece of studio equipment would use HDCD by default. Three of my HDCDs:
http://www.metal-archives.com/albums/Behemoth/Demigod/53348
http://www.metal-archives.com/albums/Macabre/Murder_Metal/29033
http://www.metal-archives.com/albums/Rotten_Sound/Exit/60143
... in addition to less Cannibal Corpse-ish bands like Dimmu Borgir, I (= Abbath from Immortal), In Flames, Machine Head, Megadeth, Meshuggah, Opeth and Soilwork. Not sure how many HDCD features were used though, these were decoded. A mistake yes.
FWIW, my copies of Vile and The Wretched Spawn are not HDCDs.
I have two 'greatest hits' CDs, one by Johnny Cash, another by Thelonius Monk, where hdcd scan shows only one encoded track on the disc.
In the reverse case, Mike Oldfield's Ommadawn HDCD, the last track is not HDCD-encoded.
This one: https://www.discogs.com/Utah-Phillips-Ani-DiFranco-Fellow-Workers/release/391967
foo_hdcd claims track 1 is a HDCD (no features employed). hdcd.exe reports that it is not a HDCD.
Furthermore, hdcd.exe decoded tracks 4-6, 10 and 13-15 as hdcd.exe were decoded by hdcd.exe
This is not a compilation where the tracks are from different "sources" in that sense; it claims to be recorded "before a live in-studio audience". I can only guess tracks 2,3,7-9 and 11-12 required less ovedubs afterwards ;-)
I have two 'greatest hits' CDs, one by Johnny Cash, another by Thelonius Monk, where hdcd scan shows only one encoded track on the disc.
I have a few "compilations" and the like with mixed HDCDness. Glenn Hughes: "Different Stages" is a double CD with precisely one HDCD track on each physical disc.
If it is possible, could you scan with ffmpeg hdcd filter and post the output? It requires building ffmpeg from source, but it will give detailed stats about the HDCD encoding.
For example:
ffmpeg -nostats -v verbose -i Neil_Young/Neil_Young-Silver_&_Gold.flac -af hdcd -f s24le /dev/null 2>&1 | grep "_hdcd_"
Output would look something like this:
[Parsed_hdcd_0 @ 0x1d8eab0] Process mode: process stereo channels together
[Parsed_hdcd_0 @ 0x1d8eab0] Force PE: off
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: counter A: 0, B: 30040, C: 30040
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: pe: 30040, tf: 9226, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg 0.0: 28980
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -0.5: 83
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -1.0: 90
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -1.5: 77
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -2.0: 57
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -2.5: 57
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -3.0: 84
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -3.5: 58
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 0: tg -4.0: 554
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: counter A: 0, B: 30040, C: 30040
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: pe: 30040, tf: 9233, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg 0.0: 28978
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -0.5: 85
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -1.0: 91
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -1.5: 77
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -2.0: 56
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -2.5: 57
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -3.0: 84
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -3.5: 58
[Parsed_hdcd_0 @ 0x1d8eab0] Channel 1: tg -4.0: 554
[Parsed_hdcd_0 @ 0x1d8eab0] HDCD detected: yes, peak_extend: enabled permanently, max_gain_adj: -4.0 dB, transient_filter: detected, detectable errors: 0
If using linux:
% git clone git://source.ffmpeg.org/ffmpeg
% cd ffmpeg
% ./configure && make
And there is a multi-file scanning script (http://wiki.hydrogenaud.io/index.php?title=High_Definition_Compatible_Digital#Scanning_a_FLAC_archive_for_HDCD_using_FFmpeg) on the wiki.
After scanning, perhaps even add your results to List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs) :)
Thanks.
After scanning, perhaps even add your results to List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs)
What a nice idea, thank you very much for creating that list. Actually I started a similar one just for myself a few years ago but I haven't updated for a very long time (available here (https://docs.google.com/spreadsheets/d/1Qc0WNd4wgeiQIEDTZZLGMrCn2Sycg8mH4eLkmMI2UA4/edit#gid=2008743414) for anyone interested), so I'll gladly contribute.
Just as a suggestion regarding the list, I think it would be really useful to add to the table the label and maybe the catalog number of each release as well to identify which pressings are actually HDCD encoded from those that aren't - or even possible, to differentiate several HDCD releases of a same disc that came out with a different set of features enabled.
Regards
... so I'll gladly contribute.
Just as a suggestion regarding the list, I think it would be really useful to add to the table the label and maybe the catalog number of each release as well to identify which pressings are actually HDCD encoded from those that aren't - or even possible, to differentiate several HDCD releases of a same disc that came out with a different set of features enabled.
Cool, very similar thoughts. :). Added columns sound good, go ahead, but lets talk about any other changes at Talk:List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=Talk:List_of_HDCD-encoded_Compact_Discs) so that we don't take this thread off-topic, away from information about HDCD rarities.
This one: https://www.discogs.com/Utah-Phillips-Ani-DiFranco-Fellow-Workers/release/391967
foo_hdcd claims track 1 is a HDCD (no features employed). hdcd.exe reports that it is not a HDCD.
Furthermore, hdcd.exe decoded tracks 4-6, 10 and 13-15 as hdcd.exe were decoded by hdcd.exe
I've scanned the sample you sent, and it is not HDCD, it just happens to have a sequence that appears to be valid HDCD. Here is the ffmpeg output:
[Parsed_hdcd_0 @ 0x22725e0] Process mode: process stereo channels together
[Parsed_hdcd_0 @ 0x22725e0] Force PE: off
[Parsed_hdcd_0 @ 0x22725e0] Channel 0: counter A: 0, B: 1, C: 1
[Parsed_hdcd_0 @ 0x22725e0] Channel 0: pe: 0, tf: 0, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 1
[Parsed_hdcd_0 @ 0x22725e0] Channel 0: tg 0.0: 1
[Parsed_hdcd_0 @ 0x22725e0] Channel 1: counter A: 0, B: 0, C: 0
[Parsed_hdcd_0 @ 0x22725e0] Channel 1: pe: 0, tf: 0, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: -1
[Parsed_hdcd_0 @ 0x22725e0] Channel 1: tg 0.0: 0
[Parsed_hdcd_0 @ 0x22725e0] HDCD detected: noChannel 0 has one valid packet that signals PE off and adjust gain 0.0. Check out Talk:List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=Talk:List_of_HDCD-encoded_Compact_Discs) § False Positives, for a bit more about this. Also check out FFmpeg ticket #5727 (https://trac.ffmpeg.org/ticket/5727) for an example of a worst case false positive.
By false positive, you mean something more than those which just have a HDCD flag but no HDCD features?
Like, it the audio does at some point in the stream, coincide with something that the HDCD spec interprets as HDCD information?
In the example you sent, there is a completely valid HDCD packet in one channel, but the packet happens to signal PE:off, LLE:0.0, so it has no effect. In the example linked in issue #5727, the coincidental valid packet signals -6.0dB gain adjust in one channel, so one channel drops volume quite unexpectedly, and in a very noticeable way. Both of the examples are really not HDCD encoded, it was just a coincidence that the bits aligned in exactly the right way to form a valid packet, and may be detected as HDCD when they are not. This is not the same as an HDCD-encoded disc that signals PE:off, LLE:0.0 in every packet (uses no features).
Right. So that CD has some tracks that are detected as HDCD by hdcd.exe, some that are not, and one that is not despite containing one HDCD packet in one channel - I think I should dig up that CD sometime to find out what is really in those tracks I let hdcd.exe decode :-/
@kode54: considering https://hydrogenaud.io/index.php/topic,79427.msg900371.html#msg900371 , it may be that there are quite a few "false positive by coincidence" cases, and that you should change foo_hdcd's behaviour towards single-channel packets?
Does anyone know what is the maximum gain that LLE can adjust? AFAIK, usually HDCDs that use it come with a gain between -2.0 and -4.0dB, but this live version of Yngwie Malsmteen - Concerto Suite... (discogs (https://www.discogs.com/es/Yngwie-Johann-Malmsteen-Concerto-Suite-For-Electric-Guitar-And-Orchestra-In-E-Flat-Minor-Live-With-T/release/2920607)), reaches up to -7.5dB in a few tracks.
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: counter A: 0, B: 44601, C: 44601
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: pe: 0, tf: 0, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg 0.0: 43048
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -0.5: 238
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -1.0: 190
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -1.5: 157
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -2.0: 111
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -2.5: 111
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -3.0: 103
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -3.5: 114
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -4.0: 107
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -4.5: 101
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -5.0: 105
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -5.5: 85
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -6.0: 45
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -6.5: 34
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -7.0: 20
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -7.5: 32
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: counter A: 0, B: 44601, C: 44601
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: pe: 0, tf: 0, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg 0.0: 43048
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -0.5: 238
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -1.0: 190
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -1.5: 157
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -2.0: 111
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -2.5: 111
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -3.0: 103
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -3.5: 114
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -4.0: 107
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -4.5: 101
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -5.0: 105
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -5.5: 85
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -6.0: 45
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -6.5: 34
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -7.0: 20
[Parsed_hdcd_0 @ 0000000000647940] Channel 1: tg -7.5: 32
[Parsed_hdcd_0 @ 0000000000647940] HDCD detected: yes, peak_extend: never enabled, max_gain_adj: -7.5 dB, transient_filter: not detected, detectable errors: 0
Wow, that is a really interesting specimen. -7.5dB is the lowest value that can be stored in the control code. There was a chansen post around here somewhere with detailed information, and I thought he said that a value below -6.0 would never actually be seen, so this is interesting. I'm mobile right now but I'll definitely look into this more when I get home.
There appear to be two types of HDCD packets:
* one that stores a 3-bit gain adjust value, 8 possible values representing -7dB through 0dB in steps of 1dB
* a second that stores a 4-bit gain adjust value, but uses it as a 3.1-bit fixed-point number, 16 possible values used as -7.5dB through 0dB in steps of 0.5dB.
So the maximum gain adjust value that can be encoded is -7.5dB. An aside: I've never seen the first type in an actual HDCD-CD, ever, but someone put it in the code who knows something more than I do.
Regarding that CHansen post I mentioned, he quotes an excerpt of the PM A/D operators' manual (http://wiki.hydrogenaud.io/index.php?title=HDCD#Regarding_the_Low-level_Gain_Adjustment_feature) (which can be read at that link), then goes on to say:
If you see a level of greater than 4.0 of gain adjustment, there was an error made in the disc where the mastering engineer mistakenly used "Special Mode". This is not only wrong, but there is no consumer equipment that can even decode this level of gain adjustment! Only by playing this back through the PM Model One (44 and 48 kHz only) or the PM Model Two (added dual- and quad-rate sampling rates) could this file be properly decoded...
If what he is saying is correct, then that CD was a mistake by an unqualified mastering engineer, or maybe they've intentionally started using the full range of gain adjust. It seems CHansen is much more a hardware expert than software, and I think it might be an interesting thought, that now there is an open-source software implementation, perhaps the full range can be used when it could not in hardware alone. :) Maybe I'm being too optimistic or something.
Another note, a fix for the previously mentioned ffmpeg issue #5727 (https://trac.ffmpeg.org/ticket/5727) has been applied, details here (https://github.com/FFmpeg/FFmpeg/commit/d574e22659bd51cdf16723a204fef65a9e783f1d). Now stereo channels are processed together, and only gain values matching in both channels are actually used.
Quite interesting info, thank you. So it means that a small portion of this album cannot be properly decoded by HDCD-capable players. According to the ffmpeg log I posted, packets that trigger gains below -4.0 aren't that many in the full album:
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -4.5: 101
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -5.0: 105
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -5.5: 85
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -6.0: 45
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -6.5: 34
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -7.0: 20
[Parsed_hdcd_0 @ 0000000000647940] Channel 0: tg -7.5: 32
That makes 377 (out of 44601), although most tracks seems to contain some of them. I just do not know if the difference in gain would be applied long enough to make it noticeable.
Maybe I'm being too optimistic naive or something.
Fixed that. :)
It does look like the kind of inadvisable use of "Special Mode" that CHansen talked about.
There appear to be two types of HDCD packets:
* one that stores a 3-bit gain adjust value, 8 possible values representing -7dB through 0dB in steps of 1dB
I think I found an example of this. The HDCD Sampler (https://www.discogs.com/es/Various-HDCD-Sampler/release/3192807) disc from 1992 contains LLE packets which go up to -7.0 in -1dB steps accross all tracks.
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: counter A: 23471, B: 0, C: 23471
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: pe: 23471, tf: 9390, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg 0.0: 22490
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -0.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -1.0: 232
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -1.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -2.0: 165
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -2.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -3.0: 104
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -3.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -4.0: 54
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -4.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -5.0: 60
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -5.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -6.0: 80
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -6.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 0: tg -7.0: 286
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: counter A: 23487, B: 0, C: 23487
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: pe: 23487, tf: 9978, almost_A: 0, checkfail_B: 0, unmatched_C: 0, cdt_expired: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg 0.0: 22506
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -0.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -1.0: 233
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -1.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -2.0: 165
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -2.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -3.0: 104
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -3.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -4.0: 54
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -4.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -5.0: 60
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -5.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -6.0: 80
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -6.5: 0
[Parsed_hdcd_0 @ 0000000000f6dae0] Channel 1: tg -7.0: 285
[Parsed_hdcd_0 @ 0000000000f6dae0] HDCD detected: yes, peak_extend: enabled permanently, max_gain_adj: -7.0 dB, transient_filter: detected, detectable errors: 0
Related to this, is there a way to know when each gain is being adjusted somehow? Currently I have foobar configured to display the HDCD info in realtime on the statusbar, however LLE uses to be triggered so briefly that is quite hard to tell when -7.0 or -6.0dB are attenuated just by looking at it.
I think I found an example of this. The HDCD Sampler (https://www.discogs.com/es/Various-HDCD-Sampler/release/3192807) disc from 1992 contains LLE packets which go up to -7.0 in -1dB steps accross all tracks.
Neat, I've never seen one that used the 'A' format packet legitimately. Counter A is the verification there. I've written a bit about the two packet formats at HDCD § Packet formats (http://wiki.hydrogenaud.io/index.php?title=High_Definition_Compatible_Digital#Packet_formats).
It is interesting that this disc is from 1992, and all the discs that I've scanned are from years later and only have B format packets. I wonder when the 16-bit format with the 4-bit target gain and XOR check was introduced.
Related to this, is there a way to know when each gain is being adjusted somehow?
I create an "analyze mode" in the ffmpeg filter that outputs a solid tone with amplitude signaling where a certain feature was used.
Then the output can be loaded along side the original in an audio editor where it is possible to see what the features were turned on at any point in the track. In the Audacity screenshot attached, the top track is the original audio, the middle track is where LLE is used, and the bottom track is where PE was used.
In this case, it is Neil Young - Great Divide (because it uses all features). You can see that the LLE is only during the fade-in.
Looks pretty good! Although I assume it isn't merged on github yet, right?
Actually I believe an analyzer like this could be pretty useful to find spots on discs which could be interesting to perform ABX tests with. Strictly speaking about LLE decoding I am not aware on any blind test focused on it and these two discs with 'extreme' gains may come in handy as starting point.
Looks pretty good! Although I assume it isn't merged on github yet, right?
Not yet, but I can send you (or anyone interested) a patch to apply manually, if you like.
The HDCD component for foobar2000 has been updated :D
Detected HDCD in my lossless archive, using recent FFmpeg (https://hydrogenaud.io/index.php?topic=112225):
- Behemoth Demigod
- Behemoth Slaves Shall Serve
- Cannibal Corpse Gore Obsessed
- In Flames Trigger
- In Flames Soundtrack to Your Escape
- John Williams Star Wars Episode III - Revenge of the Sith OST
- Machine Head The More Things Change
- Machine Head The Blackening
- Melvin Taylor Rendezvous With the Blues
- Meshuggah Catch Thirtythree
- Nine Inch Nails The Fragile
- Opeth Ghost Reveries
- Sebastian Bach Bring 'Em Bach Alive!
- Sonata Arctica Unia
- Within Temptation The Silent Force
Detected HDCD in my lossless archive, using recent FFmpeg:
...
Very cool. If you still have the output from FFmpeg could you add rows for the missing items at List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs)? If you don't want to edit the wiki, you could post the ffmpeg log on the talk page or here and I will add the items.
I'm not sure how to interpret the detailed FFmpeg ouput, so here is the raw log : http://pastebin.com/raw/UUt7wcby (http://pastebin.com/raw/UUt7wcby).
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.
"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"
I'm not sure how to interpret the detailed FFmpeg ouput, so here is the raw log : http://pastebin.com/raw/UUt7wcby (http://pastebin.com/raw/UUt7wcby).
The main lines I use to check are the last one for the HDCD features enabled and the lines with A, B and C counters to confirm it has a high number of valid packets, otherwise the cd could be a false positive. However, I just noticed two strange things on your logs:
[Parsed_hdcd_0 @ 0x391e300]
The HDCD filter is unlikely to produce a desirable result in this context.[Parsed_hdcd_0 @ 0x391e300] HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0
(bad_config)It is the first time I see them, so I am not sure why those warnings appeared.
I'm not sure how to interpret the detailed FFmpeg ouput, so here is the raw log : http://pastebin.com/raw/UUt7wcby (http://pastebin.com/raw/UUt7wcby).
I've added them to the table. Mostly they are HDCD-encoded discs that do not use any HDCD features. The Star Wars Soundtrack is a very strange case where there are a handful of completely valid HDCD packets in each track, but it is not fully HDCD, which is interesting, I will look into that. Thanks for the data.
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.
"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"
Indeed. There are many such discs. That is one of the reasons for making this detailed list. You can see which discs will light up the HDCD indicator without actually using any HDCD features.
[Parsed_hdcd_0 @ 0x391e300] The HDCD filter is unlikely to produce a desirable result in this context.
It is the first time I see them, so I am not sure why those warnings appeared.
This is a bug. It is supposed to warn you if there is some resampling or truncation in the stream which might destroy the HDCD code before the filter gets it (or undo the work after), but the warning is false in this case. s16-planar is being converted to s16, which is fine.
I've added them to the table. Mostly they are HDCD-encoded discs that do not use any HDCD features. The Star Wars Soundtrack is a very strange case where there are a handful of completely valid HDCD packets in each track, but it is not fully HDCD, which is interesting, I will look into that. Thanks for the data.
You're welcome. If you update the FFMpeg filter and want me to scan again, let me know, I have fully automated the process.
[Parsed_hdcd_0 @ 0x391e300] The HDCD filter is unlikely to produce a desirable result in this context.
It is the first time I see them, so I am not sure why those warnings appeared.
This is a bug. It is supposed to warn you if there is some resampling or truncation in the stream which might destroy the HDCD code before the filter gets it (or undo the work after), but the warning is false in this case. s16-planar is being converted to s16, which is fine.
If that helps I'm using the following command line:
ffmpeg -hide_banner -nostats -loglevel debug -threads 1 -y -i FILEPATH -af hdcd -f s24le /dev/null
And my custom built FFmpeg is:
ffmpeg version N-81284-g82e53b3 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
configuration: --prefix=/tmp/ffmpeg.T1kOLSLRjF/FFmpeg --pkg-config-flags=--static --extra-cflags=-I/tmp/ffmpeg.T1kOLSLRjF/FFmpeg/include --extra-ldflags=-L/tmp/ffmpeg.T1kOLSLRjF/FFmpeg/lib --bindir=/opt/ffmpeg/bin --enable-gpl --enable-version3 --enable-nonfree --disable-runtime-cpudetect --disable-ffserver --disable-encoder=vorbis --disable-encoder=aac --enable-x11grab --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --disable-debug --cpu=corei7
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 51.100 / 57. 51.100
libavformat 57. 46.100 / 57. 46.100
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 49.100 / 6. 49.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Well, the bug is only that the warning is issued when it need not be. I only tested with FLAC, and didn't think of WavePack using a planar sample format. Nothing will change about the decoding behavior. The patch that fixes this turned out to be unexpectedly controversial (https://patchwork.ffmpeg.org/patch/116/), so I don't know what will happen.
Very happy to report that two HDCDs that used to have a problem with foo_hdcd, "Pleasant Smell" CD Single by 12 Rounds and "For The Masses - A Tribute to Depeche Mode" by various artists, are now being decoded properly with the latest 1.17 update ("Reinstated support for arbitrary channel counts"). The problem was that the left channel was encoded louder than the right. It's all good now. And both support peak extension.
I found a short, but very useful list of which HDCDs actually have features like peak extension, and which are "fake HDCDs": http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs
"For the Masses" actually has -4.0dB gain adjust encoded in one channel on the disc. It seems like a massive error. Anyway, ffmpeg:master and the newest foo_hdcd will ignore the mismatched gain values, but still use PE, as described in this issue (https://trac.ffmpeg.org/ticket/5727).
For those interested, "analyze mode" is now in ffmpeg:master (commit (http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=b2b659b17df235d49f2d164a1a8284dae37a2133)), and is described at Analyze Mode (http://wiki.hydrogenaud.io/index.php?title=High_Definition_Compatible_Digital#Analyze_mode).
Attached is a screenshot of Audacity showing the analyze mode ouput for "For the Masses". First track: decoded HDCD, Second track: lle (process_stereo=false); third track: pe. You can see there is one track without the problem.
I've checked out my copy of Opeth - Ghost Reveries, and while it does contain the HDCD flag, there seems to be absolutely no difference between the straight redbook rip and the HDCD decoded version, apart from the expected 6dB amplitude difference. Amplified to the same level, the files have completely identical DR ratings and replaygain adjustments.
"HDCD detected: yes, peak_extend: never enabled, max_gain_adj: 0.0 dB, transient_filter: not detected, detectable errors: 0"
I do not remember the source off the top of my head, but I read that one of the Pacific Microsonics A/D converters included HDCD capability and could mistakenly trigger the flag on everything passed through it, regardless of whether it leveraged the peak extend and other features. I'll try and hunt that down.
I was reviewing the wiki articles on HDCD again and saw the term "reversible." Is there any information out there indicating conclusively that this is the equivalent to "mathematically lossless" or is this just one more reason to be annoyed with this needlessly gimmicky format?
Also, is this system tied in any way to earlier generation ADCs that may exhibit audible aliasing due to far less than ideal filter design?
PS: I like that someone used the term scam. :)
"reversible."
(Wikipedia or HA wiki?)
"reversible." Is there any information out there indicating conclusively that this is the equivalent to "mathematically lossless"
I sense a "42" here: I am not sure if I understand the question. Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?
An example: What will HDCD do to white noise at full volume?
PS: I like that someone used the term scam. :)
Funny. But should not be in the wiki I think.
I didn't think about HDCD for a while but when i remember right some decoded ones are simply a bit shift that equals -6dB. Shifting the decoded file one bit louder gives you the lossless original back.
(Wikipedia or HA wiki?)
Either/both. I'd hope my post would have motivated people to check. The HA wiki article (http://wiki.hydrogenaud.io/index.php?title=High_Definition_Compatible_Digital) points to the Wikipedia article (https://en.wikipedia.org/wiki/High_Definition_Compatible_Digital). There shouldn't be an accessibility issue.
Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?
Yes, but why only 16, when it is 20-bits that's being touted? I am not talking about the non-decoded signal, so there is no reason to make any exceptions for control data that no longer exists in the decoded data stream.
An example: What will HDCD do to white noise at full volume?
Sure, but instead of "what will HDCD do," let's be specific. What will the HDCD encoding and decoding process do? Take any commercial release that actually makes use of the entire bag of tricks (including peak extension). We've already seen plenty or releases that are crushed without decoding. When they are encoded how close are they to the original signal prior to encoding?
Funny. But should not be in the wiki I think.
Perhaps, perhaps not. The skepticism is very much valid and casual readers deserve to know that it exists.
I'm still curious about my other question regarding hardware and aliasing, as well.
I didn't think about HDCD for a while but when i remember right some decoded ones are simply a bit shift that equals -6dB. Shifting the decoded file one bit louder gives you the lossless original back.
Yes, anything mastered on the PM A/D converter would insert HDCD packets even when all the features were off. So even if every packet contained a control code that signaled {PE:off, LLE:0.0dB}. In this case, the HDCD decoder has shifted the audio 1-bit to make room for peak extend that is never used, and if you shift it back, you'll get the original. Discs like this have a yellow background in the HDCD detected column on List of HDCD-encoded Compact Discs (http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs).
"reversible."
(Wikipedia or HA wiki?)
In Wikipedia, and what it is referring to is the "soft-limiting" or scaling of the top -9dB of the 20-bit PCM into the top -3dB of the 16-bit PCM. The HDCD decoder will then "reverse" the soft-limiting. I don't think it is perfectly reversible, and I've seen some comments that the curve used in the original reverse-engineered software decoder is "approximate" based on trying to match a curve printed in the Keith Johnson paper. The open source decoder is based on that work, so it is not likely perfect either, but only good enough.
PS: I like that someone used the term scam. :)
Funny. But should not be in the wiki I think.
It really was though. Even if it was a good idea to begin with, the execution was so poor in so many ways. HDCD's existence took more than it gave by a huge margin. But there is so much music out there from the last 20 years that have been touched by this...
In this case, the HDCD decoder has shifted the audio 1-bit to make room for peak extend that is never used, and if you shift it back, you'll get the original.
Minus whatever data was discarded in the LSB in order to make room for the control data, not that the data in the LSB is likely to have any audible impact under all but extreme listening conditions and only for a minority or releases.
In Wikipedia, and what it is referring to is the "soft-limiting" or scaling of the top -9dB of the 20-bit PCM into the top -3dB of the 16-bit PCM. The HDCD decoder will then "reverse" the soft-limiting. I don't think it is perfectly reversible, and I've seen some comments that the curve used in the original reverse-engineered software decoder is "approximate" based on trying to match a curve printed in the Keith Johnson paper. The open source decoder is based on that work, so it is not likely perfect either, but only good enough.
And by that token, 16-bits without the gimmicks is also good enough. ;) But thanks for answering. :)
Do you mean that the decoded signal equals the encoded signal up to a 16 bit resolution except for the HDCD flags themselves?
Yes, but why only 16, when it is 20-bits that's being touted? I am not talking about the non-decoded signal, so there is no reason to make any exceptions for control data that no longer exists in the decoded data stream.
Concerning what the implementations deliver, bp0 gave the answer in that Christopher Key's hdcd.exe was reverse-engineered (from https://hydrogenaud.io/index.php/topic,30999.25.html and the links to doom9 I take it that he checked WMP, not only the curve in the HDCD paper) - without the chip you cannot expect it as bit-exact. And foo_hdcd does not (did not?) perfectly reproduce hdcd_exe either: https://hydrogenaud.io/index.php/topic,79427.400.html
So if the ffmpeg code emulates (?) foo_hdcd which not-completely-precisely emulates hdcd.exe which is a reverse engineering in the analogue domain, you have more than one lossy step. In practice.
But if the question is whether it could be lossless (if you knew the process exactly): well if you apply a strictly increasing transformation to a 24-bit signal before you convert to 16 bits, you could end up with roundoff errors - but you will also in practice apply dither in the process: a 16-bit signal is not bit-exact to the 16 MSBs of the 24 in any case. Scrutinizing whether it is "lossier than ordinary dithering" is surely beyond me (the Johnson/Pflaumer AES preprint claims the HDCD code is encrypted to appear as noise i.e. as part of the dither, so apart from the magic numbers Christopher Key mentions he used to reverse engineer, it need not any loss).
Which brings me to those
"20" bits: there is a "noise shaping" part of this (details unknown to me; forum member SebastianG disputes the use of term "noise shaping", see the thread). It isn't lossless - and it isn't unique to HDCD, so one could very well claim that the "resolution" gained this way is part of the HDCD "format", only what you get from using Pacific's HDCD gear or anything else that does noise shaping equally well or better.
And by the way, the AES preprint writes "approximately 19 bit precision", and I have just conjectured that "20" is because anything above 19 will require a 20th bit if one were to store in a fixed-point PCM format. (By that argument, mp3 would be a great format, wouldn't it?)
I've read 20 bits long before your post so I wasn't attributing it to you. Thanks for the summary, though.
I'm basically wondering if any of these HDCD releases would be preferable to non-HDCD masters or should one not bother to seek them out. I know I would avoid HDCD releases of the Van Halen catalog, for example (though only because they are squashed and do not use peak extension).
I'm basically wondering if any of these HDCD releases would be preferable to non-HDCD masters or should one not bother to seek them out.
Is there anything released both as HDCD and as non-HDCD, for the same "mastering"?
I don't know. It doesn't really* matter to me either way.
If there are any particularly outstanding HDCD releases, especially in relation to non-HDCD alternatives (when they exist) then I'd be interested in hearing them.
Even though opinions about versions are subjective, reading about them here would elevate my level of confidence far above anything I might read from other communities which have no sanity controls.
(*) Of course if they are from the same dynamic master and there is a 16-bit non-HDCD version that makes proper use of the dynamic range available, then sure, I'd pick the 16-bit version over HDCD if I can't be assured proper HDCD decoding within the error afforded by 16-bit with allowances provided for processes that add noise. Ideally there should be little if any difference between the two assuming proper HDCD decoding, correct? (meant as a rhetorical question and absolutely subject to TOS #8).
I have some HDCDs and dithered all of the ones with peak extension back to normalized 16 bit. I don't remember anything special that deserved more as this. The only exception may be Knopfler's Golden Heart that indeed had an extreme low noisefloor when i remember right. Not that i heard a problem but this is where you may use noiseshaped dither when keeping it at 16 bit to be safe.
I would decode HDCD prior to encoding to lossy and keep a lossless copy that was still HDCD-encoded.
"Safe" is not really ever a concern of mine. If I think I hear a problem then I ABX and refine my practices as necessary, if only on a case by case basis.
Then there is nothing to write home about. My HDCDs are as loud as other CDs and normal playback will never come close enough for the noisefloor of 16 bit to be a problem. I have no ultra silent classic in HDCD.
I would definitely choose a dynamic HDCD release over a crushed non-HDCD release unless the HDCD release was from a grossly inferior source or had some other serious problem.
Is there anything released both as HDCD and as non-HDCD, for the same "mastering"?
I even forget that I posted a possible answer to that question myself. In this thread.
https://hydrogenaud.io/index.php/topic,112348.msg925894.html#msg925894
The ten seconds I mention there, is what a HDCD decoder should take to reset. Which is potentially a big issue; what happens in the following situation?
- A CD is compiled with HDCD tracks and non-HDCD tracks
- The "master" file is written to CD, with an arbitrary offset, slightly moving the track boundaries, so that the HDCD flags from track N find their way to the very first samples of track N+1
- and so the CD is pressed.
The ten seconds I mention there, is what a HDCD decoder should take to reset. Which is potentially a big issue; what happens in the following situation?
- A CD is compiled with HDCD tracks and non-HDCD tracks
- The "master" file is written to CD, with an arbitrary offset, slightly moving the track boundaries, so that the HDCD flags from track N find their way to the very first samples of track N+1
- and so the CD is pressed.
There is supposed to be 200ms of "HDCD silence" at the beginning of each track, where the first packet is expected to appear. This is to give the decoder time to detect a packet and adjust levels if one is found.
I've seen that packets usually appear less than one second apart. The sustain timer in foo_bar uses 10 seconds, so if a non-HDCD encoded track is inserted, the last packet will continue to be used for the first 10 seconds of the track, but this is foo_hdcd specific. As the FFmpeg decoder is based on foo_hdcd, it inherited the 10 second timer, but I think it was chosen arbitrarily (but maybe kod54 will say otherwise). I've got an FFmpeg patch that allows it to be set as an option, and to lowers the default to 2000ms, but the patch hasn't been accepted/applied in master, yet. I'm still playing with it.
There is supposed to be 200ms of "HDCD silence" at the beginning of each track, where the first packet is expected to appear. This is to give the decoder time to detect a packet and adjust levels if one is found.
Beginning of each "track", are you sure? If the entire disc is HDCD, it would make much more sense to inherit the state from the previous track. (The decoder will reset in the time taken to switch a physical CD in a single physical player.) What if the beginning of track 2 is at volume above -6 dB?
I've seen that packets usually appear less than one second apart.
The Johnson/Pflaumer AES preprint mentions "in both channels within a one second interval". Makes sense.
As the FFmpeg decoder is based on foo_hdcd, it inherited the 10 second timer, but I think it was chosen arbitrarily (but maybe kod54 will say otherwise).
I just presumed that it was from hdcd.exe, which scans only the first 750 frames = 10 seconds to identify a HDCD (with the -i option). But that choice may in turn have been arbitrary.
Beginning of each "track", are you sure?
No. :-\ I may have misunderstood something I read.
Is there anything released both as HDCD and as non-HDCD, for the same "mastering"?
Just got curious, and checked from this list (http://wiki.hydrogenaud.io/index.php?title=List_of_HDCD-encoded_Compact_Discs).
Yngwie Malmsteen: Concerto for big ego and orchestra has two entries in the list, but mine is not HDCD.
Consistent with this. https://www.discogs.com/master/view/236975
My version is before the 2013 reissue, and I don't think the 1998-2000 versions were different mixes. So ... apparently a yes.
I even forget that I posted a possible answer to that question myself. In this thread.
https://hydrogenaud.io/index.php/topic,112348.msg925894.html#msg925894
Well maybe one more.