Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: HDCD Decoder (Read 327213 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: HDCD Decoder

Reply #425
It's about ten seconds into the tracks when the HDCD flag turns off.  All of the voluming halving options either cause the same volume disparity or risk clipping.  Keep in mind that the volume halving setting should be "set it and forget it;" in other words I shouldn't have to change it per-album.  I'm using FLAC.  I haven't done any processing of the files, and it seems pretty unlikely that processing has occurred upstream that would only affect the tracks after ten seconds in.

Assuming this plugin's HDCD detection is correct, it seems clear that there are mastering errors.  Given this assumption, this is still a real-life album, so the plugin should handle this failure mode if possible, correct?  I'll assume you agree.

I don't think "unfortunate track boundary" is the case here, because some of the tracks do not enable the HDCD flag and come before some of the problem tracks.  Since the tracks that do enable the HDCD flag do so from the beginning of the track, why shouldn't this plugin maintain HDCD processing for the rest of the track (in other words, maintain the volume halving until the track is complete).  We're assuming that this album is incorrectly mastered, so given that, how would this proposed fix break correctly mastered albums?  If it doesn't break the behavior in the correct HDCD case, I think this error condition should be handled as proposed only because this is a real album.

Otherwise I'd say, "Too bad, screw it.  Manually zero the 16-bit."  Or something.

Re: HDCD Decoder

Reply #426
Edit: This thread is relevant: https://hydrogenaud.io/index.php/topic,112348.0.html
And suggests that the ten seconds I refer to below (I do not edit it out) could be arbitrary and might not correspond to a real-world DAC.

It's about ten seconds into the tracks when the HDCD flag turns off. 
kode54 could weigh in on whether my suggestion is technically plausible, but I'd put my $.02 on the track boundary.

IIRC, HDCD is supposed to turn off after ten seconds with no HDCD packet. I can only assume the rationale is that a (physical) HDCD was supposed to be HDCD on all tracks, and ten seconds would give the DAC time to reset when switching to a new CD.

My hypothesis would then be that this CD has a HDCD-enabled track number N followed by a non-HDCD track number N+1, and in the mastering, the track boundary has been set a bit too far "to the left" so that a few HDCD packets from track N has ended up in the file which is supposed to be track N+1. You start track N+1, but due to the mastering quirks you do get a split second of what was supposed to be the end of track N - and there are the HDCD flags.

That said, I have encountered a similar issue when playing back the file which contained an already decoded HDCD signal. Seems to me that some HDCD packets did survive the conversion.  (Oh, and: you do not have two HDCD components installed?)

Assuming this plugin's HDCD detection is correct, it seems clear that there are mastering errors.  Given this assumption, this is still a real-life album, so the plugin should handle this failure mode if possible, correct?  I'll assume you agree.

That depends on what you mean by "correct". Do you mean it should handle it like a physical CD player and DAC (with HDCD support) would do (which I think it actually does, see below)? In that case, you have probably misunderstood something about how ("physical") HDCD works:

Since the tracks that do enable the HDCD flag do so from the beginning of the track, why shouldn't this plugin maintain HDCD processing for the rest of the track

Because a physical DAC will not maintain HDCD status "for the rest of the track"! A physical DAC does not know track boundaries, and cannot know what is "the rest of the track". That is why HDCD processing turns off a fixed time after the last HDCD packet.

Likely, a physical DAC would also "mess up" here. My conjecture is that the final master was not reviewed on a HDCD DAC, and so no-one understood it could be an issue.

Re: HDCD Decoder

Reply #427
I understand the incorrect track boundary theory, it's just that some of the HDCD-beginning-flagged tracks are preceded by completely non-HDCD ones, which seems to remove some correlation.  Isn't this the opposite of the N+1 example you describe?  Ah, but doesn't this component stop analyzing after some arbitrary amount of seconds into a track if no HDCD packets are found?

"Correct" means as close to the specification and reverse engineered information as possible, and now that we're out in the wild, accounts as much as possible for discs that incorrectly follow the specification without compromising the performance of correctly mastered discs. :D

Because a physical DAC will not maintain HDCD status "for the rest of the track"!

Therein lies our advantage.  This software has more information. :)

A physical DAC does not know track boundaries, and cannot know what is "the rest of the track". That is why HDCD processing turns off a fixed time after the last HDCD packet.

So there was never CD player + HDCD DAC combo hardware?

Here's a better proposed solution to this incorrect disc because it doesn't involve volume halving:  This component could disable itself when it reads a per-track "disable HDCD processing" tag.  Kind of like how the De-emphasis plugin works.

HDCD: Yet another example of stupid (that's redundant) over-engineering.  This is fun though, ain't it? :P

Re: HDCD Decoder

Reply #428
Here's a better proposed solution to this incorrect disc because it doesn't involve volume halving:  This component could disable itself when it reads a per-track "disable HDCD processing" tag. 

Agreed. kode54, are you there? ;)

Suggestion:
-> scan track for a HDCD tag.  If <HDCD> is "no", then do not process this track.

I would also suggest to let the component populate tags with, e.g., "PE" or "LLC" after scanning (excluding those with the "no"). In order to review later.

Also, is the ten-seconds reset what it should be?

Re: HDCD Decoder

Reply #429
A (sort of) work around is to set Advanced ->Decoding ->HDCD ->Halve output volume to Never.
In theory, there is no difference between theory and practice. In practice there is.

Re: HDCD Decoder

Reply #430
But that risks clipping.  No?  If no, then what's the point of the option in the first place.

Re: HDCD Decoder

Reply #431
But that risks clipping.  No?  If no, then what's the point of the option in the first place.

Foobar2000 can limit level according to %replaygain_track_peak% or %replaygain_album_peak% in order to prevent clipping.

Why that option? Not sure, but maybe to emulate actual HDCD behaviour?

Re: HDCD Decoder

Reply #432
That depends on whether this component clips samples above digital full scale.  The %__bitspersample% value of "24" suggests that the component outputs integer samples, which means values above 0 dBFS would be clipped.

Ring around the rosie.

Re: HDCD Decoder

Reply #433
OK, I spent some time on a problem child in my collection. Note, all tracks verify AccurateRip.

Per-track scans:
hdcd.exe reports no HDCD in no tracks.
foo_hdcd reports PE enabled at the beginning, but 0 dB peak in all tracks but numbers 5 and 12, which have no HDCD.
All these ten "lose" their HDCD status after ten seconds, according to fb2k with foo_hdcd.

Converting to image
Converted to WavPack image with embedded cuesheet using CUETools, no HDCD decoding.
Rescanned for HDCD using foo_hdcd. Tracks 2 through 8 report as HDCD. Note that this includes track 5, which was not HDCD if scanned standalone.
Same procedure, but to TAK. Now, tracks 1 through 10 report as HDCD.

Transcoding between lossless formats (using a fb2k without foo_hdcd)
Lossless is lossless if I use a fb2k without foo_hdcd. However, if I bit-compare these files using foobar2000 with foo_hdcd, they differ in some thousand samples.  That is: foo_hdcd treats the same signal (slightly) different according to codec.
Also, comparing image vs. single tracks of same codec will yield differences (naturally, as foo_hdcd converts some).

Attempted to get rid of HDCD by converting to lossywav.
Only track 2 tested. Complete failure. foo_hdcd identified as HDCD at absolutely every quality setting.

Converted with dithering
Converted to ALAC-in-mp4 using refalac, 16 bit to 16 bit with dithering. Success. No HDCD reported.


Re: HDCD Decoder

Reply #434
Okay, so false detection. Do you propose I scan entire tracks before allowing HDCD processing to be enabled?

Re: HDCD Decoder

Reply #435
Here's a better proposed solution to this incorrect disc because it doesn't involve volume halving:  This component could disable itself when it reads a per-track "disable HDCD processing" tag.  Kind of like how the De-emphasis plugin works.

Re: HDCD Decoder

Reply #436
Done. Sorry for taking so long.

Re: HDCD Decoder

Reply #437
Thank you!  No problem, you have no obligation to fulfill requests.  Doesn't mean I won't bug you, though. :)

Re: HDCD Decoder

Reply #438
Well it could be - for all that I know - that there was an outright bug that caused foo_hdcd to treat different filetypes/codecs different. At least to a complete amateur it doesn't seem very "lossless is lossless".
(Also, the ten-seconds countdown is still arbitrary?)

Thank you!  No problem, you have no obligation to fulfill requests.  Doesn't mean I won't bug you, though. :)

Seconed. All three parts.

Re: HDCD Decoder

Reply #439
The "HDCD" "no" tag doesn't work in an external m-TAGS file: http://www.m-tags.org/

It works if applied directly to the FLAC file.

Re: HDCD Decoder

Reply #440
Feel free to submit a pull request to add inter-component dependency so it can specifically ask the m-TAGS component whether its "hdcd" tag is set to "no", without using title formatting scripts. I have this:

Code: [Select]
class hdcd_postprocessor_entry : public decode_postprocessor_entry
{
public:
virtual bool instantiate( const file_info & info, decode_postprocessor_instance::ptr & out )
{
        if ( info.info_get_decoded_bps() != 16)
{
            return false;
        }
const char * encoding = info.info_get( "encoding" );
if ( !encoding || pfc::stricmp_ascii( encoding, "lossless" ) )
{
return false;
}
const char * hdcd = info.meta_get( "hdcd", 0 );
if ( hdcd && pfc::stricmp_ascii( hdcd, "no" ) == 0 )
{
return false;
}
out = new service_impl_t< hdcd_postprocessor_instance >;
return true;
}
};

Re: HDCD Decoder

Reply #441
Aw, no API?  Darn.  (You wouldn't really want a dependency on another component, would you?)

Re: HDCD Decoder

Reply #442
It would just need to attempt to instantiate a service implemented by the m-TAGS component for retrieving from it. Then it would attempt to query "hdcd" = "no". No service, no failure.

Re: HDCD Decoder

Reply #443
0116]
A (sort of) work around is to set Advanced ->Decoding ->HDCD ->Halve output volume to Never.
But that risks clipping.  No?  If no, then what's the point of the option in the first place.[/quote]
In theory, but in practice the risk is small because replaygain (and/or volume) mitigates this.
Also, as Porcus mentioned, one could use the "prevent clipping setting" of replaygain.
Finally, if you put Advanced Limiter at the end of the DSP chain, the occasional clipping (if any) will be prevented.

That depends on whether this component clips samples above digital full scale.
Foobar2000 is designed so the whole decode+DSP+volume is done in 32 float. Only at the output stage the stream is converted.

Anyway, I think the HDCD format is flawed by design and this is for me the best compromise to deal with it. Sorry I didn't check back on this sooner.
In theory, there is no difference between theory and practice. In practice there is.

Re: HDCD Decoder

Reply #444
When converting a HDCD track to a lossy format (vorbis, opus, fhg aac) with the setting Halve output volume = Never, the resulting overs are clipped. If I add a DSP that does nothing (convert mono to stereo, or foo_dsp_amplify by 0) the overs are correctly encoded. Shouldn't the output of HDCD be treated as processed by DSP by default and passed in floating-point to lossy encoders?

I would typically normalize, but suppose one forgets to do it. (Halve = Never is the correct setting to prevent large changes in loudness and inaccurate ReplayGain from being written.)

Re: HDCD Decoder

Reply #445
This is a converter issue, not one with my component.

Re: HDCD Decoder

Reply #446
Issue. (Cannot anymore remember what specs say about it.)

Tool's infamous Lateralus. Has a -4 dB gain in the fade-in at the beginning of track 1, and - AFAIK - no HDCD features anywhere else. CUETools detects it. ffmpeg detects it.

foo_hdcd (version 1.19 on fb2k1.4beta15) does not detect it because it is at the very beginning. Here is what I did:
- checked, converted to .wav an checked again.
- concatenated tracks 2+1 in that order; hacked the cuesheet to move the track boundary .4 seconds. That means, I got a copy of track 2 with the first .4 seconds of track 1 included at the end.
Now foo_hdcd detects it.


Re: HDCD Decoder

Reply #447
It only detects two consecutive HDCD frames within the first ten seconds of a given track. Otherwise, it disables itself and flushes all buffered audio.

Re: HDCD Decoder

Reply #448
It only detects two consecutive HDCD frames within the first ten seconds of a given track. Otherwise, it disables itself and flushes all buffered audio.

It identifies as HDCD everywhere, but no features except in the first second of track 1. Because it is at the beginning, foo_hdcd does not detect the gain.

Re: HDCD Decoder

Reply #449
And this is really difficult to fix, since it would require a full CD image file, and a scanner that can be told to check the beginning of the image file, before it seeks to the specified offset for the CUE sheet.