Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Title formatting/syntax for proper bitdepth (Read 19426 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Title formatting/syntax for proper bitdepth

Reply #100 –
Quote from: Porcus on
Hm, 6 dB low-level gain isn't supposed to be in CDs, so that must be "special mode" only?
Anyway, that figure is 0.100000000100111 in binary, so it gives an error 9 bits down. And when does the 6 dB gain kick in? This could get the 20th bit wrong or something like that?
Like I mentioned earlier, my take on the manual was not that special mode should not be used. And it is used in the HDCD sampler disc I have, goes to -7.0 dB. Also that number was just an example as -6 dB is generally considered halving, perfect 0.5 halving is -6.02 dB. All the gain adjustment values are inexact and as there's interpolation between the steps, they result in all possible sample values and will use any number of bits output has to offer.

I uploaded quick updates for both HDCD decoder and True Peak Scanner preview. The previous TP scanner preview build had a bug in there, if you scanned LRA values without album mode it still attempted to access album results and crashed.
HDCD decoder now allows two seconds to pass between HDCD indicator packets before giving up. I tried 0.5 seconds and 1 second, but these were too short for one test track I had. At the moment the component will print a message "HDCD signal disappeared" if it deemed a track to be HDCD but then the packets disappear. I would be interested in such tracks so the detection can be fine tuned, either to ignore fakes or adjust expectations to reality.

Also I compared old foo_hdcd results with hdcd.exe and it was actually further off than it is now. Event though very old 1.11 build actually outputted 20 bit signal like hdcd.exe.

Re: Title formatting/syntax for proper bitdepth

Reply #101 –
For HDCD "testing purposes", I guess  a "testing build" could be a good thing? One that could scan through the entire files, and report the following:

* Where the packets start and end. I don't mean every packet, but time-intervals where HDCD appear to be switched on - also if HDCD is off for some interval, report how close to track-end HDCD starts again. That could help identify identify those cases which due to index 0 appended, have packets "intended for the next track". Who knows whether that "next track" even ended up on the CD, especially if it is a compilation, but if there is HDCD at end of "file track N-1 and continuing into file track N" - i.e. "in CD track N index 0 and continuing into index 1" - then it stands to reason it was intended to be applied from first sample. But, if it then switches off quite soon?
Seems you may want some scan "as albums" functionality?

* Identify other potential false positives, like "single-channel" packets. They are likely to be ignored; and, if evidence says those do not come with any HDCD features, then they should certainly not trigger any "halve volume 'Always'"?

* Identify events where "the only HDCD feature to apply" is low-level gain in digital near-silence. If there is only the "HDCD dither" to mute, then what? Sure sure, technically it matters ... but ...?


(BTW, if you want (more) users to find it, consider taking it over to a different thread.)

Re: Title formatting/syntax for proper bitdepth

Reply #102 –
Case, I checked only RG Scanning and BPS scanning. In context menu I call "Scan RG + BPS (as albums)" - On a single track album, no RG values are written. I have set it to use the traditional ReplayGain tag fields.
I found this: If I check Truepeak scanning additionally, and then perform "Scan Truepeak + RG + BPS (as albums)" -> RG values are written.

Re: Title formatting/syntax for proper bitdepth

Reply #103 –
As more versions of both components arrive maybe it would be good to give them numbers or dates to be able to differentiate them. And it is getting harder to find the links to downloads in earlier posts (especially for potential new testers).
As for newest versions - I did not find any difference in TBS.
HDCD:
- one of my "problematic albums - the one that in Kode's versions was partially HDCD (only a few tracks) and then became non-hdcd, still is non-hdcd (both in scanner and in bottom bar; and shows 16 bit).
- the other "problematic"- fully HDCD in Kode's versions, then first track non-hdcd in your earlier version, now it is fully hdcd but the first track is not recognized by "scan for HDCD tracks". But bottom bar shows hdcd and the track is halved as the rest of the album.
- Lateralus - which for me never (with Kodes versions) showed PE, LLE or TF, in one of your previous versions showed gain for first second of first track and in the last version it shows no gain in the begining but shows it at the ending of the first track
- other hdcds - some file that were 17 now are 32 (rather those w/o PE but with LLE); Those with PE are 24 or 17 (short, acoustic or acapella)

Re: Title formatting/syntax for proper bitdepth

Reply #104 –
Quote from: wojak on
- Lateralus - which for me never (with Kodes versions) showed PE, LLE or TF, in one of your previous versions showed gain for first second of first track and in the last version it shows no gain in the begining but shows it at the ending of the first track
That's the one that a previous version would fail to recognize because it is at the very beginning. Of course the usefulness for that particular track can be questioned, but not catching fade-in because fade-in is at the beginning, that was a mistake.

Re: Title formatting/syntax for proper bitdepth

Reply #105 –
@Squeller : RG gain writing was accidentally dependant on peak scanning, fixed. I also improved float detection, it should now always manage to correctly mark when the source signal can't be presented by integer format.

About HDCD, I started implementing more advanced features to the scanner per Porcus' suggestion. The scenario about false packets on one channel causing volume halving shouldn't happen even now. The HDCD "on" mode requires HDCD flags on both channels. Though once that condition is satisfied peak extension of course doesn't need to be used on both channels.

I don't know if I'm doing something wrong, but so far my new scanner suggests the 200 ms packet interval mentioned in documents is indeed correct. In all the valid HDCD tracks I have the packets are from few milliseconds to less than 200 ms apart. For example in the track that needed adjusting the decoder to allow 2 second keep-alive timer my scanner reports:
Min interval: 0.0227 ms
Max interval: ~171 ms
Avg interval: ~45 ms

Adjusting the scanner to require control codes to be on both channels at the same time is more rare, that can give even minutes long periods where such event don't happen.

Currently scanner and the actual decoder test things a bit differently. The "library" behavior for example depends on how big audio chunks you feed to it. The DSP was modified to feed the processor 200 ms chunks at a time but current version of scanner feeds things as they come from the various input components.
Apart from that, what is the expected behavior with these weird tracks that only signal features at the beginning for a brief moment? My intention was not to treat them as HDCD as the majority of the track doesn't look like HDCD.

Re: Title formatting/syntax for proper bitdepth

Reply #106 –
Now the Clipping scanner behaves strangely - it does not detect and write a number of clips for some tracks, it also found 2 clips in one (and only) track of an album but 4 clips in an album (none of the other tracks showed any clips) - so it shows 0, 0, 2, 0, 0...for tracks and 4 for album.

Re: Title formatting/syntax for proper bitdepth

Reply #107 –
I tried replicating buggy behavior with clipping scanner but failed to do so. Then I went through all the places in code where clip counts are handled, and I didn't see any new changes having touched these. And I also don't see album count being mishandled in old code.

Are you certain there isn't some hidden file somewhere with tag data matching the album grouping pattern, as that would affect the album totals. Internally the tool sorts the tracks by grouping pattern, removes duplicates, scans, then goes through the list and treats consecutive tracks that match a single pattern as an album and computes the album based values. Album counters are reset to zero only at the beginning of every new album.

Re: Title formatting/syntax for proper bitdepth

Reply #108 –
Quote from: Case on
I tried replicating buggy behavior with clipping scanner but failed to do so. Then I went through all the places in code where clip counts are handled, and I didn't see any new changes having touched these. And I also don't see album count being mishandled in old code.

Are you certain there isn't some hidden file somewhere with tag data matching the album grouping pattern, as that would affect the album totals. Internally the tool sorts the tracks by grouping pattern, removes duplicates, scans, then goes through the list and treats consecutive tracks that match a single pattern as an album and computes the album based values. Album counters are reset to zero only at the beginning of every new album.

I do not use any sort of Libraty or reFacets. I just manually put the album (which is in a folder) to playlist and then check that playlist (all the tracks from that album) with a mouse and scan it. So if the album has 10 track I check those 10 track with mouse and scan them as album - up till now it always worked. I always use "see hidden and system files" option in Windows. My folders only consist of actual music files plus cover and dr.txt. In this particular example I scanned tht album with your previous version and it showed just 2 clips in one of the file (so the album had 2 too) then when newer version was out I re-scanned that album and it also found only 2 clips in that same track but changed album clips to 4 (just like adding the previously found clips to the new count).
There are also some files that produce "?" after scanning (not in that album)- which never happend with earlier versions - it seems to affect mainly mp3, DSD and those test files made by me by transcoding mp3 to flac (24 and 32 bit).

Re: Title formatting/syntax for proper bitdepth

Reply #109 –
Quote from: wojak on
Quote from: Case on
I tried replicating buggy behavior with clipping scanner but failed to do so. Then I went through all the places in code where clip counts are handled, and I didn't see any new changes having touched these. And I also don't see album count being mishandled in old code.

Are you certain there isn't some hidden file somewhere with tag data matching the album grouping pattern, as that would affect the album totals. Internally the tool sorts the tracks by grouping pattern, removes duplicates, scans, then goes through the list and treats consecutive tracks that match a single pattern as an album and computes the album based values. Album counters are reset to zero only at the beginning of every new album.

I do not use any sort of Libraty or reFacets. I just manually put the album (which is in a folder) to playlist and then check that playlist (all the tracks from that album) with a mouse and scan it. So if the album has 10 track I check those 10 track with mouse and scan them as album - up till now it always worked. I always use "see hidden and system files" option in Windows. My folders only consist of actual music files plus cover and dr.txt. In this particular example I scanned tht album with your previous version and it showed just 2 clips in one of the file (so the album had 2 too) then when newer version was out I re-scanned that album and it also found only 2 clips in that same track but changed album clips to 4 (just like adding the previously found clips to the new count).
There are also some files that present "?" after scanning (not in that album)- which never happend with earlier versions - it seems to affect mainly mp3, DSD and those test files made by me by transcoding mp3 to flac (24 and 32 bit).

Re: Title formatting/syntax for proper bitdepth

Reply #110 –
Quote from: Case on
Apart from that, what is the expected behavior with these weird tracks that only signal features at the beginning for a brief moment? My intention was not to treat them as HDCD as the majority of the track doesn't look like HDCD.
Is low-level extension necessarily flagged in parts of the audio where it isn't used? Say if only fade-in and fade out contains anything < -45 dB.

Much more concerning is how to deal with peak extension "upon track transitions". (And here we are in trouble if we try to replicate a hardware decoder, that does not know track transitions and switches off after ten seconds or whatever.)

Re: Title formatting/syntax for proper bitdepth

Reply #111 –
Yay, a golden "HQ" marker (i.e. "somewhat better than 44.1/16") taking real bitspersample from your plugin into account, here %truebps% :-)

X
Code: [Select]
$ifgreater($add($if2(%truebps%,%__bitspersample%),%samplerate%),44116,$set_style(back,$rgb(255,224,47),$rgb(255,224,47)),NORMAL STYLE PART)

Re: Title formatting/syntax for proper bitdepth

Reply #112 –
@wojak : is the issue reproducible? Do the results look like that on the dialog that shows the scan results? What shows up as a question mark? Remember that possible errors may get logged on console.

Re: Title formatting/syntax for proper bitdepth

Reply #113 –
Quote from: Case on
@wojak : is the issue reproducible? Do the results look like that on the dialog that shows the scan results? What shows up as a question mark? Remember that possible errors may get logged on console.

I did "remove....information from files" - all the data was removed and changed to "?" (I do not use [ ]) but that one file. The value 2 (two clips) changed to 4 for that one track, all other data including "album clips" for that track was "?" So I manually remved that value and name from the tag and rescanned the whole album. It showed 4 for that one track and 4 for the album but it also showed 32 bit float for that file and 32 (without float) for rest of the files. So I reinstalled the component again and rescanned - now all file show 32 float with 0 clips except for that one file which shows 4 (and album 4). So it seems it is now OK but I do not know what was happening earlier. Especially that the data was not properly removed. 
The dialog always shows the same values as columns. The "?" means that there is no vlue as I do not use [ ].

As for the bunch of test files (each from different album and in different audio format) - removing data from the componnt removed all but track clips (each track still had its clips) and one file (352,8/24 - dxd) still had LRA (track and album). So I removed it all manually and rescanned. It crashed foobar (dump showed that foo_truepeak" was the cause). Next scan also crashed foobar. So I reinstalled the component again and again it crashed foobar.

Edit: I think that one file started to crash with the newest component (that never happened with all earlier versions) - it is DSD256 (11,28mHz) - it crashes almost instantly when I start the scan.

Re: Title formatting/syntax for proper bitdepth

Reply #114 –
Thanks for submitting crashes. That DSD file crashing did not start with this version, I suspect you had used lower PCM samplerate with foo_input_sacd in the past. The libebur128 library I use for loudness scanning had a bug triggered by these insanely high sample rates and it attempted to access memory beyond what it had allocated.
Uploaded a new preview version with that issue fixed: https://foobar.hyv.fi/foo_truepeak_preview.fb2k-component.
This version also includes 64-bit floating point output detection when run on foobar2000 x64.
If you have strange metadata behavior you could check the file properties to see what's in there. Perhaps your file had for whatever reason multiple fields with the same name, or your display script reads data from multiple fields.

Re: Title formatting/syntax for proper bitdepth

Reply #115 –
Yes, you are right. Since you have released "foo output" I started to experiment with mainly sacd options and must have left it with max. available PCM output. The crashes have gone now. All DSD files show 64 float now. All lossy show 32 float.
As for "?" in place of clips....some of those test files had old verions of "true peak scanner clipped samples" (without "track") so those were getting 3 fields "clipped sample", "clipped samples track" and "clipped samples album" now. After manual delete all seems ok now.
Thank you very much.

Edit: One new strange thing. While playing .m2ts I get:
- 24 fot pcm;
- 32 for TrueHD and DTSHDMA - why if those two are lossless and 24;
- empty for AC3 - while AC3 from .ac3 file gets 32 float.
Edit2: I think those above come from ffmpeg input not from TBS but still they are strange.

Re: Title formatting/syntax for proper bitdepth

Reply #116 –
Quote from: Case on
Uploaded a new preview version with that issue fixed: https://foobar.hyv.fi/foo_truepeak_preview.fb2k-component.
Case, I use the ReplayGain scanning and let your plugin write the tags to the traditional RG fields. But peak values are missing then and some of my UI code relies on peaks. Could you easily add peaks?
And a question: Does Replaygain scanning in your component do the same upsampling/downsampling magic as the native RG scanner? Maybe it internally even uses the native RG scanner?
Thank you.

Re: Title formatting/syntax for proper bitdepth

Reply #117 –
If you turn off the peak scanning option there are no peaks to write.

Re: Title formatting/syntax for proper bitdepth

Reply #118 –
Quote from: Case on
If you turn off the peak scanning option there are no peaks to write.
Aww, that's rocket science and beyond me :-) After thinking for a very long time about it, I concluded to award myself a platinum facepalm.

Re: Title formatting/syntax for proper bitdepth

Reply #119 –
@Porcus : I started updating the HDCD detector, download new test version: https://foobar.hyv.fi/foo_hdcd.fb2k-component.
I didn't put the new gathered data on the result dialog as I didn't want to waste time thinking how to best format things without first asking for your thoughts. Full HDCD signal off periods are also logged, just not printed anywhere.
Best to scan a single track at a time or the console will be a mess.
In addition to things you recommended I also added frame counters that tell how many frames are affected by the effects. A frame here means a single pair of stereo samples, though apart from LLE, the effect could only affect one channel.

Re: Title formatting/syntax for proper bitdepth

Reply #120 –
Running some HDCD testing, and "Intermittent" reports don't seem completely consistent.

Seriously dumb question, but it looks to me that the order of files matters so much that I am not even sure if the properties are reported on the correct tracks - especially when a track in the list isn't any HDCD and thus doesn't show up in the report at all?

For example, I ran a group of "intermittent-only" files together. Here are results from a CD was part of that group:
* When scanned together with the group with other tracks, one track in the middle of the CD got PE "Intermittent" and the last three got TF "Intermittent"
* Scanned that track only, and it came up with PE & TF both "Disabled" and
* Scanned that whole album and nothing else: now every track has TF "Intermittent" (but no PE).

Then a different scan, including those two which I in https://hydrogenaud.io/index.php/topic,126955.msg1055135.html#msg1055135 wrote off as human error. Not so fast, swine. Those two are edits of pt. 1 and pt. 2 of Mike Oldfield: "The Wind Chimes", and they appear after each other in the playlist and in the file folder:
* Scanned together with a group of others, the second of these gives PE "Enabled"
* Scanning only those two together, none of them give that result. But also gain is different.


 

Re: Title formatting/syntax for proper bitdepth

Reply #121 –
Sorry, it's highly possible I have introduced huge bugs in the result dialog. It's a great mixture of original kode54 stuff and some newer stuff copy/pasted from my own products. As you may remember, original foo_hdcd returned the results in what appeared to be random order (they were listed in the order their scanning finished). I only used this experimental scanner on known HDCD album and was satisfied that tracks by default are printed in the right order... I'll have some more bugs to fix.

Now that you refreshed this topic I'll mention some findings I have made in the mean time.
I compared hdcd.exe output to my HDCD sampler decoded by Windows 7's Windows Media Player. Turns out hdcd.exe is bugged, it doesn't handle all control logic correctly. On the last track there was a gain change flagged only on one channel, hdcd.exe and decoders based on that ignore it, but correct decoding seems to require respecting such gain changes as long as current gain at that point isn't zero.

Also I now know why the output of old decoders, like WMP, is 20 bits. The decoding happens in 24 bit accuracy and gain control uses 12 bit accuracy. When computing the final sample value 24 bit multiplied by 12 bit would overflow 32-bit registers and they couldn't or didn't want to handle that. So they truncated the last four bits of the 24 bit value and avoided overflows that way.

I have modified the old pre-optimization kode54 sources to produce bit-perfect output with WMP, if configured to 20 bit output. But I can now with good conscience keep the output at 24 bits.

My next plan is to see if I can modify this version to run as fast as the ffmpeg-optimized version, or alternatively change the optimized version to be bit-identical with my current version. The old decoder and ffmpeg aren't accurate even to 20 bits.

Re: Title formatting/syntax for proper bitdepth

Reply #122 –
Quote from: Case on
On the last track there was a gain change flagged only on one channel, hdcd.exe and decoders based on that ignore it, but correct decoding seems to require respecting such gain changes as long as current gain at that point isn't zero.
"Single-channel" HDCD packets have been found and discussed before:
https://hydrogenaud.io/index.php/topic,79427.msg865928.html#msg865928
https://hydrogenaud.io/index.php/topic,79427.msg900371.html#msg900371
(You aren't clear about whether that particular thing has HDCD flags in both channels (but only one set to gain), or whether only one channel is flagged as being HDCD at all.)


Quote from: Case on
Also I now know why the output of old decoders, like WMP, is 20 bits. The decoding happens in 24 bit accuracy and gain control uses 12 bit accuracy. When computing the final sample value 24 bit multiplied by 12 bit would overflow 32-bit registers and they couldn't or didn't want to handle that. So they truncated the last four bits of the 24 bit value and avoided overflows that way.
(Is it plausible that the Pacific chip had something like that?)

Re: Title formatting/syntax for proper bitdepth

Reply #123 –
Quote from: Porcus on
"Single-channel" HDCD packets have been found and discussed before:
https://hydrogenaud.io/index.php/topic,79427.msg865928.html#msg865928
https://hydrogenaud.io/index.php/topic,79427.msg900371.html#msg900371
(You aren't clear about whether that particular thing has HDCD flags in both channels (but only one set to gain), or whether only one channel is flagged as being HDCD at all.)
There are valid HDCD codes on both channels constantly. This is the HDCD Sampler disc after all. The quote from the second link contradicts with actual observed behavior, thought part of the reasoning logic supports the observed behavior. When a gain change is already running, keeping the gain steady or adjusting it slightly is less audible than suddenly temporarily changing the gain to zero. Adjusting gain on one channel only, however, would never make sense.

Quote from: Porcus on
(Is it plausible that the Pacific chip had something like that?)
I don't know how advanced device it was, but 32-bit or 16-bit are the guesses I'd go with. I'd think the Microsoft implementation is close to actual hardware, or then it was simply created from highly detailed specs. But for Windows 7 or even XP era PC the 32-bit overflow would not have been a problem.

Re: Title formatting/syntax for proper bitdepth

Reply #124 –
Quote from: Case on
Quote from: Porcus on
(Is it plausible that the Pacific chip had something like that?)
I don't know how advanced device it was, but 32-bit or 16-bit are the guesses I'd go with.
(Why not 24? ... cf how the Pacific was marketed, see https://www.goodwinshighend.com/hdcd---folder/hdcd-recording-encoding-components-library/ )

Quote from: Case on
I'd think the Microsoft implementation is close to actual hardware
One rough idea about how much they cared for that ... see if its de-emphasis EQ curve is close to the spec? Not saying that is exact science, they were implemented at different times in WMP's history, and WMP didn't even get track lengths ripped right ...

Anyway, when the Pacific devices touted 120 dB resolution - pretty much 20 bits, and HDCD touted 19.2 - that may have explained why some application stopped at 20. Rightfully or not. Or - still "rightfully or not"! - produce > 20 bits but don't care squat whether the extra bits are right, because anything 20 or more is "close to actual hardware" when 20 is all you can get out on the analog side.

By the way, does that humpty-bump around 20 kHz at https://forums.stevehoffman.tv/threads/was-the-potential-of-hdcd-squandered.743909/page-2#post-18478289  relate to the flaws you have found?