i apologize if this has been asked, but i have searched using every possible term i can think of and cannot find an objective answer.
i am looking to save to a WAV file from a digital input via toslink in the native format this it is being received in; bitwise save the data.
i am not looking to really go into the value of doing this but just how to. i have an oppo bpd 93 that allows HDCD to be decoded but not passed through DAC. so i can send the decoded HDCD stream out the digital output of the player, and the plan is to capture the digital bits into a new wave file directly.
i have gotten a TOSLINK to USB converter (https://hifimediy.com/product/hifime-ur23-spdif-optical-to-usb-converter/) and can see the data coming in and actually save it to a file in Audacity; i set the preferred input to 24bit 44.1k and set the conversion dither to none. (i would expect not to have to set anything, that there should be an option that can detect what format the incoming content is in in the first place but...) i hit record, play the HDCD, and then export to WAV. i have done this a few times.
since the data is digital, never (supposedly) having been DAC'd, i'd assume the resulting files from Audactity would match, maybe not in length or null samples in the beginning and end, but there should be something of a match using the Foobar Binary Comparator. they do not match; from what i can tell it seems to same the entire thing is different.
so the question really is, despite the nonsense i have already tried - how exactly is the simplest way to simply save the incoming data from the digital input directly to a file? it seems when audacity exports it does some sort of conversion. am i just using the wrong software?
or maybe i am wrong for thinking multiple passes using the same process should create matching files? it is digital so the bits should match unless i am really missing something.
match using the Foobar Binary Comparator.
Sorry, I'm lost... You've got the decoded HDCD converted to a 24-bit WAV file, and you're comparing to what?
i captured the same disc multiple times, so there are multiple copies of the same "data".
i compared those copies, and they do not match.
in other words, doing the same thing twice (or more) but not getting the same results.
If you want to rip a CD, you use a CD-ROM drive and a secure ripping application (for Windows: EAC/dBpoweramp/CUERipper).
If you want to rip a CD, you use a CD-ROM drive and a secure ripping application (for Windows: EAC/dBpoweramp/CUERipper).
Exactly. There is nothing to be gained by capturing the "real-time" Oppo digital output. The Oppo can not pass the HDCD "expanded" bitstream. You need to play the ripped file through a DAC capable of decoding HDCD. Even then, there are limitations such as not being able to use any digital volume control ahead of the DAC lest it strip out the HDCD "flags".
Alternatively, there is a plugin for foobar2000 (https://www.foobar2000.org/components/view/foo_hdcd) to rip and decode an HDCD to a 24-bit file, but I've never used it. Pretty sure dBPoweramp can also do this.
HDCD can be decoded later, no need to run it upon ripping.
The Oppo can not pass the HDCD "expanded" bitstream. You need to play the ripped file through a DAC capable of decoding HDCD."
this is not correct, unless this does not mean what i think it means.
oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93 (https://oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93)
i already have a secure rip of the content, i prefer cueripper myself. i am aware of the multitude of HDCD software decoder options as well. afaik, software decoding is not as comprehensive as passing through a hardware decoder from what i have read here and elsewhere. if i am wrong, and the software decoder 100% does everything the hardware decoder is doing then please correct me.
i am not looking to really go into the value of doing this but just how to.
I am not doing this cause "you won't be able to hear the difference". i would like to ABX this myself, and do it where i have an actual legit simple way to compare using the ABX tool in Foobar. it's really about neurotic completeness for my collection.
let me know what you guys think thanks
Two possible sources of error:
* You are sure you did align the bitstream up well?
* Once HDCD is switched on, it will be kept alive for ten seconds. That means you can get different results if you play two tracks consecutively, vs if you play the second after a pause.
this is not correct, unless this does not mean what i think it means.
oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93 (https://oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93)
Well, this surprises me. Not only because it is different from what is printed in the BDP-93 owners manual, but also because I thought Pacific Microsonics license terms prohibited exposing the "decoded" bitstream.
Regardless, you should be able to capture the Oppo playback using Audacity. Be sure Audacity is set to record at 24/44:
Edit > Preferences > Quality > Sampling
On that same page, disable dither in both "Conversion" options. That should capture and export the output of your Oppo bit-perfectly so long as the TOSLINK>USB converter is functioning transparently. The recording gain in Audacity should be set to 100%. After recording, export to 24-bit WAV or FLAC.
Well, this surprises me. Not only because it is different from what is printed in the BDP-93 owners manual, but also because I thought Pacific Microsonics license terms prohibited exposing the "decoded" bitstream.
Regardless, you should be able to capture the Oppo playback using Audacity. Be sure Audacity is set to record at 24/44:
Edit > Preferences > Quality > Sampling
On that same page, disable dither in both "Conversion" options. That should capture and export the output of your Oppo bit-perfectly so long as the TOSLINK>USB converter is functioning transparently. The recording gain in Audacity should be set to 100%. After recording, export to 24-bit WAV or FLAC.
awesome thanks. i have a mail from support before i found that page where i specifically asked him if it would decode without DAC and i was told the BDP 93/95 and 103/105 players do this.
i will confirm this setup and run a couple passes. i am doing the whole CD in a single shot and will split tracks later (plan is to use an image cue and make the capture image length match the standard CD image and split identically...) so to get a couple passes will take a few hours. i'll give it a shot.
* Once HDCD is switched on, it will be kept alive for ten seconds. That means you can get different results if you play two tracks consecutively, vs if you play the second after a pause.
hmmm so i did queue up track one paused and wait while i setup audacity. i did not do this in the next try, so this 100% could be where I screwed up. going to try the recommendations and see how it works.
i really was looking for confirmation that it is possible to capture the data bit perfectly (or as close as possible) using this process. thanks for the help.
i am aware of the multitude of HDCD software decoder options as well. afaik, software decoding is not as comprehensive as passing through a hardware decoder from what i have read here and elsewhere. if i am wrong, and the software decoder 100% does everything the hardware decoder is doing then please correct me.
I think it depends upon which features the HDCD is using. You can rip your CDs normally and then use FB2K to scan the tracks to see which HDCD features are enabled. If the only features are volume adjustment and/or peak extension, the software-based decoding solutions should be the same as what the Oppo does. If the HDCD is using any filters, then you'd need the Oppo to decode it correctly, but this seems to be fairly rare.
HDCD software decoding was reverse engineered from Microsoft's official implementation in Windows Media Player AFAIK, so it should be correct for the features that WMP implemented. If you can get the Oppo capture method working, it would be an interesting experiment to see if the results match what you get with software decoding for a track with peak extension enabled.
If you can get the Oppo capture method working, it would be an interesting experiment to see if the results match what you get with software decoding for a track with peak extension enabled.
this was always part of the plan.
i just did two copies of the same disc. i followed the configuration suggested, i aligned the starts, and trimmed the end and this is what i get. i also thought the Binary Comparator would detect offset shifts, could have sworn i did that, but maybe i am wrong, and the tracks are misaligned?
Differences found in compared tracks.
Zero offset detected.
Comparing:
"untitled01.wav"
"untitled02.wav"
Compared 154706916 samples.
Differences found: 117542395 values, 0:00.000000 - 58:28.093311, peak: 0.225037 (-12.95 dBTP) at 53:22.082653, 2ch
Channel difference peaks: 0.170013 (-15.39 dBTP) 0.225037 (-12.95 dBTP)
File #1 peaks: 0.988647 (-0.10 dBTP) 0.988647 (-0.10 dBTP)
File #2 peaks: 0.988647 (-0.10 dBTP) 0.988647 (-0.10 dBTP)
Detected offset as 0 samples.
Total duration processed: 58:28.093
Time elapsed: 0:17.409
201.51x realtime
i followed the configuration suggested, i aligned the starts, and trimmed the end and this is what i get. i also thought the Binary Comparator would detect offset shifts
You have aligned the starts well, and foo_bitcompare says "Detected offset at 0 samples".
62 percent of the samples compared are common too. A bit hard to read, but the "154706916" samples compared are 154706916 per channel. There are two channels, so in reality it has compared 309413832 samples and found that 191871437 are common and 117542395 differ.
did a 3rd and 4th pass.
much much better results this time. not exactly sure what i did differently.
so after the "recording" finishes, i save it immediately and then on a different PC i am importing the track into audacity and then trimming the front and back and aligning the start time exactly to the start of a normal secure rip, then chopping the end off past the number of samples, then Export again to 24 bit wav and create a "new" file (keep the original recording).
going to try a couple more time, this time doing the trimming right after its recorded so as to minimize the number of import/export in audacity.
edit: i compared the original recordings and bitcompare detected the offsets and it was identical result as the trimmed files, 682 mismatch; so its not the import/export in audacity changing anything. not bad either really. comparing in Beyond Compare as hex, its got minimal differences there too. so i am close, or maybe as close as i am going to get to resproducibility. my guess is its just not "secure rip" level data coming in from the bit stream maybe?
edit 2: i split the files to the tracks, and got MD5s of tracks.
10/14 tracks match md5, 4/14 are different.
BC hex compare:
928239200 same byte(s)
1690 left orphan byte(s)
1690 right orphan byte(s)
650 difference byte(s)
Differences found in compared tracks.
Zero offset detected.
Comparing:
"untitled-3.wav"
"untitled-4.wav"
Compared 154706916 samples.
Differences found: 682 values, 5:28.322086 - 58:27.964649, peak: 0.180267 (-14.88 dBTP) at 52:40.703492, 2ch
Channel difference peaks: 0.169952 (-15.39 dBTP) 0.180267 (-14.88 dBTP)
File #1 peaks: 0.988647 (-0.10 dBTP) 0.988617 (-0.10 dBTP)
File #2 peaks: 0.988647 (-0.10 dBTP) 0.988617 (-0.10 dBTP)
Detected offset as 0 samples.
Total duration processed: 58:28.093
Time elapsed: 0:03.385
1036.32x realtime
what do you guys think? will i get any more exact or do you think this is it? or should it be 100% every time?
Which software option are you using to decode the HDCD? I don't know if they produce identical results.
everything i have done above is the hardware decode for now.
i normally would use CueTools to do the convert., but was going to try a couple others. not there yet, want to get the hardware decode capture consistent (enough).
thanks
edit - one thing i noticed on the hardware decode, the volume gain/peaks are the same as the secure rip. its not half like when cuetools does the convert. also, i have seen that dbpoweramp has a +6cB add to the HDCD conversion to accomodate so i believe its consistent with the software decode for the peak to drop by 6dB? the hardware decode is not doing that it seems.
once i have this down, i will post a secure rip and a hardware decode rip of a album with peak extension and maybe transient filter and you can software convert with whatever tools you like and compare yourself, i am sure you know 100x as much about this as me.
Ah, my mistake. I misread your comparisons as hardware capture vs software decode, not two hardware captures. Carry on! ;)
so turns out i had a thought last night that since this BDP device supports USB media, will it detect a flac rip of the CD so i don't have to use the disc transport?
i got to check this morning and turns out the HDCD light is coming on for a file as well, so no read errors with this method.
doing a couple passes now.
Yes, HDCD is flagged in-signal, so unless the device is deliberately turning off HDCD detection for that input, you should get it.
Of course you have to pass a 16-bit file that is not already HDCD decoded. (If not ... it seems to me that HDCD packets can survive HDCD decoding, so ...)
following the suggestions from Apesbrain to configure audacity to capture the data bit perfect, and using a file as source instead of disc, i was able to capture the content identically two times over, no differences found.
would this be a bitperfect way to hardware decode HDCD if the source is a verified secure rip, in theory?
anyway the real question was how to use audacity to capture digital signal as-is on a digital input and save direct to file. thanks for the help.
Curious now: how do they compare to the software decoded?
this CD does not have any features associated with it as per foobar, so there is probably no value in the one i have been testing with other than getting the process down.
with that said, i had run it through cuetools to decode and it just looks really odd to me in audacity. this is all three files.
top = HDCD from oppo, middle = normal CD image, bottom = cuetools converted hdcd
(https://i.imgur.com/rz4yf8A.jpg)
i have a disc, sonny boy williamson - portrait of a blues man (1996) that has a minimum gain -4.0, peak extension and intermittent transient filter. i will convert this and compare.
i am pretty sure i cannot post links to the content here. let me see if i can find some sample HDCD stuff.
You can upload samples in the Uploads forum:
https://hydrogenaud.io/index.php?board=35.0
then link to your upload post here. Just clip ~15 seconds of audio in Audacity and upload the WAV or compress to FLAC and upload that.
i put 15 second clips from the same song, track 1 - little girl, from 3:45-4:00.
https://hydrogenaud.io/index.php?topic=122263 (https://hydrogenaud.io/index.php?topic=122263)
Sonny Boy Williamson – Portrait Of A Blues Man
Label: Analogue Productions – CAPR 3017
Series: Revival Series – CAPR 3017
Format: CD, Compilation, HDCD
Country: Denmark
Released: 24 Sep 1996
Genre: Blues
Style: Chicago Blues
HDCD flags are as per foobar: Miminum Gain -4.0dB, Maximum Gain 0.0dB, Peak Extention - Enabled, Transient Filter - Intermittent
ffmpeg output is:
[Parsed_hdcd_0 @ 00000261458d9ec0] HDCD detected: yes, peak_extend: enabled permanently, max_gain_adj: -4.0 dB, transient_filter: detected, detectable errors: 0
let me know what you think. let me know if you want me to pass through any other SW decoder.
I don't think your CUETools and ffmpeg conversions are correct. The output should be 24 bits, but the WAV files you uploaded are 16 bits. The ffmpeg command should be:
ffmpeg -i in.wav -af hdcd -acodec pcm_s24le out.wav
I decoded CD Rip.wav with hdcd.exe and ffmpeg and uploaded them to the same thread:
https://hydrogenaud.io/index.php?topic=122263.msg1009263#msg1009263
The two decoded files are not identical, but they are close. Aside from the different headers, there were 691 differences. The hardware-decoded WAV and hdcd.exe WAV files have completely different samples, because it appears the hardware-decoded WAV has been normalised after decoding; it's much louder than the output of both hdcd.exe and ffmpeg.
ugh i was working with the whole files, which are decoded properly. I screwed up exporting the 15sec clips with Audacity and didn't changed back from 16 bit signed. the sources were 24 bit. my mistake. either way, mine are different as the full songs, but i used cuetools and ffpmeg.
i uploaded proper 15 sec clips for the CUEtools and FFMpeg decodes i did.
im curious what you think of the cuetools conversion; i uploaded it properly this time, as well as the ffmpeg that was clipped from the larger file conversion.
sorry for the confusion, way too many things at once. thanks.
The hdcd.exe and CUETools decodes are identical. Aside from the headers, our two ffmpeg decodes are also identical. There are still 691 differences between the hdcd.exe/CUETools and the ffmpeg decodes. I suppose they could be some kind of metadata, but I'm not very familiar with the structure of WAV files.
The "HW Decode" and the "CD Rip" are bit-identical except that the "HW Decode" is padded to 24 bit. So the hardware DAC has done nothing to it.
There are still 691 differences between the hdcd.exe/CUETools and the ffmpeg decodes. I suppose they could be some kind of metadata, but I'm not very familiar with the structure of WAV files.
When foo_bitcompare says 691 different samples, it is the audio and has nothing to do with metadata. But see the peaks:
Differences found: 691 values, 0:00.520522 - 0:14.864467, peak: 0.000002 (-114.95 dBTP) at 0:00.520658, 1ch
Channel difference peaks: 0.000002 (-114.95 dBTP) 0.000002 (-114.95 dBTP)
So, they differ in the 20th bit. Makes me believe it is round-off. And that ffmpeg puts more noise in the 20th bit, by the compressed sizes.
Which are quite "interesting" when you compare encoders:
1 870 208 SW Decode - CUETools.flac
1 887 338 SW Decode - CUETools.tak
2 001 618 SW Decode - ffmpeg.flac
2 064 470 SW Decode - CUETools.wv
2 129 060 SW Decode - CUETools.ofr
2 173 188 SW Decode - ffmpeg.tak
2 549 680 SW Decode - ffmpeg.wv
2 790 492 SW Decode - ffmpeg.ofr
2 925 898 SW Decode - ffmpeg.m4a
2 925 898 SW Decode - CUETools.m4a
flac -8pe smaller than TAK -p4m smaller than WavPack -hhx6 smaller than OptimFROG --preset max (smaller than OptimFROG --preset 10, so it isn't a weirdo that fools max), anyone?
The "HW Decode" and the "CD Rip" are bit-identical except that the "HW Decode" is padded to 24 bit. So the hardware DAC has done nothing to it.
His CD player doesn't actually support HDCD decoding, then?
When foo_bitcompare says 691 different samples, it is the audio and has nothing to do with metadata.
I compared with a hex editor, not foobar2000, so I didn't have access to any information about the meaning of the differences. Thanks for the clarification.
The "HW Decode" and the "CD Rip" are bit-identical except that the "HW Decode" is padded to 24 bit. So the hardware DAC has done nothing to it.
His CD player doesn't actually support HDCD decoding, then?
That is a possibility. Or potentially that the peak extend flags are false positives.
But, had it been me, I would have looked for user errors first, clumsy as I am.
@koawmfot , the procedure seems to be here: https://www.oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93 Sure you have enabled it?
It is kind of hard to have a false positive with HDCD version b, since it does a complex LFSR dance to control packet integrity.
The "HW Decode" and the "CD Rip" are bit-identical except that the "HW Decode" is padded to 24 bit. So the hardware DAC has done nothing to it.
are you saying its not really decoding anything? do you think it would be worth while to try and capture it as 16bit in audacity and see if it is bit aligned to the original source?
EDIT: what tool are you using to figure that out?
But, had it been me, I would have looked for user errors first, clumsy as I am. @koawmfot , the procedure seems to be here: https://www.oppodigital.com/KnowledgeBase.aspx?KBID=44&ProdID=BDP-93 Sure you have enabled it?
its enabled. i referenced that KB earlier in the thread. i also have a mail from support confirm that it should decode HDCD without doing DAC.
HDCD ON (https://imgur.com/FzG3coj)
the HDCD light is coming on too. I am going to try changing the Coax/Digital output to LPCM (from bitstream), see if is different. nothing to lose. (EDIT: This did not change anything, and i believe it is specific to DSD output. same MD5 on the new capture.)
i have another disc with peak extension, Cat Power - Moon Pix (1998, Label: Matador, Catalog#: OLE 286-2); no mention its HDCD on the case too. this has no gain and no transient filter, only peak extension. i can try that if you're interested. if you have a track or anything you want to pass along i can try to convert too, PM me?
as far as user error, that's entirely possible, but this is very basic. optical out on the oppo, to the optical->USB on the PC, and Audacity set to Quality 44.1/24bit, none for dither. hit record, play the track, align the start and set total number samples and export.
(there is one odd thing, and it is when it is playing the CD, it does say "Track Type: HDCD, Channel Type: Stereo 44.1k 16b". i would have thought it would say 24b, but then it could jsut be the pre-decode data?)
qq - is there a tool or app that can show me what the bitstream info is as it is coming in to the PC?
The "HW Decode" and the "CD Rip" are bit-identical except that the "HW Decode" is padded to 24 bit. So the hardware DAC has done nothing to it.
are you saying its not really decoding anything? do you think it would be worth while to try and capture it as 16bit in audacity and see if it is bit aligned to the original source?
EDIT: what tool are you using to figure that out?
i grabbed a 16bit import of the bitstream and it does not match the source file at all.
Replaygain values in attached pic. You can see the software decodes have done the thing where they halve the levels to make room for the peak extension. HW decode has not. So yeah looks like the HW decode isn't doing it's job.
I believe i know why the HW Decode did not work as expected.
Yes, HDCD is flagged in-signal, so unless the device is deliberately turning off HDCD detection for that input, you should get it.
Of course you have to pass a 16-bit file that is not already HDCD decoded. (If not ... it seems to me that HDCD packets can survive HDCD decoding, so ...)
From what i can tell the player is NOT decoding HDCD for file-based input. so it must be deliberately
turning off HDCD detection for that input.
I had said the HDCD light was on when playing the USB files.... turns out if there is a CD in the player that is HDCD, it will ALWAYS show HDCD light, no matter what content is playing.
so as @porcus said, user error. anyway, i need to now burn some clean discs to try to copy in (most of my collection is digital only rips at this point, media has been long gone.)
I will get a new HW Decoded sample up shortly.
thanks
Replaygain values in attached pic. You can see the software decodes have done the thing where they halve the levels to make room for the peak extension.
Hmmm... Now I'm wondering how this relates to HDCDs with the minimum gain -4.0 dB flag. Most of my HDCDs have both peak extension and -4.0 dB min, but one disc has only peak extension. Would you happen to know how this influences the decoding process, if peak extension always requires cutting the volume in half?
I'm not the best person to ask but my understanding is the "gain" cuts the volume of the quiet parts.
As for peak extension, if you didn't lower the volume first you would end up with clipping as the peaks have nowhere to extend to.
Yes, my question is since you already have to cut the volume by half for peak extension to work, what's the point of the -4.0 dB minimum flag?
That is at the quiet end. Peak extension is at the loud end.
The point? To get more out of the bits overall.
(Disregarding the alternative answer: "pointless, you don't need more than 96 dB properly dithered". Oh, and: part of those "20 dB" promised - which weren't 20, it were >19 hence rounded up to 20 - was the dithering. So it was for the full HDCD process, including that part which also is performed when decimating 24 bits to 16 without HDCD - it is not apples to apples, it is apples to non-dithered oranges.)
Yes, my question is since you already have to cut the volume by half for peak extension to work, what's the point of the -4.0 dB minimum flag?
The gain isn't applied unconditionally to the whole signal, it begins to get applied only when the signal drops below a certain volume threshold, and it scales progressively as the volume gets lower, until the full 4dB is applied.
The gain isn't applied unconditionally to the whole signal
Interesting. Thanks.
I have successfully and reproducibly captured the HW decoded output.
new HW decode file is here: https://hydrogenaud.io/index.php?topic=122263.msg1009436#msg1009436 (https://hydrogenaud.io/index.php?topic=122263.msg1009436#msg1009436)
Two passes with the same clean burned CD (to minimize read errors) i was able to get (mostly) identical copies of the disc in decoded HDCD. the differences are in the end of the data not in any of the audio, so all tracks matched MD5 from both extractions, except the last track which has issues in 1008 samples in the dead space after the last track.
Differences found in compared tracks.
Zero offset detected.
Comparing:
"\\infinity\continuum\music\proper\williamson, sonny boy\{1996} portrait of a blues man\test\sbw-clean-fulldisc-p01.wav"
"\\infinity\continuum\music\proper\williamson, sonny boy\{1996} portrait of a blues man\test\sbw-clean-fulldisc-p02.wav"
Compared 103338060 samples.
Differences found: 1008 values, 39:03.148707 - 39:03.179138, peak: 0.000031 (-90.31 dBTP) at 39:03.148707, 2ch
Channel difference peaks: 0.000031 (-90.31 dBTP) 0.000031 (-90.31 dBTP)
File #1 peaks: 0.895081 (-0.96 dBTP) 0.857697 (-1.33 dBTP)
File #2 peaks: 0.895081 (-0.96 dBTP) 0.857697 (-1.33 dBTP)
Detected offset as 0 samples.
Total duration processed: 39:03.267
Time elapsed: 0:12.731
184.07x realtime
I'd consider this successful extraction of the content.
Can anyone tell what is happening here that maybe SW decode does not do or where there may be gaps? love to hear what you guys think.
I can also add more samples, of other HDCDs with different feature profiles? let me know, and if you want to pass something along i can test too. just PM. Sonny Boy is the only disc i have with gain at all, and the only one with all the features.
One big observation I had on SW v HW decode... not with this particular disc, but on one with an HDCD flag, but NO HDCD features detected - it will NOT half the levels to make room for peak extension. My guess is any disc without peak extension enabled will not have volume levels adjusted. i will create a few sample rips and post.
thanks//