HydrogenAudio

CD-R and Audio Hardware => CUETools => Topic started by: bennetng on 2022-09-22 17:05:20

Title: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-22 17:05:20
reference: https://hydrogenaud.io/index.php?msg=1016166
@Porcus did you test the files? foo_verifier, foo_bitcompare and flac.exe -t showed anomalies.
Use -V so that the encoder throws errors when MD5 mismatch. So far only this CD has problem, either in separate files or in one image. Removing --vbr the files encoded properly.
Code: [Select]
5 out of 29 files converted with major problems.

Source: "F:\My Music\Game Music\Einhander CD\11 - 突破[Breakthrough].flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters) : "H:\11. 突破[Breakthrough].flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "F:\Software\Player & Codec\CUETools\CUETools.Flake.exe" -q --ignore-chunk-sizes -8 -V --vbr 4 - -o "temp-4071E17331D7A04CACDD90B682F08D96.flac"
  Working folder: H:\
  Conversion failed: The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters

Source: "F:\My Music\Game Music\Einhander CD\13 - 払暁[Dawn].flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters) : "H:\13. 払暁[Dawn].flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "F:\Software\Player & Codec\CUETools\CUETools.Flake.exe" -q --ignore-chunk-sizes -8 -V --vbr 4 - -o "temp-322EDDD47174604E85390D5779FAD085.flac"
  Working folder: H:\
  Conversion failed: The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters

Source: "F:\My Music\Game Music\Einhander CD\16 - 焦燥[Impatience].flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters) : "H:\16. 焦燥[Impatience].flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "F:\Software\Player & Codec\CUETools\CUETools.Flake.exe" -q --ignore-chunk-sizes -8 -V --vbr 4 - -o "temp-F7CE828604ACCD4696C7DC7910958EC2.flac"
  Working folder: H:\
  Conversion failed: The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters

Source: "F:\My Music\Game Music\Einhander CD\17 - 熱圏[Thermosphere].flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters) : "H:\17. 熱圏[Thermosphere].flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "F:\Software\Player & Codec\CUETools\CUETools.Flake.exe" -q --ignore-chunk-sizes -8 -V --vbr 4 - -o "temp-F171283C44598C4BA3A0616461640EF7.flac"
  Working folder: H:\
  Conversion failed: The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters

Source: "F:\My Music\Game Music\Einhander CD\21 - 無重量[Zero Gravity].flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters) : "H:\21. 無重量[Zero Gravity].flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "F:\Software\Player & Codec\CUETools\CUETools.Flake.exe" -q --ignore-chunk-sizes -8 -V --vbr 4 - -o "temp-CDA2793624EC41419FFF9CBE5BC0AE02.flac"
  Working folder: H:\
  Conversion failed: The encoder has terminated prematurely with code 5 (0x00000005); please re-check parameters



MOD edit: added reference post
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Bogozo on 2022-09-22 19:20:41
Indeed. Short sample from this CD, that cannot be correctly encoded with --vbr 4 - https://www.dropbox.com/s/13ghip3ia1hc26h/24.%20Muddle%20-%20Continue%20Count.flac?dl=1
Who is not lazy enough to report to CUETools developer? @korth maybe?  :))
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-22 19:25:18
Thanks. This really scared me and I checked if my RAM are corrupted or not.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Porcus on 2022-09-22 22:35:44
That's quite a W.T.F**k(e)...

It has already been reproduced - and I can confirm the clip @Bogozo posted makes errors with all variable block sizes from 1 to 4 in two different CUETools versions including 2.1.5 (where CUETools.flake.exe had slightly different syntax, that doesn't mean anything else was changed). Nothing wrong with other random files found at this computer where I am now, so I suppose it has to be provoked by something strange. Might check the "test corpus" when I am back at that computer.

korth has already moved this thread, so I assume it is duly noted.

Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Porcus on 2022-09-23 10:11:16
Found the same in my files, in three albums (Flute Music; High Tech Choruses; Jan Johansson's Folkvisor). Checking all --vbr settings from 1 to 4, only one file verified, Jan Johansson at --vbr 3.


I also decoded a few troublesome files with both ffmpeg and reference FLAC, and they agree. So the blame is at the Flake encoder, and luckily not in the reference decoder.

Edit: Might be helpful for debugging to notice that foo_bitcompare gives differences down at -90.31 dB, i.e. the LSB. Also in the Jan Johansson CD where --vbr 3 is correct, the others fail at thousand to two thousand samples, ruling out "random bits happen to hit the correct".
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-23 11:36:57
I just found several English phonetics tutorial CD tracks demonstrating vowels, consonants etc have the same issue as well.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Wombat on 2022-09-23 15:09:59
When CUEtools flake was new not many people switched from official flac. I use it ten years now and reported encoding bugs but vbr was never tested here.
flacCL only used vbr in non subset levels above 8 afaik.
@Gregory S. Chudov  back when it was new surely could have fixed it when more people were interested back then.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-23 15:52:37
Speaking of flacCL...
https://www.audiosciencereview.com/forum/index.php?threads/why-isnt-wavpack-more-popular.31465/post-1111063
I have about 16GB of non-commercial personal archive in 16/48 and 24/48 encoded with flacCL, so the files cannot be verified with any online database. I just scanned all of them and there is no MD5 error. I can't remember the used encoding settings.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Wombat on 2022-09-23 16:16:23
@korth
Why did you edit? I PMd a bit with Mr. Chudov. His name is more like Grigory and "Greg" just simpler to use in the country he moved to.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Bogozo on 2022-09-23 16:30:36
Speaking of flacCL...
https://www.audiosciencereview.com/forum/index.php?threads/why-isnt-wavpack-more-popular.31465/post-1111063
I have about 16GB of non-commercial personal archive in 16/48 and 24/48 encoded with flacCL, so the files cannot be verified with any online database. I just scanned all of them and there is no MD5 error. I can't remember the used encoding settings.
By the way there was bug in FLACCL, preventing correct encoding of some files. It was presented in versions pre-2.1.5 and early builds of 2.1.5 - https://sourceforge.net/p/cuetoolsnet/bugs/22/
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-23 16:42:00
Code: [Select]
Last modified : 2014-03-02 02:19:23
Duration : 9:42:09.694 (1 676 625 299 samples)
Sample rate : 48000 Hz
Channels : 2
Bits per sample : 16
Avg. bitrate : 666 kbps
Codec : FLAC
Encoding : lossless
Tool : CUETools FLACCL 2.1.5
Embedded cuesheet : no
So my files are not completely safe. See the timestamp. Anyway, I can't do anything about it now, too late.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Porcus on 2022-09-23 16:58:42
So my files are not completely safe. See the timestamp.
As of now, all discovered errors have led to MD5 mismatch - or worse: even if you switched off MD5ing, the FLACCL bugreport (https://sourceforge.net/p/cuetoolsnet/bugs/22/) indicated that the stream itself would be corrupted.

Anyway, I can't do anything about it now, too late.
Not if they don't verify. Well at least, that would require the bug to be "reversible".

Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: korth on 2022-09-23 17:11:34
@korth
Why did you edit? I PMd a bit with Mr. Chudov. His name is more like Grigory and "Greg" just simpler to use in the country he moved to.
@Gregory S. Chudov is his HA user name. I edited for a forum "mention"(assuming he has notification enabled).
I suppose I could've left both Gregory and Grigory in the post though.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Wombat on 2022-09-23 17:25:49
All cool korth, sorry for my overreaction. Sometimes less coffee...
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-23 17:36:14
I can't remember the used encoding settings.
Googled myself and found that I posted in a local forum mentioned a foobar preset in 2013 that worked well for my GPU:
https://www.hkepc.com/forum/viewthread.php?fid=25&tid=1946635
Code: [Select]
-q -9 --verify --slow-gpu --task-size 256 --cpu-threads 2 - -o %d
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: NetRanger on 2022-09-23 20:13:42
@c72578 - Have u noticed this thread?
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Porcus on 2022-09-23 21:07:27
Reported at https://github.com/gchudov/cuetools.net/issues/220
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: bennetng on 2022-09-27 15:26:59
https://vgmdb.net/album/5066
https://www.youtube.com/playlist?list=PLyxqUxT0goirVyF6FILtXMKpcQfOwCC5d
CUETools.Flake 2.2.2 failed to encode with --vbr, but now I can predict what kinds for files can benefit from changing the default block size.
With --vbr, this CD has even more error tracks when using presets -1 to -3 than -5 to -8. Only -0 results in no error. The 0-8 presets affect stereo decorrelation (independent, estimate, evaluate, search). Anything other than independent results in error.
Title: Re: CUETools.Flake 2.2.2 MD5 mismatch in encoded flac CD image
Post by: Porcus on 2022-10-23 09:57:27
Variable block size is getting removed: https://github.com/gchudov/cuetools.net/pull/223
If it cannot be repaired immediately, removing it is better than keeping it - and I don't even know if anyone will miss it. (Except for testing purposes, and those can keep their old builds.)