HydrogenAudio

CD-R and Audio Hardware => CUETools => Topic started by: Porcus on 2021-03-25 10:57:54

Title: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-25 10:57:54
1) Choosing libflac and put the slider at compression level 6.

2) Afterwards, recompressing with flac --best --force *.flac (that's a version 1.3.1 of reference flac.exe).
Getting around 0.51 ...
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: korth on 2021-03-25 12:33:48
https://xiph.org/flac/documentation_tools_flac.html
Quote
--best    Highest compression. Currently synonymous with -8
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-25 19:26:14
Sorry, slider at "8". But still, -6 being DOUBLE the size of -8 would be so "abysmal" that you should not believe it.

Indeed, recompressing with "-0" improves enormously over CUETools-generated -8. (Yes, -0 over -8, not the other way around.)
CUETools-generated with the "8" setting, after which I removed all tags and optimized layout using fb2k (that affects the HTOA with padding, I guess).
Bitrates:
830 for CUETools 2.1.8 at setting "8"
630 for flac.exe -0
583 for flac.exe -8


Pasting from command line window:

Code: [Select]
c:\copy4flac-0>flac --force -0 *.flac

flac 1.3.1, Copyright (C) 2000-2009  Josh Coalson, 2011-2014  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

flac (0) (HTOA) - Copy.flac: wrote 8544 bytes, ratio=47,378
FAILURE: Compression failed (ratio 47,378, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
flac (1) - Copy.flac: wrote 39438228 bytes, ratio=0,696
flac (10) - Copy.flac: wrote 23047738 bytes, ratio=0,791
flac (11) - Copy.flac: wrote 16080409 bytes, ratio=0,907
flac (12) - Copy.flac: wrote 13939740 bytes, ratio=0,901
flac (2) - Copy.flac: wrote 28144850 bytes, ratio=0,881
flac (3) - Copy.flac: wrote 24087487 bytes, ratio=0,697
flac (4) - Copy.flac: wrote 29653427 bytes, ratio=0,688
flac (5) - Copy.flac: wrote 18441542 bytes, ratio=0,768
flac (6) - Copy.flac: wrote 31910628 bytes, ratio=0,782
flac (7) - Copy.flac: wrote 13865914 bytes, ratio=0,682
flac (8) - Copy.flac: wrote 26342544 bytes, ratio=0,763
flac (9) - Copy.flac: wrote 10463287 bytes, ratio=0,736
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: korth on 2021-03-27 15:38:00
I could not duplicate your result (showing last 3 of 10 tested rips but other results consistent)
All source test files from CUETools 2.1.8 compressed using libFLAC -8
then
flac -8 --force *.flac (using flac.exe 1.3.1)
all ratios are ≥ 1.000

Code: [Select]
E:\>flac -8 --force E:\test08\*.flac

flac 1.3.1, Copyright (C) 2000-2009  Josh Coalson, 2011-2014  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01.flac: wrote 38820757 bytes, ratio=1.000
02.flac: wrote 38321296 bytes, ratio=1.000
03.flac: wrote 37511600 bytes, ratio=1.000
04.flac: wrote 27624624 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
05.flac: wrote 31336520 bytes, ratio=1.000
06.flac: wrote 21499974 bytes, ratio=1.000
07.flac: wrote 26099143 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
08.flac: wrote 26090780 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
09.flac: wrote 25965161 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).

Code: [Select]
E:\>flac -8 --force E:\test09\*.flac

flac 1.3.1, Copyright (C) 2000-2009  Josh Coalson, 2011-2014  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

00. (HTOA).flac: wrote 27867 bytes, ratio=0.999
01.flac: wrote 4578024 bytes, ratio=1.000
02.flac: wrote 18047899 bytes, ratio=1.000
03.flac: wrote 20906307 bytes, ratio=1.000
04.flac: wrote 42327147 bytes, ratio=1.000
05.flac: wrote 25447212 bytes, ratio=1.000
06.flac: wrote 43811398 bytes, ratio=1.000
07.flac: wrote 46834980 bytes, ratio=1.000
08.flac: wrote 21281388 bytes, ratio=1.000
09.flac: wrote 23251622 bytes, ratio=1.000
10.flac: wrote 12663375 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).

Code: [Select]
E:\>flac -8 --force E:\test10\*.flac

flac 1.3.1, Copyright (C) 2000-2009  Josh Coalson, 2011-2014  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01.flac: wrote 75194955 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
02.flac: wrote 45266060 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
03.flac: wrote 33099510 bytes, ratio=1.000
04.flac: wrote 29421938 bytes, ratio=1.000
05.flac: wrote 71845318 bytes, ratio=1.000
FAILURE: Compression failed (ratio 1.000, should be < 1.0).
This happens for some files for one or more of the following reasons:
 * Recompressing an existing FLAC from a higher to a lower compression setting.
 * Insufficient input data  (eg, very short files, < 10000 frames).
 * The audio data is not compressable (eg a full range white noise signal).
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: itisljar on 2021-03-28 11:56:18
I vaguely remember something similar happening few years ago, but I don't remember in what context. I just know that We had to download libflac.dll (or something like it) from rarewares to fix that.
FOUND THE THREAD!
https://hydrogenaud.io/index.php?topic=117421.0
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-28 14:39:09
Oh thanks - and that was myself, and I totally forgot it. I.Am.Getting.Demented! (I still remember that St. Matthew Passion repair though. CUETools <3 .)

I did a fresh download now with the _HF1 version linked to from http://cue.tools/wiki/CUETools_Download . Same thing, and problem vanishes by replacing CUETools_2.1.8_HF1\CUETools_2.1.8\plugins\x64\libFLAC_dynamic.dll

Testing with a vastly different album (some black metal) as an example - yes it is CDDA that produces 1408:
1408 by CUETools at "0"
1381 by CUETools at "8"
1075 by flac.exe version 1.3.1 at -0
1013 by flac.exe version 1.3.1 at -0
1013 by CUETools with the 1.3.3 .dll provided in this build (https://hydrogenaud.io/index.php?topic=118008.msg991626#msg991626).

Note, the latter .dll is much bigger.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Adil on 2021-03-28 20:07:11
Sorry for my 'abysmal' ignorance, but all that is state here is good or very bad?

I ask because I am fan of CUETools.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-28 21:51:54
Bad - but lossless. I can fix it by recompressing using flac.exe.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: korth on 2021-03-28 22:11:24
problem vanishes by replacing CUETools_2.1.8_HF1\CUETools_2.1.8\plugins\x64\libFLAC_dynamic.dll
So does replacing the dll fix the problem for you or not? You still have to re-compress?
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-28 22:26:49
does replacing the dll fix the problem for you or not?
It does.

But when answering someone who asks an unspecified "good or very bad" question, I certainly want to make the point that whatever CUETools did, is lossless. It might be bad compression-wise, but it is not so bad that it is dangerous to use.

Recompress what CUETools has converted, and all is fine.


Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: korth on 2021-03-28 22:38:09
The problem is apparently specific to certain chipsets (which is bad for you but not necessarily bad for everyone) and fixable in such cases by replacing the dll.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-28 23:17:44
in such cases
Of course it should be fixed not only "in such cases", but for everyone - by replacing the dll in the version offered for download, with one that doesn't malfunction. I have had that problem with two chipsets of which one was a not-too-ancient Dell laptop - I cannot be the only one and I wasn't (https://hydrogenaud.io/index.php?topic=117421.msg969822#msg969822).

I am not whining for myself, as it is not my problem anymore - all the rips that CUETools ever repaired for me (many, as I bought a lot of my CDs second-hand, scratches included) were duly tagged, so that was done overnight. But others who use it to convert a collection to/from img.flac+img.cue might have a job to do.

And to everyone else who might want to recompress: Stay off the "flac frontend" application, it is not safe (https://hydrogenaud.io/index.php?topic=99803.msg921798#msg921798). For novices to safely recompress flac files, use an application like e.g. foobar2000 to write a copy to a [different rootdirectory]\%directory%\%filename% and then, use https://www.foobar2000.org/components/view/foo_bitcompare to ensure that new and old are the same. Only then overwrite.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: c72578 on 2021-03-29 12:23:57
@Porcus  Thanks for reporting this issue.
libFlac will be updated in CUETools.
Currently, flac is at commit https://github.com/gchudov/flac/commit/45f7d41a8a87a743e79ee26143e8231e0edb5be5 in CUETools
whereby the build at Rarewares (https://www.rarewares.org/lossless.php) (flac_dll-1.3.3.git-ce6dd6b5) is using a newer version, at commit https://github.com/xiph/flac/commit/ce6dd6b5732e319ef60716d9cc9af6a836a4011a
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: c72578 on 2021-03-29 21:13:06
Please find an updated build of CUETools for testing in this PR:
https://github.com/gchudov/cuetools.net/pull/93
libFLAC has been updated to 1.3.3 git ce6dd6b, which is the same version as used in the current builds of libFLAC at RareWares.
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: Porcus on 2021-03-30 00:32:14
Please find an updated build of CUETools for testing in this PR:
https://github.com/gchudov/cuetools.net/pull/93
libFLAC has been updated to 1.3.3 git ce6dd6b, which is the same version as used in the current builds of libFLAC at RareWares.

Looks OK on this computer (https://hydrogenaud.io/index.php?topic=117421.msg969791#msg969791) (Intel H97)! All the following tests ended up with 1.000 when trying to recompress using version 1.3.1 flac.exe at same setting as the CUETools slider:
One high-bitrate album (black metal) at -0 and at -8
One lower-bitrate album (classical organ works, presumed near-mono) at -0, -8 and -3.
I have probably never even applied the "-3" before.

Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: NetRanger on 2021-03-30 06:55:07
Please find an updated build of CUETools for testing in this PR:
https://github.com/gchudov/cuetools.net/pull/93
libFLAC has been updated to 1.3.3 git ce6dd6b, which is the same version as used in the current builds of libFLAC at RareWares.


FLAC V1.3.3 git-27c61570 is now available over at RareWares
Title: Re: CUETools 2.1.8's flac produces abysmal compression ratio
Post by: c72578 on 2021-04-04 07:40:25
Thanks for the info about the new builds at RareWares. In the meantime, the pull request has been updated, using the same commit 27c6157 of xiph/flac. An new build for testing has been added to the PR: https://github.com/gchudov/cuetools.net/pull/93