Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares (Read 895 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Hi all,
A really weird bug I encountered while compressing some wavs to flac: On some small files in particular, the encoder appears to freeze for several minutes; the command prompt shows the typical Flac.exe welcome message but no compression progress is shown. During this time Flac.exe's memory usage climbs up to 4 gb at times! And when it finally encodes the file, the output is ridiculously huge (300 kb wav becomes 1 or 2 gb flac!) So yeah, there's definitely a problem. I guess the silver lining is the file doesn't appear broken, just bloated beyond belief. X86 or X64 builds don't seem to make a  difference. Not sure what else I should be testing or what other info is needed about my system etc.

I've attached a troublesome  wav file. If I compress this with -6 or more, it breaks. Can someone confirm this?

I happened to have an old copy of Flac 1.3.2 lying around and it seems to be fine. Pretty sure I got that from rarewares too.

Edit: Just downloaded the newest 1.3.3 from this thread, and that seems to be fine at the moment. Will have to test it more to be sure.
Edit 2: Hmm seems to be doing the same thing but on different files.
For now I'll downgrade to 1.3.2 but I hope to figure this out...

Any advice?
Thanks!

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #1
I've attached a troublesome  wav file. If I compress this with -6 or more, it breaks. Can someone confirm this?
confirmed

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #2
I'll compile and upload latest git for you to check against. I'll post here when it's available.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #3
New compiles now at rarewares. :)

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #4
The new 32-bit compile still does not work. I tried removing all metadata from the wav, no improvement. After leaking memory encode fails:

test-.wav: ERROR during encoding
state = FLAC__STREAM_ENCODER_FRAMING_ERROR

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #5
The new 64-bit compile with -8 produced a 1GB File with the Test.flac but no error message.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #6
I confirm that both 32bit and 64bit compiles fail on the test.wav, above. Similarly NetRanger's GCC 32bit and 64bit compiles of the same git generation also fail, so this is a FLAC issue. The last version I have that encodes this correctly is flac-v1.3.3.git-ce6dd6b, so I shall revert to this on rarewares.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #7
I can confirm it also.

But i tested to convert the 'test.wav' file to 16-bit instead of 24-bit and then it converted perfectly, no issues at all. Wonder why FLAC reacts like it do with that 24-bit wav file when converting it to 24-bit FLAC.

I have never stumbled over any files b4 that FLAC have had issues with.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #8
Huh, glad I'm not going crazy then!

Thanks john33 for updating the version on Rarewares. I'll test it at some point soon and run it on other similar files ot make sure it is truly stable.

I wonder what's actually going wrong though?

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #9
It fails with block size of 4096 (flac -8 -b 4096), other block sizes works fine. Hmm, weird...
flac -8 | lame -V2

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #10
Seems to be the combination of block size 4096 with partial_tukey(2)? Even changing the "(2)" resolves it.

Started out with  -l 12 -b 4096 -m -r 6 -A tukey(0.5);partial_tukey(2);punchout_tukey(3) as that is "-8", and fiddled a bit.
Removing partial_tukey(2) resolved it. Tweaked -l even up to 32 and -r even up to 15 (with --lax). Sure, no prob. Including partial_tukey(n) for n=1,3,4,5,6 is not an issue. Even including the whole load of other functions and partial_tukey(otherthantwo) in the end ... OK. Worked my way to
--lax -l 32 -b 4096  -r 15 -m  -A bartlett;bartlett_hann;blackman;blackman_harris_4term_92db;connes;flattop;gauss(0.2);hamming;hann;kaiser_bessel;nuttall;rectangle;triangle;tukey(0.5);punchout_tukey(2);partial_tukey(4) -p -e
which took some seconds, but sure, no prob. (Got down to 112973 by using arguments gauss(0.1) and tukey(0.2) and punchout_tukey(5) and partial_tukey(6).)

But  -l 2 -b 4096  -r 1   -A partial_tukey(2) is A Bad Thing.



Version tried: the Win64 of the download from https://hydrogenaud.io/index.php?topic=118008.msg1001140#msg1001140
High Voltage socket-nose-avatar

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #11
Nice find! I took a quick look and it seems this sample triggers a problem that I've come across a while ago: http://lists.xiph.org/pipermail/flac-dev/2020-July/006448.html However, I thought this would only occur with 'hacked' code, but you found a sample that triggers this on current git as well. It seems there is some underlying problem here too.

After bisecting this, I found that one of my own commits is the culprit: https://github.com/xiph/flac/commit/ae288c067c03bc3404eff67063958bfe854f5a01 I'm not sure how this change causes the problem, but I will definitely dive into it.

Forget that last bit, that only changes the compression ratio display. With that fix reverted the file is still bloated, but FLAC doesn't display it as such. It is the commit before that: https://github.com/xiph/flac/commit/ced7f6829d14e38128bf0ba66412cc0541246c46 Apparently this commit is wrong and it actually worsens the problem it was supposed to fix....
Music: sounds arranged such that they construct feelings.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #12
Here's a pull request that should fix this problem:

https://github.com/xiph/flac/pull/251
Music: sounds arranged such that they construct feelings.

Re: horrendous file bloating when using Flac 1.3.3 binaries from Rarewares

Reply #13
Good news! I'll recompile once it becomes available.

 

 
SimplePortal 1.0.0 RC1 © 2008-2021