Skip to main content

Topic: Merge FLAC files limited to max 2.00GB? (Read 26384 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Merge FLAC files limited to max 2.00GB?
I'm trying to merge 2 large FLAC files of about 1.5GB into 1, the resulting FLAC file should be about 3GB. However I get the following error:

Code: [Select]
1 out of 2 tracks converted with major problems.

Source: "C:\bigfile1.flac"
  Track converted successfully.
Source: "C:\bigfile2.flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters) : "C:\Image.flac"
  Additional information:
  Encoder stream format: 48000Hz / 2ch / 24bps
  Command line: "C:\Foobar2000\flac.exe" -s --ignore-chunk-sizes -8 - -o "Image.flac"
  Working folder: C:\
   Conversion failed: The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters

The resulting merged image.flac file is cut off at exactly 2.00GB. I've checked both files and they decode fine on their own, and I have the same problem with merging other big flac files too. Hard drive space is sufficient.

As I have many FLAC files >2.00GB its likely not a FLAC limitation but something hardcoded into Foobar2000. Is there any particular reason for this and if not, can this limit please be removed?  Or does someone have a better idea to merge these files?

(Using Win7 x64, Foobar2000 v1.1, C2D + 8GB RAM, NTFS)
  • Last Edit: 02 October, 2010, 01:43:28 PM by SoleBastard

  • Andreasvb
  • [*][*][*][*][*]
Merge FLAC files limited to max 2.00GB?
Reply #1
Seems to be a limitation in Windows.

See this post, http://www.hydrogenaudio.org/forums/index....st&p=483172
Windows 10 Pro x64 // foobar2000 1.3.10

Merge FLAC files limited to max 2.00GB?
Reply #2
Well, eac3to has no problem converting DTSHD or TrueHD files into >2GB FLAC files using the reference libFLAC.dll 1.2.1. Thats why I don't think either Windows or FLAC is the problem but rather Foobar2000 itself.
  • Last Edit: 02 October, 2010, 01:56:07 PM by SoleBastard

  • Andreasvb
  • [*][*][*][*][*]
Merge FLAC files limited to max 2.00GB?
Reply #3
Is the flac.exe the same version?
Windows 10 Pro x64 // foobar2000 1.3.10

Merge FLAC files limited to max 2.00GB?
Reply #4
If you mean if the flac.exe file used by Foobar2000 has the same version as the libFLAC.dll used by eac3to then yes, both are the latest 1.2.1.

  • Andreasvb
  • [*][*][*][*][*]
Merge FLAC files limited to max 2.00GB?
Reply #5
What if you try from cmd?
Then you know if it's foobar2000 or not.
Windows 10 Pro x64 // foobar2000 1.3.10

Merge FLAC files limited to max 2.00GB?
Reply #6
What do you mean 'from cmd'? I don't know how to do this using flac.exe as I can't seem to find the correct option in 'flac -help'

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Merge FLAC files limited to max 2.00GB?
Reply #7

Merge FLAC files limited to max 2.00GB?
Reply #8
Thanks for the link lvqcl (and I apologize for not finding the thread myself).

I just merged the same files using WavPack and this resulted in a nice 2.25GB file that plays fine. So I guess I was wrong by blaming Foobar2000 but its actually the windows build of FLAC 1.2.1. I guess I'll fire up my Ubuntu boot to try merge the FLACs in there.

However, that still doesn't explain why eac3to can make >2GB FLAC files in Windows without any problems. I'll ask the creator of that program how he manages to do that (see here).
  • Last Edit: 02 October, 2010, 03:56:09 PM by SoleBastard

  • Case
  • [*][*][*][*][*]
  • Developer (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #9
Here's FLAC 1.2.1 modified to work with file sizes larger than 2 GB. The executable is larger than the official binary because it's statically linked MSVC 2010 compile.

Edit: removed attachment.
  • Last Edit: 03 October, 2010, 02:56:52 AM by Case

Merge FLAC files limited to max 2.00GB?
Reply #10
Thanks Case, I tried using your flac.exe with Foobar2000 v1.1. However, when the temp file is slightly above 2GB it gives a similar error code as above. To make sure the correct flac.exe is used by Foobar I created a custom convert profile in Foobar and pointed it at your flac.exe. I guess there are some more problems than just the flac.exe file.

As a temporary work-around I merged to big FLAC files into 1 .w64 file with foobar2000. I then converted this .w64 file to flac using eac3to 3.24. Awkward, but it works.
  • Last Edit: 02 October, 2010, 06:00:00 PM by SoleBastard

  • Squeller
  • [*][*][*][*][*]
Merge FLAC files limited to max 2.00GB?
Reply #11
I'm also interested in a solution. 

libFLAC uses the C stdlib for file i/o.  even on my XP box with VS 2005, microsoft's stdlib implementation is still limited to 2 GB (i.e. no 64-bit off_t).

I'm reluctant to add win-specific calls to libFLAC just because MS is intentionally sabotaging portability.  every other build of flac works with large files.

(source)

  • kode54
  • [*][*][*][*][*]
  • Administrator
Merge FLAC files limited to max 2.00GB?
Reply #12
Microsoft's C standard library has extensions for 64-bit file offsets as of some revision from years ago.

  • Case
  • [*][*][*][*][*]
  • Developer (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #13
Sorry about that. I didn't test piped encoding for larger than 2 GB files. Input file size check broke when "(long)(-1)" got changed to "(__int64)(-1)". Fixed version attached.

The fix involves defining fseeko as _fseeki64 and ftello as _ftelli64 and changing off_t to __int64.

Edit: attachment removed. More off_t to __int64 changes required. Also stat to _stati64 and fstat to _fstat64.
  • Last Edit: 03 October, 2010, 04:58:54 AM by Case

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Merge FLAC files limited to max 2.00GB?
Reply #14
Sorry, output FLAC file is still limited by 2,281,813,291 bytes.

  • Case
  • [*][*][*][*][*]
  • Developer (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #15
It worked for me in a quick test but I should have learned from the first mistake not to be too hasty. If source file was too large the encoded file had no md5 checksum or length info. My original diagnosis of the problem was incorrect.
Now all off_t types are changed to __int64 and remaining 32 bit stuff (stat and fstat) changed to 64 bit versions (_stati64, _fstati64). I can convert 5.61 GB WAV file to ~3 GB FLAC file fine in foobar with pipes but WAV file reader seems to have some issues. Encoding the wav directly from command line reports some incomplete sample error at the end and file lengths don't match in foobar's binary comparator.

Edit: removed attachment.
  • Last Edit: 03 October, 2010, 07:47:33 AM by Case

Merge FLAC files limited to max 2.00GB?
Reply #16
Sorry, still doesn't work, same non-descriptive error as above.

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Merge FLAC files limited to max 2.00GB?
Reply #17
flac - --ignore-chunk-sizes -o a.flac < image.wav    =>

-: WARNING: skipping unknown sub-chunk 'ДвЧв' (use --keep-foreign-metadata to keep)
-: ERROR during read while skipping over unsupported sub-chunk
  • Last Edit: 03 October, 2010, 06:01:56 AM by lvqcl

  • Case
  • [*][*][*][*][*]
  • Developer (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #18
I could dublicate SoleBastard's issue immediately with 24 bit source files and went digging into FLAC's WAV reader. It was designed to only handle 4 GB source files. Attached version no longer has that limit.
[ Specified attachment is not available ]

Merge FLAC files limited to max 2.00GB?
Reply #19
Yay, this one seems to work fine!  I'll double-check if everything is encoded correctly later on but thanks a lot Case!

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Merge FLAC files limited to max 2.00GB?
Reply #20
I just merged the same files using WavPack and this resulted in a nice 2.25GB file that plays fine.


It's interesting that foobar2000 shows incorrect bitrate of very big wavpack files (> 4 GB?). Winamp and wvunpack -c show the same incorrect bitrate, too.

Merge FLAC files limited to max 2.00GB?
Reply #21
FLAC file seems 100%, excellent work Case!

It's interesting that foobar2000 shows incorrect bitrate of very big wavpack files (> 4 GB?). Winamp and wvunpack -c show the same incorrect bitrate, too.


Can confirm here with a 5.3 GB file and the latest WavPack, bitrate is way off (too low).

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #22
Can confirm here with a 5.3 GB file and the latest WavPack, bitrate is way off (too low).

It's a known problem that WavPack cannot handle files over 4 GB. It was discussed here in detail (and with some workarounds). Note that this a problem with the decoding library only; there is nothing limiting in the file format (and the files are fine).

I basically got this working last summer (along with handling .CAF files) but I still have not gotten a chance to finish it up and release it, but am hopeful for this year.

David

  • Squeller
  • [*][*][*][*][*]
Merge FLAC files limited to max 2.00GB?
Reply #23
Thanks Case, I'll test it soon. I'll merge some single files albums to flacs>2GB.

  • seanyseansean
  • [*][*][*][*]
  • Members (Donating)
Merge FLAC files limited to max 2.00GB?
Reply #24
I could dublicate SoleBastard's issue immediately with 24 bit source files and went digging into FLAC's WAV reader. It was designed to only handle 4 GB source files. Attached version no longer has that limit.
(Attachment Link)


Thank you, Case. And nice to see an old name around here too!