FLAC v1.3.3 (August 4th, 2019)
https://xiph.org/flac/changelog.html
https://xiph.org/flac/download.html
No Official Windows binaries available at this time.
FLAC v1.3.3
Built on August 04, 2019, GCC 9.1.0
Latest commit included : f764434
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
Thanks NR :D
In case anyone's wondering, this release does not contain any improvements in compression (or at least not at --best), so there's no utility in re-encoding your existing 1.3.x files.
I am, however, pleased that it should no longer spam my console output when I specify -f --no-error-on-compression-fail.
I did a VS 2017 compile done from the source of the official 1.3.3 Xiph release.
I don't do these things often atm. It would be nice someone can check it.
It should be a bit faster against a GCC version.
I did a VS 2017 compile done from the source of the official 1.3.3 Xiph release.
I don't do these things often atm. It would be nice someone can check it.
It should be a bit faster against a GCC version.
Works OK on Windows 7 32 bit and core i3 3245. Indeed a bit faster than build from NetRanger - ~156x vs ~148x
Wombat's build is indeed a bit faster compared to NetRanger's build.
Same file encoded at level 8
Wombat's build: 174,91x
NetRanger's build: 156.85x
BTW, using x64
Thanks for testing how it works in w32 and x64!
This compile i only did because i see no sign of official 1.3.3 Windows binaries and some may also like these.
In no way i want to compete with Netranger and his nice effort to always deliver recent GIT builds. Cheers!
Can I know what's the difference between NR binaries and those at Rarewarez? The latter seem be bigger.
Made with different compliers. John33 use ICL for his compiles.
Is there going to be an official FLAC 1.3.3 version that gets released on the official site?
v1.3.4 is around the corner so to say.
Verion 1.3.3 was borked because the version number in CMakeLists.txt did
not get updated.
https://github.com/xiph/flac/pull/119
https://github.com/xiph/flac/pull/119
Thanks for the info.
The compiled versions report as libFLAC 1.3.3 20190804 and one day later as libFLAC 1.3.4 20190805
I wonder why 1.3.4 didn't make it to any official xiph page yet?
'erikd' haven't pushed it yet on Github.
https://github.com/xiph/flac/pull/119
Thanks for the info.
The compiled versions report as libFLAC 1.3.3 20190804 and one day later as libFLAC 1.3.4 20190805
I wonder why 1.3.4 didn't make it to any official xiph page yet?
https://github.com/xiph/flac/issues/111#issuecomment-521431000
Verion 1.3.3 was borked because the version number in CMakeLists.txt did
not get updated.
So, Erik cares about typo in CMakeLists.txt, but he doesn't care about fact that offical site still hosts 32-bit binary of 1.3.2 for windows with bug (that was fixed in git more than 2 years ago) that prevents encoding on 32-bit system if size of source PCM stream is larger than 4GB?
Erik has already made it quite clear that he couldn't care any less about proprietary ("dead") operating systems.
BTW, FLAС still needs --channel-map=none to encode some non-standard channels configurations (e.g. 4.1) and still --channel-map=none is not documented.
I discovered some 24-bit WAV files that are NOT decoded properly after encoding-decoding using the flac 1.3.3 Windows 64-bit binaries at rarewares. Specifically the fmt chunk is messed up and does not contain the correct information. Unfortunately they are wav files I cannot release to the public. Flac 1.3.2 from xiph works correctly.
More specifically, the original wav header looks like this:
52 49 46 46 86 8C 80 02 57 41 56 45 RIFF†Œ€.WAVE
But after encoding and decoding with flac 1.3.3 it looks like this:
52 49 46 46 9E 8C 80 02 57 41 56 45 RIFFžŒ€.WAVE
The fmt chunk in the original wav is:
66 6D 74 20 10 00 00 00 01 00 02 00 fmt ........
but in the encoded/decoded wav it's:
66 6D 74 20 28 00 00 00 FE FF 02 00 fmt (...þÿ..
It looks like flac is "fixing" the wav file to be a correct 24-bit PCM file with the 0xFFFE flag, but I don't know if "fixing" files is appropriate or not.
I believe this was intentional.
IIRC, FLAC is not storing the "fmt" header itself, but reconstructs it instead, and in this commit from 2017, it was modified to use WAVEFORMATEXTENSIBLE for bits different than 8 or 16
https://git.xiph.org/?p=flac.git;a=commit;h=bb750734287a4079ca3de9ff85c71cc62160ac46
So it all depends on what kind of compatibility do you need when decoding a flac file.
I wouldn't have even noticed except that when batch converting some files I was double checking the conversion by running a SHA256 sum on the original wav and comparing it against a SHA256 sum on the decoded flac file. I don't mind if flac is going to fix files, but it will make it more difficult to make sure decoded PCM is bit-for-bit identical to the original. I'm not the type who can just "trust" that it is. I suppose I can just compare the data chunk, but that's a lot more difficult from a batch file than comparing the entire file.
FLAC v1.3.3-Git-2019-09-15
Built on September 18, 2019, GCC 9.2.0
Latest commit included : 5598543
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
FLAC v1.3.3-Git-2019-10-07
Built on October 08, 2019, GCC 9.2.0
Latest commit included : 2e7931c
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
You can compile libFLAC_dynamic.dll? Thanks
You can compile libFLAC_dynamic.dll? Thanks
Can't do that, the 'media auto-build suite' doesn't offer any such option.
FLAC v1.3.3-Git-2019-10-10
Built on October 10, 2019, GCC 9.2.0
Latest commit included : 952d511
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
You can compile libFLAC_dynamic.dll? Thanks
This is a x64 build using MSVC 2017 (v141) SDK version 10.0.17763.0 from the 2019-10-06 commit (2e7931c2). Tried using the latest commit but it breaks on windows_unicode_filenames.c
Please backup the current dll you're using in case this doesn't work for you
LibFLAC_dynamic x64
MSVC 2019 (v142) SDK version 10.0.18362.0 from the 2019-10-06 commit b84ff55 (2019-10-23)
FLAC v1.3.3-Git-2019-10-23
Built on November 03, 2019, GCC 9.2.0
Latest commit included : b84ff55
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.18362.0
commit b84ff55 (2019-10-23)
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.18362.0
commit 2907d49 (2019-11-11)
FLAC v1.3.3-Git-2019-11-15
Built on November 16, 2019, GCC 9.2.0
Latest commit included : cdcf0d5
https://xiph.org/flac/
https://git.xiph.org/?p=flac.git;a=summary
https://github.com/xiph/flac/commits/master
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.18362.0
commit ed1b67b (2019-11-18)
FLAC v1.3.3-Git-2019-11-19
Built on November 19, 2019, GCC 9.2.0
Latest commit included : 3bb5d8c
https://xiph.org/flac/
https://git.xiph.org/?p=flac.git;a=summary
https://github.com/xiph/flac/commits/master
Date: 2019-11-20
Commit: f706f28
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Commit d518e13a1f (2019-11-21)
We're waiting NetRanger. :)
Date: 2019-11-21
Commit: d518e13
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Date: 2019-11-24
Commit: b02e159
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Date: 2019-11-30
Commit: 6455e47
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Date: 2019-12-09
Commit: a9d9f4d
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
FLAC v1.3.3-Git-2019-12-08
Built on December 10, 2019, GCC 9.2.0
Latest commit included : a3d8927
https://xiph.org/flac/
https://git.xiph.org/?p=flac.git;a=summary
https://github.com/xiph/flac/commits/master
FLAC v1.3.3-Git-2019-12-22
Built on December 23, 2019, GCC 9.2.0
Latest commit included : 0dfe235
https://xiph.org/flac/
https://git.xiph.org/?p=flac.git;a=summary
https://github.com/xiph/flac/commits/master
Date: 2019-12-27
Commit: cffe389
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Date: 2019-12-27
Commit: cffe389
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
does there exist amd optimized flac (or ogg vorbis) builds somewhere? i would like to see them performing in contrast to the builds offered at rarewares or in this thread.
Date: 2020-03-06
Commit: c8fca6b
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
Date: 2020-03-06
Commit: c8fca6b
Compiler: MSVC 2019 (v142) SDK version 10.0.18362.0
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
virustotal reports ten positives
see https://www.virustotal.com/gui/file/54e596b27c2b631617521cb3e425b117647e71ac5f083237990531842d21ebe9/detection
Of course it does. That's what it's good at doing. Scaring people. Especially people who use MSVC >2015. Better report that false positive to Microsoft, and all the others will just go away with a rescan.
metaflac.exe in flac-win-1.3.3-c8fca6b-2020-03-06\Win32 triggers a false positive
FLAC v1.3.3-Git-2020-03-06
Built on March 08, 2020, GCC 9.2.0
Latest commit included : 1d5299d
https://xiph.org/flac/
https://git.xiph.org/?p=flac.git;a=summary
https://github.com/xiph/flac/commits/master
1.3.3 was released over 6 months ago and never even made the news on the homepage, that's a little sad IMHO
https://xiph.org/flac/news.html
FLAC 1d5299d (https://github.com/xiph/flac/tree/1d5299d67b4f4d8172b2fab702696bce1082b91a) MSYS2/mingw-w64-x86_64 2020-03-23 (64bit, GCC 9.3)
Tuned for modern Intel CPUs and linked against similarly built libogg 1.3.4.
Simple recompression benchmark on an AMD 2700X:
This build:
./timer64.exe ./flac.exe --verify --best --exhaustive-model-search --qlp-coeff-precision-search ./test.flac -o test_grieverv.flac
test.flac: Verify OK, wrote 377669674 bytes, ratio=0.992
Kernel Time = 1.171 = 0%
User Time = 335.281 = 99%
Process Time = 336.453 = 99% Virtual Memory = 15 MB
Global Time = 337.578 = 100% Physical Memory = 14 MB
NetRanger's build:
./timer64.exe ./flac.exe --verify --best --exhaustive-model-search --qlp-coeff-precision-search ./test.flac -o test_netranger.flac
test.flac: Verify OK, wrote 377669674 bytes, ratio=0.992
Kernel Time = 1.531 = 0%
User Time = 409.140 = 99%
Process Time = 410.671 = 99% Virtual Memory = 15 MB
Global Time = 411.926 = 100% Physical Memory = 14 MB
Build information:
Environment variables:
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1 -D_FORTIFY_SOURCE=0"
CFLAGS="-march=x86-64 -O3 -fno-plt -fno-semantic-interposition -fno-stack-protector -falign-functions=32 -flto -ffat-lto-objects -mtune=skylake -pipe"
CXXFLAGS="-march=x86-64 -O3 -fno-plt -fno-semantic-interposition -fno-stack-protector -falign-functions=32 -flto -ffat-lto-objects -mtune=skylake -pipe"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed -pipe"
FLAC configuration:
LDFLAGS+=" -Wl,-Bstatic"
./configure \
--disable-shared \
--enable-static \
--disable-xmms-plugin \
--disable-rpath \
--prefix=${MINGW_PREFIX} \
--build=${MINGW_CHOST} \
--host=${MINGW_CHOST} \
--target=${MINGW_CHOST}
FLAC 1d5299d (https://github.com/xiph/flac/tree/1d5299d67b4f4d8172b2fab702696bce1082b91a) MSYS2/mingw-w64-x86_64 2020-03-25 (64bit, GCC 9.3)
Rebuilt with more aggressive GCC optimizations taken from GentooLTO (https://github.com/InBetweenNames/gentooLTO).
Compared to the previous build, it's around 4.2% faster on the previously tested file which saved another ~14 seconds.
Build information:
Environment variables:
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1 -D_FORTIFY_SOURCE=0"
CFLAGS="-march=x86-64 -O3 -fno-common -fno-plt -fno-semantic-interposition -fno-stack-protector -fno-math-errno -fno-trapping-math -falign-functions=32 -fdevirtualize-at-ltrans -fgraphite-identity -floop-nest-optimize -fipa-pta -flto -ffat-lto-objects -mtune=skylake -pipe"
CXXFLAGS="-march=x86-64 -O3 -fno-common -fno-plt -fno-semantic-interposition -fno-stack-protector -fno-math-errno -fno-trapping-math -falign-functions=32 -fdevirtualize-at-ltrans -fgraphite-identity -floop-nest-optimize -fipa-pta -flto -ffat-lto-objects -mtune=skylake -pipe"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed -pipe"
FLAC configuration:
LDFLAGS+=" -Wl,-Bstatic"
./configure \
--disable-shared \
--enable-static \
--disable-xmms-plugin \
--disable-rpath \
--prefix=${MINGW_PREFIX} \
--build=${MINGW_CHOST} \
--host=${MINGW_CHOST} \
--target=${MINGW_CHOST} \
--disable-stack-smash-protection
Speedup is great, from average 125x with Netranger's on my i7-2600 to 145x with GrieverV's build here.
On my i7-6700, Case's build is still the fastest...
https://hydrogenaud.io/index.php?topic=115900.msg974249#msg974249
His build just doesn't work on my computer. It starts, shows help, but when it needs to encode, it just... exits.
His build just doesn't work on my computer. It starts, shows help, but when it needs to encode, it just... exits.
-march=haswell
Means that the binary has been fully optimized for the Haswell CPU family and is unlikely to run on any other family unless they have full support for the instruction sets GCC enables for the Haswell family. Fortunately, I believe all current Intel CPUs since Haswell support the Haswell optimized code generated by GCC.
@GrieverV Just tested your latest compile against Case's here on my i7-6700 with my set of 40 WAV files (1.77 GB).
All reading/writing is from/to SSD, but since I do at least 5 runs, the WAVs are likely to be cached.
flac_Griever_1d5299d.exe
Kernel Time = 2.203 = 5%
User Time = 38.453 = 91%
Process Time = 40.656 = 96% Virtual Memory = 14 MB
Global Time = 42.164 = 100% Physical Memory = 14 MB
Case´s build:
Kernel Time = 2.562 = 6%
User Time = 35.390 = 90%
Process Time = 37.953 = 96% Virtual Memory = 14 MB
Global Time = 39.238 = 100% Physical Memory = 14 MB
I have no idea what causes this difference (platform specific compile, compiler version used, compiler optimisations ...)
What I noticed is that your binary uses way less "kernel time" and more "user time" than Case's. But interpreting these figures is way beyond my knowledge...
@sundance Their toolchain is likely very different so there could be a lot of possible explanations, but the most likely would be -march=haswell and GCC 4.9.
If I remember correctly, there was a big performance regression on GCC 5 with FLAC encoding and it's possible GCC never recovered it or further regressed. I've attached a build that was built with -march=haswell and -funroll-loops if you want to try it. As far as I could tell, there wasn't much difference with -funroll-loops alone when I tested.
Getting pretty close now:
flac_GrieverV_1d5299d_haswell.exe
Kernel Time = 2.390 = 5%
User Time = 36.468 = 90%
Process Time = 38.859 = 96% Virtual Memory = 14 MB
Global Time = 40.137 = 100% Physical Memory = 14 MB
did some testing of my own with GrieverV's aggressively optimized Skylake 1d5299d 1.3.3 flac.exe. I don't use -ep so I left it out and compared it to my up until recently used flac 1.3.2 (official xiph generic windows binary, flac-1.3.2-win.zip). SSD to SSD, i7-7700K, it's 25 % faster! (I know, apples and oranges, but I switched from this 1.3.2 to GrieverV's 1.3.3 just now and I'm gonna stick with it, so this is my real life improvement)
PS C:\Program Files (x86)\foobar2000\Encoders\skylakeflac> Measure-Command { ..\flac.exe --verify --best E:\flac_reenc_AIO\Image.wav -o E:\flac_reenc_AIO\Image_new.flac }
flac 1.3.2
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
Image.wav: Verify OK, wrote 2816137583 bytes, ratio=0,789
Days : 0
Hours : 0
Minutes : 1
Seconds : 50
Milliseconds : 85
Ticks : 1100852308
TotalDays : 0,00127413461574074
TotalHours : 0,0305792307777778
TotalMinutes : 1,83475384666667
TotalSeconds : 110,0852308
TotalMilliseconds : 110085,2308
PS C:\Program Files (x86)\foobar2000\Encoders\skylakeflac> Measure-Command { .\flac_1d5299d_skylake_2020-03-25.exe --verify --best E:\flac_reenc_AIO\Image.wav -o E:\flac_reenc_AIO\Image_new.flac }
flac 1.3.3
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
Image.wav: Verify OK, wrote 2816138302 bytes, ratio=0,789
Days : 0
Hours : 0
Minutes : 1
Seconds : 23
Milliseconds : 653
Ticks : 836538482
TotalDays : 0,000968215835648148
TotalHours : 0,0232371800555556
TotalMinutes : 1,39423080333333
TotalSeconds : 83,6538482
TotalMilliseconds : 83653,8482
wow, this is really impressive. seems like there is always room for further optimizations.
flac2flac conversion of 25 tracks in foobar via wine on i5-ivy.
netranger compile:
Track converted successfully.
Total encoding time: 0:22.661, 239.69x realtime
grieverv compile:
Track converted successfully.
Total encoding time: 0:17.350, 313.07x realtime
thanks a lot @grieverv.
FLAC v1.3.3-Git-2020-04-06
Built on April 07, 2020, GCC 9.3.0
Latest commit included : dd975a9
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://git.xiph.org/?p=flac.git;a=summary
FLAC 7a35c52 (https://github.com/xiph/flac/tree/7a35c528495b6d8a51e87e007f6810b5553101bc) MSYS2/mingw-w64-x86_64 2020-04-07 (64bit, GCC 9.3)
Aggressively optimized builds tuned for modern processors. Included is a generic x86_64 build and an Intel Haswell optimized build that will only run on Haswell or newer Intel CPUs and Zen or newer AMD CPUs. All FLAC tests passed for both builds.
Compared to my previous builds, the generic build is now built with -funroll-loops and FLAC is now configured with --enable-64-bit-words.
Build information:
Environment variables:
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1 -D_FORTIFY_SOURCE=0"
CFLAGS="-march=x86-64 -O3 -fno-common -fno-plt -fno-semantic-interposition -fno-stack-protector -fno-math-errno -fno-trapping-math -falign-functions=32 -fdevirtualize-at-ltrans -fgraphite-identity -floop-nest-optimize -fipa-pta -flto -ffat-lto-objects -funroll-loops -m64 -mtune=skylake -pipe"
CXXFLAGS="$CFLAGS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed ${CFLAGS}"
FLAC configuration:
./configure \
--disable-shared \
--enable-static \
--disable-xmms-plugin \
--disable-rpath \
--prefix=${MINGW_PREFIX} \
--build=${MINGW_CHOST} \
--host=${MINGW_CHOST} \
--target=${MINGW_CHOST} \
--enable-64-bit-words \
--disable-stack-smash-protection
This is likely to be my last build so I've included my makepkg configuration and PKGBUILDs for libogg and flac.
no changes in speed with latest netranger build - but grievervs build got tamed from 313x to 264x.
Track converted successfully.
Total encoding time: 0:20.609, 263.56x realtime
same setup as before.
@GrieverV:
Well done, mate! This time your Haswell compile is the fastest I ever tested!
These are my test results (under today's full moon conditions ;) )
flac133_ GrieverV_7a35c528.exe:
Kernel Time = 2.921 = 7%
User Time = 35.781 = 89%
Process Time = 38.703 = 96% Virtual Memory = 14 MB
Global Time = 40.034 = 100% Physical Memory = 14 MB
flac133_ GrieverV_7a35c528_haswell.exe:
Kernel Time = 2.359 = 6%
User Time = 35.359 = 90%
Process Time = 37.718 = 96% Virtual Memory = 14 MB
Global Time = 39.087 = 100% Physical Memory = 14 MB
flac133_Case.exe:
Kernel Time = 2.812 = 7%
User Time = 35.109 = 89%
Process Time = 37.921 = 96% Virtual Memory = 14 MB
Global Time = 39.223 = 100% Physical Memory = 14 MB
@sanskrit44: What CPU are you running, and which of GrieverV's compiles did you test?
@sanskrit44: What CPU are you running, and which of GrieverV's compiles did you test?
it is a i5-3230m running the non-haswell build.
I see consistent gains on a 2700X with this test file: BIS1447-002-flac_16.flac (https://web.archive.org/web/20200408091726/https://www.eclassical.com/custom/eclassical/files/BIS1447-002-flac_16.flac)
--best: 4.5%
--verify --best: 5.8%
--verify --best --qlp-coeff-precision-search: 2.5%
--verify --best --exhaustive-model-search --qlp-coeff-precision-search: 1.2%
The builds I tested were generic x86_64 from 2020-03-25 (https://hydrogenaud.io/index.php?topic=118008.msg981392#msg981392) and 2020-04-07 (https://hydrogenaud.io/index.php?topic=118008.msg981823#msg981823). The differences between the two are just -funroll-loop and enabling 64 bit words for FLAC (upstream commits do not have any effect).
@sanskrit44 Were your tests perhaps done at different times? Background processes can change the results and each build should be tested at least three times to ensure the results are consistent.
Are you able to reproduce similar results with different FLAC files or the one linked above?
Are you able to reproduce similar results with different FLAC files or the one linked above?
i retested flac2flac with foobar and booted into a fresh system for each run.
mingw-w64-x86_64-flac-git-1.3.3.r3906.1d5299d6:
my set of 25 flacs:
Total encoding time: 0:17.488, 310.60x realtime
25x flac testfile:
Total encoding time: 0:32.065, 326.21x realtime
mingw-w64-x86_64-flac-git-1.3.3.r3909.7a35c528:
my set of 25 flacs:
Total encoding time: 0:16.978, 319.93x realtime
25x flac testfile:
Total encoding time: 0:30.801, 339.59x realtime
looking much better now :)
why are those called haswell builds when they are compiled with -mtune=skylake?
FLAC 1d5299d MSYS2/mingw-w64-x86_64 2020-03-25 (64bit, GCC 9.3)
PS C:\Program Files (x86)\foobar2000\Encoders> Measure-Command { .\flac.exe --verify --best E:\Musik_reenc_AIO\Image.w64 -o E:\Musik_reenc_AIO\imageout.flac }
flac 1.3.3
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
Image.w64: Verify OK, wrote 4116685875 bytes, ratio=0,779
Days : 0
Hours : 0
Minutes : 2
Seconds : 16
Milliseconds : 485
Ticks : 1364857502
TotalDays : 0,00157969618287037
TotalHours : 0,0379127083888889
TotalMinutes : 2,27476250333333
TotalSeconds : 136,4857502
TotalMilliseconds : 136485,7502
FLAC 7a35c52 MSYS2/mingw-w64-x86_64 2020-04-07 (64bit, GCC 9.3) haswell/skylake on i7-7700K
PS C:\Program Files (x86)\foobar2000\Encoders> Measure-Command { .\flacneu\flac.exe --verify --best E:\Musik_reenc_AIO\Image.w64 -o E:\Musik_reenc_AIO\imageout2.flac }
flac 1.3.3
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
Image.w64: Verify OK, wrote 4116686510 bytes, ratio=0,779
Days : 0
Hours : 0
Minutes : 1
Seconds : 58
Milliseconds : 800
Ticks : 1188007802
TotalDays : 0,00137500903009259
TotalHours : 0,0330002167222222
TotalMinutes : 1,98001300333333
TotalSeconds : 118,8007802
TotalMilliseconds : 118800,7802
why are those called haswell builds when they are compiled with -mtune=skylake?
I only included the generic builds flags in the build info since all that's done for Haswell is changing march and removing mtune. The binary is still built for Haswell and the relevant makepkg configuration is included in the download.
You can think of -
mtune=skylake as a more modern default for x86_64 CPUs since every Intel CPU from the last decade will likely show benefits over generic/x86_64; AMD Zen also shows improvements and I wouldn't be surprised if bulldozer benefits, too.
Size difference?
Size difference?
Executables are larger due to enabling loop unrolling, FLAC compression is different due to -march=haswell.
I was able to narrow the FLAC compression difference down to the -fma flag. AVX+AVX2 is fine and FLAC built with -fma and configured with --disable-avx --disable-asm-optimizations has the same difference so it's probably GCC/FMA's fault.
Yeah I meant compression difference. Was surprised as FLAC does integer math only doesn't it? It made me wonder how recompile could make the resulting files differ...
AFAIU, decoding is integer only but libFLAC uses floating point unless FLAC__INTEGER_ONLY_LIBRARY (https://github.com/xiph/flac/blob/c6318e9dd3f7a91f40340911bcf57bf36768910e/src/libFLAC/include/private/float.h#L42) macro is defined.
For those watching this thread, I'll probably be releasing new builds soon with FMA disabled so compression is reproducible with other builds (confirmed MSVC reproduces same compression). I also noticed I had -D__USE_MINGW_ANSI_STDIO=1 disabled for the previous builds, although I'll leave that disabled since it incurs a very minor performance penalty that shouldn't be necessary for ogg/FLAC.
I'll have to build, test, package and benchmark before I release, however I did see ~4-5% gain on a 2700X --verify --best after disabling FMA.
FLAC 7a35c52 (https://github.com/xiph/flac/tree/7a35c528495b6d8a51e87e007f6810b5553101bc) MSYS2/mingw-w64-x86_64 2020-04-12 (64bit, GCC 9.3)
Aggressively optimized builds tuned for modern processors. Included is a generic x86_64 build and an Intel Haswell optimized build that will only run on Haswell or newer Intel CPUs and Zen or newer AMD CPUs. All FLAC tests passed for both builds.
Changes:
- Rebuilt against a newer Mingw-w64 git snapshot
- Haswell binaries are now built without FMA support which should allow reproducible compression sizes with other builds
The new Haswell build is 8.1% faster than the previous Haswell build and 5% faster than the generic build on a 2700X (--verify --best) with this test file: BIS1447-002-flac_16.flac (https://web.archive.org/web/20200408091726/https://www.eclassical.com/custom/eclassical/files/BIS1447-002-flac_16.flac)
Build information:
Environment variables for generic:
CPPFLAGS="-D_FORTIFY_SOURCE=0"
CFLAGS="-march=x86-64 -O3 -fno-common -fno-plt -fno-semantic-interposition -fno-stack-protector -fno-math-errno -fno-trapping-math -falign-functions=32 -fdevirtualize-at-ltrans -fgraphite-identity -floop-nest-optimize -fipa-pta -flto -ffat-lto-objects -funroll-loops -m64 -mtune=skylake -pipe"
CXXFLAGS="$CFLAGS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed ${CFLAGS}"
Environment variables for Haswell:
CPPFLAGS="-D_FORTIFY_SOURCE=0"
CFLAGS="-march=haswell -O3 -fno-common -fno-plt -fno-semantic-interposition -fno-stack-protector -fno-math-errno -fno-trapping-math -falign-functions=32 -fdevirtualize-at-ltrans -fgraphite-identity -floop-nest-optimize -fipa-pta -flto -ffat-lto-objects -funroll-loops -m64 -mno-fma -pipe"
CXXFLAGS="$CFLAGS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed ${CFLAGS}"
FLAC configuration:
./configure \
--disable-shared \
--enable-static \
--disable-xmms-plugin \
--disable-rpath \
--prefix=${MINGW_PREFIX} \
--build=${MINGW_CHOST} \
--host=${MINGW_CHOST} \
--target=${MINGW_CHOST} \
--enable-64-bit-words \
--disable-stack-smash-protection
I'd appreciate it if anyone else can confirm the compression difference is now gone and how much of a performance hit or increase Intel CPUs receive.
@GrieverV:
- The compression difference is gone; same FLACs as with NetRanger's binaries here
- On my i6700 I can not confirm a speed increase (haswell build); a quick test resulted in ~ the same performance as Case's build, but your FMA enabled build is still a bit faster (~1%).
Pardon my ignorance: What does this "FMA support" do? I was under the impression that compiler options can (and should) have an impact on the binary size/speed, but change the output files of a lossless encoder (although both files are valid FLACs with identical MD5)? Isn't this algorithm fixed by the source code?
Thanks for confirming and testing on Intel!
I'm not a math person and floating-point (approximate) stuff is like black magic to me, but as far as I understand, fused multiply-add (https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation#Fused_multiply%E2%80%93add) can result in slight differences due to different rounding compared to the usual multiply-add. Most modern CPUs should have hardware support for fused multiply-add operations which can be used through the FMA instruction set and GCC can make use of those instructions when performing optimizations by specifying the -mfma flag which is enabled when -march=haswell is set. Compilers usually have options (https://gcc.gnu.org/wiki/FloatingPointMath) that affect floating-point math.
libFLAC uses floating-point math for encoding that can be optimized by GCC and it seems enabling FMA will change the result in some cases.
What must I do to make compile compatible with SandyBridge CPU, but not with generations before?
Is there some step-by-step which I can follow?
I've tried yesterday, by installing MinGW and setting things up, but it just wouldn't work.
@itisljar Assuming you're using MSYS2, all you have to do is change -march=haswell to -march=sandybridge in the makepkg_mingw64.conf included with the Haswell build, copy to msys64/etc/, start MSYS2 MinGW 64-bit shell and run 'MINGW_INSTALLS=mingw64 makepkg-mingw -sic' in each of the folders containing the PKGBUILDs. It's easiest to copy the libogg/ and flac-git/ folders to msys64\home\<user> so all you have to do is 'cd <folder>/'
If you're using plain Mingw-w64, you'll need to set the environment variables manually and refer to the PKGBUILDs to see how ogg and FLAC are built.
Somewhat step-by-step for MSYS2:
1) Install MSYS2
2) Start MSYS2 MinGW 64-bit shell
3) Run 'pacman -Syyu' and then close the shell
4) Run 'pacman -Syu'
5) Install mingw toolchain, cmake and nasm 'pacman -S --needed mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-nasm'
6) Modify makepkg_mingw64.conf and change march=haswell to march=sandybridge
7) Copy makepkg_mingw64.conf to msys64/etc/
8) Copy libogg/ and flac-git/ to msys64\home\<user>
9) Using the MSYS2 MinGW 64-bit shell, enter the libogg/ directory by 'cd libogg/'
10) Build and install libogg with 'MINGW_INSTALLS=mingw64 makepkg-mingw -sic'
11) Repeat steps 9 and 10 for flac-git (cd ../flac-git/)
12) You should have a *.pkg.tar.xz archive containing the built libogg/FLAC inside the folder containing the PKGBUILDS (libogg/ flac-git/)
edit: Fair warning, anti-viruses hate MSYS2 and may completely break your MSYS2 install when you try to build any package (curl.exe, bash.exe, etc.). You might want to add an exception for the MSYS2 folder while building packages and then remove it from exceptions when done to avoid any security issues in the future while not using MSYS2.
@GrieverV:
Thank you very much for shedding some light on FMA optimisation. Very interesting stuff.
Btw, here are the results of my ususal test set:
Intel Core i7-6700 CPU @ 3.40GHz, 24 GB RAM, Samsung SSD 850 EVO 500GB
Windows 10 Pro 64-bit, Version 1909, Build 18363.720
flac133_case.exe:
39.303 sec (avg. runtime/5 runs, fastest run: 39.228 sec)
total size of FLAC files: 1.167.279.038 bytes -> different build!
flac133_GrieverV_7a35c528_haswell_FMA.exe (07.04.2020):
38.964 sec (avg. runtime/5 runs, fastest run: 38.814 sec)
total size of FLAC files: 1.167.276.028 bytes
flac133_GrieverV_7a35c528_haswell_noFMA.exe (12.04.2020):
39.169 sec (avg. runtime/5 runs, fastest run: 39.055 sec)
total size of FLAC files: 1.167.278.241 bytes
flac133_GrieverV_7a35c528.exe (12.04.2020):
40.192 sec (avg. runtime/5 runs, fastest run: 40.068 sec)
total size of FLAC files: 1.167.278.241 bytes
Somewhat step-by-step for MSYS2:
Oh, thank you very much! I will try these in next few days!
Can't edit the post, but... do I edit these:
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
LDFLAGS="-pipe"
Like this?
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=x86-64 -mtune=sandybridge -O2 -pipe"
CXXFLAGS="-march=x86-64 -mtune=sandybridge -O2 -pipe"
LDFLAGS="-pipe"
Or like this?
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=sandybridge -mtune=generic -O2 -pipe"
CXXFLAGS="-march=sandybridge -mtune=generic -O2 -pipe"
LDFLAGS="-pipe"
sundance: From what I understand FLAC uses floating point math in Linear Predictive Coding. Different floating point instructions have different precision, plus there are rounding errors. Most lossless encoders try to predict the next sample(s) and store error in the resulting file - due to slightly different math the predictions differ, so stored correction information differs, and so file size is different, but still lossless. Hopefully...
In case of lossy encoders, different compile may produce different file too, but in this case decoded file will be different too, inaudibly though. Hopefully...
Thanks a lot for your explanation. That makes perfect sense.
Somehow I tended to think that lossless compression and (imprecise) floating point calculations can't go together...
FLAC uses integer math not floating point. From the WIKIpedia article:
- The FLAC format supports only integer samples, not floating-point. It can handle any PCM bit resolution from 4 to 32 bits per sample, any sampling rate from 1 Hz to 65,535 Hz in 1 Hz increments or from 10 Hz to 655,350 Hz in 10 Hz increments, and any number of channels from 1 to 8.[9] To Date (Vers. 1.3.3 of the reference encoder), FLAC encoding is limited to 24 bits per sample since no encoder for 32 bits per sample exists.[10]
- Channels can be grouped in some cases, for example stereo and 5.1 channel surround, to take advantage of interchannel correlations to increase compression.
- CRC checksums are used for identifying corrupted frames when used in a streaming protocol. The file also includes a complete MD5 hash of the raw PCM audio in its STREAMINFO metadata header. FLAC allows for a Rice parameter between 0 and 16.
- FLAC uses linear prediction to convert the audio samples. There are two steps, the predictor and the error coding. The predictor can be one of four types (Zero, Verbatim, Fixed Linear and Finite Impulse Response[dubious – discuss] (FIR) Linear). The difference between the predictor and the actual sample data is calculated and is known as the residual. The residual is stored efficiently using Golomb-Rice coding. It also uses run-length encoding for blocks of identical samples, such as silent passages.
Thanks
The FLAC format uses only integer math, but FLAC encoders can use floating-point math to select the (integer) parameters used for the linear prediction.
FLAC v1.3.3-Git-2020-05-03-37e675b
Built on May 03, 2020, GCC 9.3.0
Latest commit included : 37e675b
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://gitlab.xiph.org/xiph/flac
FLAC v1.3.3-Git-2020-05-14-ce6dd6b
Built on May 14, 2020, GCC 10.1.0
Latest commit included : ce6dd6b
https://xiph.org/flac/
https://github.com/xiph/flac/commits/master
https://gitlab.xiph.org/xiph/flac
1.3.3.r3913.ce6dd6b5-1, gcc.exe (Rev3, Built by MSYS2 project) 10.1.0
-march=native on "Coffee Lake", but should be skylake compatible
$ gcc -march=native -Q --help=target | grep march
-march= skylake
Known valid arguments for -march= option:
Test -8ep
NetRanger's Built on May 14, 2020, GCC 10.1.0 - 10.91x
FLAC v.1.3.3 win64 ICL 19 compile - by John33, 10.08.2019 - 10.87x
flac-1.3.2-win.zip 2017-01-01 from site - 13.07x
This build - 14.53x
1.3.3 was released over 6 months ago and never even made the news on the homepage, that's a little sad IMHO
https://xiph.org/flac/news.html
thanks, 1.3.3 is now on the news page :)
What about official win compiles?
What about official win compiles?
Erik has already made it quite clear that he couldn't care any less about proprietary ("dead") operating systems.
I wonder whats smarter: claiming that Windows is dead, or not caring about most of your users ???
I can't say I'm particularly happy about his stance either, but John33's compiles at RareWares (https://www.rarewares.org/lossless.php) have been the de facto Windows compiles for quite some time, anyway; until a few years ago, they trounced the "official" build in performance, to boot.
John33 use ICL compile win32 and win64. FLAC Frontend (http://flacfrontend.sf.net/)
Erik has already made it quite clear that he couldn't care any less about proprietary ("dead") operating systems.
How old is he, 15? :) oh, nevermind, we can get windows compiles elsewhere.
Anything have a compiler visual studio? i'm try use cmake with ninja but some component not found
I seem to be a little late to the party here!! ;)
I can generate Intel compiles and/or Visual Studio compiles. What does everybody want?
I seem to be a little late to the party here!! ;)
I can generate Intel compiles and/or Visual Studio compiles. What does everybody want?
I'd prefer Intel compiles (that's what you currently have on the RW site, correct?), but either would be great. :)
I seem to be a little late to the party here!! ;)
I can generate Intel compiles and/or Visual Studio compiles. What does everybody want?
I'd prefer Intel compiles (that's what you currently have on the RW site, correct?), but either would be great. :)
OK, I'll get to it tomorrow. :) I'll post here when they're available.
I'll get to this when I am able but the git repositories at xiph.org are inaccessible at the moment. :(
I'll get to this when I am able but the git repositories at xiph.org are inaccessible at the moment. :(
Are you looking at the new GitLab instance? https://gitlab.xiph.org/xiph/flac
The old cgit interface is no more, IIRC.
I was, but discovered the error in my ways!! ;)
New compiles now at Rarewares. :)
Thanks, John :D
New compiles now at Rarewares. :)
Thanks, but the libFLAC links are pointing to the FLAC executables downloads ;)
New compiles now at Rarewares. :)
Thanks, but the libFLAC links are pointing to the FLAC executables downloads ;)
Thanks for the 'heads up', it's fixed now. :)
More haste, less speed is the moral of that, I think!
What is 1.3.4?
https://github.com/xiph/flac/tree/topic/release-1.3.4
Compiled with MSVS 2019 v142 SDK 10.0.19041.0
Win32 & Win64 with libFLAC_dynamic.dll
Hi guys! I want to download v1.3.3 but on xiph site I found command-line tools only. Why there's no exe file of 1.3.3 for windows?
You mean a GUI, right?
Why there's no exe file of 1.3.3 for windows?
https://hydrogenaud.io/index.php?msg=974323
https://hydrogenaud.io/index.php?msg=974432
Hi Korth! I don't understood...So I have to download 1.3.4?
Note: flac.exe is a command-line tool.
As I interpret the message, an issue with the official 1.3.3 source code prevented an official Windows build.
So stick with 1.3.2, wait for 1.3.4, or use one of the unofficial builds for 1.3.3.
You need FLAC.EXE anyway
There is a GUI: https://sourceforge.net/projects/flacfrontend/
Very interesting! :o
Note: flac.exe is a command-line tool.
As I interpret the message, an issue with the official 1.3.3 source code prevented an official Windows build.
So stick with 1.3.2, wait for 1.3.4, or use one of the unofficial builds for 1.3.3.
Ook! I'll keep v1.3.2
There is a GUI: https://sourceforge.net/projects/flacfrontend/
Avoid. I had it eating some of my files: https://hydrogenaud.io/index.php?topic=99803.msg921798#msg921798
Hi. I'm using the old frontend (GUI for Windows). I've encoded with this GUI, over 15K songs and never had an issue. I am attaching it, if someone needs it. I also put the x64 tools from flac 1.3.4 found here, posted by "Ozz" , which also works great for me.
Cheers.
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.19041.0
commit bfd4f13 (2020-12-17)
FLAC v1.3.3-Git-2021-03-15-27c6157
Built on March 15, 2021, GCC 10.2.0
Latest commit included : 27c6157
https://github.com/xiph/flac/commits/master
Thanks, NR 8)
Sorry for the noob question :)) I am on Windows 10. I would like to update flac.exe that ships with Flac Frontend to version 1.3.3 but I do not know whether I should use the 64-bit or 32-bit version. Any help? Thanks.
Sorry for the noob question :)) I am on Windows 10. I would like to update flac.exe that ships with Flac Frontend to version 1.3.3 but I do not know whether I should use the 64-bit or 32-bit version. Any help? Thanks.
Open a command-line window (hold down Win key and press r, then type "cmd" without quotation marks, and press Enter).
In the command-line window, type
wmic os get osarchitecture
and press enter.
Open a command-line window (hold down Win key and press r, then type "cmd" without quotation marks, and press Enter).
In the command-line window, type
wmic os get osarchitecture
and press enter.
Thanks. Windows 10 is 64-bit and Flac Frontend is 32-bit. So which flac.exe version should I use?
Since the frontend simply calls the command line encoder it shouldn't make any difference. Start with the 64 bit version. If you encounter a problem, which you shouldn't, switch to the 32 bit version.
Here I go reading and answering the question, without noticing that the front-end was mentioned.
@gilibaus , beware the front-end; I had it eating my files: post 116 up here, or https://hydrogenaud.io/index.php?topic=99803.msg921798#msg921798
I don't know what it does wrong, but I never had the CLI do this. Does the front-end rename even incompletely converted temp files to overwrite the original?
Rather, use e.g. foobar2000,, bitcompare to the original and only then delete.
@john33 and
@Porcus thanks for your help
I’m curious what happened to 1.3.4... it seems we went backwards to 1.3.3 again?
Following. Doing some CD rips and want to make sure I'm staying up to date with these things. Currently using the 1.3.2 that ships with EAC.
I’m curious what happened to 1.3.4... it seems we went backwards to 1.3.3 again?
https://github.com/xiph/flac/issues/111#issuecomment-507073018
https://github.com/xiph/flac/issues/111#issuecomment-521431000
https://github.com/xiph/flac/pull/119
FLAC v1.3.3-Git-2021-06-23-eba0ff8
Built on June 23, 2021, GCC 10.3.0
Latest commit included : eba0ff8
FLAC v1.3.3-Git-2021-06-25-bab58c3
Built on June 26, 2021, GCC 10.3.0
Latest commit included : bab58c3
FLAC v1.3.3-Git-2021-06-26-37d1a62
Built on June 30, 2021, GCC 10.3.0
Latest commit included : 37d1a62
https://github.com/xiph/flac/commits/master
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.19041.0
commit 313ab58
(2021-07-11)
Any advice on how to build flac statically in msys/mingw?
My commands:
#!/bin/bash
export CC=clang CXX=clang++
git clone https://github.com/xiph/flac.git --recursive
cd flac
cmake -B build -G Ninja -S ./ \
-DCMAKE_BUILD_TYPE=Release \
-DWITH_OGG=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_DOCS=OFF \
-DINSTALL_MANPAGES=OFF \
-DCMAKE_EXE_LINKER_FLAGS='-fstack-protector-strong -static'
ninja -C build
FLAC v1.3.3-Git-2021-07-11-313ab58
Built on July 12, 2021, GCC 10.3.0
https://github.com/xiph/flac/commits/master
@john33 i noticed a small increase in compression after installing your compile for FLAC 1.3.3 over my previous vanilla 1.3.2 install. Is that just the regular difference between the two versions or did you tweak anything, do you have an update log available somewhere?
The difference is very minor, about 5000 bytes in 239 files, but still.
@john33 i noticed a small increase in compression after installing your compile for FLAC 1.3.3 over my previous vanilla 1.3.2 install. Is that just the regular difference between the two versions or did you tweak anything, do you have an update log available somewhere?
The difference is very minor, about 5000 bytes in 239 files, but still.
Hmmm, interesting, but not of my doing. It is perhaps possibly down to the Intel compiler although I tend to doubt it. Much more likely to be some change in the library.
FLAC v1.3.3-Git-2021-07-25-b358381
Built on July 25, 2021, GCC 10.3.0
Latest commit included : b358381
https://github.com/xiph/flac/commits/master
Wow, imagine my surprise when I tried to register here using an old throw-away handle only to find I'd already been here over a decade ago to post some moldy old desktop screenshots:
I can confirm it's present in the US pressing of the album too:
(http://img.photobucket.com/albums/v491/sheik124/TakeABow.jpg)
I'm curious about the file size differences I see vs. 1.3.2 and also when comparing john33's build with either Griever's or Ozz's. Also curious why john33 rolled back to the commit he did. I probably should've tried to match the three builds by commit but I could only find ce6dd6b on rarewares and 7a35c528 looks like the last one Griever posted...can the compiler used actually make a difference in the output file size? To be clear, this has nothing to do with the "bloat" bug mentioned in this topic. (https://hydrogenaud.io/index.php?topic=121349.0)
Griever's is fastest on my old machine by a decent enough amount for hi-res files that I'll probably stick with it. I saw it hit the mid 70x range in fb2k while the other two were high 60s, very scientific! john33's build seems to make 96/24 files that are ~3 KB larger than the other two for whatever reason. Sometimes the 1.3.3 builds are collectively larger than 1.3.2 for 96/24, and sometimes they're smaller. All three builds are faster than 1.3.2. I tested using the same rip of the same track from that post I made here
14 years ago, I also tried the 2015 96/24 release, and another Hi-Res track. Obviously can'tshare any of the copyrighted test files themselves.
There was only a <1 KB difference between any of the builds or 1.3.2 for 44/16, using Muse's "Take a Bow" from that old post. Entirely inconsequential unless I'm right at the boundary between requiring an extra NTFS cluster.
Oh no...31.3 MB (32,821,248 bytes) "on disk" for all of the above in any case, so it's not going to be noticeable. No, I am not going to check my library for FLAC files that are "
some multiple of 4 KB minus <1 KB" in size, and I'm pretty sure the drive on my NAS with most of the music on it has 64KB blocks either way, so even the 96/24 differences aren't
really differences. Somewhat comically, at least for these three files, the 1.3.2 files as a whole win...
Test File // Version | Griever 7a35c528 | Ozz 313ab58 | john33 ce6dd6b | 1.3.2 |
01. Take a Bow.flac 96/24 | 107 MB (112 904 836 bytes) | 107 MB (112 904 853 bytes) | 107 MB (112 907 505 bytes) | 107 MB (112 910 775 bytes) |
01 Take A Bow.flac 44/16 CD | 31.2 MB (32 819 379 bytes) | 31.2 MB (32 819 370 bytes) | 31.2 MB (32 819 357 bytes) | 31.2 MB (32 819 331 bytes) |
01. One Last Kiss 96/24 | 83.5 MB (87 594 621 bytes) | 83.5 MB (87 594 348 bytes) | 83.5 MB (87 597 426 bytes) | 83.5 MB (87 581 227 bytes) |
Total | 222.5 MB (233 318 836 bytes) | 222.5 MB (233 318 571 bytes) | 222.5 MB (233 324 288 bytes) | 222.5 MB (233 311 333 bytes) |
Diff | +7,503 bytes | +7,238 bytes | +12,955 bytes | 0 |
Build Name | mingw-w64-x86_64-flac-git-1.3.3.r3909.7a35c528 | FLAC_1.3.3_313ab58-Ozz | flac-v1.3.3.git-ce6dd6b-x64 | flac-1.3.2-win |
realized this explorer screenshot was pointless after checking the exact file sizes and adding them as a table courtesy of a very convenient BBCode table generator
(https://i.postimg.cc/htBQvYtZ/Untitled.png)
The only time I re-encode anything these days anyways is when certain Hi-Res download sites give you
bizarre, larger than PCM padded? 4616 kbps 96/24 FLAC files for whatever reason. I don't think this is the bloat bug either, the entire album was like this, as if it was just PCM with ogg headers. "Bigger is better." Those get beat with the Level 8 stick out of spite. Apologies for not comparing all of them more, editing the hardlink to flac.exe is a pain in the butt.
Beautiful World [1.3.1 - Source] | 180 MB (189 548 998 bytes) |
Beautiful World [PCM] | 175 MB (183 586 602 bytes) |
Beautiful World [1.3.2 Level 8] | 116 MB (122 368 039 bytes) |
Beautiful World [Griever Level 8] | 116 MB (122 369 621 bytes) |
Beautiful World [Griever Level 0] | 132 MB (138 737 844 bytes) |
Thank you to all three of you for compiling these BTW!
There was only a <1 KB difference between any of the builds or 1.3.2 for 44/16, using Muse's "Take a Bow" from that old post.
Just to be sure that it isn't padding or seektable ... tried
metaflac.exe --dont-use-padding --remove-all?
(It will also remove the vendor (version) string, so make a copy first.)
Those get beat with the Level 8 stick out of spite.
I use -8 pretty much always, so I have to resort to -e as spite switch.
There was only a <1 KB difference between any of the builds or 1.3.2 for 44/16, using Muse's "Take a Bow" from that old post.
Just to be sure that it isn't padding or seektable ... tried
metaflac.exe --dont-use-padding --remove-all
?
(It will also remove the vendor (version) string, so make a copy first.)
You know, I don't think I've ever used metaflac or even flac.exe itself, it's usually piped through via foobar or EAC or something.
If you ever want a lazy way to do operations like that for all FLAC files in a folder + its subfolders,
FOR /F "tokens=*" %G IN ('dir /b /s *.flac') DO metaflac.exe --dont-use-padding --remove-all "%G"
No fancy table this time
Before:
2021-09-18 01:04 PM 32,819,331 01 Take A Bow 1.3.2.flac
2021-09-18 01:09 PM 32,819,357 01 Take A Bow flac-v1.3.3.git-ce6dd6b-x64.flac
2021-09-18 01:07 PM 32,819,370 01 Take A Bow 1.3.3_313ab58-Ozz.flac
2021-09-18 01:10 PM 32,819,379 01 Take A Bow mingw-w64-x86_64-flac-git-1.3.3.r3909.7a35c528.flac
After:
2021-09-18 11:46 PM 32,689,426 01 Take A Bow 1.3.2.flac
2021-09-18 11:46 PM 32,689,452 01 Take A Bow flac-v1.3.3.git-ce6dd6b-x64.flac
2021-09-18 11:46 PM 32,689,474 01 Take A Bow mingw-w64-x86_64-flac-git-1.3.3.r3909.7a35c528.flac
2021-09-18 11:46 PM 32,689,476 01 Take A Bow 1.3.3_313ab58-Ozz.flac
The "giant" 4616 kbps FLAC out of curiosity:
2019-10-22 06:18 PM 189,548,998 01 - Beautiful World.flac - Before
2021-09-18 11:53 PM 183,454,510 01 - Beautiful World.flac - After
2021-09-18 03:09 PM 183,586,602 01 - Beautiful World.wav - Still tagged with embedded album art too BTW...
2021-09-18 03:05 PM 446,533 heart station.jpg
for /r %G IN (*.flac) works as well I think. So the differences stayed about the same, and that is not the explanation. (Reason why I asked is this thread (https://hydrogenaud.io/index.php?topic=120906.msg996787#msg996787).)
But the Beautiful World must have had a 5.5 gigabytes padding or something? Given that the art was that 446553 file. (You can find art size by Mp3tag.)
dBpoweramp has the option to encode to "uncompressed" FLAC (I don't remember how, does maybe it forces VERBATIM for everything ... everything but wasted bits I think). Probably it was implemented to attract audiophools or maybe to shut them up.
It is possible to get out that information by checking the output of flac -a - which would be a text file same order of magnitude as the file or something, that is for the curious.
"uncompressed"
I think it's a WAV wrapped in a FLAC. I could be wrong.
EZ CD Audio has that option too.
"uncompressed"
I think it's a WAV wrapped in a FLAC. I could be wrong.
Yep, you are ;) I don't think that is even possible, but read: https://forum.dbpoweramp.com/showthread.php?45797-Flac-uncompressed-question
Apparently the "uncompressed" FLAC cannot switch off FLAC's "wasted bits" feature. I.e. if you fit a 16-bit signal into a 24-bit WAV (or AIFF), it will pad up eight bits with zeroes; flac.exe will notice that the eight bits are "wasted" and treat it as a 16 bit signal (this is in each subframe header, the
file will be flagged as 24-bit). Other codecs offer that as well, more in section 2.3 of http://www.audiograaf.nl/losslesstest/Lossless%20audio%20codec%20comparison%20-%20revision%204.pdf . There seems no way to pass options to the reference flac.exe to force wasted bits to zero.
EZ
Unsurprising. Whenever that guy sees a buzzword at dBpoweramp, he copies it - to the point of calling his ripping engine "AccurateCDDA" (http://web.archive.org/web/20100524030625/http://www.poikosoft.com/product_details.html) to leech on AccurateRip.
Just avoid.
EZ CD is also currently the only way to use Fraunhofer's xHE-AAC encoder. I asked Fraunhofer. No, they are not into selling personal licenses.
I have created a FLAC 64 bit installer with user-friendly functions:
(https://u.cubeupload.com/Riccardo/3f8Immagine.png)
(https://u.cubeupload.com/Riccardo/Immagine2.png)
(https://u.cubeupload.com/Riccardo/Immagine3.png)
(https://u.cubeupload.com/Riccardo/Immagine4.png)
(https://u.cubeupload.com/Riccardo/Immagine5.png)
(https://u.cubeupload.com/Riccardo/Immagine6.png)
(https://u.cubeupload.com/Riccardo/Immagine7.png)
(https://u.cubeupload.com/Riccardo/Immagine8.png)
I have created a FLAC 64 bit installer with user-friendly functions:
(https://u.cubeupload.com/Riccardo/Image.png)
(https://u.cubeupload.com/Riccardo/Image2.png)
(https://u.cubeupload.com/Riccardo/Image3.png)
(https://u.cubeupload.com/Riccardo/Image4.png)
(https://u.cubeupload.com/Riccardo/Image5.png)
(https://u.cubeupload.com/Riccardo/Image6.png)
(https://u.cubeupload.com/Riccardo/Image7.png)
(https://u.cubeupload.com/Riccardo/Image8.png)
Now the installer is easier to read and the .png image takes up less space.
End of 2021.
No official stable release for the most popular operating system in world.
Heh...
I guess it's time to edit all Wikipedias that FLAC doesn't support Windows anymore.
...so, finally, I don't exact understand which build is recommended (compression efficiency oriented) to use inside CUEtools 2.1.9: any suggestion ?
...so, finally, I don't exact understand which build is recommended (compression efficiency oriented) to use inside CUEtools 2.1.9: any suggestion ?
So that isn't obvious. Not even what you mean by "compression efficiency" (are you taking time into account, or are you only interested in "smallest file for reasonable time, but time isn't crucial as long as it is reasonable"?).
CUETools have a couple of alternative encoders which
once upon a time compressed better and/or faster. Meanwhile both CUETools flake and the reference implementation have improved - and hardware has improved, and with that the assessment of what time is "reasonable". Some testing done in 2017: https://hydrogenaud.io/index.php?topic=114084.msg939675#msg939675
* FLACCL can utilize GPU though CUDA. Depending on what GPU and what CPU you have, that may or may not be faster.
* CUETools flake is a further development of the original flake, a 3rd-party FLAC implementation (also to a large extent incorporated into ffmpeg's FLAC encoder)
Flake does certain things better - like, it uses double precision to compute autocorrelation. More discussion at https://hydrogenaud.io/index.php?topic=120158.msg999746#msg999746 where ktf has published a few flac.exe betas (warning: beta is beta is beta) that incorporate double precision. From that thread you can see that double precision costs a bit extra time. You can also see that there are potential great improvements for hi-rez,
which CUETools does not support. (CUETools flake does, but if your point was recommendation for CUETools the application, then hirez improvements are irrelevant.)
Note that flake 9 to 11 produce non-subset (https://xiph.org/flac/format.html#subset) streams.
can someone build the latest flac.exe with a recent compiler and aggressively optimized for Intel Comet Lake-S? i9-10850K to be exact. does not have to be able to run on older CPUs
FLAC 1.3.3 + LibFLAC_dynamic.dll (x86 & x64)
MSVC 2019 (v142) SDK version 10.0.19041.0
commit 79e462f
(2022-02-09)
Thanks, Ozz
FLAC v1.3.3-Git-2022-02-11-e8143ab
Built on February 11, 2022, GCC 11.2.0
MSVC 2019 (v142) SDK version 10.0.19041.0
Just curious, do you build these with the sln files or with CMake?
Intel 19 compiles of flac-V1.3.3-e8143ab5:
www.rarewares.org/files/lossless/flac-V1.3.3-e8143ab5-x64.zip (http://www.rarewares.org/files/lossless/flac-V1.3.3-e8143ab5-x64.zip)
www.rarewares.org/files/lossless/flac-V1.3.3-e8143ab5-x86.zip (http://www.rarewares.org/files/lossless/flac-V1.3.3-e8143ab5-x86.zip)
www.rarewares.org/files/lossless/flac_dll-v1.3.3-e8143ab5-x64.zip (http://www.rarewares.org/files/lossless/flac_dll-v1.3.3-e8143ab5-x64.zip)
www.rarewares.org/files/lossless/flac_dll-v1.3.3-e8143ab5-x86.zip (http://www.rarewares.org/files/lossless/flac_dll-v1.3.3-e8143ab5-x86.zip)
:) (All built using the VS sln.)
FLAC v1.3.3-Git-2022-02-14-a2fe43f
Built on February 14, 2022, GCC 11.2.0
(All built using the VS sln.)
Note that building with the sln files supplied in git or tarball will be deprecated in favor of building with sln files configured with CMake starting with release 1.3.4. See
https://github.com/xiph/flac/blob/29e5b507894c4fce8c59ded5cd2f633a7dc5bfed/README#L214-L216
and
https://github.com/xiph/flac/blob/dee83e0633da5984c5a3450564050132fe14e13d/doc/html/changelog.html#L49
Configuring through CMake still has some issues that need to be worked out, but it wouldn't surprise me if 1.3.4 is the last release to support building through MSVC without using CMake. (There's a bunch of CMake+MSVC fixes here (https://github.com/xiph/flac/pull/267) but those probably won't be applied before release 1.3.4)
(All built using the VS sln.)
Note that building with the sln files supplied in git or tarball will be deprecated in favor of building with sln files configured with CMake starting with release 1.3.4. ...
Noted, thanks. It will still be a pain in the rear having to modify all the compiler options as it is now.
Noted, thanks. It will still be a pain in the rear having to modify all the compiler options as it is now.
That, or you could make sure those compiler options are included in the CMake config, i.e. have your changed merged. If you want, I can help with that. The major benefit of CMake is that you can configure a build instead of having to modify the sln files by hand to change build behaviour.
Noted, thanks. It will still be a pain in the rear having to modify all the compiler options as it is now.
That, or you could make sure those compiler options are included in the CMake config, i.e. have your changed merged. If you want, I can help with that. The major benefit of CMake is that you can configure a build instead of having to modify the sln files by hand to change build behaviour.
And, thanks again. I'll give it a try and see how I get on. ;)
I'm building with custom Xcode project files instead of CMake. Probably excluding all the assembly and intrinsics, too. Is ARM optimization ever happening, or is that unnecessary?
Is ARM optimization ever happening, or is that unnecessary?
There is quite a bunch of ARM64 intrinsics routines for encoding waiting to be merged. Execution time with these is halved.
Suggestion from Peter, implement
FLAC__bitreader_read_unary_unsigned__LZCNT, using __lzcnt intrinsics provided by the compiler, and in the case of Clang, surround the implemented function with:
#pragma clang attribute push (__attribute__((target("lzcnt"))), apply_to=function)
...
#pragma clang attribute pop
And detect the instruction at runtime with the cpuid function, like all other optional x86 code paths.
BitScanReverse and such do not automatically optimize to lzcnt on the supporting Haswell or K10 processors unless you hard enable it at compile time. This optimization brings approximately a 20-30% improvement in decode time. Not too much to be impressed by when the figures are a comparison between 8000x and 12000x realtime, but maybe worth implementing anyway.
Clang, at least from Xcode, requires the instructions to be enabled per function, or per module. Per function is best in this case, since the requisite function will only be called if cpuid flags it present. In the case of Macs, this would be every Mac with at least a Haswell processor. Not including Apple Silicon running under Rosetta, though. But that already has a clz instruction which gets compiled in automatically for the __clz intrinsic function.
#pragma clang attribute push (__attribute__((target("lzcnt"))), apply_to=function)
...
#pragma clang attribute pop
It would be even nicer (less code clutter) if Xcode's clang supports the target_clones attribute. Can't find anything about it. That has been in GCC for quite a while now and the next vanilla clang will support it too.
What does target_clones do? Note that this cannot be enabled for the entire project, as that would enable unwanted effects that should only be switched on with cpuid detection.
Here, have a patch:
https://github.com/losnoco/Cog/blob/main/ThirdParty/flac/patches/libflac-lzcnt.patch
target_clones duplicates a function for several architectures or instruction sets, without needing to add any runtime code to select. Selection is done by some code added by the compiler or in the C library. You can for example add target_clones("default,lzcnt") to a function to have it compile a non-lzcnt and an lzcnt version without having to add any code at all.
Then by all means, try that. I don't think MSVC supports it, either, though.
Noted, thanks. It will still be a pain in the rear having to modify all the compiler options as it is now.
That, or you could make sure those compiler options are included in the CMake config, i.e. have your changed merged. If you want, I can help with that. The major benefit of CMake is that you can configure a build instead of having to modify the sln files by hand to change build behaviour.
The existing sln files, with the switch to the Intel compiler, compile 'out-of-the-box' when the 'ogg' directory and library are placed as specified. They binaries are also built without runtime dependencies. The current cmake files do not find the ogg directory, or the library in the places specified and, excluding ogg from the build, the resulting binaries are runtime dependent. I would expect that the cmake environment would at least generate sln files that match the current included versions. Without that I see no point or benefit in switching to the cmake process.
Here's a PR that I think will fix most of these problems, at least the ogg one: https://github.com/xiph/flac/pull/267/commits/e418e54689713a1845c21174589790edda8c0ead
I presume aside from these errors, building went fine? That's a good start already. O:)
Thanks, I'll let you know. ;)
After much trial and error, the following 'FindOgg.cmake' works:
find_package(PkgConfig)
pkg_check_modules(_OGG QUIET ogg)
file(GLOB _OGG_DIR "./include")
file(GLOB _OGG_STATIC_LIB "./objs/Release/lib")
find_path(
OGG_INCLUDE_DIR
NAMES "ogg/ogg.h"
PATHS ${_OGG_DIR}
# HINTS "${_OGG_DIR}/include"
)
find_library(
OGG_LIBRARY
NAMES libogg_static.lib
# PATH ${_OGG_STATIC_LIB}
HINTS "${_OGG_STATIC_LIB}"
)
mark_as_advanced(
OGG_INCLUDE_DIR
OGG_LIBRARY)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Ogg
REQUIRED_VARS OGG_INCLUDE_DIR OGG_LIBRARY
VERSION_VAR _OGG_VERSION)
if(OGG_FOUND AND NOT TARGET Ogg::ogg)
add_library(Ogg::ogg UNKNOWN IMPORTED)
set_target_properties(Ogg::ogg PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${OGG_INCLUDE_DIR}"
IMPORTED_LOCATION "${OGG_LIBRARY}")
endif()
and static libs are built, and statically linked executables. The usual dlls are not, however, built. Nevertheless, good progress!
MSVC 2019 (v142) SDK version 10.0.19041.0
Just curious, do you build these with the sln files or with CMake?
SLN only
If you want to build libogg + libFLAC DLL with MSVC, you just need to do something like this:
cmake -B build/dir -G Ninja -DBUILD_SHARED_LIBS=yes -DCMAKE_PREFIX_PATH=path/to/libogg/dir
BUILD_SHARED_LIBS is a standard cmake option to enable shared library:
https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html
CMAKE_PREFIX_PATH is a standard cmake option to let it know where to search libraries:
https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html
Messages concerning v1.3.4 were split to a new topic FLAC (https://hydrogenaud.io/index.php?board=67.0) - https://hydrogenaud.io/index.php?topic=122179.0