Skip to main content
Topic: FLAC v1.3.3 (Read 98130 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: FLAC v1.3.3

Reply #75
FLAC 7a35c52 MSYS2/mingw-w64-x86_64 2020-04-12 (64bit, GCC 9.3)

Aggressively optimized builds tuned for modern processors. Included is a generic x86_64 build and an Intel Haswell optimized build that will only run on Haswell or newer Intel CPUs and Zen or newer AMD CPUs. All FLAC tests passed for both builds.

Changes:
  • Rebuilt against a newer Mingw-w64 git snapshot
  • Haswell binaries are now built without FMA support which should allow reproducible compression sizes with other builds

The new Haswell build is 8.1% faster than the previous Haswell build and 5% faster than the generic build on a 2700X (--verify --best) with this test file: BIS1447-002-flac_16.flac

Build information:
Spoiler (click to show/hide)

I'd appreciate it if anyone else can confirm the compression difference is now gone and how much of a performance hit or increase Intel CPUs receive.

Re: FLAC v1.3.3

Reply #76
@GrieverV:
- The compression difference is gone; same FLACs as with NetRanger's binaries here
- On my i6700 I can not confirm a speed increase (haswell build); a quick test resulted in ~ the same performance as Case's build, but your FMA enabled build is still a bit faster (~1%).

Pardon my ignorance: What does this "FMA support" do? I was under the impression that compiler options can (and should) have an impact on the binary size/speed, but change the output files of a lossless encoder (although both files are valid FLACs with identical MD5)? Isn't this algorithm fixed by the source code?

Re: FLAC v1.3.3

Reply #77
Thanks for confirming and testing on Intel!

I'm not a math person and floating-point (approximate) stuff is like black magic to me, but as far as I understand, fused multiply-add can result in slight differences due to different rounding compared to the usual multiply-add. Most modern CPUs should have hardware support for fused multiply-add operations which can be used through the FMA instruction set and GCC can make use of those instructions when performing optimizations by specifying the -mfma flag which is enabled when -march=haswell is set. Compilers usually have options that affect floating-point math.

libFLAC uses floating-point math for encoding that can be optimized by GCC and it seems enabling FMA will change the result in some cases.

Re: FLAC v1.3.3

Reply #78
What must I do to make compile compatible with SandyBridge CPU, but not with generations before?
Is there some step-by-step which I can follow?
I've tried yesterday, by installing MinGW and setting things up, but it just wouldn't work.
Error 404; signature server not available.

Re: FLAC v1.3.3

Reply #79
@itisljar
Assuming you're using MSYS2, all you have to do is change -march=haswell to -march=sandybridge in the makepkg_mingw64.conf included with the Haswell build, copy to msys64/etc/, start MSYS2 MinGW 64-bit shell and run 'MINGW_INSTALLS=mingw64 makepkg-mingw -sic' in each of the folders containing the PKGBUILDs. It's easiest to copy the libogg/ and flac-git/ folders to msys64\home\<user> so all you have to do is 'cd <folder>/'

If you're using plain Mingw-w64, you'll need to set the environment variables manually and refer to the PKGBUILDs to see how ogg and FLAC are built.

Somewhat step-by-step for MSYS2:
Spoiler (click to show/hide)

edit: Fair warning, anti-viruses hate MSYS2 and may completely break your MSYS2 install when you try to build any package (curl.exe, bash.exe, etc.). You might want to add an exception for the MSYS2 folder while building packages and then remove it from exceptions when done to avoid any security issues in the future while not using MSYS2.

Re: FLAC v1.3.3

Reply #80
@GrieverV:
Thank you very much for shedding some light on FMA optimisation. Very interesting stuff.

Btw, here are the results of my ususal test set:
Code: [Select]
Intel Core i7-6700 CPU @ 3.40GHz, 24 GB RAM, Samsung SSD 850 EVO 500GB
Windows 10 Pro 64-bit, Version 1909, Build 18363.720

flac133_case.exe:
39.303 sec (avg. runtime/5 runs, fastest run: 39.228 sec)
total size of FLAC files: 1.167.279.038 bytes -> different build!

flac133_GrieverV_7a35c528_haswell_FMA.exe (07.04.2020):
38.964 sec (avg. runtime/5 runs, fastest run: 38.814 sec)
total size of FLAC files: 1.167.276.028 bytes

flac133_GrieverV_7a35c528_haswell_noFMA.exe (12.04.2020):
39.169 sec (avg. runtime/5 runs, fastest run: 39.055 sec)
total size of FLAC files: 1.167.278.241 bytes

flac133_GrieverV_7a35c528.exe (12.04.2020):
40.192 sec (avg. runtime/5 runs, fastest run: 40.068 sec)
total size of FLAC files: 1.167.278.241 bytes



Re: FLAC v1.3.3

Reply #82
Can't edit the post, but... do I edit these:

Code: [Select]
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
LDFLAGS="-pipe"

Like this?

Code: [Select]
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=x86-64 -mtune=sandybridge -O2 -pipe"
CXXFLAGS="-march=x86-64 -mtune=sandybridge -O2 -pipe"
LDFLAGS="-pipe"

Or like this?

Code: [Select]
#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D__USE_MINGW_ANSI_STDIO=1"
CFLAGS="-march=sandybridge -mtune=generic -O2 -pipe"
CXXFLAGS="-march=sandybridge -mtune=generic -O2 -pipe"
LDFLAGS="-pipe"
Error 404; signature server not available.

Re: FLAC v1.3.3

Reply #83
sundance: From what I understand FLAC uses floating point math in Linear Predictive Coding. Different floating point instructions have different precision, plus there are rounding errors. Most lossless encoders try to predict the next sample(s) and store error in the resulting file - due to slightly different math the predictions differ, so stored correction information differs, and so file size is different, but still lossless. Hopefully...
In case of lossy encoders, different compile may produce different file too, but in this case decoded file will be different too, inaudibly though. Hopefully...

Re: FLAC v1.3.3

Reply #84
Thanks a lot for your explanation. That makes perfect sense.
Somehow I tended to think that lossless compression and (imprecise) floating point calculations can't go together...

Re: FLAC v1.3.3

Reply #85
FLAC uses integer math not floating point.  From the WIKIpedia article:

  • The FLAC format supports only integer samples, not floating-point. It can handle any PCM bit resolution from 4 to 32 bits per sample, any sampling rate from 1 Hz to 65,535 Hz in 1 Hz increments or from 10 Hz to 655,350 Hz in 10 Hz increments, and any number of channels from 1 to 8.[9] To Date (Vers. 1.3.3 of the reference encoder), FLAC encoding is limited to 24 bits per sample since no encoder for 32 bits per sample exists.[10]
  • Channels can be grouped in some cases, for example stereo and 5.1 channel surround, to take advantage of interchannel correlations to increase compression.
  • CRC checksums are used for identifying corrupted frames when used in a streaming protocol. The file also includes a complete MD5 hash of the raw PCM audio in its STREAMINFO metadata header. FLAC allows for a Rice parameter between 0 and 16.
  • FLAC uses linear prediction to convert the audio samples. There are two steps, the predictor and the error coding. The predictor can be one of four types (Zero, Verbatim, Fixed Linear and Finite Impulse Response[dubious – discuss] (FIR) Linear). The difference between the predictor and the actual sample data is calculated and is known as the residual. The residual is stored efficiently using Golomb-Rice coding. It also uses run-length encoding for blocks of identical samples, such as silent passages.

Thanks

Re: FLAC v1.3.3

Reply #86
The FLAC format uses only integer math, but FLAC encoders can use floating-point math to select the (integer) parameters used for the linear prediction.


 

 
SimplePortal 1.0.0 RC1 © 2008-2020