Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC v1.3.4 (Read 52673 times) previous topic - next topic - Topic derived from FLAC v1.3.3
0 Members and 1 Guest are viewing this topic.

Re: FLAC v1.3.4

Reply #152 –
Quote from: Porcus on
Challenge accepted. Awaiting the

c:\bin\timer64.exe flac -8f -A subdivide_tukey(250) Napalm_Death__1987__Scum__12__You_suffer.flac
*cough* The "CD version" is actually much longer. Which makes for a difference, when the above "250" encodes at 1/7th of realtime speed (-p to get it down to 1/34th)

With @NetRanger's freshly posted build.

Re: FLAC v1.3.4

Reply #154 –
FLAC v1.3.4-0bf7282 git compiles now at Rarewares. :)

Re: FLAC v1.3.4

Reply #155 –
Quote from: john33 on
FLAC v1.3.4-0bf7282 git compiles now at Rarewares. :)
I see you found the problem with the WinXP toolset in MSVC :( I didn't cross my mind this would also affect your compiles.

Did you stumble on anything else? If you have any additions for the CMake configuration specific for the Intel Compiler, let me know.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #156 –
Some findings about -r. While -8 -r 8 doesn't help much on normal music (if at all) compared to -8, on noise-like material it can significantly improve compression, while it doesn't hurt speed much. Build from this post was used - https://hydrogenaud.io/index.php/topic,122179.msg1014061.html#msg1014061
Merzbow - Venereology
-8 - 505 926 247 bytes / 1341 kbps avg
-8 -r 8 - 478 585 859 bytes / 1269 kbps avg

Less extreme example - https://aylwin.bandcamp.com/album/farallon. Unlike Merzbow, this is music, just insanely compressed/distorted/clipped.
-8 - 405 730 765 bytes / 1229 kbps avg
-8 -r 8 - 394 267 319 bytes / 1195 kbps avg

Re: FLAC v1.3.4

Reply #157 –
Quote from: ktf on
Quote from: john33 on
FLAC v1.3.4-0bf7282 git compiles now at Rarewares. :)
I see you found the problem with the WinXP toolset in MSVC :( I didn't cross my mind this would also affect your compiles.

Did you stumble on anything else? If you have any additions for the CMake configuration specific for the Intel Compiler, let me know.
To be honest, not having an XP system, real or VM, I really don't know, but I erred on the side of caution. ;)

I haven't found any other issues but, call me old-fashioned, I continue to use MSVS project files that I maintain and will amend as required. While I can see a benefit in the 'global' sense to using Cmake, it offers me no benefit when I can work with the MSVS project files 'in my sleep'. ;)

Re: FLAC v1.3.4

Reply #158 –
Thanks for keeping compatibility in mind. Keeping up with projects that constantly update their compilers and required runtime libraries can be tedious and frustrating, especially if you're using an older OS. :)

Re: FLAC v1.3.4

Reply #159 –
Quote from: john33 on
To be honest, not having an XP system, real or VM, I really don't know, but I erred on the side of caution. ;)
For anyone wondering: the only thing that could be noticed in day-to-day use is that the progress bar didn't display the compression ratio properly. However, some more advanced uses (with --skip and --until for example) didn't work because of this bug. Programs that used libFLAC directly found all sorts of weird behaviour. This went unnoticed for quite some time, but the way @john33 build now should evade all such problems.

The builds with MinGW (from @NetRanger) do not have these problems, as they do not use the platform toolset with the bug.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #160 –
Quote from: Bogozo on
Less extreme example - https://aylwin.bandcamp.com/album/farallon. Unlike Merzbow, this is music, just insanely compressed/distorted/clipped.
-8 - 405 730 765 bytes / 1229 kbps avg
-8 -r 8 - 394 267 319 bytes / 1195 kbps avg

What the actual f**c?  This is the weirdest compression result that doesn't come from chiptune. (And people: the album is for free, so go ahead play with it!)
Look at these bitrates:

1230 for FLAC -5  <--- FLAC results measured with 1.3.1, as the new build doesn't make for much
1230 for TTA
1229 for CUETools.FLACCL (2.2.2) at -11 --lax
1229 for Monkey's "High"
1227 for TAK -p4m
1227 for FLAC -8pe
1227 for MPEG-4 ALS at default and at the only slightly smaller and utterly slow -7 -p
1226 for Monkey's "Insane"
1226 for ALAC by ffmpeg
1225 for ALAC by refalac
1213 for WavPack -hhx
1207 for WavPack -hx
1206 for WavPack -hx4 (397 757 506 is smaller than -hx6 at 397 833 058 is smaller than -hhx4 at 397 863 164 is smaller than -hhx6  at 397 957 370).
1195 for FLAC -r 8 (= -5 -r 8)
1187 for CUETools.Flake (2.2.2) at -11 --lax
1154 for OptimFROG --preset 5
1150 for OptimFROG --preset 8
1142 for OptimFROG --preset 10
1139 for OptimFROG default (= --preset 2!!)
1131 for FLAC -r 9 --lax
1090-ish for SAC at default. A codec that isn't usable for playback.
1073 for FLAC -r 10 --lax (only the Stellar Descent tracks benefit, the first track is to the byte same size)
1058 for FLAC -r 11 --lax (ditto) - and same byte count for -r 12. 1071 for -r 12,12 enforcing the "12" as minimum.
1058 for FLAC -8 -r 11 --lax -  and same byte count going to -r 12
1052 for FLAC -8pe -r12 --lax, didn't try 11. New FLAC build improves 12 kilobytes.

Stepping up "r" means the block is partitioned further, but interestingly: "partitioning" the block by -b 2048 does only harm (... as usual; as does -b 8192; FLAC seems well tuned for the default 4096 on CDDA.)

Now if you didn't think this was weird enough:
The results are even more extreme for the Stellar Descent tracks (2 and 3):
flac -8pe -r 12 improves 17.8 resp. 21 percent over -8pe
... and > 10 percent over OptimFROG and 7 percent over sac.


What kind of distortion are they applying?!



As for Venereology, the live track 3 "I Lead You Towards Glorious Times" is the least compressible piece of music I ever bought, so I've used it in a few tests:
https://hydrogenaud.io/index.php/topic,120158.msg1001917.html#msg1001917 and the next reply #51 (therein I also did something about -r)
https://hydrogenaud.io/index.php/topic,122040.msg1010086.html#msg1010086

Re: FLAC v1.3.4

Reply #162 –
Hint: it is not "ALAC". And it is certainly nothing as offensive as something pronounced anywhere close to doubleyou-emm-ey-ell.


And I had my jaw dropped so much that ...
Quote from: Porcus on
(And people: the album is for free, so go ahead play with it!)
... I forgot parentheses around "with".

Re: FLAC v1.3.4

Reply #164 –
Quote from: Porcus on
What kind of distortion are they applying?!
Looking at the waveform, it seems a large part of the track is almost a square wave. So, it isn't far from chiptune.

Quote from: Porcus on
1131 for FLAC -r 9 --lax
1090-ish for SAC at default. A codec that isn't usable for playback.
1073 for FLAC -r 10 --lax (only the Stellar Descent tracks benefit, the first track is to the byte same size)
1058 for FLAC -r 11 --lax (ditto) - and same byte count for -r 12. 1071 for -r 12,12 enforcing the "12" as minimum.
1058 for FLAC -8 -r 11 --lax -  and same byte count going to -r 12
1052 for FLAC -8pe -r12 --lax, didn't try 11. New FLAC build improves 12 kilobytes.
Interesting. The r value goes up exponentially. So -r12 means: split the block up into 2^12 partitions, which is 4096 partitions. That is equal to the blocksize, and doesn't really make sense. Furthermore, FLAC limits r internally, as specified by the format. The number of samples in the first partition has the predictor order subtracted, and the number of samples in a partition cannot be zero. Therefore, -r12 can only lead to 4096 partitions when a fixed predictor with order 0 is used, which isn't very useful here. When a predictor order of 1 is used, -r12 is internally limited to -r11, and when a predictor order of 2 is used this is further limited to -r10, with a predictor order of 4 to -r9 and so forth.

It seems reasonable to assume -r12 won't differ from -r11.

However, the fact that -r11 is smaller than -r10 is really stunning indeed. It means that for each 2 samples, one rice parameter is stored, which is a 4-bit overhead AND it means the predictor order is no larger than 1. It is very rare this pays of, but it does here.

So, why does FLAC outperform very advanced codecs like SAC here? Because these high partition orders make FLAC able to 'switch' extremely quickly if the signal does so too. If you look at the signal, it is almost a square wave: you have some samples (about 10) with exactly the same value, then a upwards slope for a few samples, than some more samples with exactly the same value, then a downward slope. With an extremely high partition order, FLAC can spend few bits on the samples with the exact same value and spend more bits on the slopes. If you use -r6 like the presets do, all these samples need to use the same number of bits (more exact: the same rice parameter) to be described.

It seems this is something only FLAC does, apparently, if it beats all other codecs. It is however specific to certain kinds of music, it seems.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #165 –
Quote from: ktf on
However, the fact that -r11 is smaller than -r10 is really stunning indeed.
The fact that -r 12,12 gives smaller files than -r 10 is even more jaw-dropping then?
(... on both tracks 2 and 3 and the average, but not on track 1)
Same thing happens on -b16384 -r 14,14 which improves over -b16384 -r12.

I didn't keep good enough track of what I tested -b 8192 on ... yes -r 12 improves on -b 8192 and so that makes sense.
Encoding the first file with --lax  -r 11 and the two Stellar Descent tracks with --lax -b 32768 -r 14, and we are down to 1057 even without -8ep.  Putting -8ep there doesn't improve enough to get it past 1057.

ffmpeg doesn't make magic here (I mean it still beats TAK -p4m so don't complain, but nothing near the flac -r 11 sizes). More figures:
1226 ffmpeg default and ffmpeg -compression_level 12
1206-ish for La -high
1204 ffmpeg -compression_level 12 -lpc_type cholesky -lpc_passes 9
 lowest git flac build without directly accessing -A
1080-ish for SAC with --sparse-pcm, ran overnight. It - and OptimFROG - outcompresses FLAC by 3 to 4 percent on the first track, so again it is 2 and 3 that are the big WTFs

Re: FLAC v1.3.4

Reply #167 –
Quote from: NetRanger on
FLAC git-5d1402ea 20220831
Built on August 31, 2022, GCC 12.2.0
For anyone interested, this can be considered a release candidate. I don't plan to change anything else except documentation, changelog etc. for the next release, except when new bugs are found.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #168 –
FLAC-1.4.0.RC1-git5d1402e compiles at Rarewares.

Re: FLAC v1.3.4

Reply #169 –
Quote from: ktf on
For anyone interested, this can be considered a release candidate. I don't plan to change anything else except documentation, changelog etc. for the next release, except when new bugs are found.

Decision already made I guess, but I think the following deserves a second thought - admittedly, without giving a shot at my illiteracy at reading code:
Quote
* Consider whether subdivide_tukey(n) only shall mean to compute n and its divisors; that is, subdivide_tukey(8) splits into 8, recycles to form combinations of the 8 (that includes 1, 2 and 4 and would make for 15, am I right? The 8ths don't mean it computes all 2^8-1 combinations?)

(An alternative to 1,...,n and to "n and its divisors" could be the least common denominator of (1/1,...,1/n). 1, 2, 6, 12, 60 and then who cares that the next is 60 too.)

Reason for considering this:
My hunch is that the "3" would have to calculate "all three thirds" and "both halves" (five in total) before it starts re-using calculations - and those five aren't much less than calculating all six sixths and combining them for thirds and halves?

(On the other hand: if tapering is different on the "three thirds" routine than on the "two halves" routine - possibly too different from the "single" routine - what then? Is it? If so, it would mean that you fire three differently tapered "single tukey" windows at the block, and it is up to testing to tell whether that is bad use of time.)

Re: FLAC v1.3.4

Reply #170 –
Quote from: Porcus on
Reason for considering this:
My hunch is that the "3" would have to calculate "all three thirds" and "both halves" (five in total) before it starts re-using calculations - and those five aren't much less than calculating all six sixths and combining them for thirds and halves?
No, that is not how it reuses calculations.

First is calculates a tukey window for the whole block, by default tukey(0.25) when using subdivide_tukey(2) and tukey(0.166) when using subdivide_tukey(3). Then, it calculates a tukey window for the first and second half of the block. No reuse here yet, but even this calculation is implemented faster then it was before. Then, (when using subdivide_tukey(3) or higher) it calculated a tukey window for 3 thirds of the block, and then subtracts each of these from the original tukey to get the 'punchout' variants. For subdivide_tukey(4) this is repeated. Practically, the punchout variants are now 'free' in a sense that they don't need to calculate autocorrelation by going over the samples but by simply subtracting previous calculations.

It is not possible to combine sixths to get halves and thirds because there is some overlap.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #171 –
Quote from: ktf on
Practically, the punchout variants are now 'free' in a sense that they don't need to calculate autocorrelation by going over the samples but by simply subtracting previous calculations.

It is not possible to combine sixths to get halves and thirds because there is some overlap.

Dumb question but: if you tried to combine, it would not be the same - well then, wouldn't that mean you get more slightly different windowing functions pretty much for "free"? Say:
Does that "overlap" mean that adding up the two halves of the "partial_tukey(2)", would get you a different "whole block but tapered" windowing function than the tukey(.25) and the tukey(.5) already calculated - and it would be essentially "free" but isn't done? And if you simulated the first part of a "punchout_tukey(2)" by taking the the tukey(.25) and subtracting the second half - that would be different than taking the first half, and it would be essentially "free" but isn't done?

Re: FLAC v1.3.4

Reply #172 –
Quote from: Porcus on
Dumb question but: if you tried to combine, it would not be the same - well then, wouldn't that mean you get more slightly different windowing functions pretty much for "free"?
I probably shouldn't have used the phrase "for free". Yes, flac doesn't have to calculate the autocorrelation but it still needs to calculate the residual and find the smallest possible rice partitioning, so it makes no sense to just try all possible combinations.
Music: sounds arranged such that they construct feelings.

Re: FLAC v1.3.4

Reply #173 –
Quote from: john33 on
FLAC-1.4.0.RC1-git5d1402e compiles at Rarewares.

I used that to reencode a file I encoded with an 1.3.4 version from the last weeks (don't know which one, but the size of the flac.exe is 1.052.160 bytes), and the resulting file is *way* bigger (6.88MB vs. 7.2MB). I did it through the fb2k menu, so maybe there was a change where parameters get silently dropped somewhere? I used level 8.
 

Re: FLAC v1.3.4

Reply #174 –
Stupid question, but: sure it is level 8? Asking because many users have been playing around with different fb2k versions lately, major beta coming out.