Hello,
I have used FLAC casually for many years, to archive and to play a modest collection of albums.
Recently, I have had some problems and experiences that I am hoping some enthusiasts could address.
First, and simplest, I wanted to determine the compression ratio of particular files, that is, the the size used for the compressed audio within a file divided by the size that would be required if uncompressed. I use the standard command-line tools, running on a Linux desktop, and I found no way to report this simple information. I am forced to decompress the file to determine the uncompressed size.
Second, and more mysterious, I wanted to utilize the encoder's capabilities for greater compression to reduce the size consumed by the collection, even if only by a few percent of the total size. I used the command line tool to transcode the source FLAC files to new targets, while including the -8 switch. Astonishingly, around half of the result files were larger than the corresponding source. This effect was reported as a warning by the tool, and could be confirmed by examining the results. In fact, over all albums, the average size of the result file is only %0.6 smaller than the source. The files were originally encoded with a tool that I assume invoked the standard FLAC library, as I am not aware of any alternative. Certainly no commercial FLAC encoder, if any such thing exists, was used. I believe that the files were originally encoded with default parameters, which suggests that encoding them so as to target highest possible compression would make, contrary to the actual effect I observed, every one smaller, and by a wider margin. What of course is most strange is that even if the original encoding was aggressive, the result of a transcoding operation such as the one I performed should not make them larger, only, in the worst case, fail to make them smaller. But perhaps I do not understand the issues correctly. Have I missed any relevant details? Can anyone account for these peculiar observations?
Thanks.
[...] I wanted to utilize the encoder's capabilities for greater compression to reduce the size consumed by the collection, even if only by a few percent of the total size. [...]
-8 -e -p
but! no "few". only a hundredth of a percent. and! time/power costs are MUCH greater than the benefits of compression.
if you are talking about saving "few" percent - it's time to buy a new hdd.
I originally included the -e. switch I am seeing very similar effects even including -p. The extra option appears to have little practical benefit.
My collection is currently compressed at very close to 50% ratio. My understanding is that FLAC compression can achieve much higher ratios with more aggressive encoding settings, in some cases less that 40%. I am surprised that I have not been able to achieve this effect easily. Meanwhile, I am even more perplexed that my current files are somehow already encoded at better rates than what I can make the encoder do again.
I also would mention that as I watch the encoder run, it is often showing rates 10%-20% higher at the initial part of the an album, though those rates drop substantially around the point that encoding is 20% or 30% complete for that album. This observation is yet another baffling effect I am seeing. Does it provide any hints about what might be happening?
Hopefully someone can shine some light on these problems. Thanks.
You will not see much difference in compression ratio with slower settings with the FLAC encoder. To reduce file sizes you might need to change the format or downsample if you have high resolution recordings. TAK is a good format, but AFAIK not supported on Linux. Better compression ratios are possible with quiet sound, less high frequency content (not cymbals, or trumpets), and low stereo separation. A quiet intro would compress better, which might explain the better ratio you observed at the start of your example album.
Possible reasons why existing files had slightly better compression:
Encoder tried a list of -A windows.
Used smaller block size.
Used zero padding.
An alternate encoder such as ffmpeg (https://hydrogenaud.io/index.php/topic,117257.0.html) or cuetools was used.
compress ratio depends on source. in my experience: flac -8
Sabaton - Metalizer - 76.57%
Blackmore's Night - Ghost Of A Rose - 63.51%
Mylene Farmer - L'autre - 59.28%
classical music, maybe, less 50%
" I am surprised that I have not been able to achieve this effect easily."
difference between (flac -0) and (flac -8) approximately 6%...
TAK is better 2-3% than flac...
A quiet intro would compress better, which might explain the better ratio you observed at the start of your example album.
The effect is consistent though not universal, across a larger set of albums. It is not limited to one incidental occurrence.
Possible reasons why existing files had slightly better compression:
An alternate encoder such as ffmpeg (https://hydrogenaud.io/index.php/topic,117257.0.html) or cuetools was used.
So other implementations are available that might be more efficient than the reference? Are you sure that ffmpeg or cuetools doesn't simply embed or link with libflac? Does any serious possibility exist that I can get substantially superior results with the
combination of a different encoder
and more aggressive parameters?
I am aware of Flake, FLACCL (GPU accelerated) and ffmpeg encoders.
Substantially superior results will only be possible in special cases with unusual source material.
As was discussed in the other thread, ffmpeg achieved better results on an upsampled recording lacking high frequencies entirely (high relative to the sampling rate, example: Vangelis - "Nocturne"). TAK will better FLAC on near mono recordings where the difference between channels is small but non-zero (old music, some hip-hop), on surround sound, and WavPack will be superior on rare 18-bit material padded to 24-bit where the LSBs switch together (example: Eagles - "The Studio Albums 1972-1979").
I agree with m14u, that it is not worth the effort to recompress existing FLAC files, except in special circumstances where you know the original used low settings, or has excessive metadata embedded. There is also a chance of a user mistake, error or bug and the source is accidentally deleted during a mass conversion.
FWIW, here's the command line I use:
flac -8 -l 12 -b 4096 -m -e -r 6
That's the way it used to be before FLAC 8 was dumbed down awhile back.
First, and simplest, I wanted to determine the compression ratio of particular files, that is, the the size used for the compressed audio within a file divided by the size that would be required if uncompressed. I use the standard command-line tools, running on a Linux desktop, and I found no way to report this simple information. I am forced to decompress the file to determine the uncompressed size.
I guess that one of your tools would give you the song length, channels, sampling rate and bit depth (at least ffmpeg does). You can take that to calculate the original bitrate and file size (excluding metadata and container). For example for audio CD and 100s song:
100 * 44100 * 2 * 16 = 141120000 bit = 141120000 bit / 8 = 17640000 byte
(song length in sec) * (sampling rate in Hz) * (number of channels) * (bit depth) = (size in bit) = (size in bit) / 8 = (size in byte)
My collection is currently compressed at very close to 50% ratio. My understanding is that FLAC compression can achieve much higher ratios with more aggressive encoding settings, in some cases less that 40%.
No. 50 percent with -8, then forget all about 40 - regardless of codec. For 40, you need different music.
Have a look at the charts here: http://www.audiograaf.nl/downloads.html
The non-classical part of my CDDA collection compresses around 65 percent. The classical music part at 45. "Material is everything" is a slight exaggeration, but you get the idea.
FWIW, here's the command line I use:
flac -8 -l 12 -b 4096 -m -e -r 6
flac -h
-8, --compression-level-8, --best Synonymous with -l 12 -b 4096 -m -r 6 -A tukey(0.5) -A partial_tukey(2) -A punchout_tukey(3)
I originally asked how to find the compression ratio using the basic flac tools because that is all I was aware might be effective for finding that information, even though I failed to determine how. I have no special affinity for those tools, however, so if ffmpeg has this functionality, then it helps greatly to say how to use it for that purpose in addition to saying simply that it can be used.
I originally asked how to find the compression ratio using the basic flac tools because that is all I was aware might be effective for finding that information, even though I failed to determine how. I have no special affinity for those tools, however, so if ffmpeg has this functionality
I'd use foobar200 for this purpose. On Linux it is possible to use it with Wine. Formula for calculation of compression ratio (in %) will be
$muldiv(%__bitrate%,100,$div($mul(%__samplerate%,%__bitspersample%,%__channels%),1000))'%' You can display this info as custom column in playlist view.
No way to do it without a GUI and without WINE? I thought it should be a basic feature of a tool set. Always been able to do it for all kinds of other compression and archive containers.
for 44.1/16/2ch
(flac`s bitrate) / 1411 * 100 = ratio?
ffmpeg flac less effective than "original" flac
ffmpeg flac less effective than "original" flac
not exactly :)
ffmpeg 4.1.3 flac compression 12 / flac 1.3.2 -8
Sabaton - Metalizer - 76.76% / 76.57% (-0.19)
Blackmore's Night - Ghost Of A Rose - 63.65% / 63.51% (-0.14)
Mylene Farmer - L'autre - 58.87% / 59.28% (+0.41)
it makes sense to check ffmpeg on your collection
ffmpeg 4.1.3 flac compression 12
Do not compare that to flac -8. If nothing has changed, ffmpeg creates non-subset files at levels 11 and 12, so normal users would likely stay safe and don't go beyond 10. https://hydrogenaud.io/index.php/topic,108100.msg887080.html#msg887080
If you really feel like going beyond subset, then there's the lax option.
Edit: And if you really, really want another percent, then consider whether FLAC is that sacred. And some people do embed ridiculous-size album art in every track in an album ...
My collection is currently compressed at very close to 50% ratio. My understanding is that FLAC compression can achieve much higher ratios with more aggressive encoding settings, in some cases less that 40%.
No. 50 percent with -8, then forget all about 40 - regardless of codec. For 40, you need different music.
Have a look at the charts here: http://www.audiograaf.nl/downloads.html
The understanding I had that FLAC can get ratios much better than 50% was not intended to apply to music that specifically had already been found to compress at 50% with -8 encoding settings. My music was encoded long ago with unknown settings, likely the default of whatever software package was used, so it is my guess that they were not particularly aggressive. The claim of much better ratios was not specific to any particular input.
I understand that if music encodes with -8 at some ratio, then that ratio would be very close to the best possible encoding. Such is how the -8 option is documented.
My situation now is one of is trying to determine what amount of saving I can get by re-encoding, and how best to do it. I was hopeful for much more than a few percent, but I could still benefit from so little, since one of the potential targets is my mobile device. A few percent difference could completely determine whether another album fits on the device. For the hard disk only, I agree with the earlier post that a few percent is not a helpful savings.
In my encoding job, the -8 option produced negligible savings, and overall worse results compared to the source for many albums. This effect led me to consider that maybe my test was somehow invalid, and that by considering or resolving some other issue, I might somehow realize my hope of non-negligible, even significant, savings.
If you still want significant further size reduction in flac, you might look into lossyWAV.
You might be able to detect weak compression preset by scanning a set of files for the block size with metaflac.
$ metaflac --show-max-blocksize test.flac
1152
Or look for uncompressed FLAC (it exists) by the bitrate vs uncompressed bitrate as recommended above using Foobar.
Or look for encoder version <= 1.1.2 with 16-bit or <= 1.2.0 with 24-bit.
As for --lax, I don't think it is useful. It allows to specify a blocksize over 4608, but that leads to worse compression with the reference encoder. Usually 2048 to 4096 (default best) is optimal. ffmpeg can use 8192 blocks though.
I still think it is not worth the effort to scan the entire collection for a small percentage of poorly compressed files. You will not gain much over the default -5 preset and a recent encoder.
You might be able to detect weak compression preset by scanning a set of files for the block size with metaflac.[...]
"this is wonderful idea, but it did`t the work"(c)
flac -h
0-2 - b 1152
3-8 - b 4096
except detection 0-2 level...
It would detect the worst settings, where it might be justified to take any action. I suggested based on the fact that I found such files in my collection. Regarding the encoder versions, compression was significantly improved either in 1.1.3 or 1.1.4, and with RICE2 in 1.2.1 for 24-bit. It might be possible to look for uncompressed "verbatim" frames, but I don't know how to automate it.
I originally asked how to find the compression ratio using the basic flac tools because that is all I was aware might be effective for finding that information, even though I failed to determine how. I have no special affinity for those tools, however, so if ffmpeg has this functionality, then it helps greatly to say how to use it for that purpose in addition to saying simply that it can be used.
If you have ffmpeg installed you will probably have ffprobe.
Then with
ffprobe YourInputFileHere -hide_banner
you'll get something like
Input #0, flac, from 'q:\temp\temp.flac':
Duration: 00:04:16.22, start: 0.000000, bitrate: 939 kb/s
Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
You can parse that anyway you want or if it's easier:
>ffprobe "q:\temp\temp.flac" -hide_banner -show_streams
you'll get
Input #0, flac, from 'q:\temp\temp.flac':
Duration: 00:04:16.22, start: 0.000000, bitrate: 939 kb/s
Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
[STREAM]
index=0
codec_name=flac
codec_long_name=FLAC (Free Lossless Audio Codec)
profile=unknown
codec_type=audio
codec_time_base=1/44100
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
sample_fmt=s16
sample_rate=44100
channels=2
channel_layout=stereo
bits_per_sample=0
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/44100
start_pts=0
start_time=0.000000
duration_ts=11299439
duration=256.223107
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=16
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
[/STREAM]
You can even change the output format with e.g. -print_format json (available formats: default, compact, csv, flat, ini, json, xml).
If you know exactly what values you want you could use something like:
ffprobe -v error "q:\temp\temp.flac" -show_entries stream=duration:stream=channels:stream=sample_rate:stream=sample_fmt -of default=noprint_wrappers=1:nokey=1
that will give you just
s16
44100
2
256.223107
[...]
that will give you just
s16
44100
2
256.223107
...
mediainfo ?
...
I only have MediaInfo GUI installed so I simply used an installed ffmpeg/ffprobe to generate an example that the thread starter asked for.
My music was encoded long ago with unknown settings, likely the default of whatever software package was used, so it is my guess that they were not particularly aggressive.
OK. Settings may have been -8 originally though.
It is most often possible to find out what FLAC version is used to encode, but not the setting. If you are on Windows, I suggest foobar2000 - not only is it a good player, but it is also one where you get a lot of support here. It can display tool used (see attachment), and it can convert everything too.
I understand that if music encodes with -8 at some ratio, then that ratio would be very close to the best possible encoding. Such is how the -8 option is documented.
Yeah, pretty much.
There are limits to what whatever codec can achieve. FLAC did not target the absolute maximum compression level, but rather decoding speed (= battery life!). Imprecisely speaking, a heavier option will search further for patterns to compress, and that takes time; there is a heavier setting than "-8" that can do a "more brute force" search for a fewer-bytes representation, but that is slow. For practical purposes, a you would rather have it perform a "clever search" over functions that are tested to perform well.
-8 was revised some time ago, because there were found functions that seem to do a better job on the average. That does not rule out that new -8 does slightly worse than old -8.
Furthermore, if you compare file size after re-encodings, there is something called "padding". By default, the reference encoding pads up the metadata block with an empty 8 kB to make room for re-tagging; when that is spent, it will have to re-write the entire file, and that takes much more time than just overwriting a small part of a file. You can switch off in the encoding process, but I wouldn't.
My situation now is one of is trying to determine what amount of saving I can get by re-encoding, and how best to do it.
Here is what I would do, assuming that you have hard drive space to spare:
- Use foobar2000 or the audiotester.exe utility to check integrity of your lossless files. (If they are corrupt, you do not want to re-encode; re-encoding will leave the false impression that everything was right.)
- Use foobar2000 to convert with -8 with a naming structure that completely mirrors the one you had. (Any corrupt ones: copy rather than re-encode.)
- Use foobar2000 with foo_bitcompare to compare - bit by bit - old against new. Then you have safeguarded against any wrong overwrites.
Now you have a backup. If you absolutely want to keep the file-by-file smaller one, then use a copy utility that skips overwriting based on size.
(There is a "FLAC frontend", but it has eaten files of mine. Avoid.)
one of the potential targets is my mobile device.
If you cannot fit it all: Go lossy. mp3 or something.
Once in a while this topic arises and the last time i checked CUEtools flake -8 compressed slightly better here compared to flac -8 -ep on a mixed genre test corpus while being much faster.
It creates standard spec files but uses bigger blocksizes for high samplerate material.
You may use it with -b 4096 then for a blocksize of 4096 for everything like reference flac does.
FWIW, here's the command line I use:
flac -8 -l 12 -b 4096 -m -e -r 6
That's the way it used to be before FLAC 8 was dumbed down awhile back.
FLAC 1.3.2 with -8 -e -p
Talking Heads - Stop Making Sense (1999 Re-Issue)
01-Psycho Killer / 874 kbps
02-Heaven / 868 kbps
03-Thank You for Sending Me an Angel / 994 kbps
04-Found a Job / 940 kbps
05-Slippery People / 917 kbps
06-Burning Down The House / 960 kbps
07-Life During Wartime / 977 kbps
08-Making Flippy Floppy / 977 kbps
09-Swamp / 906 kbps
10-What a Day That Was / 953 kbps
11-This Must Be the Place (Naive Melody) / 942 kbps
12-Once in a Lifetime / 973 kbps
13- Genius of Love / 950 kbps
14-Girlfriend Is Better / 953 kbps
15-Take Me to the River / 933 kbps
16-Crosseyed and Painless / 978 kbps
Average bitrate: 945 kbps
FLAC 1.3.2 with flac -8 -l 12 -b 4096 -m -e -p -r 6
01-Psycho Killer / 874 kbps
02-Heaven / 868 kbps
03-Thank You for Sending Me an Angel / 994 kbps
04-Found a Job / 940 kbps
05-Slippery People / 917 kbps
06-Burning Down The House / 960 kbps
07-Life During Wartime / 977 kbps
08-Making Flippy Floppy / 977 kbps
09-Swamp / 906 kbps
10-What a Day That Was / 953 kbps
11-This Must Be the Place (Naive Melody) / 942 kbps
12-Once in a Lifetime / 973 kbps
13- Genius of Love / 950 kbps
14-Girlfriend Is Better / 953 kbps
15-Take Me to the River / 933 kbps
16-Crosseyed and Painless / 978 kbps
Average bitrate: 945 kbps
-----
So, basically identical. 527.4 MB for the album either way as reported by Nautilus, with foobar2000 having gone over them with Optimize File Layout and Minimize File Size. Same encoding time either way.
I'd like to point out that running over the files again when someone used an older FLAC encoder, you usually get smaller files and faster encode times with -8 -e -p, or maybe even just setting 8 by itself, and you don't have to deal with unwieldy command lines and dead slow encoding. Recent FLAC releases have improved compression.
Commentary for others, including OP:
WavPack Extra High (even more so with some additional processing, but after Extra High x3 you see severe diminishing returns for the encoding time) can still beat FLAC -8 -ep, but by about 1.25% or less on a typical album. foobar2000 is inaccurate when it says processing time has no effect on file size with WavPack. Extra Processing level 1 will get you the most bang for your buck when going past Extra High. Files might be 100 KB smaller than simply Extra High without severe slowdowns.
Most of the other lossless codecs either do worse than FLAC, have horrible licenses, or take an insane amount of CPU time to decode (sometimes all of the above) and aren't worth screwing around with.
If you need to free space, you could try WavPack, but for the most part FLAC -8 -ep is going to be your best bet because it is open source, has low decode requirements, and is pretty much universally supported.
Basically, with proprietary lossless codecs, you spend weeks encoding your media library over again so you can store 1% more music in a format where you might not ever get your data back out of it and it probably won't play on much of anything. (If it does, it might hog the processor and drain your battery.) And it may not actually be lossless because we can't tell what the QA process is to ensure that it's not hitting corner cases and silently corrupting your data. If that's your thing, go for it.
FLAC decodes quickly no matter what level you encoded it at while other formats, like Monkey's Audio, get a little extra compression at higher settings at the cost of causing whatever you're trying to play them back on to have a stroke. Someone encoded some files at APE level Insane or whatever and playing them caused my Core i7-6560U laptop to start stuttering. I got them over to FLAC as soon as possible and for like 2% larger files, they take almost no CPU time to play. :P
With compression, the 80/20 rule is very much in play. Almost anything should be fast enough to encode at FLAC level 8 at several hundred times real speed without the -e -p switches now, so if nothing else, just use FLAC 8.
Once in a while this topic arises and the last time i checked CUEtools flake -8 compressed slightly better here compared to flac -8 -ep on a mixed genre test corpus while being much faster.
It creates standard spec files but uses bigger blocksizes for high samplerate material.
You may use it with -b 4096 then for a blocksize of 4096 for everything like reference flac does.
I found some stuff that was encoded with Flake the other day and ran it through FLAC (1.3.2) -8 -e -p and it compressed better with FLAC. I guess your mileage may vary. I'd be skeptical of alternative FLAC encoders because there's a comprehensive test suite for reference libFLAC and probably not much testing at all with alternative encoders. Since the point is to preserve the original data, you may just want to play it safe and use the official software.
I checked to see if the rips passed AccurateRIP and they did in that case. Granted, it's not a comprehensive test, but I wouldn't make a habit of using software like Flake.
Simply encoding and decoding several files and matching MD5 sums does not a test suite make. ;)
I found some stuff that was encoded with Flake the other day and ran it through FLAC (1.3.2) -8 -e -p and it compressed better with FLAC. I guess your mileage may vary. I'd be skeptical of alternative FLAC encoders because there's a comprehensive test suite for reference libFLAC and probably not much testing at all with alternative encoders. Since the point is to preserve the original data, you may just want to play it safe and use the official software.
I checked to see if the rips passed AccurateRIP and they did in that case. Granted, it's not a comprehensive test, but I wouldn't make a habit of using software like Flake.
Simply encoding and decoding several files and matching MD5 sums does not a test suite make. ;)
In guess you missed the older threads over here. CUEtools flake is very well tested for several years now and had known minor errors fixed immediately when reported.
It is not about simply encoding several files.
For size tests you should use a recent CUEtools 2.16 or 2.17 compile encoding some of your files and not rely on some flake stuff you found the other day.
I found some stuff that was encoded with Flake the other day and ran it through FLAC (1.3.2) -8 -e -p and it compressed better with FLAC. I guess your mileage may vary. I'd be skeptical of alternative FLAC encoders because there's a comprehensive test suite for reference libFLAC and probably not much testing at all with alternative encoders. Since the point is to preserve the original data, you may just want to play it safe and use the official software.
I checked to see if the rips passed AccurateRIP and they did in that case. Granted, it's not a comprehensive test, but I wouldn't make a habit of using software like Flake.
Simply encoding and decoding several files and matching MD5 sums does not a test suite make. ;)
In guess you missed the older threads over here. CUEtools flake is very well tested for several years now and had known minor errors fixed immediately when reported.
It is not about simply encoding several files.
For size tests you should use a recent CUEtools 2.16 or 2.17 compile encoding some of your files and not rely on some flake stuff you found the other day.
I'm sorry for implying that Flake has problems if that's how you took it.
What I meant to say was:
Reference libFLAC has such a comprehensive test battery that must be passed before any release goes out that it takes hundreds of hours for it to run. On top of that, most software that supports FLAC uses reference by incorporation, which means that it has the most users, which means that corner cases are most likely found and fixed already, considering that reference libFLAC goes back 18 years if you go by when the bitstream format for FLAC was frozen and even longer if you count all development time.
It's more complex than some people have banged around on it a little bit and it doesn't seem to do anything horrible.
In Linux distributions, one of the reasons I stick with the default file system is that it's the default for reasons. Sure, there are geewhizbang alternatives (that have weaknesses to go along with their features), but one thing about the default file system is....it just kind of has to work, and if big corporations like Google and others are tossing around exabytes of data on it and it's used internally in billions of Android devices, and this and that, it tells me that squeaky wheels are going to get grease. Perhaps unsurprisingly, some of the worst data loss situations I've ever heard of came about when people went too far off the beaten path and started using some file systems that were "supported", kind of, but not widely used.
As far as the Flake encoder goes, I think Winamp used it, but Winamp ceased being relevant a long time ago, and I had to look up what this CUE Tools thing was, and it looks all Windows and .NET-ey and so my interest level is dropping. My interest level is dropping.
To be honest, after thousands of FLAC albums, I was surprised to see Flake having been used on a few, so that kind of tells me what kind of real world exposure it's getting.
If you are still interested in testing the encoder performance you may use this stand alone executable encoder over here https://hydrogenaud.io/index.php/topic,106446.msg903939.html#msg903939
I runs under wine and doesn't need net to run.
If you are still interested in testing the encoder performance you may use this stand alone executable encoder over here https://hydrogenaud.io/index.php/topic,106446.msg903939.html#msg903939
I runs under wine and doesn't need net to run.
Hooked CueTools Flake up to foobar2000 1.4.4 beta 1 on my Core i7-6560U laptop on Fedora 30 on an nvme SSD and the Ext4 file system. Both codecs used four encoder jobs (2 physical cores with hyperthreading) running at High priority.
Talking Heads - Stop Making Sense (1999 Re-Issue)
FLAC source converted to FLAC 8 using both libFLAC 1.3.2 and CueTools Flake
Reference libFLAC 1.3.2 with git patches up to November of 2018 as provided by Rarewares in their x86-64 bundle with SSE 3 optimizations compiled by the Intel Compiler:
Time: 15 seconds.
Average Bitrate: 946 kbps
Finished size (Reported by Nautilus after Optimize File Layout and Minimize File Size used in foobar2000): 528 MB
CueTools Flake:
Time: 55 seconds
Average Bitrate: 946 kbps
Finished size (Reported by Nautilus after Optimize File Layout and Minimize File Size used in foobar2000): 527.8 MB
-----------
Conclusion: Reference libFLAC is well over 3 times faster than CueTools Flake at setting 8 and the difference in file size and bitrate is negligible between the two. Reference libFLAC is also more thoroughly tested, as I mentioned previously.
I guess the good news is that according to the audio MD5 provided by foobar2000, Flake didn't _corrupt_ the output of the FLAC files it made, but this is one test with one album. Even if Flake doesn't have any potential data corruption issues, it is a very slow encoder.
-----------
Additional:
libFLAC 1.3.2 (same setup) setting -8 -e -p
Time: 6 Minutes 14 Seconds
Average Bitrate: 945 kbps
Finished size (Reported by Nautilus after Optimize File Layout and Minimize File Size used in foobar2000): 527.4 MB
Conclusion: Adding -e -p to libFLAC ended up resulting in 0.06% smaller files. That's 6/100ths of 1%. The cash value of a coupon in most states. And it requires 25 times as much encoding time to get this.
If the math holds up, you'd reclaim 153.6 megabytes or so on a 256 GB FLAC collection by running them through again with -e -p. Worth it?
Well, at the price of SD card space, that's about 2 cents worth of space.
And some people do embed ridiculous-size album art in every track in an album ...
I hate that so much when I get FLACs like that because the only easy way I've found to get rid of it is to remove the image file by file in Easytag and then toss the files in foobar2000 and hit "Optimize File Layout and Minimize File Size". There's probably an easier way.
Sometimes the difference isn't shocking, but other times you can weed out 7-8 MB or more of useless embedded album art from a given album.
Can't you do "Remove all pictures" in batch in Foobar2000?
I usually use Metaflac in frontah, but it is a 4 step process anyway. Add some text around 500 bytes, remove metadata block 'picture', remove padding, remove the text to keep a little bit of padding for future tag edits.
Can't you do "Remove all pictures" in batch in Foobar2000?
I usually use Metaflac in frontah, but it is a 4 step process anyway. Add some text around 500 bytes, remove metadata block 'picture', remove padding, remove the text to keep a little bit of padding for future tag edits.
For some reason, Easytag will remove the pictures in that they're still in the file but Nautilus can't see them anymore (I presume nothing can.). So that's all kinds of unhelpful. After using Easytag to do that, doing an Optimize File Layout and Minimize File Size in foobar2000 removes them for good, along with all of the padding, as it is essentially writing out a new file with a new tag.
But now that you mention it, a remove all pictures was in foobar2000 the entire time. d'oh!.
Jóhann Jóhannsson (2011) The Miners' Hymns
i8700k
flac 1.32 -8, ~12sec 208.518.631 Bytes
CUEtools flake -8, ~23sec 207.553.190 Bytes
flac 1.32 -ep -8, ~111sec 207.839.232 Bytes
Removed padding and optimized on all with foobar.
CUEtools is not to shabby against -ep here. I found on silent material CUEtools flake can do things more efficient as even flac -ep
You see thats why one should use a test corpus. I normaly drop ~50-100 albums in for conclusions.
Even this only represents its behaviour here with my music that i report so maybe others can compare.
Back to removing album art, you have to remove it then Optimize File Layout and Minimize Size in foobar2000.
Two step process. Still.... :P I just tried it. I wish embedded art was never allowed. What's wrong with folder.jpg?
I wish embedded art was never allowed. What's wrong with folder.jpg?
also: let's put tags into .txt files, who needs more than just plain audio stream in a file?
I wish embedded art was never allowed. What's wrong with folder.jpg?
also: let's put tags into .txt files, who needs more than just plain audio stream in a file?
Roll on WAV!
Two step process. Still....
But now you have no padding and a spelling correction or replaygain scan will cause rewriting of the entire file. I think the Optimize layout should leave a little bit automatically, but it is what it is.
Tags that apply to the entire release (album artist, cat number) could go into a single file, but the risk of losing them is too great, and a special application or plugin is needed to read such a format. With pictures it is the other way around. Any program can display or replace cover.jpg, its size is a rough indication of the quality. Annoying to see the cover.jpg not displayed because there is a low quality version embedded and forgotten.
comrades, by comparing flac8 & flake8 you compare warm and soft. if you take flac (max-compress), do it for flake (max-compress - 12).
comrades, by comparing flac8 & flake8 you compare warm and soft. if you take flac (max-compress), do it for flake (max-compress - 12).
No! CUEtools flake -8 creates standard files. Anything higher creates non-standard files that may not play everywhere. Besides that forcing CUEtools flake to lax settings is not really optimized and more like a leftover from vintage flake.
All explained in old threads over here.
[...] may not play everywhere.[...]
on hardwares? yes, in "old threads over here" says that. without "proof". but it's the little things.
there is not a word about it in the original question.
different between flake8/9/10 only in key (-m) - 3/6/5. anybody test this moment?
I wish embedded art was never allowed. What's wrong with folder.jpg?
also: let's put tags into .txt files, who needs more than just plain audio stream in a file?
A little bit of plaintext is basically nothing compared to the 1.5 MB JPEGs that I've seen certain people put into EACH music file * 100. :) At this point, you're talking 5-8% of a typical FLAC file being the freaking album art. Even if they're only 100 KB each, it's still obnoxious and people should cut it out.
That being said, I find ALAC rips too where freaking iTunes used the tags as an open sewer for a bunch of gibberish. Just the facts, ma'am.
I scoop out all of the iTunes crap from the tag before converting them to FLAC 8, and then optimize the FLAC files.
Any halfway decent music player app should be able to load lyrics in real time from a website without storing them in each and every file. Interestingly, iTunes lyrics tags don't seem to be picked up in any of my players, even though metaflac will happily copy them into an empty field if you let it. And worse, the iTunes lyrics are often wrong.
comrades, by comparing flac8 & flake8 you compare warm and soft. if you take flac (max-compress), do it for flake (max-compress - 12).
No! CUEtools flake -8 creates standard files. Anything higher creates non-standard files that may not play everywhere. Besides that forcing CUEtools flake to lax settings is not really optimized and more like a leftover from vintage flake.
All explained in old threads over here.
I haven't tried non-Subset in Flake, but when I did it up in FLAC, it brought each file down by about 100 KB at the cost of encoding at 0.8x of real time on my Core i7 6560U with four encoder processes running.
Even if I had the time to deal with non-Subset, the fact that decoders are not required to play them makes the proposition too iffy for my taste.
The point of Subset was mainly to prevent FLAC from turning into something more like Monkey's Audio where hardware couldn't play it back in real time in exchange for like 1 or 2% better compression. By making non-Subset difficult to use, it prevented people from accidentally stumbling into a situation where their FLAC files won't play on something.
Two step process. Still....
But now you have no padding and a spelling correction or replaygain scan will cause rewriting of the entire file. I think the Optimize layout should leave a little bit automatically, but it is what it is.
Tags that apply to the entire release (album artist, cat number) could go into a single file, but the risk of losing them is too great, and a special application or plugin is needed to read such a format. With pictures it is the other way around. Any program can display or replace cover.jpg, its size is a rough indication of the quality. Annoying to see the cover.jpg not displayed because there is a low quality version embedded and forgotten.
Actually, I've edited the tags after optimizing them and it does seem to leave some, because it did not have to write out a whole new file.
In fact, I went over them again and it said the size did not change. :)
"Tags in a single file", is, I think a reply to someone else? Anyway, that's essentially what you have with a FLAC image of an entire album and a CUE sheet.
I hate finding stuff like this because it means I _have to_ split it up myself. All so they could save like 1 MB and most of the time they didn't even use FLAC 8 along with it. :)
There's too many people out there using CD ripping software and FLAC who very obviously haven't the faintest idea of what they're doing.
I wish embedded art was never allowed. What's wrong with folder.jpg?
also: let's put tags into .txt files, who needs more than just plain audio stream in a file?
Roll on WAV!
I hope that was a joke. An audio collection in WAV/AIFF files is about the dumbest possible way to store them, but it's your money. :)
In fact, I'm not even sure why Apple ever created a lossless format. Much less one that's shockingly less efficient than something that already existed. It's not like any of their products have plenty of storage laying around unless it's a Mac, and even then, not guaranteed.
To Wombat: I'm not sure that "most efficient ALAC codec" is worth bragging about since there's essentially no case where ALAC could possibly beat FLAC for interoperability and robustness, except on Apple stuff that has no real storage anyway where no lossless codec could really be useful, where they have chosen not to support real standards, again. Most efficient ALAC codec means you're the best looking horse in the glue factory.
That being said, a simple conversion from ALAC (encoded with Apple's refalac) to FLAC -8 generally nets a reduction of about 15-18 kbps in my experience. Or 4-8 MB across an album. Well worth the 10 seconds to get it out of Apple's format.
ALAC is open source now, so there's no point bickering about software freedom either way, but it's not a good codec. Even if you have found a way to make the bitrate and performance comparable to FLAC, ALAC would still have no error robustness or even a way to detect corruption, thanks to the MP4 container.
With FLAC, I can ask if anything in my library is corrupt and know about it a minute or two later (impressive for such a large library) and then I can stomp anything that is corrupt with a backup copy. With ALAC, you may never know.
Even worse, from what I've read, ALAC files with more than a little corruption generally become entirely unplayable and it takes a lot to completely destroy a FLAC file even if you only had one copy, because there might be a dropout of 1/10th of a second somewhere, but it recovers the stream.
As usual, Apple "engineering" by marketing department and Not Invented Here mentality at its best.
If you have anything in ALAC it's best to use refalac to decompress it and then feed it into FLAC, and then to check again to make sure it matches AccurateRIP or something.
To Wombat: I'm not sure that "most efficient ALAC codec" is worth bragging about since there's essentially no case where ALAC could possibly beat FLAC for interoperability and robustness, except on Apple stuff that has no real storage anyway where no lossless codec could really be useful, where they have chosen not to support real standards, again. Most efficient ALAC codec means you're the best looking horse in the glue factory.
That being said, a simple conversion from ALAC (encoded with Apple's refalac) to FLAC -8 generally nets a reduction of about 15-18 kbps in my experience. Or 4-8 MB across an album. Well worth the 10 seconds to get it out of Apple's format.
Are you talking to my old upload thread? Back in 2014 when i posted this CUEtools ALAC -10 was almost equal to flac -8. At this time indeed it was the best compressing ALAC encoder for CD material i knew of.
I never used it but ALAC i can imagine is still the main lossless format for many.
I wish embedded art was never allowed. What's wrong with folder.jpg?
also: let's put tags into .txt files, who needs more than just plain audio stream in a file?
Roll on WAV!
I hope that was a joke.
Yeah, like that embedded comment ...
I use folder.jpeg quite a lot, but some releases have per-track art (https://en.wikipedia.org/wiki/The_Slip_(album)#Artwork). If you don't like embedded, don't use it.
In fact, I'm not even sure why Apple ever created a lossless format. Much less one that's shockingly less efficient than something that already existed.
Neither am I, but Microsoft also wanted their own thing.
One hypothesis: probably because Apple landed on the MP4 container. Which is video ready and ... and Apple had DRM in MP4, maybe that is a reason?
Yeah, it is mediocre at best (and sucks when it comes to integrity check ...) So what? Apple users do as they are told ... and how many would need an extra hard drive anyway? Lossless audio - outside BluRay discs - is a niche product.
FLACout - http://advsys.net/ken/utils.htm
Is a flac optimizer that can squeeze out a bit more efficiency at the cost of a lot of time
FLACout - http://advsys.net/ken/utils.htm
Is a flac optimizer that can squeeze out a bit more efficiency at the cost of a lot of time
It didn't really work the last time we tried https://hydrogenaud.io/index.php/topic,106129.msg868366.html#msg868366
About FLAC optimization -- I played with that for a while to see what could be done. I experimented with the apodization functions available -- and could usually see some improvement by adding some functions, and using some of the complete search type things.. If I got more than a few percent better on a normal file, I was happy that there was some progress.
For day-to-day, where it isn't practical to take real-time to compress a file, the default -8 does pretty good. Maybe a few percent can be had, at a cost of 10X more more the amount of compression time.
I seem to remember that adding blackman_harris_4term_92dB & kaiser_bessel sometimes helps (btw, I use those window functions on my software DolbyA decoder for certain subtile characteristics -- but those characteristics might not be why those apodization functions seemed to have helped.)
John
If I remember correctly, the current -8 also came about after experimenting with some apodization functions, and finding a pretty good combination?
TLDR;
If you have files with reduced bitdepth like Lossywav processed files you cannot achieve smaller files with recompression unless you use the right blocksize. For CD recordings this is 512. Lossywav files need some special attention.
I recently reencoded all my FLACs which were encoded with reference libFLAC 1.2.1 20070917 -8 to reference libFLAC 1.3.2 20170101 -8 and overall it saved about 8 GB or 0,35 %
before
2.228.270 MB
after
2.220.534 MB
(the data is somewhat not representative because I my last "optimize file layout + minimize file size" for EVERY file is some years back but the embedded cover art usually doesn't get changed ever)
I first started with -8 -ep which gave me about 5x realtime speed which I calculated would have taken a month, so I canceled and switched to -8 (without -ep) which went up to 500x realtime speed (5 yo quad core + HT)
https://xiph.org/flac/changelog.html
FLAC 1.3.1 (25-Nov-2014)
- New apodization functions partial_tukey and punchout_tukey for improved compression (Martijn van Beurden).
- Retuned compression presets to incorporate new apodization functions (Martijn van Beurden).