Skip to main content

Topic: Something that's been bugging me about FLAC... (Read 5708 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Something that's been bugging me about FLAC...
So, I have been using FLAC for many, many years now and I have ripped countless CDs and encoded them to FLAC for archive purposes as well as general listening. In all these years, something has been bothering me that I don't understand about this codec and I have decided to post here and see if I can get an answer to this nagging question. Not sure why I waited this long to bring this up, but anyways, here goes...

As you all know, FLAC has various "compression" settings which you can select when encoding the file (which never made sense to me since this is supposed to be a lossless codec). Now I understand that these "compression" levels have been likened to how WinZip compresses a file. I sorta get the analogy but there is something else here that still doesn't make sense to me. To illustrate my question/point better, I did a little experiment and listed my results below. I ripped the same exact song at 4 different "compression" levels and the resulting bitrate that I got for each rip is listed as well.

(Uncompressed) - 1412 kbps
Level 0 - 986 kbps
Level 5 (default) - 933 kbps
Level 8 - 931 kbps

So, based on these results, here is what is bothering me. Between the uncompressed setting and the level 8 setting, there is a loss of almost 500 kbps of data. In my mind, that is quite a substantial amount. I totally understand that whatever setting you chose will ultimately affect the end result bitrate of the file. I get that. What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".

How is that possible? What is in those ~500 kbps worth of data that is getting shaved off? I just don't understand how you can be losing this much data and still have the resulting file be considered lossless. Maybe it's too technical of an answer for me to understand but I figured if anyone would be able to answer this for me, the folks on the forum here would.

So, yeah... this has been nagging at me for awhile. Any help to make me understand this better would be greatly appreciated.

Oh and btw.... Merry Christmas! 
  • Last Edit: 25 December, 2011, 03:25:43 PM by TempestGarden

  • saratoga
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #1
What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".


FLAC uses a data compression, which basically stores information more succinctly:

http://en.wikipedia.org/wiki/Data_compression

  • benski
  • [*][*][*][*][*]
  • Developer
Something that's been bugging me about FLAC...
Reply #2
From a layman's point of view, most of the initial compression results from the fact that there is a lot of similarity between the left and right channels and therefore redundant data.  For example, instruments that are "center panned" (vocals, kick drum, bass guitar) are duplicated in both channels.

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
Something that's been bugging me about FLAC...
Reply #3
It’s called lossless compression for a reason.

http://en.wikipedia.org/wiki/Lossless_data_compression

Why are you using FLAC at all, never mind for many years, if you have no idea what it’s for? Please do the most basic research before using something and before asking questions like this.
  • Last Edit: 25 December, 2011, 03:36:00 PM by db1989

Something that's been bugging me about FLAC...
Reply #4
Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.
  • Last Edit: 26 December, 2011, 07:29:47 PM by db1989

  • saratoga
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #5
Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.


While I'm sure you didn't mean to be rude, if you come and just ask someone to find you information you could easily get from the forums, this website's own FAQ, wikipedia, the flac website, or thousands of other places on google, its relatively inconsiderate of everyone else's time.  People will probably be about as considerate back to you.

  • psycho
  • [*][*][*]
Something that's been bugging me about FLAC...
Reply #6
TempestGarden, imagine this scenario.

You have a simple text file with the content "I love you, I love you, I love you.". That's 35 characters = 35 bytes. Some very simple form of lossless compression would be to do the folowing (and something like that is happening also with FLAC, plus a lot of more complicated calculaions): The algorithm detects repeating pattern "I love you" and it stores it like this: "I love you"=a, then it detects the comma and does ", "=b and then it stores the data like this:
I love you=a
, =b
ababa.
That's 24 characters = 24 bytes.

You see, none of the information has been lost, because when you decode this, you'll get the exact same file as you had before compression.

I hope this helps.
lame -V 0

Something that's been bugging me about FLAC...
Reply #7
Yes, this does help and is a great example! I really appreciate you writing this in an easy to understand example. Thank you very much.
  • Last Edit: 26 December, 2011, 07:28:42 PM by db1989

  • Roseval
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #8
The compression ratio is a source of misunderstanding.
A lot of people thing that it works like the bitrates  in MP3 so more or less loss.
The compression ratio simply tells how many CPU FLAC  is allowed to use to find the best possible compression (linear prediction).
The  more time is allowed the higher the compression.
TheWellTemperedComputer.com

Something that's been bugging me about FLAC...
Reply #9
So, based on these results, here is what is bothering me. Between the uncompressed setting and the level 8 setting, there is a loss of almost 500 kbps of data. In my mind, that is quite a substantial amount. I totally understand that whatever setting you chose will ultimately affect the end result bitrate of the file. I get that. What makes no sense to me at all though, is how a file (at level 8 setting) can lose almost 500 kbps of data and still be considered "lossless".

Oh and btw.... Merry Christmas! 

If I'm reading you right, you're not actually struggling with the concept of lossless compression as such, but you feel that "lossless" should always result in exactly the same compressed file size, as nothing is discarded (perhaps not an unreasonable basic assumption on the face of it, regardless of the peevishness and discourtesy you've encountered).

Try thinking of a steel coil spring held between the palms of your outstretched hands - you can make it shorter by squeezing harder, but the more you compress it, the higher the price you pay in terms of effort you have to put in. However, regardless of how much or how little you squeeze it, the substance of the spring remains exactly the same, and in each case it will return to exactly the same state when it's released.

Similarly, with FLAC (or WinZip) you can achieve varying amounts of compression, but higher compression will require more work to be done - you'll need a more powerful CPU to achieve higher compression in a given amount of time, or conversely you'll need more time for a given amount of CPU power. Like the spring, though, all the original data is still there, regardless of the degree to which it's compressed.

It's really about juggling the resources (disk space/time/CPU power) available to you, and choosing the setting which is most appropriate for your individual needs.


Something that's been bugging me about FLAC...
Reply #10
^ Another great, easy to understand example. Thank you as well for explaining it to me like this.

The compression ratio is a source of misunderstanding.
A lot of people thing that it works like the bitrates  in MP3 so more or less loss.


Yes, I think you are spot on here. I believe what you said is exactly where the source of my misunderstanding came from.
  • Last Edit: 26 December, 2011, 07:29:01 PM by db1989

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
Something that's been bugging me about FLAC...
Reply #11
Maybe because I didn't fully understand how it worked which was made very clear in my first post. I have been using it THINKING I understood it.

Thanks for the "warm" response btw, I really appreciate it.

I wasn’t intending to be rude. I had to ask because I honestly don’t understand what you thought FLAC was doing for you if not reducing the size of your tracks (compression) whilst preserving the audio exactly (lossless), and why you used it when you were unaware of its purpose in this way. I’m glad that you’re beginning to understand lossless compression now, but I’m genuinely unsure how you could spend years using it without having gained any idea of its purpose, and (as has been said) it is something for which explanations (such as those you have received in this thread) are readily available, including this from the official site of FLAC:
Quote
FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio is compressed in FLAC without any loss in quality. This is similar to how Zip works, except with FLAC you will get much better compression because it is designed specifically for audio
Further information is available on its FAQ.

  • mjb2006
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #12
I think what the OP needs is just to understand that the official flac encoder's "compression levels" are actually aliases for certain combinations of settings; they are not a single setting being adjusted on a linear scale. From the documentation:

-0 .. -8   = Fastest compression .. highest compression. The default is -5.
-0, --compression-level-0   = Synonymous with -l 0 -b 1152 -r 3
-1, --compression-level-1   = Synonymous with -l 0 -b 1152 -M -r 3
-2, --compression-level-2   = Synonymous with -l 0 -b 1152 -m -r 3
-3, --compression-level-3   = Synonymous with -l 6 -b 4096 -r 4
-4, --compression-level-4   = Synonymous with -l 8 -b 4096 -M -r 4
-5, --compression-level-5   = Synonymous with -l 8 -b 4096 -m -r 5
-6, --compression-level-6   = Synonymous with -l 8 -b 4096 -m -r 6
-7, --compression-level-7   = Synonymous with -l 8 -b 4096 -m -e -r 6
-8, --compression-level-8   = Synonymous with -l 12 -b 4096 -m -e -r 6

The meaning of -l, -b, etc. can be found in the docs.

  • Speckmade
  • [*]
Something that's been bugging me about FLAC...
Reply #13
I love you=a
, =b
ababa.
That's 24 characters = 24 bytes.

Now this is an example of removing one type of so-called redundancy that isn't actually used with lossless audio compression AFAIK.
You might find it fascinating to find out not only about the already mentioned (stereo) channel decoupling, but also the prediction that takes advantage of the correlation between subsequent samples, and entropy coding that deals with values that appear more or less often (as an effect of the decorrelation).
Although, if you're unfamiliar with pulse-code modulation (PCM) you might wanna find out about that first...

  • saratoga
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #14
That is a simple form of entropy coding, which is used in FLAC.
  • Last Edit: 26 December, 2011, 07:29:17 PM by db1989

  • slks
  • [*][*][*][*]
Something that's been bugging me about FLAC...
Reply #15
To paraphrase what's already been said -

The different FLAC level settings (-0 through -8) just spend more computing power to search for increasingly more efficient ways to pack the same data. The higher the setting, the more CPU cycles (time) will be used to compress the file to as little bits as possible (smaller filesize).

If you are still concerned, you could compare your original, uncompressed .wav file to FLACs that have been encoded and decoded at various compression levels. Going through them, bit by bit, you will see that they are identical. foobar2000 has a plugin to do this easily, I believe it was called something like "Bit-compare".

  • halb27
  • [*][*][*][*][*]
Something that's been bugging me about FLAC...
Reply #16
A simple model for lossless audio compression:
Instead of describing each wave sample as a 16 bit integer as is done with wav files, it can be describred as the difference to the previous wave sample.  As the difference is a small value usually it can usually be described with far less than 16 bit. All it takes is an efficient and flexible coding scheme for encoding the difference values.
Real life lossless codecs like FLAC are more sophisticated taking in account more than 1 of the previous samples and the correlelation between left and riigh channel as was said before.
  • Last Edit: 26 December, 2011, 05:43:19 AM by halb27
lame3995o -Q1

  • Ron Jones
  • [*][*][*][*]
Something that's been bugging me about FLAC...
Reply #17
Try thinking of a steel coil spring held between the palms of your outstretched hands - you can make it shorter by squeezing harder, but the more you compress it, the higher the price you pay in terms of effort you have to put in. However, regardless of how much or how little you squeeze it, the substance of the spring remains exactly the same, and in each case it will return to exactly the same state when it's released.

That's a fantastic analogy. I'll have to remember that one.

I wasn’t intending to be rude. I had to ask because I honestly don’t understand what you thought FLAC was doing for you if not reducing the size of your tracks (compression) whilst preserving the audio exactly (lossless), and why you used it when you were unaware of its purpose in this way.

He was aware of this, as was stated in his first post:
Quote
I just don't understand how you can be losing this much data and still have the resulting file be considered lossless.

  • Speckmade
  • [*]
Something that's been bugging me about FLAC...
Reply #18

ababa

...isn't actually used...

That is a simple form of entropy coding, which is used in FLAC.

I don't understand - this doesn't sound right.
This is a dictionary coder. It replaces strings of arbitrary length with place holders whereas entropy coding replaces (fixed-lenght) symbols. FLAC uses Rice coding and Run-length encoding.

  • TBeck
  • [*][*][*][*][*]
  • Developer
Something that's been bugging me about FLAC...
Reply #19
I have to agree.

I don't know of any lossless audio compressor utilizing something like this.

Well, ok, once i read LPAC is coding two consecutive zeros as one bit, because no appropriate rice code for very low volume signals (consisting mostly of zeros) exists. But this is a very specific or unique way to deal with quite rare signals.
  • Last Edit: 26 December, 2011, 06:51:14 PM by db1989

  • Speckmade
  • [*]
Something that's been bugging me about FLAC...
Reply #20
no appropriate rice code for very low volume signals

FLAC uses RLE for that, as I understand.