HydrogenAudio

Lossless Audio Compression => FLAC => Topic started by: xdesirex22 on 2012-04-20 00:41:35

Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-20 00:41:35
Hello everyone, i have recently started converting all of my WAV/AIFF files to FLAC to save space, as i have read from various sources that FLAC is indeed true lossless. I am using XLD on Mac, with the settings seen here in this image:

(http://farm8.staticflickr.com/7192/6948728852_9ea382917a.jpg) (http://www.flickr.com/photos/46751632@N04/6948728852/)
XLD (http://www.flickr.com/photos/46751632@N04/6948728852/) by tstirling7 (http://www.flickr.com/people/46751632@N04/), on Flickr

Will these settings yield a true lossless conversion? I need to confirm because at some point i am going to be deleting my WAV/AIFF files, and i want to be completely sure that i am not making a mistake when i do this. I was listening to both the AIFF and FLAC of a track today, and i could've sworn the AIFF sounded a little bit louder and punchier, but i can be a bit paranoid (for lack of a better word) at times about things like this so i could've been hearing things.

Also, i have a second question. I read on a forum somewhere that audio conversion programs aren't perfect so with each conversion comes a small (possibly inaudible) loss in quality. This made sense to me. but my question is, How much does this loss in quality really matter? Is this even true with a program like XLD? thank you for your time.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: RobertoDomenico on 2012-04-20 01:07:31
If comparing AIFF and FLAC you must perform a ABX test otherwise you're falling to the placebo effect. Unless something went wrong with the conversion there is no way you'll be able to hear any different between lossless formats.

Your settings are fine. Your FLAC files will be perfect conversions.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-20 01:09:58
Ah, that is wonderful news. You have no idea what a clear confirmation like that means to me. I would be crushed if i learned that i ended up destroying my entire lossless library.

What is an ABX test? Sounds like something i should know about
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: tpijag on 2012-04-20 01:16:25
abx answer (http://www.google.com/cse?cx=partner-pub-4544327213918729%3Aeg2ypbddpax&ie=ISO-8859-1&q=ABX+test&sa=Search&siteurl=www.hydrogenaudio.org%2Fforums%2Findex.php%3Fshowtopic%3D94592%26pid%3D793094%26st%3D0%26%23entry793094&ref=www.hydrogenaudio.org%2Fforums%2Findex.php%3Fact%3DSearch%26nav%3Dlv%26CODE%3Dshow%26searchid%3D6c2621bd9a0f9bb07a153301b9c0b3d4%26search_in%3Dtopics%26result_type%3Dtopics%26lastdate%3D0#gsc.tab=0&gsc.q=ABX%20test&gsc.page=1)
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: RobertoDomenico on 2012-04-20 01:29:04
There is a quick guide here that will be helpful to a beginner XLD Guide (http://www.torrentech.org/index.php?act=Help&CODE=01&HID=68)
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Carledwards on 2012-04-20 01:45:56
There is a quick guide here that will be helpful to a beginner XLD Guide (http://www.torrentech.org/index.php?act=Help&CODE=01&HID=68)


Thanks for the link. Handy.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: 2Bdecided on 2012-04-20 12:27:14
You don't need to ABX lossless files - the process should be mathematically lossless. To verify, you just need to compare before and after file to ensure that they are bit-for-bit binary identical. No need to even listen to them.

Cheers,
David.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: db1989 on 2012-04-20 13:08:12
To verify, you just need to compare before and after file to ensure that they are bit-for-bit binary identical.
Compare the PCM streams that result from the two files, that is; don’t compare the raw WAV and compressed files, because that’s just asking for trouble.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: icstm on 2012-04-20 13:49:40
Yes compare the PCM streams and realise that all this formats are is clever long-word algorithm compression techniques. Think ZIP for music. Even in ZIP you have compression factors that are to do with how much computation should be done, but how much of the original data should be lost.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: db1989 on 2012-04-20 14:19:26
but not how much of the original data should be lost (because none ever is).
Fixed?

In which case, yes, and it provides a further opportunity to reassure xdesirex22 that the compression level solely affects the computational power and techniques used during compression; this may create a different FLAC file, but the stream that you would get from decoding it (note: some formats decode more slowly files that were compressed to a higher degree) would be exactly the same as that from any other FLAC file and, by definition, exactly the same as the original uncompressed stream.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-22 08:49:47
how can i compare pcm stream? is there a program that can do this?
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: db1989 on 2012-04-22 09:30:11
foobar2000, load and select two files, right-click, “Bit-compare”.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: skamp on 2012-04-22 10:26:46
You'll need the binary comparator (http://www.foobar2000.org/components/view/foo_bitcompare).
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: evereux on 2012-04-22 10:39:38
And a Windows PC or emulator since he is on a Mac.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nessuno on 2012-04-22 12:00:47
And a Windows PC or emulator since he is on a Mac.


This actually makes things easier: flac -d -c filename.flac | md5

If the computed values for both flac files are the same, so the decoded output streams are! 
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Chinch on 2012-04-22 12:25:59
First, to answer every question you have about this, reference my very detailed post regarding these very questions (including real life examples, results and even screenshots)-- I went into a lot of technical (well, technical to those needing to learn this stuff) points of the concept/process, but I like to think that I did it in a very easy to follow, elementary way. So I strongly urge you to read it, it may help you out tremendously; and who doesn't want to learn more about this stuff? Non-technophiles, but screw them!

Everything you ever cared to know about FLAC files, lossless encoding, proving the process is indeed LOSSLESS, including checksums/hashes (SFV, MD5, SHA-1, etc), bit-level comparison, and many more details that will make you a better person (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=94187&view=findpost&p=790714)

Yes, agreed with the few above -- FLAC files have an internal MD5 signature that ONLY considers the AUDIO STREAM block of the file, and hashes on it. It discards any artwork, vorbis comments, cuesheets... so when comparing two FLAC files, and viewing the MD5 values, if they match, then you will have an equivalently reliable result to using the binary comparator (which, I use all the time just because it's quicker at times, nothing wrong with the plugin!)

Alternative way to the above post, just for knowledge... -d switch decodes the file and -c decodes it to stdout (standard output) then the output is piped, but you (speak up if I'm somehow wrong) could just as easily use the -t (test) parameter:

-t, --test                        Same as -d except no decoded file is written

Here's a test I did on a randomly picked album:

>flac -t Deftones - Diamond Eyes - 2010.flac

Deftones - Diamond Eyes - 2010.flac: ok


or for more information, if you want an output file, then you can use the -a switch, but I do not recommend it, because a) you're not going to understand it anyway b) the output will be a .ANA file, and it's gonna be huge:

-a, --analyze                    Same as -d except an analysis file is written

Do not forget, if you want to check that the file is encoded and written to disk properly during ENCODE, simply use the -V (verify) switch, which will encode a block, write to disk, then turn right around and decode that block and compare that value to the other value still in memory of the unencoded original content. If you're using any decent computer made within the last 10 years, just use -8 for compression... your encoding time is going to probably be the same, but you'll save several megs of HD space per file depending on size. Either way, it's the old storage vs. CPU/processing trade-off law. Except in this case, with the CPU time "sacrifice" being so negligible, your best interest is to save hard drive space, which yields you a greater value in the end. That's my reasoning for using maximum compression always. I always use -V -8 to verify the encode and compress it to the max. Again, don't forget to read the post in the link at the top. You'll see exactly how and why LOSSLESS is LOSSLESS. Compression does NOT mean loss of any original data, by very definition, lossless means NO LOSS of the original content. Same as a ZIP file or RAR archive. The files shrink in size, but you don't have corrupted files extracting always because it removed data from what was originally there...
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nessuno on 2012-04-22 12:48:01
Alternative way to the above post, just for knowledge... -d switch decodes the file and -c decodes it to stdout (standard output) then the output is piped, but you (speak up if I'm somehow wrong) could just as easily use the -t (test) parameter:


Yes, but if I'm not wrong, the OP wanted to test if different XLD flac encoding options (so different flac files from the same source) have any result on decoded stream, so my test involves an actual decoding to PCM phase.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nessuno on 2012-04-22 14:10:37
Yes, but if I'm not wrong, the OP wanted to test if different XLD flac encoding options


If instead he wants to compare PCM data between an AIFF or WAV file and a FLAC one on Mac, he can use the compare function of the shntool into xACT (http://xact.scottcbrown.org/) application.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: greynol on 2012-04-22 17:08:43
This actually makes things easier: flac -d -c filename.flac | md5

metaflac --show-md5sum
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nessuno on 2012-04-22 19:17:12
This actually makes things easier: flac -d -c filename.flac | md5

metaflac --show-md5sum


Ok, we both know such a test is useless because FLAC is lossless, period.

The one showed by metaflac is a value computed at compress time, possibly from the input stream itself, and stored into the file, so stricto sensu not a fingerprint of the resulting stream after the decode process, as seen by an "external observer".
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-22 20:14:39
ok, a lot of this went over my head. are these tests necessary? I just want to make sure that im not screwing up my files. Im literally just opening them in XLD with the conversion options shown in the original post. I can't download Foobar because i am on a mac.

Is xact a comparable program? im downloading that right now.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: greynol on 2012-04-22 20:25:47
Ok, we both know such a test is useless because FLAC is lossless, period.

Why then is this discussion being fueled with paranoia?
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: db1989 on 2012-04-22 20:28:17
ok, a lot of this went over my head. are these tests necessary?
Not really, unless you’re someone who makes a point of always testing/verifying everything for yourself and/or doesn’t trust the concrete mathematics and testimonies of countless other users!

Quote
I just want to make sure that im not screwing up my files.
You’re not. Lossless is lossless. Again: Compression level affects computation and file-size, not quality.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-22 20:34:37
Quote
You’re not. Lossless is lossless. Again: Compression level affects computation and file-size, not quality.

i see. so if that's the case, then why wouldn't one always opt for the highest compression level?
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: garym on 2012-04-22 20:39:15
Quote
You’re not. Lossless is lossless. Again: Compression level affects computation and file-size, not quality.

i see. so if that's the case, then why wouldn't one always opt for the highest compression level?


probably no reason not to these days. But in the not so distant past, there were a few players that  would choke a bit on the processing required to decode the -8 compressed FLAC file. For example, Some Squeezebox players used to choke a bit on 24/96 FLAC files if they were maximum compression, but not at -5 compression. Nothing to do with lossless, just the processing power within the Squeezebox players.  Since then, a firmware update has solved this problem.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: db1989 on 2012-04-22 20:42:24
Some people don’t want, or care, to use a non-default setting to gain what might be a negligible saving of space but also a sizeable increase in the processing power and time required. It’s a game of juggling numbers.

Have a look at the tests others have previously done, or—better—test a few files for yourself, assessing whether the additional time is worth the reduction in space.

It’s no more complex than that, certainly not a case of there being some shady reason for the choice. (Analogy: the option to read audio CDs at different speeds was recently invoked by a woo-wizard as evidence of this having some bearing on quality [in non-exceptional cases].)
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nessuno on 2012-04-22 21:02:49
Is xact a comparable program? im downloading that right now.


XLD is a CD ripper, AFAIK the only one on Mac that uses AccurateRip verify, and a format transcoder.
xACT is a collection of utilities, a sort of Swiss Army knife to manage audio files, maybe in a less straightforward way.
They both support some batch operations.
They both use internally the same Core Audio routines (for AAC, ALAC, AIFF etc...), FLAC reference distribution, Lame mp3 etc… so when doing the same operations their results are exactly the same. (edit: except ripping, for the reason above).

To use one or the other depends actually on your needs.
Title: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xdesirex22 on 2012-04-22 23:45:44
i appreciate all the help. I've learned a lot from this thread.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: xaos on 2018-09-14 11:38:47
Okay, I just had to create an account in case someone like me wants to find out if FLAC is truly lossless and ends up in this crazy thread. None of the commands recommended to the OP have any relevance to whether or not the FLAC files are going to have the same quality as the original .wav or .aif files. They will merely show whether or not the different compression settings for the FLAC files contain the same amount of information when output.

To find out if your original files and FLAC files are identical in quality (which they should always be), all you have to do is  import the two files on separate tracks in any DAW (even Audacity, which is free),, and invert the phase on one. When you play the play the file, you should hear absolutely nothing. This is the only sure fire way to be positive you are getting exactly the same thing with the FLAC file as you are the .wav or .aif files. Phase inversion will cancel the audio signal out completely. This is how noise cancellation headphones work. If you've placed both files at the exact beginning of the project, leave all the volume and panning controls alone, and do nothing but invert one file, silence will tell you that you've got a perfect copy of your original. Anything you hear is different in one file or the other.

Just be careful if you use Audacity not to do anything with your original files. Changes you make with it are permanent. It's a destructive editor, meaning that it actually applies the changes you make in the program to the file you placed in it.

MODERATION

Recommendation removed, per TOS #14.  I recommend you figure out how to use Audacity before you continue to say things that make you look stupid.

/MODERATION
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Chibisteven on 2018-09-14 12:26:52
Okay, I just had to create an account in case someone like me wants to find out if FLAC is truly lossless and ends up in this crazy thread. None of the commands recommended to the OP have any relevance to whether or not the FLAC files are going to have the same quality as the original .wav or .aif files. They will merely show whether or not the different compression settings for the FLAC files contain the same amount of information when output.

To find out if your original files and FLAC files are identical in quality (which they should always be), all you have to do is  import the two files on separate tracks in any DAW (even Audacity, which is free),, and invert the phase on one. When you play the play the file, you should hear absolutely nothing. This is the only sure fire way to be positive you are getting exactly the same thing with the FLAC file as you are the .wav or .aif files. Phase inversion will cancel the audio signal out completely. This is how noise cancellation headphones work. If you've placed both files at the exact beginning of the project, leave all the volume and panning controls alone, and do nothing but invert one file, silence will tell you that you've got a perfect copy of your original. Anything you hear is different in one file or the other.

Just be careful if you use Audacity not to do anything with your original files. Changes you make with it are permanent. It's a destructive editor, meaning that it actually applies the changes you make in the program to the file you placed in it.

There's an even easier way to check a compressed file if it's the same as the original.
https://www.foobar2000.org/components/view/foo_bitcompare

In fact you can batch check 2 different sets quickly and efficiently for any errors and redo the affected files that didn't get converted properly.  No need to do anything more than that.  Very handy tool if you're switching your lossless library from one existing lossless format to a different one and have to convert thousands of files in the process.

On a side note: Why does this topic exist?  FLAC is lossless.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Garathor on 2018-09-14 12:50:54
Okay, I just had to create an account in case someone like me wants to find out if FLAC is truly lossless and ends up in this crazy thread. None of the commands recommended to the OP have any relevance to whether or not the FLAC files are going to have the same quality as the original .wav or .aif files. They will merely show whether or not the different compression settings for the FLAC files contain the same amount of information when output.

To find out if your original files and FLAC files are identical in quality (which they should always be), all you have to do is  import the two files on separate tracks in any DAW (even Audacity, which is free),, and invert the phase on one. When you play the play the file, you should hear absolutely nothing. This is the only sure fire way to be positive you are getting exactly the same thing with the FLAC file as you are the .wav or .aif files. Phase inversion will cancel the audio signal out completely. This is how noise cancellation headphones work. If you've placed both files at the exact beginning of the project, leave all the volume and panning controls alone, and do nothing but invert one file, silence will tell you that you've got a perfect copy of your original. Anything you hear is different in one file or the other.

Just be careful if you use Audacity not to do anything with your original files. Changes you make with it are permanent. It's a destructive editor, meaning that it actually applies the changes you make in the program to the file you placed in it.

This is absurd and meaningless. First of all, there is not going to be any difference between the original file and the flac file, because flac is in fact a lossless format. And if you really need to see proof of that, you can simply decode a flac file and compare it to the original:
Code: [Select]
$ flac -o flacfile.flac original.wav
$ flac -o decoded.wav -d flacfile.flac
$
$ sha256sum original.wav decoded.wav
ccae65206b1b5320a7e19cc90c3b8ac8bb73d335be013e884bae7dcca3343c9d  original.wav
ccae65206b1b5320a7e19cc90c3b8ac8bb73d335be013e884bae7dcca3343c9d  decoded.wav
$
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Porcus on 2018-09-14 14:47:36
Gosh. Let me disregard the age of the original posting ...

Those who just go ahead and make the claim that the conversion will be lossless, can download this .wav file (http://www.mauvecloud.net/sounds/pcm3244s.wav), convert it to FLAC using foobar2000 and bit-compare. Guess what.
Then they can open a command-line window and try to do the same using flac.exe.

The OP was not using flac.exe directly, the OP was using a given application with given settings. Did anybody test whether that application with those settings shown in the screenshot, actually do what flac.exe would do? Or perhaps what foobar2000 would do? (Which I think should be considered a bug, by the way. @Peter , are you there?)
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Deathcrow on 2018-09-14 14:59:06
Gosh. Let me disregard the age of the original posting ...

Those who just go ahead and make the claim that the conversion will be lossless, can download this .wav file (http://www.mauvecloud.net/sounds/pcm3244s.wav), convert it to FLAC using foobar2000 and bit-compare. Guess what.
Then they can open a command-line window and try to do the same using flac.exe.

The OP was not using flac.exe directly, the OP was using a given application with given settings. Did anybody test whether that application with those settings shown in the screenshot, actually do what flac.exe would do? Or perhaps what foobar2000 would do? (Which I think should be considered a bug, by the way. @Peter , are you there?)


The OP of this very old thread was just confused about what lossless compression means and was also suffering under the very normal placebo effect (induced by bad advice from his acquaintances, about how converting files *always* leads to some reduction in quality).

There's no reason to assume that the files he created with XLD weren't perfectly identical encodings and I don't think there's much point in investigating the behaviour of a certain piece of software from >6 years ago.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Porcus on 2018-09-14 15:26:51
There's no reason to assume that the files he created with XLD weren't perfectly identical encodings

Is there any reason to assume that files created with foobar2000 are not perfectly identical?
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Deathcrow on 2018-09-14 16:33:35
Is there any reason to assume that files created with foobar2000 are not perfectly identical?

No, of course not. Why are we even talking about Foobar2000? As far as I can see people only suggested fb2k's bit comparator plugin to verify the validity of the encodes... which is a good idea.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Case on 2018-09-14 17:48:42
@Porcus Peter is well aware that there are theoretical audio formats that can't be piped through the player losslessly. Fortunately no one in their sane minds uses 32-bit integer or 64-bit float as audio format. You may find some comfort in knowing that the next version will report in properties whether a 32-bit file is in float or integer format.

PS: Even audio editors I have (Adobe Audition CS6 and latest Audacity) will simply load the file quietly as 32-bit floating point. No one should be surprised as 32-bit integer format is pure insanity. 32-bit float won't clip and it will still allow 150 dB of bit-perfect dynamic range in the non-clipping range.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Rotareneg on 2018-09-14 19:56:10
Just be careful if you use Audacity not to do anything with your original files. Changes you make with it are permanent. It's a destructive editor, meaning that it actually applies the changes you make in the program to the file you placed in it. I recommend downloading...

TOS 14. (https://hydrogenaud.io/index.php/topic,3974.html#post_tos14)

That is entirely false! Audacity absolutely does NOT destructively change the original files you load in it.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: j7n on 2018-09-14 20:50:33
It's a destructive editor, meaning that it actually applies the changes you make in the program to the file you placed in it. I recommend downloading [..] That way you're sure to keep your original files safe.
You would still have to press Save in a destructive editor (like Sound Forge) to commit any changes. Working on a sound file is as safe as working in a word processor. A DAW is called non-destructive because edits like trims and deletions are just instructions in a project file, and could be tweaked or undone in the future. But a DAW doesn't usually provide accurate sample-level editing capabilities, effects are typically realtime and hard to apply to a selection. An audio production application doesn't replace a destructive stereo editor. I never used Audacity, but heard that is also project-based, with export/eender, like a mixture between the two types.

Sound Forge and Izotope RX will open and show 32-bit int files, such as those output by SoX. Compressing these temporary work files losslessly is a waste of time though.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Nikaki on 2018-09-14 21:28:56
I vote for closing this nonsense thread.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Porcus on 2018-09-14 23:10:41
@Porcus Peter is well aware that there are theoretical audio formats that can't be piped through the player losslessly.
And - at least sometimes - issue transcoding warnings. It should warn here too. The infamous "lossless is lossless!" does not have credibility if lossless isn't lossless.

I raised this very question back in 2011 (https://hydrogenaud.io/index.php/topic,85943.0.html), and the reactions were quite hostile. First to me for even asking, and then to where it was due. Violating the "lossless is lossless" maxime without warning is bad and unnecessary.
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: lvqcl on 2018-09-14 23:30:54
foobar2000 also doesn't show any warning for 24->16 bit transcoding, or when some DSP is active, or when RG is applied during transcoding. Should it?
Title: Re: Is FLAC really lossless? / Do compression settings make a difference?
Post by: Porcus on 2018-09-15 05:09:33
foobar2000 also doesn't show any warning for 24->16 bit transcoding, or when some DSP is active, or when RG is applied during transcoding. Should it?
You have to actively choose to output to 16 bits, or to apply some DSP or to change volume. Just like if I re-dither 16 -> 16 bit transcoding, right?