HydrogenAudio

Lossless Audio Compression => FLAC => Topic started by: BFG on 2012-12-01 17:51:02

Title: Residual Partition aka Rice option
Post by: BFG on 2012-12-01 17:51:02
I'm interested in getting the maximum possible compression with FLAC, regardless of encode time.  According to the documentation (http://flac.sourceforge.net/documentation_tools_flac.html#encoding_options), a setting of up to -r 16 is possible.  Yet anything over -r 8 results in an error code for me.  Am I missing something?
Title: Residual Partition aka Rice option
Post by: skamp on 2012-12-01 17:55:59
Add --lax to your command line.

Quote
The encoding parameters specified do not conform to the FLAC Subset and may not be streamable or playable in hardware devices.  If you really understand the consequences, you can add --lax to the command-line options to encode with these parameters anyway.  See http://flac.sourceforge.net/format.html#subset (http://flac.sourceforge.net/format.html#subset)
Title: Residual Partition aka Rice option
Post by: Glenn Gundlach on 2012-12-01 18:52:10
I'm interested in getting the maximum possible compression with FLAC, regardless of encode time.  According to the documentation (http://flac.sourceforge.net/documentation_tools_flac.html#encoding_options), a setting of up to -r 16 is possible.  Yet anything over -r 8 results in an error code for me.  Am I missing something?


Why waste the time to do this? Save a few KBytes on a disc? Last week I bought a 3 TB USB3 drive for $99. That's 3.3 CENTS per gigabyte. Each KByte you save will save you 3.3 MILLIONTHS of a penny. Given the way files are stored on the disc you may reap _zero_ savings - same number of clusters used. Obsessive compulsive behavior......  I'm just saying...
Title: Residual Partition aka Rice option
Post by: Porcus on 2012-12-01 20:31:40
Why waste the time to do this?


Because of a portable player that is almost full?
Title: Residual Partition aka Rice option
Post by: skamp on 2012-12-01 20:43:43
Because of a portable player that is almost full?


Use lossyFLAC
Title: Residual Partition aka Rice option
Post by: dhromed on 2012-12-01 21:55:43
Why waste the time to do this?


Proof of concept for the inquisitive mind.
Title: Residual Partition aka Rice option
Post by: BFG on 2012-12-02 00:43:56
Proof of concept for the inquisitive mind.

Exactly.  And, I'm limited to 32GB on my FLAC-enabled phone

Thanks for the info.  I didn't realize -r settings >8 required the --lax setting.  I'll stick with 8 then.
Title: Residual Partition aka Rice option
Post by: Makaki on 2013-06-04 01:13:36
It seems it's not always the case that higher values of -r give better compression.

I used the following commands for my test:
flac.exe -V --best -f Closer.wav -> 46333044 bytes
flac.exe -V --best -r 6 -f Closer.wav -> 46333044 bytes (this was to confirm the default)
flac.exe -V --best -r 7 -f Closer.wav -> 46333405 bytes
flac.exe -V --best -r 8 -f Closer.wav -> 46333613 bytes

So maybe different files may give different results, but it's not always the case. I finally did test -r 15 with --lax:
flac.exe -V --best -r 9 --lax -f Closer.wav -> 46331566 bytes
flac.exe -V --best -r 10 --lax -f Closer.wav -> 46298363 bytes
flac.exe -V --best -r 11 --lax -f Closer.wav -> 46253224 bytes
... same results for 11-15
flac.exe -V --best -r 15 --lax -f Closer.wav -> 46253224 bytes

These did show an increase in compression with each value, I'm curious why 7 and 8 would not do the same.
Title: Residual Partition aka Rice option
Post by: BFG on 2013-06-04 05:21:36
Interesting results.  I would suspect - though haven't tested - that varying the LPC order (-l), block size (-b) and LP coefficient search (-p or -q) would further affect the results.
Title: Residual Partition aka Rice option
Post by: compleat on 2013-09-18 04:47:06
Hello, I'm new to the forums (first post actually). Since I wanted to ask something regarding this option I thought there was no need to make a new thread, so I'll just post it here.
Following this http://www.stanford.edu/group/farmshare/cg...p/FlacLikeABoss (http://www.stanford.edu/group/farmshare/cgi-bin/wiki/index.php/FlacLikeABoss) , I tried using "-r 0,16", but flac says that the only valid values are from 0 to 15 inclusive (when the documentation says otherwise).
The only thing I could find regarding this was this bug report http://sourceforge.net/p/flac/bugs/352/ (http://sourceforge.net/p/flac/bugs/352/)
If anybody has any idea regarding this I'd appreciate it (or if the options from that link aren't the optimal ones to achieve the smallest possible filesize, I'd love to hear which are the ones  )
Edit: if it's of any relevance, I'm using the 1.3.0 version, downloaded with arch linux's package manager.
Title: Residual Partition aka Rice option
Post by: ktf on 2013-09-18 05:16:37
but flac says that the only valid values are from 0 to 15 inclusive (when the documentation says otherwise).

The tool is right, the documentation is wrong: 15 is the max here. Where did you find 16 in the docs? Only the man page and flac -H or somewhere else as well?
Title: Residual Partition aka Rice option
Post by: compleat on 2013-09-18 14:12:51
As you said, apart from -H and man page, http://xiph.org/flac/documentation_tools_flac.html (http://xiph.org/flac/documentation_tools_flac.html)
It even says "Use -r 0,16 to force the highest degree of optimization". I take it it used to be possible but now it's no longer? should I use -r 0,15 instead?
Title: Residual Partition aka Rice option
Post by: tuffy on 2013-09-18 16:13:49
I'm surprised it goes up to 15.  The Rice residual partition order is a 4 bit field and the 0xF value is reserved for an escape code.
Title: Residual Partition aka Rice option
Post by: ktf on 2013-09-18 16:42:13
should I use -r 0,15 instead?

Well, I wouldn't recommend it anyway (I don't think it will gain you much, but you can try anyway) but if you want to allow FLAC to use any number of partitions allowed by the spec, you should use that. Chances are very, very small any file you convert will have a partition order of 15 in any block. Partition order 15 means 32768 partitions, which is not going to work with a blocksize of only 4096.

I'm surprised it goes up to 15.  The Rice residual partition order is a 4 bit field and the 0xF value is reserved for an escape code.

The rice partition order is different from the rice parameter, which for the standard rice partition (not RICE2, which is used only for 24-bit input) can be 0 to 14 with 15 being an escape code.

edit:
Quote
I take it it used to be possible but now it's no longer? should I use -r 0,15 instead?

It never did, it's just a mistake. Looking at the bug report you mentioned, it looks to me like even Josh himself confuses the Rice partition order with the Rice parameter. Anyway, using -r 0,15 is the same as -r 15, the first parameter (the zero) is optional and 0 by default.
Title: Residual Partition aka Rice option
Post by: compleat on 2013-09-18 18:06:21
should I use -r 0,15 instead?

Well, I wouldn't recommend it anyway (I don't think it will gain you much, but you can try anyway) but if you want to allow FLAC to use any number of partitions allowed by the spec, you should use that. Chances are very, very small any file you convert will have a partition order of 15 in any block. Partition order 15 means 32768 partitions, which is not going to work with a blocksize of only 4096.

Well, I did try with those parameters (the ones on the link above but changing -r 0,16 to -r 0,15) on a flac I had downloaded from bandcamp, and the filesize was a bit smaller (1 mb less on a gross aproximation, 69 vs 68 mb if using ls -lh to see the filesizes, don't remember right now exactly how many bytes). I was thinking in making a shell script to convert all my flac music to it, after all I can just leave it running in the background as I do other stuff, don't see a reason why not having the highest possible compression.
Always wondered why if I converted a flac file downloaded from bandcamp using the -8 option (using dBpoweramp on windows) the filesize always ended up higher. It's the first time I've managed to shrink it lol (albeit using the command line on linux).
Title: Residual Partition aka Rice option
Post by: ktf on 2013-09-18 18:21:07
[...] and the filesize was a bit smaller (1 mb less on a gross aproximation, 69 vs 68 mb if using ls -lh to see the filesizes, don't remember right now exactly how many bytes). [...] It's the first time I've managed to shrink it lol (albeit using the command line on linux).

You should compare apples with apples. Maybe those bandcamp files were encoded with FLACCL or Flake/FFMPEG, which are different encoders. To properly compare, you should first re-encode the files with FLAC to -8 and then to FLAC -8 -r 15. I'm pretty sure you'll see pretty much no difference, just like Makaki here (http://www.hydrogenaudio.org/forums/index.php?showtopic=98153&view=findpost&p=836148). Thing is, the -r switch doesn't do that much.

Anyway, I've just submitted a patch to the flac-dev list, this documentation error should be fixed in git soon (and in FLAC 1.3.1 of course)
Title: Residual Partition aka Rice option
Post by: compleat on 2013-09-18 18:50:11
Just gave it a try now, test8 is using the -8 option, r15 is using all the other parameters (including -r 0,15 obviously). This is from ls (deleted the unnecessary info)

106583276 test.wav
71390078 test8.flac
71102858 test_r15.flac

Difference is about... 280kb, pretty negligible I guess