Skip to main content

Topic: LossyFLAC yielding higher bitrate than FLAC alone (Read 7305 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Hex144
  • [*]
LossyFLAC yielding higher bitrate than FLAC alone
Hi

I'm not aware of this issue being known so here it goes...
I noticed that LossyWAV + FLAC (with flac block size of 512) may produce larger files than FLAC alone for some simple music material that already encodes at low bitrates with FLAC.

tools used: lossyWAV 1.3.0, FLAC 1.2.1
example bitrates for a track which consists of solo piano (44.1 kHz, 16 bit, stereo):

Code: [Select]
flac -8     375
-q E, -8    393
-q S, -8    390

flac -5     379
-q E, -5    395
-q S, -5    393


Any thoughts?
  • Last Edit: 09 May, 2013, 11:52:32 AM by Hex144

  • Dynamic
  • [*][*][*][*][*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #1
This is known in the development threads, particularly with clean recordings of simple classical and solo tonal instruments at modest volume levels. It is a result of the shorter block length (512 samples versus 4096 for FLAC which reduces the lossless encoder's efficiency a little) and the relative lack of bits removed by the lossy process compared to loud, spectrally-dense materials (partly because of the volume level, partly because there may be areas of the spectrum with no content, causing lossyWAV to assume there's a very low noise floor and no scope to remove bits.

It's possible that the more aggressive settings such as -C (eConomic) -P (portable) and -X (eXtraportable) would still offer a reduction, especially on the most recent versions that employ adaptive noise shaping to increase the bit-depth reduction and reduce bitrate (-X reaching around 310 kbps typically).

It's not something we worry about unduly as the files are small in any case and the loss of compression on these is far, far smaller than the compression advantage gained on more typical music. This effect doesn't seem to be observed unless the lossless FLAC bitrate is less than about 500 kbps or so.
Dynamic – the artist formerly known as DickD

  • Hex144
  • [*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #2
So what I'm getting is that it's not really LossyWAV's "fault", but FLAC's, with its small block size.
I'm more interested in the higher quality presets, for archiving music, and this thing seems to happen on many of the quieter tracks (for example, using 'Extreme', a Chopin nocturne can grow by 7% while probably not being bit-for-bit lossless). Guess I'll have to be careful.

Thanks for replying.
  • Last Edit: 09 May, 2013, 01:23:49 PM by Hex144

  • carpman
  • [*][*][*][*][*]
  • Developer
LossyFLAC yielding higher bitrate than FLAC alone
Reply #3
Might be useful:
http://www.hydrogenaudio.org/forums/index....st&p=718766

My view:
Personally on solo piano music, I skip lossyWAV (TAK on its [own] performs very well, approx 350 - 550 kbps).

C.
PC = TAK + LossyWAV  ::  Portable = Lame MP3

  • halb27
  • [*][*][*][*][*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #4
The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
For quiet classical music and very high lossyWAV quality settings the situation described may occur not very seldom.
But you can always compare the file size lossyWAV | FLAC and FLAC, and use the more appropriate files, or decide a priori according to the musical contents as carpman said.
  • Last Edit: 09 May, 2013, 03:45:59 PM by halb27
lame3995o -Q1

  • Hex144
  • [*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #5
Thanks Halb and Carpman.

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
LossyFLAC yielding higher bitrate than FLAC alone
Reply #6
The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
It's needed to make lossyWAV more efficient, but it still works with a 4096-sample blocklength - and in these rare cases, works better. Whether it works well enough to bother with lossyWAV at all depends on the source material. A solo piano recording with significant background noise (e.g. an old analogue recording) might benefit considerably. A pristine digital recording with low noise would benefit less. It is theoretically possible, even with the same blocksize, for lossyWAV to increase the bitrate - but I've yet to see it.

Even without lossyWAV, FLAC itself could benefit a little from adjusting its blocksize for optimal efficiency depending on the input signal, but no FLAC encoder bothers. FLAC decoders are supposed to handle it.

Cheers,
David.


  • jensend
  • [*][*][*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #7
As a spectator, early on in LossyWAV's development I wondered about the 512 blocksize. Does the noise floor really change that much from one 10ms period to the next? Trawling through the archives I see 2BDecided started off with 4096 sample blocks because it was flac's default, then switched to 1024 sample blocks when that looked substantially better in his testing, but I wonder how extensive the testing was that fixed on 512.*

Might the optimal lossywav blocksize depend on musical content/genre and on lossywav aggressiveness? I think so. With the current lossywav, some classical music I had on hand compressed better with flac -b 512 when using lossywav 'high' quality but did better with -b 1024 when using lossywav 'extreme'. (In both cases the difference was <1% though.)

4096 samples is understandably too big- and regular FLAC doesn't benefit much from it either. Most of the benefit flac can get from >512 sample blocks is already achieved at 1024 (similarly, 512-sample blocks already eliminate most of the overhead from 256-sample blocks).

*Here's the only stuff I found about that in a quick search: this post refers to a post that's not in its thread anymore- maybe this one?

  • Hex144
  • [*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #8
The small FLAC blocksize is a requirement which originates from lossyWAV in order to make lossyWAV useful.
It's needed to make lossyWAV more efficient, but it still works with a 4096-sample blocklength - and in these rare cases, works better.

Thinking about it, it makes sense since lossyWAV does little to no pre-processing. So encoding a few "problem" files using flac's default blocksize led to the same bitrate as the flac original (no big surprise)...

  • Dynamic
  • [*][*][*][*][*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #9
On more thing - I seem to recall that TAK introduced features for improving lossyTAK compression by noticing when the bit-depth stayed the same for consecutive blocks of 512 and increasing the block size for those blocks adaptively, so TAK might not show the same 7% increase. TAK tends to compress a little bit more than FLAC anyway.

(I may be misremembering - it was only from a glance through TAK changelogs or HA forum threads - I never tried it)

The problem lies then with compatibility - TAK is not widely supported, while FLAC is.

I guess, if anyone felt it worthwhile, FLAC encoders could implement an adaptive block-size mode that looks at bit-depth in 512-sample chunks and reduces the block size to suit the consecutive blocks of equal bit-depth, while reverting to 4096 (or the set blocksize) in other cases. This would retain 4096-sample blocks when possible and handle lossyWAV efficiently with relatively little processor overhead. The only potential problem is if we discovered any decoders that fail to handle changing block size properly.
  • Last Edit: 12 May, 2013, 04:35:20 AM by Dynamic
Dynamic – the artist formerly known as DickD

  • lvqcl
  • [*][*][*][*][*]
  • Developer
LossyFLAC yielding higher bitrate than FLAC alone
Reply #10
There's also --merge-blocks option in Wavpack.

  • TBeck
  • [*][*][*][*][*]
  • Developer
LossyFLAC yielding higher bitrate than FLAC alone
Reply #11
On more thing - I seem to recall that TAK introduced features for improving lossyTAK compression by noticing when the bit-depth stayed the same for consecutive blocks of 512 and increasing the block size for those blocks adaptively, so TAK might not show the same 7% increase. TAK tends to compress a little bit more than FLAC anyway.

(I may be misremembering - it was only from a glance through TAK changelogs or HA forum threads - I never tried it)

Early beta releases of TAK 2.1.0 contained a dedicated LossyWav-Codec (basically a variation of the main codec). Advantages:
  • About 2 percent better compression for my test corpus.
  • Usually not more than 0.1 percent compression penality for the afore-noted problem files.

This codec had been removed because there seemed to be little interest.

Maybe i will reintegrate it sometime.

  • Dynamic
  • [*][*][*][*][*]
LossyFLAC yielding higher bitrate than FLAC alone
Reply #12
Thanks, TBeck. I guess the demand is going to be limited what with lossyWAV's modest use and TAK's limited device support compared to FLAC (and Wavpack which is in Rockbox) so lumbering people with extra features for a small userbase isn't terribly worthwhile unless lossyTAK becomes more widely used.

Thanks, lvqcl. Using Wavpack with --merge-blocks option may be a better option than FLAC for a user with a Rockbox DAP and this type of content. I noticed the switch was absent from one example in the lossyWAV Wiki (now edited) but was present in the fb2k and EAC lossyWV examples, so I'm glad you mentioned it here.
Dynamic – the artist formerly known as DickD