HydrogenAudio

hi. what is the average filesize reduction when LossyWav is used prior to lossless encoding?

losslessenconding(WAV)=100%
losslessenconding(lossy.WAV)= aprox. ... ?


Given that there are plenty of lossless encoders, my question concerns the encoder which has the best performance (on average) in terms of final file size (not in terms of savings compared to the original size (withot lossywav)).

Best lossless encoder: TAK.
Second best lossless encoder and most universally usable: FLAC.

Saving of lossyWAV -P --altpreset | lossless is 50-55% that of lossless on average when used on pop music. (52% is saved on my test set of various old and new pop music for which plain FLAC -8 results in an average bitrate of 788 kbps).
Saving is better for 'wild' pop music when the lossless codecs' performance is worse.
Saving is worse for music with a lot of quiet and/or non-complex parts in it when the lossless codecs' performance is good to very good. This applies to many tracks of classical music or music originating from one or very few instruments.

My collection averages 882kbit/s in lossless FLAC and 378kbit/s lossyFLAC (--portable --altpreset) (i.e. 42.8% that of FLAC or 26.8% that of WAV).

thanks folks for your much-to-the-point answers. exactly what I hoped for.

@halb27: do the 50-55% that you mentioned refer to TAK or all lossless encoders on average?

My experience is with FLAC.
Improving with TAK doesn't change the result very much however.
As you can see from what Nick.C and I wrote percentage saved depends more heavily on source material, and this comes mainly from the performance of pure lossless encodings which varies a lot (lossyWAV | lossless in contrary results in a more stable bitrate dpending a lot less on source material).
50-55% saving is more of my personal experience with pop music containing a lot of ballads and a lot of music originating from the time before the loudness war. With today's popular music especially of the vivid kind saving is better, it's more like what Nick.C wrote, or even better.

halb27 and Nick.C,
you both mentioned FLAC as your preferred encoder for lossyWav.

Do you know if better compression can be achieved if another lossless encoder is is used in conjunction with lossyWav? What would be the best one in terms of smalles file size ?

If you read post #2 again, you'll get your answer.

I wouldn't see why 

I wouldn't see why

Best lossless encoder: TAK.
Second best lossless encoder and most universally usable: FLAC.

Maybe I should have explicitly said: best resp. second best lossless encoder for encoding the lossyWAV result.

As well as Nick.C as myself are using lossyWAV on a mobile DAP. That's why we use FLAC.
If I'd use lossyWAV for archiving on a pc system I'd use TAK. (Temporarily I really did, but not much later ended up re-encoding [being a bit too interested in the lossyWAV progress in order not to do so], so I had to collect my lossless tracks from backups and partially had to re-rip).

Maybe I should have explicitly said: best resp. second best lossless encoder for encoding the lossyWAV result.

Well, yes at first I guessed that's what you wanted to say. But then you went on arguing:

"My experience is with FLAC.
Improving with TAK doesn't change the result very much however.",

... which finally left me with not much of a clue.

How can TAK be the "best lossless encoder for encoding the lossyWAV result" when - simultaneously - "TAK doesn't change the result very much however" ?

My test for LossyWAV 1.2.0 --standard:

FLAC -8 -b 512 : 450 kbps
TAKC -e -p4m -fsl512: 430 kbps.

The difference is 450/430 = 1.0465 => 4.65%.

TAK yields the best performance on the lossyWAV result, but the difference isn't big usually from what I remembered.
lvqcl's results show however that the difference is bigger than what I memorized (though I didn't use -p4m).

Anyway:
If you do know that lacking TAK support isn't an issue, and if you want to have the filesize advantage of TAK no matter if you call it small or not-so-small: I suggest to use TAK.
In any other case: I suggest to use FLAC.

TAK best in terms of compression ratio

FLAC not too far behind in that comparison, while being much more compatible.

Does that make more sense now?


(Looks like i was slow this time!)

yes, the last 3 posts made it clear for me now. thank you!

as a follow-up question: lossyWav uses a blocksize of 512. Can this be changed? I couldn't see an appropriate setting in http://wiki.hydrogenaudio.org/index.php?ti...cation_settings (http://wiki.hydrogenaudio.org/index.php?title=Lossywav#Application_settings)

And for a codec to benefit from lossyWav, is it enough that it can be set to a blocksize of 512 manually, or does it ALSO have a built-in function similar to FLAC's "wasted bits" feature (i.e. question is if the codec must meet both requirements) ?

The blocksize is automatic and equates to approximately 11.6msec for 44.1kHz material and 10.6msec for 48kHz material. This is due to the default 1.5msec and 20msec (approximate) FFT window lengths used for the analysis.

For lossless encoding, the blocksize can be set to anything that you want, however for optimum encoding it should be set to 512 for 44.1/48kHz material, 1024 for 88.2/96kHz material and 2048 for 176.4/192kHz material.

And for a codec to benefit from lossyWav, is it enough that it can be set to a blocksize of 512 manually, or does it ALSO have a built-in function similar to FLAC's "wasted bits" feature (i.e. question is if the codec must meet both requirements) ?

WMA Lossless doesn't have 'blocksize' option yet it can benefit from lossyWAV.
On the other side, CUETools.ALACEnc have blocksize option, but cannot effectively compress files after lossyWAV.

So, only the latter feature is absolutely necessary.

WMA Lossless doesn't have 'blocksize' option yet it can benefit from lossyWAV.On the other side, CUETools.ALACEnc have blocksize option, but cannot effectively compress files after lossyWAV.

So, only the latter feature is absolutely necessary.

I see, the latter is a must but both would be desirable.

What about .ape ? The wiki says it's no supported.
Is there anything I can do about this (parameters, etc.) ?
Or is anything planned for a future version?


I fail to see why a series of zeros would NOT be duly compressed by ANY lossless codec. A simple run-length encoding would do this and isn't that part of any lossless codec? [confused]

The blocksize is automatic and equates to approximately 11.6msec for 44.1kHz material and 10.6msec for 48kHz material. This is due to the default 1.5msec and 20msec (approximate) FFT window lengths used for the analysis.

For lossless encoding, the blocksize can be set to anything that you want, however for optimum encoding it should be set to 512 for 44.1/48kHz material, 1024 for 88.2/96kHz material and 2048 for 176.4/192kHz material.

thank you!
is it correct to speak of a "virtual blocksize" in this case? because I didn't know that .wav files have something like a "blocksize" at all? I mean, it's not the same thing as "blocksize" for mp3 for example, right? I know, the question is very badly phrased, but can you understand it, or should I try to rephrase ?

Is there anything I can do about this (parameters, etc.) ?

No.

Or is anything planned for a future version?

According to Changelog (http://www.monkeysaudio.com/versionhistory.html), the last time encoder engine was improved was in ver. 3.99 (April 29, 2004): "Improved entropy coder for increased compression."

I see. thanks, lvqcl.

As Nick wrote for deciding on how many least significant bits can be rounded to zero lossyWAV has to do FFT analyses. These require a certain block which is 512 samples long for the usual 44.1 or 48 kHz sampled source material. The calculated number of bits that can be removed are removed for all the samples of the block, so in order for lossyWAV to be efficient the blocksize must be that small.
So lossyWAV blocksize at first is just an internal detail of the lossyWAV mechanism. But in its consequence the number of bits removed in the WAV data share the same block structure. For best efficiency of the overall process the block size of the lossless codec should be 512 samples as well. If another blocksize is used the lossless codec can only take advantage of the smallest number of bits removed among all the lossyWAV blocks that contribute to the current lossless block: waste!

In practice your choice is TAK, FLAC, or wvPack, and wvPack isn't very attractive as it desn't work well with small blocks.

that was an excellent explanation to me.

So lossyWAV blocksize at first is just an internal detail of the lossyWAV mechanism.

right. that's why I tried to coin the term "virtual blocksize" because it is a variable used internally during the processing (with all the subsequent implications you described), but once you see the final .wav output, there are no "boundaries" ... the blocks are "invisible" or "virtual" ... hence my use of words.

In practice your choice is TAK, FLAC, or wvPack, and wvPack isn't very attractive as it desn't work well with small blocks.

I like your "in practice" approach. thanks.

For lossless encoding, the blocksize can be set to anything that you want,

how ? I used the --longhelp switch, but I couldn't find a parameter for setting the blocksize.

Nick was talking about the blocksize setting of the lossless codec, from lossyWAV's point of view (that is neglecting the possibilities for such a thing with the specific lossless encoder used).

Yes, I think I got that. My question ("how ? I used the --longhelp switch, but I couldn't find a parameter for setting the blocksize.") is still relevant though, or not?

The codec-block-size is automatically defined within lossyWAV dependent on the sample-rate of the material being processed. There is no reason to change it manually.

the reason would be to serve a lossless codec, which you can't manually set to bocksize 512. So instead of adapting the codec to the needs of lossyWav, you would adapt adapt lossyWav to the needs of the codec.

IIRC the only codec that is compatible with LossyWAV but doesn't have blocksize option is WMA Lossless.

IIRC the only codec that is compatible with LossyWAV but doesn't have blocksize option is WMA Lossless.

well that would be a field of application then. Plus, I'd like to test for myself with different lossless codecs.

So...

how ? I used the --longhelp switch, but I couldn't find a parameter for setting the blocksize.

[sarcasm]
Oh, there's a very easy way you can do that:  Get the source code, the compiler, modify it at your pleasure and taste, and do not come here asking why it doesn't improve the result
[/sarcasm]


You know... you start to be nitpicking..

[sarcasm]
Oh, there's a very easy way you can do that:  Get the source code, the compiler, modify it at your pleasure and taste, and do not come here asking why it doesn't improve the result
[/sarcasm]

your sarcasm is inappropriate.
Nick.C himself said:

For lossless encoding, the blocksize can be set to anything that you want,

So I was merely asking : HOW ?

I don't see why this is nitpicking. I was just asking for an element of information that was missing in his statement.

I was expecting a parameter in lossywav.exe,
e.g.

--blocksize 1024

but since I couldn't find any such parameter, ... well I asked.

So I was merely asking : HOW ?

He means a switch in a lossless encoder, not in LossyWAV.

the reason would be to serve a lossless codec, which you can't manually set to bocksize 512. So instead of adapting the codec to the needs of lossyWav, you would adapt adapt lossyWav to the needs of the codec.

Which lossless codec, that makes use of the "wasted bits" feature, does not allow you to change the blocksize of the encoder? If the blocksize of the lossless encoder is different from that of the lossyWAV processing then all that will happen is that the output will be encoded slightly less efficiently by the lossless codec. There will not be a switch in lossyWAV (unless forked) which allows the user to select a processing codec-block-size. A larger codec-block-size results in less bits removed as the effect of a low bin result for one of the FFT analyses will affect the bits-to-remove from more samples when blocks are longer.

@chrizoo:

Judging from your numerous posts in numerous threads you initiated you're obviously very interested in audio compression.
BUT: It would be very welcome if you slow down a bit and appreciate a bit more the many answers you get and really try to understand them.
If - like in this thread - you're asking things again and again that have been answered JAZ's kind of an answer is going to start being adequate.

Quote

So I was merely asking : HOW ?

He means a switch in a lossless encoder, not in LossyWAV.

sorry, yes I see you are right.  I was asking if and how the blocksize of 512 can be changed in lossyWav and when Nick.C answered "the blocksize can be set to anything that you want" I thought he would answer my question, whereas he actually talked about something else.

Which lossless codec, that makes use of the "wasted bits" feature, does not allow you to change the blocksize of the encoder?

according to lvqcl:

IIRC the only codec that is compatible with LossyWAV but doesn't have blocksize option is WMA Lossless.

Not that I would use WMA though. I'd just would be like to make my own tests. Theory is good. Practice is better.

If the blocksize of the lossless encoder is different from that of the lossyWAV processing then all that will happen is that the output will be encoded slightly less efficiently by the lossless codec.

that's why I was trying to find out how to match the blocksize and if the only way to do so was for the encoder to accomodate lossyWAV's blocksize or if the reverse would also be possible.

There will not be a switch in lossyWAV (unless forked) which allows the user to select a processing codec-block-size. A larger codec-block-size results in less bits removed as the effect of a low bin result for one of the FFT analyses will affect the bits-to-remove from more samples when blocks are longer.

OK, thanks for clearing that up.

Judging from your numerous posts in numerous threads you initiated you're obviously very interested in audio compression.
BUT: It would be very welcome if you slow down a bit and appreciate a bit more the many answers you get and really try to understand them.

Not to mention trying some research by yourself before asking another umpteen questions.

It would be very welcome if you ... appreciate a bit more the many answers you get and really try to understand them. If - like in this thread - you're asking things again and again that have been answered ...

I do genuinely appreciate all answers I get here (I actually observed that I say like 2-3 times more often "thank you" than most folks whose questions are answered) and I really try to understand them.

And if I'm "asking things that have been answered" (assuming you mean "answered to me" and not in general, somewhere, someplace), then obviuously only because I lack intelligence or knowledge for a better comprehension of the answers. Either that or the answers given don't answer the questions asked, which sometims happens to be the case, too.

But that's not a big deal. In the latter case I try to rephrase my question and if someone is kind and patient enough to bear with me, I always reached a point where I understood the answers (or the answerer my - maybe badly phrased - question).

Not to mention trying some research by yourself before asking another umpteen questions.

well I actually do both