Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: WavPack Lossy - Arbitrary Bitrates (Read 17419 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

WavPack Lossy - Arbitrary Bitrates

I'm just experimenting a bit, for the first time, with WavPack and its hybrid mode.  This is probably one of the most innovative codecs that I've seen, and I really hope that some of the portables and such in the near future will support this.  (Neuros perhaps?)

I'm thinking of using the lossy mode so that I can create files at an arbitrary bitrate that will be completely transparent, and also be transcodable without any perceptable loss.  I've read that 384kb/s is "optimal" for this, but for my purposes, I have enough room (for archival of, say, 20 74 min. CDs onto DVD) to go up to about 420 kb/s.

I think that this might be safer, but I'm wondering if there could be problems with some applications to use weird arbitrary bitrates like this, in WavPack.  I guess my concern stems from the fact that some implementations of Ogg Vorbis don't handle the high-bitrate files all too well.

Also, would anyone be willing to argue that, even if transcoding in the future, going above 384 is overkill?

WavPack Lossy - Arbitrary Bitrates

Reply #1
Glad you like WavPack! 

The bitrates in WavPack aren't rigidly fixed, so when you select 320 for example, you actually get somewhere between about 320 and 335. It's really just a target, so any value is as good as any other. 420 would be better than 384. In fact, it would be a little over 2 dB better.

I don't think that high bitrates like that are overkill, especially considering that lossless is often over 1024!

Portable support is what I am working on next, with early emphasis on the iriver H120 although the Neuros should be possible also. Pointing out the virtures of WavPack to the developers there never hurts! 

Of course, on portables the higher the bitrate the more often the disk is going to spin up.

WavPack Lossy - Arbitrary Bitrates

Reply #2
Hi bryant:
Glad to see you, thanks for replying!

I don't know exactly what you meant by 2 dB better... is this SNR you're talking about?  I probably am far, far more ignorant than you in matters of audio engineering.

Also, I'm really glad you're working on portable support.  That will no doubt be a huge boon to your codec.  FYI, I plugged wavpack on the neuros forums:
http://www.neurosaudio.com/community/forum...p?TOPIC_ID=2212

Perhaps a bugzila request should be put in for it, so people could vote on it.


P.S., I'm the one that e-mailed you about a codec comparison for your website, but I'm thinking now that there may not be a point, because it seems that you are prepared to release a new version shortly.

WavPack Lossy - Arbitrary Bitrates

Reply #3
I've been trying to get WavPack hybrid mode to work in foobar2000 with the diskwriter.  I'm having some problems with it...  it appears to work great, then as it finishes up, foobar2000 reports that it failed, and the wv and wvc files are erased.

These are the settings I've set up: (all the [ ] are unchecked boxes)
Encoder: C:\WINDOWS\wavpack.exe
Extension: wv
parameters: -b420hx -c - %d
Format: lossless
Highest BPS: 16
Tag: default
[ ] Pass floating point data
[ ] Encoder requires accurate length

Display name:
WavPack Hybrid -b420hx

Generic Settings
[ ] Hide console window
[ ] Never use source file BPS

WavPack Lossy - Arbitrary Bitrates

Reply #4
use these:

-hb420x -c - %d

encoder requires = yes

WavPack Lossy - Arbitrary Bitrates

Reply #5
Apparently this encoder DOES require accurate length.  Thanks shadowking!

I'd like to know what that option does, exactly.

WavPack Lossy - Arbitrary Bitrates

Reply #6
Quote
Apparently this encoder DOES require accurate length.   Thanks shadowking!

I'd like to know what that option does, exactly.
[a href="index.php?act=findpost&pid=285923"][{POST_SNAPBACK}][/a]

I don't actually understand why that option is there either, unless there is somehow more work required for the decoder to determine the accurate length in advance. However, I did find that even then it sometimes won't work (transcoding from MP3, I think) and so I added the new -i option to WavPack to ignore the length in the header. Add that to your command and you can skip the "accurate length" option.

Yes, 2 dB better SNR. In lossy mode the noise is reduced about 1 dB for every 14 kbps higher bitrate.

Thanks for posting on Neuros; I need to contact them myself also (I am in contact with the RockBox guys).

I will respond to your e-mail also...

WavPack Lossy - Arbitrary Bitrates

Reply #7
bryant:

I've been really testing out wavpack now.  It's nice how much flexibility there is in being able to encode in different ways.  I find that the default level of compression isn't terribly competitive (i.e. with flac, monkey's), but the encoder is quite fast.  I like how one can tweak it quite a bit... with the h and x options, the compression ratio can exceed flac, but still doesn't come quite close to monkey's. 

For the x option, I thought it worth commenting that your documentation confused me. 
Quote
This option accepts an optional numberic parameter from 1 to 6 that overrides the default amount of "extra" processing done. The defaults were choosen to provide the greatest "bang for the buck" and are -x6 for "fast" mode, -x4 for the normal mode and -x3 for the "high" mode.


I just tried option -hb420x1, and it was fairly quick.  Then I tried -hb420x6, and... well... let's just say, I'll have a look in the morning to see if it's finished.  It's extremely slow!    I'm excited to see what kinds of improvements that might make in the compression ratio over x1.

I didn't expect it to be quick, I was amply warned, but I had the impression from your documentation, and the fact that x6 is called "quick" mode, that x6 was fast and x1 packed more.  The way it is in reality makes more intuitive sense, but your documentation seems to indicate otherwise.  Am I wrong, or did somebody make a mistake when writing the docs?

WavPack Lossy - Arbitrary Bitrates

Reply #8
No it works like this:

Extra processing modes are: 1-6 (1 is least ,6 is the max)

wavpack compression modes: fast (-f), normal, high (-h)

Optimum extra processing settings for these modes: x6 - fast, x4 - normal, x3 - high

WavPack Lossy - Arbitrary Bitrates

Reply #9
So, shadowking, you're saying that the 1-6 settings work backwards if you use the -f or -h switches? 

WavPack Lossy - Arbitrary Bitrates

Reply #10
Quote
So, shadowking, you're saying that the 1-6 settings work backwards if you use the -f or -h switches? 
[a href="index.php?act=findpost&pid=286085"][{POST_SNAPBACK}][/a]

No. IIRC when you use -f flag the default x level is 6. If you use normal compression (no f or h switch) the default x level is 4. With h switch the x level is 3.

WavPack Lossy - Arbitrary Bitrates

Reply #11
Wow, the more I read, the more confused I get.  I thought h and x were completely unrelated options.  All I know is that x1 compressed my wv less, and was fairly quick, and x6 compressed it a tad more and took a very long time.  Can someone confirm this, or did my particular copy reverse engineer itself just so that I'd be really confused and end up arguing with others?

I'm guessing that you guys would know much more than me, but is it possible that something was changed in a recent revision of WavPack, maybe?

WavPack Lossy - Arbitrary Bitrates

Reply #12
This is normal. x1 = least processing = fastest, x6 = most processing = slowest. -x is totaly optional and not related to any compression modes. However if applied without value it will default to a level depending on the compression mode - in the case of -h it will be x3.

WavPack Lossy - Arbitrary Bitrates

Reply #13
As a new user of wavpack, I too was confused by the documentation on the -x switch. Now after you explained it, I went back and read it again, and now I understood it.

It needs to be re-worded or something. Originally I thought when I use -h, it automatically applied x3 (hence 'default'), but then I tested it and -hx is a lot slower than -h. And -h6, well, it still wasn't done after I came back from lunch, so it was too long for my taste  But the documentation says the default for -fast is -x6, that was just confusing, how can -fast use -x6 which extremely slow.... The documentation should probably mention that the extra time -x requires also depends on the mode.

I decided in the end to use -h, it's very fast and provides slightly better compression than flac -8. The -x just adds too much to the processing for very little gain in my test.

 

WavPack Lossy - Arbitrary Bitrates

Reply #14
Unless the size means really much to you, I guess you can safely drop -x in the lossless mode, because it's lossless anyway and -x will increase compression ratio only a little. But the original post is about the high-bitrate lossy mode. If I'm not wrong (pls correct me if my understanding is wrong), -x is always recommended (safer) in the lossy mode while -h can be dropped if the bitrate is very high. Let me quote the explanation from the Usage Guide:

Quote
...the "extra" mode (-x) deserves a special mention. Most of the time, the "extra" mode makes only modest improvements because WavPack has already been finely tuned for ordinary music. However, sometimes there is an instrument or sound situation that does not compress well with the standard settings and the "extra" mode can make a significant improvement. In lossless mode this would slightly improve the overall compression ratio and would probably go unnoticed. However, in lossy mode that difficult section might trigger clearly audible noise to be added, and in this case the "extra" mode would save the day by greatly reducing the noise in the exact spot where it might have been audible.

Observe that the author recommended to always use -x for High quality, and that especially in a very high bitrate -h is not needed but -x is still recommended:
Code: [Select]
Here is a chart of recommended settings for the various useful bitrates:

Bitrate     High quality   Faster Encode   Faster Decode
-------     ------------   -------------   -------------
256 kbps    -hb256x        -hb256          -b256x
320 kbps    -hb320x        -hb320          -b320x
384 kbps    -b384x         -hb384          -fb384x

WavPack Lossy - Arbitrary Bitrates

Reply #15
slightly ot, but why dont we ever see wavpack lossy or other lossless/lossy hybrid formats in listening tests.  i understand that at high bitrates, listening tests become extremely difficult, and the results are inconsequential, but even at lower bitrates, ive never come across one of these "hybrid" lossys in a listening test.  have i just not been looking hard enough, or is there some other reason these tests aren't conducted?
a windows-free, linux user since 1/31/06.

WavPack Lossy - Arbitrary Bitrates

Reply #16
VCSkier, agreed:  I'd like to see some listening tests of WavPack lossy as well.  And not only that, but I'd like to see listening tests of WavPack lossy transcoded to other lossy formats.  (Can you even imagine how long such a test would take though?  There are lots of possible permutations there.)

BTW, Liisachan, good info.  It makes sense that x can improve lossy quality - probably more-so for lower bitrates (i.e. under 320).

I'm still not clear on a few things, but here's another question:
Is "x" more effective when used without the h option? 
I.e. would -x6 make more of a difference in compression, from the default mode, as compared to the difference between -h and -hx6?

I almost confused myself, so to clarify:
Say
wavpack test.wav = 500MB
and
wavpack -h test.wav = 490MB
(So -h reduces the filesize by 10MB)

Then how would
wavpack -x6 compare with -hx6?

Obviously the -x6 would have an advantage in decoding speed over -h, even, but I'm not sure what -h is.  Does the -h option actually use -x?

edit:
From my previous example, it appears that:
wavpack -x test.wav = 494MB
(-x reduces my example by 6 MB)

So -x and -h appear to be independent of each other.  A slow way to shave off a  small amount of data.  It might not be worth the extra time, in most cases, but at least it doesn't hit you again at decode time.  I might use it for a transcoding job where I'm going away for the weekend or something.

WavPack Lossy - Arbitrary Bitrates

Reply #17
Quote
I almost confused myself, so to clarify:
Say
wavpack test.wav = 500MB
and
wavpack -h test.wav = 490MB
(So -h reduces the filesize by 10MB)

Then how would
wavpack -x6 compare with -hx6?

Obviously the -x6 would have an advantage in decoding speed over -h, even, but I'm not sure what -h is.  Does the -h option actually use -x?
[a href="index.php?act=findpost&pid=286549"][{POST_SNAPBACK}][/a]


-x6 would yeild less size reductoin than -hx6.  -h does not use -x unless you specify -x.  if you use -hx (w/ no number after the x), -x3 will be used, because it is the default setting for -x when used w/ -h (according to bryant, -x3 offers the "most bang for the buck" for -h).  you could also specify a different -x value, for more or less extra processing.  hope that helps...
a windows-free, linux user since 1/31/06.

WavPack Lossy - Arbitrary Bitrates

Reply #18
I'll try to clarify the docs on the extra mode in the next release. In the meantime here's a quick summary:

There are three basic modes in WavPack: fast, normal, and high. The modes indicate the complexity of the decompression (or decoding). High is the most complex and is about half the speed of the normal mode, but compresses better in lossless and gives better quality in lossy. Fast is somewhat simpler than the normal mode and decompresses somewhat faster, but also is a little worse in compression and quality. WavPack is normally a symmetrical compressor, which means that basically the same process is used for both encoding and decoding, and so they take about the same time.

The extra option is exactly that: it does extra processing during encoding. It works in any of the three modes to improve them (in both compression and quality) and it can be very slow. However, it doesn't change the actual mode used so the resulting files will decode just as fast as they would without the extra option; it's free in a sense. I recommend that the extra option simply be used by itself, that is with no numeric parameter. The extra mode tends to improve the faster modes more than the higher modes (simply because there's more room for improvement in the faster modes).

There is an optional numeric parameter for the extra mode that (like I said above) I don't generally advise using. This is because it is not just a linear scale; instead it actually turns on different types of optimization. It is ordered so that it does more the higher the number, but that doesn't mean that the amount of time spent will give a corresponding improvement in performance. For example, in the normal mode, I just measured these:

Code: [Select]
normal: created john.wv in 1.92 secs  (lossless, 39.32%)
-x1:    created john.wv in 3.50 secs  (lossless, 39.32%)
-x2:    created john.wv in 5.81 secs  (lossless, 39.32%)
-x3:    created john.wv in 11.53 secs (lossless, 39.53%)
-x4:    created john.wv in 25.98 secs (lossless, 40.28%)
-x5:    created john.wv in 39.48 secs (lossless, 40.29%)
-x6:    created john.wv in 82.66 secs (lossless, 40.33%)

You can see that even though the processing time just about doubles each increment of the extra value, virtually all of the improvement occurs at -x4. Before and after that you don't get much for the time spent, so when you select -x in the normal mode you get level 4 by default. Similar reasoning was used to choose the default extra level for the fast (-x6) and high (-x3) modes.

WavPack Lossy - Arbitrary Bitrates

Reply #19
very well put.  bryant, you are the king of lossless audio compression! 
a windows-free, linux user since 1/31/06.

WavPack Lossy - Arbitrary Bitrates

Reply #20
Quote
very well put.  bryant, you are the king of lossless audio compression! 
[a href="index.php?act=findpost&pid=286572"][{POST_SNAPBACK}][/a]
Thanks!    Sorry the original docs were not as clear...

WavPack Lossy - Arbitrary Bitrates

Reply #21
Thanks for the update bryant.  That's much more clear, and you could probably virtually copy it right over parts of your existing documentation.

Bryant, I have another question for you that relates more directly to my original question of this post:  Does using the -h switch possibly compromize playback on hardware, or slow computer systems?  Or is it likely that any hardware supporting wavpack would be sufficiently advanced enough to play back -h encoded wavpack smoothly? 

For normal purposes, I feel that the -h option offers plenty of improvement for the time it takes, but that is one consideration I might have.

WavPack Lossy - Arbitrary Bitrates

Reply #22
Quote
Portable support is what I am working on next, with early emphasis on the iriver H120 although the Neuros should be possible also.[a href="index.php?act=findpost&pid=285847"][{POST_SNAPBACK}][/a]

Please, don't forget to release the final 4.2 version before that 
In theory, there is no difference between theory and practice. In practice there is.

WavPack Lossy - Arbitrary Bitrates

Reply #23
Quote
Bryant, I have another question for you that relates more directly to my original question of this post:  Does using the -h switch possibly compromize playback on hardware, or slow computer systems?  Or is it likely that any hardware supporting wavpack would be sufficiently advanced enough to play back -h encoded wavpack smoothly? 
[a href="index.php?act=findpost&pid=286586"][{POST_SNAPBACK}][/a]

That's a good question, and I don't have a definitive answer. The "high" mode is about twice as complex (CPU-wise) to decode as the normal mode, so obviously there are processors that can decode the normal mode but can't decode "high", but whether any of these will ever be encountered in a portable is anyone's guess.

Also, I don't really have a very good idea how WavPack decoding compares to other lossless and lossy decoders because WavPack is currently pure C (i.e. no embedded machine code) which, I believe, is not the case with FLAC, Monkey's Audio, MPC, etc. I do know that with respect to more easily quantifiable parameters (memory footprint and integer size requirements) it is very competitive. It will be interesting to see how it performs when optimized.

Fortunately, the actual difference between the "fast" and "high" modes is isolated to one small loop which will be the first target for optimization. This means that the difference between the modes will probably be reduced in any optimized version.

And, no, despite the fact that I almost can't wait to get going on the embedded stuff, the 4.2 release takes priority. If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 

WavPack Lossy - Arbitrary Bitrates

Reply #24
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 
[{POST_SNAPBACK}][/a]

This is not a bug but just a wish, but (like i said before) I'd like to see better Unicode support in the future

Currently, for instance if you have both Japanese filename and French filename, wavpack.exe can't handle them (actually it's not wavpack's fault--Windows is the one who is silly)--as you can see in this screencap:

[a href="http://mion.wisnet.ne.jp/tmp/wv.html]http://mion.wisnet.ne.jp/tmp/wv.html[/url]




Keep up a great job!