HydrogenAudio

Lossless Audio Compression => WavPack => Topic started by: Supacon on 2005-03-26 20:42:10

Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-26 20:42:10
I'm just experimenting a bit, for the first time, with WavPack and its hybrid mode.  This is probably one of the most innovative codecs that I've seen, and I really hope that some of the portables and such in the near future will support this.  (Neuros perhaps?)

I'm thinking of using the lossy mode so that I can create files at an arbitrary bitrate that will be completely transparent, and also be transcodable without any perceptable loss.  I've read that 384kb/s is "optimal" for this, but for my purposes, I have enough room (for archival of, say, 20 74 min. CDs onto DVD) to go up to about 420 kb/s.

I think that this might be safer, but I'm wondering if there could be problems with some applications to use weird arbitrary bitrates like this, in WavPack.  I guess my concern stems from the fact that some implementations of Ogg Vorbis don't handle the high-bitrate files all too well.

Also, would anyone be willing to argue that, even if transcoding in the future, going above 384 is overkill?
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-03-26 21:13:39
Glad you like WavPack! 

The bitrates in WavPack aren't rigidly fixed, so when you select 320 for example, you actually get somewhere between about 320 and 335. It's really just a target, so any value is as good as any other. 420 would be better than 384. In fact, it would be a little over 2 dB better.

I don't think that high bitrates like that are overkill, especially considering that lossless is often over 1024!

Portable support is what I am working on next, with early emphasis on the iriver H120 although the Neuros should be possible also. Pointing out the virtures of WavPack to the developers there never hurts! 

Of course, on portables the higher the bitrate the more often the disk is going to spin up.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-26 22:40:40
Hi bryant:
Glad to see you, thanks for replying!

I don't know exactly what you meant by 2 dB better... is this SNR you're talking about?  I probably am far, far more ignorant than you in matters of audio engineering.

Also, I'm really glad you're working on portable support.  That will no doubt be a huge boon to your codec.  FYI, I plugged wavpack on the neuros forums:
http://www.neurosaudio.com/community/forum...p?TOPIC_ID=2212 (http://www.neurosaudio.com/community/forum/topic.asp?TOPIC_ID=2212)

Perhaps a bugzila request should be put in for it, so people could vote on it.


P.S., I'm the one that e-mailed you about a codec comparison for your website, but I'm thinking now that there may not be a point, because it seems that you are prepared to release a new version shortly.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-27 04:20:06
I've been trying to get WavPack hybrid mode to work in foobar2000 with the diskwriter.  I'm having some problems with it...  it appears to work great, then as it finishes up, foobar2000 reports that it failed, and the wv and wvc files are erased.

These are the settings I've set up: (all the [ ] are unchecked boxes)
Encoder: C:\WINDOWS\wavpack.exe
Extension: wv
parameters: -b420hx -c - %d
Format: lossless
Highest BPS: 16
Tag: default
[ ] Pass floating point data
[ ] Encoder requires accurate length

Display name:
WavPack Hybrid -b420hx

Generic Settings
[ ] Hide console window
[ ] Never use source file BPS
Title: WavPack Lossy - Arbitrary Bitrates
Post by: shadowking on 2005-03-27 04:32:56
use these:

-hb420x -c - %d

encoder requires = yes
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-27 04:38:12
Apparently this encoder DOES require accurate length.  Thanks shadowking!

I'd like to know what that option does, exactly.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-03-27 08:11:39
Quote
Apparently this encoder DOES require accurate length.   Thanks shadowking!

I'd like to know what that option does, exactly.
[a href="index.php?act=findpost&pid=285923"][{POST_SNAPBACK}][/a]

I don't actually understand why that option is there either, unless there is somehow more work required for the decoder to determine the accurate length in advance. However, I did find that even then it sometimes won't work (transcoding from MP3, I think) and so I added the new -i option to WavPack to ignore the length in the header. Add that to your command and you can skip the "accurate length" option.

Yes, 2 dB better SNR. In lossy mode the noise is reduced about 1 dB for every 14 kbps higher bitrate.

Thanks for posting on Neuros; I need to contact them myself also (I am in contact with the RockBox guys).

I will respond to your e-mail also...
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-27 13:57:39
bryant:

I've been really testing out wavpack now.  It's nice how much flexibility there is in being able to encode in different ways.  I find that the default level of compression isn't terribly competitive (i.e. with flac, monkey's), but the encoder is quite fast.  I like how one can tweak it quite a bit... with the h and x options, the compression ratio can exceed flac, but still doesn't come quite close to monkey's. 

For the x option, I thought it worth commenting that your documentation confused me. 
Quote
This option accepts an optional numberic parameter from 1 to 6 that overrides the default amount of "extra" processing done. The defaults were choosen to provide the greatest "bang for the buck" and are -x6 for "fast" mode, -x4 for the normal mode and -x3 for the "high" mode.


I just tried option -hb420x1, and it was fairly quick.  Then I tried -hb420x6, and... well... let's just say, I'll have a look in the morning to see if it's finished.  It's extremely slow!    I'm excited to see what kinds of improvements that might make in the compression ratio over x1.

I didn't expect it to be quick, I was amply warned, but I had the impression from your documentation, and the fact that x6 is called "quick" mode, that x6 was fast and x1 packed more.  The way it is in reality makes more intuitive sense, but your documentation seems to indicate otherwise.  Am I wrong, or did somebody make a mistake when writing the docs?
Title: WavPack Lossy - Arbitrary Bitrates
Post by: shadowking on 2005-03-27 14:25:14
No it works like this:

Extra processing modes are: 1-6 (1 is least ,6 is the max)

wavpack compression modes: fast (-f), normal, high (-h)

Optimum extra processing settings for these modes: x6 - fast, x4 - normal, x3 - high
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-27 20:17:38
So, shadowking, you're saying that the 1-6 settings work backwards if you use the -f or -h switches? 
Title: WavPack Lossy - Arbitrary Bitrates
Post by: askoff on 2005-03-27 21:51:59
Quote
So, shadowking, you're saying that the 1-6 settings work backwards if you use the -f or -h switches? 
[a href="index.php?act=findpost&pid=286085"][{POST_SNAPBACK}][/a]

No. IIRC when you use -f flag the default x level is 6. If you use normal compression (no f or h switch) the default x level is 4. With h switch the x level is 3.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-28 12:28:43
Wow, the more I read, the more confused I get.  I thought h and x were completely unrelated options.  All I know is that x1 compressed my wv less, and was fairly quick, and x6 compressed it a tad more and took a very long time.  Can someone confirm this, or did my particular copy reverse engineer itself just so that I'd be really confused and end up arguing with others?

I'm guessing that you guys would know much more than me, but is it possible that something was changed in a recent revision of WavPack, maybe?
Title: WavPack Lossy - Arbitrary Bitrates
Post by: shadowking on 2005-03-28 14:04:57
This is normal. x1 = least processing = fastest, x6 = most processing = slowest. -x is totaly optional and not related to any compression modes. However if applied without value it will default to a level depending on the compression mode - in the case of -h it will be x3.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: wildgoose on 2005-03-29 01:59:25
As a new user of wavpack, I too was confused by the documentation on the -x switch. Now after you explained it, I went back and read it again, and now I understood it.

It needs to be re-worded or something. Originally I thought when I use -h, it automatically applied x3 (hence 'default'), but then I tested it and -hx is a lot slower than -h. And -h6, well, it still wasn't done after I came back from lunch, so it was too long for my taste  But the documentation says the default for -fast is -x6, that was just confusing, how can -fast use -x6 which extremely slow.... The documentation should probably mention that the extra time -x requires also depends on the mode.

I decided in the end to use -h, it's very fast and provides slightly better compression than flac -8. The -x just adds too much to the processing for very little gain in my test.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Liisachan on 2005-03-29 02:56:13
Unless the size means really much to you, I guess you can safely drop -x in the lossless mode, because it's lossless anyway and -x will increase compression ratio only a little. But the original post is about the high-bitrate lossy mode. If I'm not wrong (pls correct me if my understanding is wrong), -x is always recommended (safer) in the lossy mode while -h can be dropped if the bitrate is very high. Let me quote the explanation from the Usage Guide (http://www.wavpack.com/wavpack_doc.htm):

Quote
...the "extra" mode (-x) deserves a special mention. Most of the time, the "extra" mode makes only modest improvements because WavPack has already been finely tuned for ordinary music. However, sometimes there is an instrument or sound situation that does not compress well with the standard settings and the "extra" mode can make a significant improvement. In lossless mode this would slightly improve the overall compression ratio and would probably go unnoticed. However, in lossy mode that difficult section might trigger clearly audible noise to be added, and in this case the "extra" mode would save the day by greatly reducing the noise in the exact spot where it might have been audible.

Observe that the author recommended to always use -x for High quality, and that especially in a very high bitrate -h is not needed but -x is still recommended:
Code: [Select]
Here is a chart of recommended settings for the various useful bitrates:

Bitrate     High quality   Faster Encode   Faster Decode
-------     ------------   -------------   -------------
256 kbps    -hb256x        -hb256          -b256x
320 kbps    -hb320x        -hb320          -b320x
384 kbps    -b384x         -hb384          -fb384x
Title: WavPack Lossy - Arbitrary Bitrates
Post by: VCSkier on 2005-03-29 03:01:29
slightly ot, but why dont we ever see wavpack lossy or other lossless/lossy hybrid formats in listening tests.  i understand that at high bitrates, listening tests become extremely difficult, and the results are inconsequential, but even at lower bitrates, ive never come across one of these "hybrid" lossys in a listening test.  have i just not been looking hard enough, or is there some other reason these tests aren't conducted?
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-29 03:52:18
VCSkier, agreed:  I'd like to see some listening tests of WavPack lossy as well.  And not only that, but I'd like to see listening tests of WavPack lossy transcoded to other lossy formats.  (Can you even imagine how long such a test would take though?  There are lots of possible permutations there.)

BTW, Liisachan, good info.  It makes sense that x can improve lossy quality - probably more-so for lower bitrates (i.e. under 320).

I'm still not clear on a few things, but here's another question:
Is "x" more effective when used without the h option? 
I.e. would -x6 make more of a difference in compression, from the default mode, as compared to the difference between -h and -hx6?

I almost confused myself, so to clarify:
Say
wavpack test.wav = 500MB
and
wavpack -h test.wav = 490MB
(So -h reduces the filesize by 10MB)

Then how would
wavpack -x6 compare with -hx6?

Obviously the -x6 would have an advantage in decoding speed over -h, even, but I'm not sure what -h is.  Does the -h option actually use -x?

edit:
From my previous example, it appears that:
wavpack -x test.wav = 494MB
(-x reduces my example by 6 MB)

So -x and -h appear to be independent of each other.  A slow way to shave off a  small amount of data.  It might not be worth the extra time, in most cases, but at least it doesn't hit you again at decode time.  I might use it for a transcoding job where I'm going away for the weekend or something.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: VCSkier on 2005-03-29 04:23:07
Quote
I almost confused myself, so to clarify:
Say
wavpack test.wav = 500MB
and
wavpack -h test.wav = 490MB
(So -h reduces the filesize by 10MB)

Then how would
wavpack -x6 compare with -hx6?

Obviously the -x6 would have an advantage in decoding speed over -h, even, but I'm not sure what -h is.  Does the -h option actually use -x?
[a href="index.php?act=findpost&pid=286549"][{POST_SNAPBACK}][/a]


-x6 would yeild less size reductoin than -hx6.  -h does not use -x unless you specify -x.  if you use -hx (w/ no number after the x), -x3 will be used, because it is the default setting for -x when used w/ -h (according to bryant, -x3 offers the "most bang for the buck" for -h).  you could also specify a different -x value, for more or less extra processing.  hope that helps...
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-03-29 07:17:02
I'll try to clarify the docs on the extra mode in the next release. In the meantime here's a quick summary:

There are three basic modes in WavPack: fast, normal, and high. The modes indicate the complexity of the decompression (or decoding). High is the most complex and is about half the speed of the normal mode, but compresses better in lossless and gives better quality in lossy. Fast is somewhat simpler than the normal mode and decompresses somewhat faster, but also is a little worse in compression and quality. WavPack is normally a symmetrical compressor, which means that basically the same process is used for both encoding and decoding, and so they take about the same time.

The extra option is exactly that: it does extra processing during encoding. It works in any of the three modes to improve them (in both compression and quality) and it can be very slow. However, it doesn't change the actual mode used so the resulting files will decode just as fast as they would without the extra option; it's free in a sense. I recommend that the extra option simply be used by itself, that is with no numeric parameter. The extra mode tends to improve the faster modes more than the higher modes (simply because there's more room for improvement in the faster modes).

There is an optional numeric parameter for the extra mode that (like I said above) I don't generally advise using. This is because it is not just a linear scale; instead it actually turns on different types of optimization. It is ordered so that it does more the higher the number, but that doesn't mean that the amount of time spent will give a corresponding improvement in performance. For example, in the normal mode, I just measured these:

Code: [Select]
normal: created john.wv in 1.92 secs  (lossless, 39.32%)
-x1:    created john.wv in 3.50 secs  (lossless, 39.32%)
-x2:    created john.wv in 5.81 secs  (lossless, 39.32%)
-x3:    created john.wv in 11.53 secs (lossless, 39.53%)
-x4:    created john.wv in 25.98 secs (lossless, 40.28%)
-x5:    created john.wv in 39.48 secs (lossless, 40.29%)
-x6:    created john.wv in 82.66 secs (lossless, 40.33%)

You can see that even though the processing time just about doubles each increment of the extra value, virtually all of the improvement occurs at -x4. Before and after that you don't get much for the time spent, so when you select -x in the normal mode you get level 4 by default. Similar reasoning was used to choose the default extra level for the fast (-x6) and high (-x3) modes.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: VCSkier on 2005-03-29 07:30:34
very well put.  bryant, you are the king of lossless audio compression! 
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-03-29 07:42:04
Quote
very well put.  bryant, you are the king of lossless audio compression! 
[a href="index.php?act=findpost&pid=286572"][{POST_SNAPBACK}][/a]
Thanks!    Sorry the original docs were not as clear...
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-29 08:42:33
Thanks for the update bryant.  That's much more clear, and you could probably virtually copy it right over parts of your existing documentation.

Bryant, I have another question for you that relates more directly to my original question of this post:  Does using the -h switch possibly compromize playback on hardware, or slow computer systems?  Or is it likely that any hardware supporting wavpack would be sufficiently advanced enough to play back -h encoded wavpack smoothly? 

For normal purposes, I feel that the -h option offers plenty of improvement for the time it takes, but that is one consideration I might have.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: GeSomeone on 2005-03-29 11:45:10
Quote
Portable support is what I am working on next, with early emphasis on the iriver H120 although the Neuros should be possible also.[a href="index.php?act=findpost&pid=285847"][{POST_SNAPBACK}][/a]

Please, don't forget to release the final 4.2 version before that 
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-03-29 19:58:15
Quote
Bryant, I have another question for you that relates more directly to my original question of this post:  Does using the -h switch possibly compromize playback on hardware, or slow computer systems?  Or is it likely that any hardware supporting wavpack would be sufficiently advanced enough to play back -h encoded wavpack smoothly? 
[a href="index.php?act=findpost&pid=286586"][{POST_SNAPBACK}][/a]

That's a good question, and I don't have a definitive answer. The "high" mode is about twice as complex (CPU-wise) to decode as the normal mode, so obviously there are processors that can decode the normal mode but can't decode "high", but whether any of these will ever be encountered in a portable is anyone's guess.

Also, I don't really have a very good idea how WavPack decoding compares to other lossless and lossy decoders because WavPack is currently pure C (i.e. no embedded machine code) which, I believe, is not the case with FLAC, Monkey's Audio, MPC, etc. I do know that with respect to more easily quantifiable parameters (memory footprint and integer size requirements) it is very competitive. It will be interesting to see how it performs when optimized.

Fortunately, the actual difference between the "fast" and "high" modes is isolated to one small loop which will be the first target for optimization. This means that the difference between the modes will probably be reduced in any optimized version.

And, no, despite the fact that I almost can't wait to get going on the embedded stuff, the 4.2 release takes priority. If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Liisachan on 2005-03-29 21:53:05
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=286714")

This is not a bug but just a wish, but (like i said before) I'd like to see better Unicode support in the future

Currently, for instance if you have both Japanese filename and French filename, wavpack.exe can't handle them (actually it's not wavpack's fault--Windows is the one who is silly)--as you can see in this screencap:

[a href="http://mion.wisnet.ne.jp/tmp/wv.html]http://mion.wisnet.ne.jp/tmp/wv.html[/url]




Keep up a great job!
Title: WavPack Lossy - Arbitrary Bitrates
Post by: echo on 2005-03-30 00:13:40
I just wanted to see what happened with all mode-x combinations. Here's what I got:
Code: [Select]
fast mode:
created test-f.wv   in  0.14 secs (lossless, 39.23%)
created test-fx1.wv in  0.31 secs (lossless, 39.23%)
created test-fx2.wv in  0.52 secs (lossless, 39.20%)
created test-fx3.wv in  0.60 secs (lossless, 39.20%)
created test-fx4.wv in  1.13 secs (lossless, 42.04%)
created test-fx5.wv in  1.50 secs (lossless, 42.04%)
created test-fx6.wv in  1.79 secs (lossless, 42.04%)

normal mode:
created test.wv     in  0.16 secs (lossless, 41.27%)
created test-x1.wv  in  0.42 secs (lossless, 41.41%)
created test-x2.wv  in  1.04 secs (lossless, 41.60%)
created test-x3.wv  in  1.87 secs (lossless, 42.02%)
created test-x4.wv  in  3.29 secs (lossless, 42.45%)
created test-x5.wv  in  4.77 secs (lossless, 42.45%)
created test-x6.wv  in  9.57 secs (lossless, 42.44%)

high mode:
created test-h.wv   in  0.36 secs (lossless, 42.06%)
created test-hx1.wv in  0.97 secs (lossless, 42.26%)
created test-hx2.wv in  1.83 secs (lossless, 42.30%)
created test-hx3.wv in  7.50 secs (lossless, 42.69%)
created test-hx4.wv in 13.58 secs (lossless, 42.74%)
created test-hx5.wv in 19.20 secs (lossless, 42.74%)
created test-hx6.wv in 39.52 secs (lossless, 42.78%)

Yeah, I know, it's only one sample and a small one too... But I think it illustrates the point that plain normal mode is almost as fast and almost as efficient as anything else...

Anyway, thanks for the clarification Bryant, I was wondering what all those -x combinations were about. Your post was most educating.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: wildgoose on 2005-03-30 00:46:52
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 


Just a suggestion,

How about use -t for "testing/verifying" the integrity of the wv file for wvunpack.exe? This would be easier to remember and most compression/encoding programs follow this standard....

Thanks!
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Duble0Syx on 2005-03-30 02:23:08
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 
[a href="index.php?act=findpost&pid=286714"][{POST_SNAPBACK}][/a]

I've only been using the beta 4 since yesterday, but I've encoded several albums worth of music.  I use -hxm with EAC and so far no trouble.  It seems release ready to me.  I can't comment on the hybrid or lossy modes since I don't use either though.  I think I'll try out the lossy just for fun.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-30 09:50:20
Quote
I just wanted to see what happened with all mode-x combinations. Here's what I got:
Code: [Select]
fast mode:
created test-f.wv   in  0.14 secs (lossless, 39.23%)
created test-fx1.wv in  0.31 secs (lossless, 39.23%)
created test-fx2.wv in  0.52 secs (lossless, 39.20%)
created test-fx3.wv in  0.60 secs (lossless, 39.20%)
created test-fx4.wv in  1.13 secs (lossless, 42.04%)
created test-fx5.wv in  1.50 secs (lossless, 42.04%)
created test-fx6.wv in  1.79 secs (lossless, 42.04%)
...

[a href="index.php?act=findpost&pid=286783"][{POST_SNAPBACK}][/a]


Echo, I was wondering how you made this table of results.  Is there some kind of batchfile that you used for this, or did you actually sit there and manually record the times and such?  I'd like to do a test like this with some very large files (CD Images), but let it run overnight.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-30 10:09:49
Quote
Is "x" more effective when used without the h option? 
I.e. would -x6 make more of a difference in compression, from the default mode, as compared to the difference between -h and -hx6?


Echo kindly (and perhaps unknowingly) provided the required data to answer my question from before.  Since, perhaps, I was the only one that could understand it anyways, I'll answer it now:

YES!

the -x option definitely appears to have more impact in normal, and especially in fast modes than it does in high mode.  So while -hx won't be much different from -h, -fx will be a more substantial improvement over -f.

Note how -fx is almost the same size at -h, while having a much better decode speed: (It will probably take just over 3X as long to encode -fx4 as -h though).
Code: [Select]
fast mode:
test-f.wv   39.23%
test-fx6.wv 42.04%

normal mode:
test.wv     41.27%
test-x4.wv  42.45%

high mode:
test-h.wv   42.06%
test-hx3.wv 42.69%


This means that you have some nice capabilities with WavPack, because you can get great compression ratios while still having a very fast decode speed.  WavPack thus demonstrates another way that it has a degree of flexibility that is possibly unmatched by anything else out there.  Nice!
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-30 11:24:00
Hmm...  I've been doing my own tests of WavPack lossless using the 46 samples from rjamorim's listening tests.  (183MB).
They *do not* closely correspond to the results that echo got - the ones which made me so excited a little while ago, but it still confirms that the answer to my original question is yes.  The extra (-x) modes are more effective the faster the encoding setting.

Code: [Select]
Type    Bytes        Size %   Change EncTime X Incr*
WAV     192,156,428  100.00%  -       -         -
wv.f    116,620,743  60.69%  -39.31% 33         -
wv.fx1  116,247,737  60.50%  -0.19%  36       10%      
wv.fx2  116,151,681  60.45%  -0.05%  40       22%    
wv.fx3  116,151,681  60.45%   0.00%  41       25%      
wv.fx4  114,715,073  59.70%  -0.75%  85      160%  
wv.fx5  114,486,325  59.58%  -0.12%  112     240%    
wv.fx6  114,480,801  59.58%  -0.00%  136     316%    
wv.norm 113,662,299  59.15%  -0.43%  29         -      
wv.x1   113,393,555  59.01%  -0.14%  40       38%    
wv.x2   113,315,235  58.97%  -0.04%  63      116%    
wv.x3   113,089,013  58.85%  -0.12%  108     271%    
wv.x4   112,044,993  58.31%  -0.54%  262     803%  
wv.x5   111,973,775  58.27%  -0.04%  398    1275%  
wv.x6   111,891,125  58.23%  -0.04%  839    2794%  
wv.h    111,071,996  57.80%  -0.43%  32         -      
wv.hx1  110,920,564  57.72%  -0.08%  80      150%  
wv.hx2  110,836,849  57.68%  -0.04%  141     339%    
wv.hx3  110,562,922  57.54%  -0.14%  633    1874%  
wv.hx4  110,490,641  57.50%  -0.04%  1142   3462%  
wv.hx5  110,466,641  57.49%  -0.01%  1614   4934%  
wv.hx6  110,426,449  57.47%  -0.02%  3422  10574%

EncTime is in seconds
*X Incr. is the increase in time it takes to encode in that -x mode over the regular non-x mode (normal/-f/-h)
100% = Double the Time


Some interesting things to note:
-fx6 is bigger than normal, and much bigger than -h.

-The difference between -f and -fx6 is about 2.0 MB / 1.1%
-The difference between normal and -x6 is about 1.7 MB / 0.9%
-The difference between -h and -hx6 is about 0.6 MB / 0.3%

The f-fx6,normal-x6,h-hx6 sequence produces filesizes that consistently get progressively smaller.  As bryant said, with the x setting, the most significant changes always occur on -x4, except that -hx3 is slightly more significant than -hx4, and is a great measure faster, hence the default being set to x3 for -h mode.

-The difference between the biggest (-f) and smallest (-hx6) is about 6 MB, or 3.2%.
-fx2 and fx3 produced the exact same file sizes.  Is this a trivial bug, maybe? 

Edit:
Added the relevant times to the table above.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: disgustipated on 2005-03-30 12:41:09
Quote
I'll try to clarify the docs on the extra mode in the next release. In the meantime here's a quick summary:

There are three basic modes in WavPack: fast, normal, and high. The modes indicate the complexity of the decompression (or decoding). High is the most complex and is about half the speed of the normal mode, but compresses better in lossless and gives better quality in lossy. Fast is somewhat simpler than the normal mode and decompresses somewhat faster, but also is a little worse in compression and quality. WavPack is normally a symmetrical compressor, which means that basically the same process is used for both encoding and decoding, and so they take about the same time.

The extra option is exactly that: it does extra processing during encoding. It works in any of the three modes to improve them (in both compression and quality) and it can be very slow. However, it doesn't change the actual mode used so the resulting files will decode just as fast as they would without the extra option; it's free in a sense. I recommend that the extra option simply be used by itself, that is with no numeric parameter. The extra mode tends to improve the faster modes more than the higher modes (simply because there's more room for improvement in the faster modes).

There is an optional numeric parameter for the extra mode that (like I said above) I don't generally advise using. This is because it is not just a linear scale; instead it actually turns on different types of optimization. It is ordered so that it does more the higher the number, but that doesn't mean that the amount of time spent will give a corresponding improvement in performance. For example, in the normal mode, I just measured these:

Code: [Select]
normal: created john.wv in 1.92 secs  (lossless, 39.32%)
-x1:    created john.wv in 3.50 secs  (lossless, 39.32%)
-x2:    created john.wv in 5.81 secs  (lossless, 39.32%)
-x3:    created john.wv in 11.53 secs (lossless, 39.53%)
-x4:    created john.wv in 25.98 secs (lossless, 40.28%)
-x5:    created john.wv in 39.48 secs (lossless, 40.29%)
-x6:    created john.wv in 82.66 secs (lossless, 40.33%)

You can see that even though the processing time just about doubles each increment of the extra value, virtually all of the improvement occurs at -x4. Before and after that you don't get much for the time spent, so when you select -x in the normal mode you get level 4 by default. Similar reasoning was used to choose the default extra level for the fast (-x6) and high (-x3) modes.
[a href="index.php?act=findpost&pid=286569"][{POST_SNAPBACK}][/a]


Nice explanation Bryant. I love wavpack. I have always just used the -h option on its own.  Its a very flexible encoder and there is pretty much a setting for everyone, depending on individual needs.

I hope portable developers will start to take notice and develop wavpack implementations. Also, I'd like to see the pro-audio industry embrace it too -  plugins for wavelab/logic/samplitude etc would be nice !

Please keep up the good work with wavpack.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: echo on 2005-03-30 13:26:22
Quote
Echo, I was wondering how you made this table of results.  Is there some kind of batchfile that you used for this, or did you actually sit there and manually record the times and such?  I'd like to do a test like this with some very large files (CD Images), but let it run overnight.
[a href="index.php?act=findpost&pid=286884"][{POST_SNAPBACK}][/a]

I'm afraid I did it manually. They were only about 20 encodes, so it wasn't a big deal.

Quote
Hmm...  I've been doing my own tests of WavPack lossless using the 46 samples from rjamorim's listening tests.  (183MB).
They *do not* closely correspond to the results that echo got - the ones which made me so excited a little while ago, but it still confirms that the answer to my original question is yes.  The extra (-x) modes are more effective the faster the encoding setting.
[a href="index.php?act=findpost&pid=286896"][{POST_SNAPBACK}][/a]

Your results are of cource more valid than mine. As I said mine was only one sample. Interesting how the compressed sizes scale like that. Do you also have the corresponding encoding times? Do they scale the same way or are they mixed like mine? Anyway thanks for your results.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-03-30 13:30:25
Quote
Your results are of cource more valid than mine. As I said mine was only one sample. Interesting how the compressed sizes scale like that. Do you also have the corresponding encoding times? Do they scale the same way or are they mixed like mine? Anyway thanks for your results.

Uh, well... I could probably give you what I got from my timestamps, but they wouldn't be that useful, because halfway through doing this, I switched from transcoding WavPack high to encoding plain Wavs.

The next time I do this, I'll need to record my encoding times.  All I can say is that -f is damned fast, and -hx6 is painfully slow.  Everything else is something in between
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-04-01 01:16:31
Okay, check the table above.  I added the encode times to it now.  I'm *really* good at wasting time
Title: WavPack Lossy - Arbitrary Bitrates
Post by: echo on 2005-04-01 15:43:20
Quote
Okay, check the table above.  I added the encode times to it now.   I'm *really* good at wasting time
[a href="index.php?act=findpost&pid=287374"][{POST_SNAPBACK}][/a]

Many thanks Supacon! I also calculated the Encoding time/Compression ratio (Compression is 100-size reported by wavpack). It seems that the best ratio is for normal mode closely followed by -h.
Code: [Select]
Mode   Compression (%)   Enc. Time (s)   Time/Compression
f      39,31             33               0,84            
fx1    39,5              36               0,91            
fx2    39,55             40               1,01            
fx3    39,55             41               1,04            
fx4    40,3              85               2,11            
fx5    40,42             112              2,77            
fx6    40,42             136              3,36            
norm   40,85             29               0,71            
x1     40,99             40               0,98            
x2     41,03             63               1,54            
x3     41,15             108              2,62            
x4     41,69             262              6,28            
x5     41,73             398              9,54            
x6     41,77             839             20,09          
h      42,2              32               0,76            
hx1    42,28             80               1,89            
hx2    42,32             141              3,33            
hx3    42,46             633             14,91          
hx4    42,5              1142            26,87          
hx5    42,51             1614            37,97          
hx6    42,53             3422            80,46

What is a little strange about the encoding times is that normal is faster than -f and also -h is (just a bit) faster than -f!
Title: WavPack Lossy - Arbitrary Bitrates
Post by: bryant on 2005-04-01 17:59:38
Quote
What is a little strange about the encoding times is that normal is faster than -f and also -h is (just a bit) faster than -f!
[a href="index.php?act=findpost&pid=287498"][{POST_SNAPBACK}][/a]
I've seen this kind of thing sometimes. Usually the problem is that disk I/O is taking too high a portion of the time. You'll also sometimes see large variations from one run to the next because of fragmented disk, cacheing, etc.

I have found a few solutions for this. Although one could argue that the numbers aren't really fair anymore, these methods are important to me to help determine the actual factors that affect performance.

1. Have the destination and source files on different physical drives.
2. For WavPack, pipe the output to "nul" (and possibly use only the second run, or the best of three):

Code: [Select]
 > wavpack -f testfile - > nul


3. For WvUnpack, use the -v option to verify only
4. To test the "roundtrip" time use a command-line like this:

Code: [Select]
 > wavpack -fq testfile - | wvunpack -v -


BTW, thanks for all the testing! I get a little tired doing it all myself... 
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-04-01 18:48:50
Yeah, I noticed some BIG inconsistencies in my timing results, so take them with a grain of salt.  I'm running these on my laptop, so it's probably not the speediest disk i/o in the world.  I was pretty surprised by some times, and the fact that -h was faster than -f and such.  I rationalized that in my head by saying that the real difference must be on the decode side, but I still haven't tested the times for that. (That'll probably come.)

On the neuros forums, Josh Coalson, myself, and another lad were discussing comparisons of WavPack to FLAC, and he mentioned that FLAC has a test mode built in that compensates for factors like disk I/O and such...
http://www.neurosaudio.com/community/forum...212&whichpage=6 (http://www.neurosaudio.com/community/forum/topic.asp?TOPIC_ID=2212&whichpage=6)
Title: WavPack Lossy - Arbitrary Bitrates
Post by: guruboolez on 2005-04-01 19:04:30
Quote
Yeah, I noticed some BIG inconsistencies in my timing results, so take them with a grain of salt.  I'm running these on my laptop, so it's probably not the speediest disk i/o in the world.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=287555")

You can limit the problem by using the buffering function of foobar2000' speedmeter. I did it for my own test, and I achieved stable results (at least on a desktop computer with enough RAM).
My own results (limited to some profiles) are more coherent:
[a href="http://www.foobar2000.net/lossless/alphabetic.htm]http://www.foobar2000.net/lossless/alphabetic.htm[/url]
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-04-01 20:49:40
Very nice guruboolez:  That's a really slick web page that you've got there.  It'd be nice to have something like that on the WavPack website itself.

You didn't give much detail about what you encoded, however... I'm guessing from what I know about you, and from the compression ratios that it's some kind of classical music.
Title: WavPack Lossy - Arbitrary Bitrates
Post by: guruboolez on 2005-04-01 20:52:42
The complete list is available on the bottom of the main page (http://www.foobar2000.net/lossless/) and extra-informations are also present on the detailed table (http://www.foobar2000.net/lossless/details.htm).
Title: WavPack Lossy - Arbitrary Bitrates
Post by: Supacon on 2005-04-02 03:11:31
Oh, great.  Guess I should have poked around a little more first... OF COURSE you'd have all that detail at hand.  Nice page, btw, but my french isn't really up to snuff these days.  I think I get the gist though.