Skip to main content
Topic: WavPack Lossy - Arbitrary Bitrates (Read 14612 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

WavPack Lossy - Arbitrary Bitrates

Reply #25
I just wanted to see what happened with all mode-x combinations. Here's what I got:
Code: [Select]
fast mode:
created test-f.wv   in  0.14 secs (lossless, 39.23%)
created test-fx1.wv in  0.31 secs (lossless, 39.23%)
created test-fx2.wv in  0.52 secs (lossless, 39.20%)
created test-fx3.wv in  0.60 secs (lossless, 39.20%)
created test-fx4.wv in  1.13 secs (lossless, 42.04%)
created test-fx5.wv in  1.50 secs (lossless, 42.04%)
created test-fx6.wv in  1.79 secs (lossless, 42.04%)

normal mode:
created test.wv     in  0.16 secs (lossless, 41.27%)
created test-x1.wv  in  0.42 secs (lossless, 41.41%)
created test-x2.wv  in  1.04 secs (lossless, 41.60%)
created test-x3.wv  in  1.87 secs (lossless, 42.02%)
created test-x4.wv  in  3.29 secs (lossless, 42.45%)
created test-x5.wv  in  4.77 secs (lossless, 42.45%)
created test-x6.wv  in  9.57 secs (lossless, 42.44%)

high mode:
created test-h.wv   in  0.36 secs (lossless, 42.06%)
created test-hx1.wv in  0.97 secs (lossless, 42.26%)
created test-hx2.wv in  1.83 secs (lossless, 42.30%)
created test-hx3.wv in  7.50 secs (lossless, 42.69%)
created test-hx4.wv in 13.58 secs (lossless, 42.74%)
created test-hx5.wv in 19.20 secs (lossless, 42.74%)
created test-hx6.wv in 39.52 secs (lossless, 42.78%)

Yeah, I know, it's only one sample and a small one too... But I think it illustrates the point that plain normal mode is almost as fast and almost as efficient as anything else...

Anyway, thanks for the clarification Bryant, I was wondering what all those -x combinations were about. Your post was most educating.

WavPack Lossy - Arbitrary Bitrates

Reply #26
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 


Just a suggestion,

How about use -t for "testing/verifying" the integrity of the wv file for wvunpack.exe? This would be easier to remember and most compression/encoding programs follow this standard....

Thanks!

WavPack Lossy - Arbitrary Bitrates

Reply #27
Quote
If anyone is waiting to report any last minute bugs in the beta, this is the time to do it! 
[a href="index.php?act=findpost&pid=286714"][{POST_SNAPBACK}][/a]

I've only been using the beta 4 since yesterday, but I've encoded several albums worth of music.  I use -hxm with EAC and so far no trouble.  It seems release ready to me.  I can't comment on the hybrid or lossy modes since I don't use either though.  I think I'll try out the lossy just for fun.

WavPack Lossy - Arbitrary Bitrates

Reply #28
Quote
I just wanted to see what happened with all mode-x combinations. Here's what I got:
Code: [Select]
fast mode:
created test-f.wv   in  0.14 secs (lossless, 39.23%)
created test-fx1.wv in  0.31 secs (lossless, 39.23%)
created test-fx2.wv in  0.52 secs (lossless, 39.20%)
created test-fx3.wv in  0.60 secs (lossless, 39.20%)
created test-fx4.wv in  1.13 secs (lossless, 42.04%)
created test-fx5.wv in  1.50 secs (lossless, 42.04%)
created test-fx6.wv in  1.79 secs (lossless, 42.04%)
...

[a href="index.php?act=findpost&pid=286783"][{POST_SNAPBACK}][/a]


Echo, I was wondering how you made this table of results.  Is there some kind of batchfile that you used for this, or did you actually sit there and manually record the times and such?  I'd like to do a test like this with some very large files (CD Images), but let it run overnight.

WavPack Lossy - Arbitrary Bitrates

Reply #29
Quote
Is "x" more effective when used without the h option? 
I.e. would -x6 make more of a difference in compression, from the default mode, as compared to the difference between -h and -hx6?


Echo kindly (and perhaps unknowingly) provided the required data to answer my question from before.  Since, perhaps, I was the only one that could understand it anyways, I'll answer it now:

YES!

the -x option definitely appears to have more impact in normal, and especially in fast modes than it does in high mode.  So while -hx won't be much different from -h, -fx will be a more substantial improvement over -f.

Note how -fx is almost the same size at -h, while having a much better decode speed: (It will probably take just over 3X as long to encode -fx4 as -h though).
Code: [Select]
fast mode:
test-f.wv   39.23%
test-fx6.wv 42.04%

normal mode:
test.wv     41.27%
test-x4.wv  42.45%

high mode:
test-h.wv   42.06%
test-hx3.wv 42.69%


This means that you have some nice capabilities with WavPack, because you can get great compression ratios while still having a very fast decode speed.  WavPack thus demonstrates another way that it has a degree of flexibility that is possibly unmatched by anything else out there.  Nice!

WavPack Lossy - Arbitrary Bitrates

Reply #30
Hmm...  I've been doing my own tests of WavPack lossless using the 46 samples from rjamorim's listening tests.  (183MB).
They *do not* closely correspond to the results that echo got - the ones which made me so excited a little while ago, but it still confirms that the answer to my original question is yes.  The extra (-x) modes are more effective the faster the encoding setting.

Code: [Select]
Type    Bytes        Size %   Change EncTime X Incr*
WAV     192,156,428  100.00%  -       -         -
wv.f    116,620,743  60.69%  -39.31% 33         -
wv.fx1  116,247,737  60.50%  -0.19%  36       10%      
wv.fx2  116,151,681  60.45%  -0.05%  40       22%    
wv.fx3  116,151,681  60.45%   0.00%  41       25%      
wv.fx4  114,715,073  59.70%  -0.75%  85      160%  
wv.fx5  114,486,325  59.58%  -0.12%  112     240%    
wv.fx6  114,480,801  59.58%  -0.00%  136     316%    
wv.norm 113,662,299  59.15%  -0.43%  29         -      
wv.x1   113,393,555  59.01%  -0.14%  40       38%    
wv.x2   113,315,235  58.97%  -0.04%  63      116%    
wv.x3   113,089,013  58.85%  -0.12%  108     271%    
wv.x4   112,044,993  58.31%  -0.54%  262     803%  
wv.x5   111,973,775  58.27%  -0.04%  398    1275%  
wv.x6   111,891,125  58.23%  -0.04%  839    2794%  
wv.h    111,071,996  57.80%  -0.43%  32         -      
wv.hx1  110,920,564  57.72%  -0.08%  80      150%  
wv.hx2  110,836,849  57.68%  -0.04%  141     339%    
wv.hx3  110,562,922  57.54%  -0.14%  633    1874%  
wv.hx4  110,490,641  57.50%  -0.04%  1142   3462%  
wv.hx5  110,466,641  57.49%  -0.01%  1614   4934%  
wv.hx6  110,426,449  57.47%  -0.02%  3422  10574%

EncTime is in seconds
*X Incr. is the increase in time it takes to encode in that -x mode over the regular non-x mode (normal/-f/-h)
100% = Double the Time


Some interesting things to note:
-fx6 is bigger than normal, and much bigger than -h.

-The difference between -f and -fx6 is about 2.0 MB / 1.1%
-The difference between normal and -x6 is about 1.7 MB / 0.9%
-The difference between -h and -hx6 is about 0.6 MB / 0.3%

The f-fx6,normal-x6,h-hx6 sequence produces filesizes that consistently get progressively smaller.  As bryant said, with the x setting, the most significant changes always occur on -x4, except that -hx3 is slightly more significant than -hx4, and is a great measure faster, hence the default being set to x3 for -h mode.

-The difference between the biggest (-f) and smallest (-hx6) is about 6 MB, or 3.2%.
-fx2 and fx3 produced the exact same file sizes.  Is this a trivial bug, maybe? 

Edit:
Added the relevant times to the table above.

WavPack Lossy - Arbitrary Bitrates

Reply #31
Quote
I'll try to clarify the docs on the extra mode in the next release. In the meantime here's a quick summary:

There are three basic modes in WavPack: fast, normal, and high. The modes indicate the complexity of the decompression (or decoding). High is the most complex and is about half the speed of the normal mode, but compresses better in lossless and gives better quality in lossy. Fast is somewhat simpler than the normal mode and decompresses somewhat faster, but also is a little worse in compression and quality. WavPack is normally a symmetrical compressor, which means that basically the same process is used for both encoding and decoding, and so they take about the same time.

The extra option is exactly that: it does extra processing during encoding. It works in any of the three modes to improve them (in both compression and quality) and it can be very slow. However, it doesn't change the actual mode used so the resulting files will decode just as fast as they would without the extra option; it's free in a sense. I recommend that the extra option simply be used by itself, that is with no numeric parameter. The extra mode tends to improve the faster modes more than the higher modes (simply because there's more room for improvement in the faster modes).

There is an optional numeric parameter for the extra mode that (like I said above) I don't generally advise using. This is because it is not just a linear scale; instead it actually turns on different types of optimization. It is ordered so that it does more the higher the number, but that doesn't mean that the amount of time spent will give a corresponding improvement in performance. For example, in the normal mode, I just measured these:

Code: [Select]
normal: created john.wv in 1.92 secs  (lossless, 39.32%)
-x1:    created john.wv in 3.50 secs  (lossless, 39.32%)
-x2:    created john.wv in 5.81 secs  (lossless, 39.32%)
-x3:    created john.wv in 11.53 secs (lossless, 39.53%)
-x4:    created john.wv in 25.98 secs (lossless, 40.28%)
-x5:    created john.wv in 39.48 secs (lossless, 40.29%)
-x6:    created john.wv in 82.66 secs (lossless, 40.33%)

You can see that even though the processing time just about doubles each increment of the extra value, virtually all of the improvement occurs at -x4. Before and after that you don't get much for the time spent, so when you select -x in the normal mode you get level 4 by default. Similar reasoning was used to choose the default extra level for the fast (-x6) and high (-x3) modes.
[a href="index.php?act=findpost&pid=286569"][{POST_SNAPBACK}][/a]


Nice explanation Bryant. I love wavpack. I have always just used the -h option on its own.  Its a very flexible encoder and there is pretty much a setting for everyone, depending on individual needs.

I hope portable developers will start to take notice and develop wavpack implementations. Also, I'd like to see the pro-audio industry embrace it too -  plugins for wavelab/logic/samplitude etc would be nice !

Please keep up the good work with wavpack.

WavPack Lossy - Arbitrary Bitrates

Reply #32
Quote
Echo, I was wondering how you made this table of results.  Is there some kind of batchfile that you used for this, or did you actually sit there and manually record the times and such?  I'd like to do a test like this with some very large files (CD Images), but let it run overnight.
[a href="index.php?act=findpost&pid=286884"][{POST_SNAPBACK}][/a]

I'm afraid I did it manually. They were only about 20 encodes, so it wasn't a big deal.

Quote
Hmm...  I've been doing my own tests of WavPack lossless using the 46 samples from rjamorim's listening tests.  (183MB).
They *do not* closely correspond to the results that echo got - the ones which made me so excited a little while ago, but it still confirms that the answer to my original question is yes.  The extra (-x) modes are more effective the faster the encoding setting.
[a href="index.php?act=findpost&pid=286896"][{POST_SNAPBACK}][/a]

Your results are of cource more valid than mine. As I said mine was only one sample. Interesting how the compressed sizes scale like that. Do you also have the corresponding encoding times? Do they scale the same way or are they mixed like mine? Anyway thanks for your results.

WavPack Lossy - Arbitrary Bitrates

Reply #33
Quote
Your results are of cource more valid than mine. As I said mine was only one sample. Interesting how the compressed sizes scale like that. Do you also have the corresponding encoding times? Do they scale the same way or are they mixed like mine? Anyway thanks for your results.

Uh, well... I could probably give you what I got from my timestamps, but they wouldn't be that useful, because halfway through doing this, I switched from transcoding WavPack high to encoding plain Wavs.

The next time I do this, I'll need to record my encoding times.  All I can say is that -f is damned fast, and -hx6 is painfully slow.  Everything else is something in between

WavPack Lossy - Arbitrary Bitrates

Reply #34
Okay, check the table above.  I added the encode times to it now.  I'm *really* good at wasting time

WavPack Lossy - Arbitrary Bitrates

Reply #35
Quote
Okay, check the table above.  I added the encode times to it now.   I'm *really* good at wasting time
[a href="index.php?act=findpost&pid=287374"][{POST_SNAPBACK}][/a]

Many thanks Supacon! I also calculated the Encoding time/Compression ratio (Compression is 100-size reported by wavpack). It seems that the best ratio is for normal mode closely followed by -h.
Code: [Select]
Mode   Compression (%)   Enc. Time (s)   Time/Compression
f      39,31             33               0,84            
fx1    39,5              36               0,91            
fx2    39,55             40               1,01            
fx3    39,55             41               1,04            
fx4    40,3              85               2,11            
fx5    40,42             112              2,77            
fx6    40,42             136              3,36            
norm   40,85             29               0,71            
x1     40,99             40               0,98            
x2     41,03             63               1,54            
x3     41,15             108              2,62            
x4     41,69             262              6,28            
x5     41,73             398              9,54            
x6     41,77             839             20,09          
h      42,2              32               0,76            
hx1    42,28             80               1,89            
hx2    42,32             141              3,33            
hx3    42,46             633             14,91          
hx4    42,5              1142            26,87          
hx5    42,51             1614            37,97          
hx6    42,53             3422            80,46

What is a little strange about the encoding times is that normal is faster than -f and also -h is (just a bit) faster than -f!

WavPack Lossy - Arbitrary Bitrates

Reply #36
Quote
What is a little strange about the encoding times is that normal is faster than -f and also -h is (just a bit) faster than -f!
[a href="index.php?act=findpost&pid=287498"][{POST_SNAPBACK}][/a]
I've seen this kind of thing sometimes. Usually the problem is that disk I/O is taking too high a portion of the time. You'll also sometimes see large variations from one run to the next because of fragmented disk, cacheing, etc.

I have found a few solutions for this. Although one could argue that the numbers aren't really fair anymore, these methods are important to me to help determine the actual factors that affect performance.

1. Have the destination and source files on different physical drives.
2. For WavPack, pipe the output to "nul" (and possibly use only the second run, or the best of three):

Code: [Select]
 > wavpack -f testfile - > nul


3. For WvUnpack, use the -v option to verify only
4. To test the "roundtrip" time use a command-line like this:

Code: [Select]
 > wavpack -fq testfile - | wvunpack -v -


BTW, thanks for all the testing! I get a little tired doing it all myself... 

WavPack Lossy - Arbitrary Bitrates

Reply #37
Yeah, I noticed some BIG inconsistencies in my timing results, so take them with a grain of salt.  I'm running these on my laptop, so it's probably not the speediest disk i/o in the world.  I was pretty surprised by some times, and the fact that -h was faster than -f and such.  I rationalized that in my head by saying that the real difference must be on the decode side, but I still haven't tested the times for that. (That'll probably come.)

On the neuros forums, Josh Coalson, myself, and another lad were discussing comparisons of WavPack to FLAC, and he mentioned that FLAC has a test mode built in that compensates for factors like disk I/O and such...
http://www.neurosaudio.com/community/forum...212&whichpage=6

WavPack Lossy - Arbitrary Bitrates

Reply #38
Quote
Yeah, I noticed some BIG inconsistencies in my timing results, so take them with a grain of salt.  I'm running these on my laptop, so it's probably not the speediest disk i/o in the world.
[{POST_SNAPBACK}][/a]

You can limit the problem by using the buffering function of foobar2000' speedmeter. I did it for my own test, and I achieved stable results (at least on a desktop computer with enough RAM).
My own results (limited to some profiles) are more coherent:
[a href="http://www.foobar2000.net/lossless/alphabetic.htm]http://www.foobar2000.net/lossless/alphabetic.htm[/url]

WavPack Lossy - Arbitrary Bitrates

Reply #39
Very nice guruboolez:  That's a really slick web page that you've got there.  It'd be nice to have something like that on the WavPack website itself.

You didn't give much detail about what you encoded, however... I'm guessing from what I know about you, and from the compression ratios that it's some kind of classical music.

WavPack Lossy - Arbitrary Bitrates

Reply #40
The complete list is available on the bottom of the main page and extra-informations are also present on the detailed table.

WavPack Lossy - Arbitrary Bitrates

Reply #41
Oh, great.  Guess I should have poked around a little more first... OF COURSE you'd have all that detail at hand.  Nice page, btw, but my french isn't really up to snuff these days.  I think I get the gist though.

 
SimplePortal 1.0.0 RC1 © 2008-2019