HydrogenAudio

Lossless Audio Compression => WavPack => Topic started by: bryant on 2006-08-14 03:49:23

Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-14 03:49:23
First, I'd like to thank Guru and the Mods for the dedicated WavPack forum. Very cool! 

Despite the fact that it seems pretty quiet lately on WavPack development, there has actually been a lot going on. Today I'm posting another 4.4 alpha that has some major changes. I am also finishing up a reorganization of the source code and the creation of an official library and API, including a Windows dll.

The new alpha has a completely new high mode. Its decoding complexity falls just about halfway between the default mode and the old high mode, but its compression is closer to the high side. The old high mode is still available and is called very high (-hh).

This new mode is particularly suited for use on the iPod using Rockbox. The old high mode files play, but use so much CPU to decode that there isn't enough left over to perform audio effects (like EQ) or even allow a peakmeter on the display. There are plans to eventually use both CPU cores on Rockbox, but I think this new mode is a much better compromise between decoding demand and compression (even on the PC).

Also, the extra mode is completely redone. The old mode was useful for creating the smallest possible WavPack file for a given quality, but was so slow that it was useless for most applications (and the -x levels that were fast enough to be usable usually didn't do anything). The new extra mode has only three levels (1 to 3) and they are much faster than before, although they can't quite match the old mode's ultimate compression ratio. But, just using -x by itself (which is equivalent to -x1) makes such an improvement with so little extra processing that I am thinking of having it always on (once it's a little more optimized).

This version uses different decorrelation filter tuning in all modes, so don't be surprised if you see differences between this and the previous. For my corpus this results in a small average improvement, but there are certainly files that do worse (although with -x now usable all the time, this shouldn't be an issue).

I have not incorporated any of the MMX enhancements, and in fact have not done any optimization on this yet, so I expect some more speed gains (and the new -x mode is a good candidate for threading).

This is an alpha (and not well tested either), so please compress responsibly.

Any feedback or results are very much appreciated. 

http://www.wavpack.com/wavpack44a2.zip (http://www.wavpack.com/wavpack44a2.zip)
Title: WavPack 4.4 alpha 2 for Windows
Post by: askoff on 2006-08-14 07:20:16
Are you planing to add MMX optimizations in WavPack 4.4 final or some later version? These unofficial tests with 4.3 version and MMX optimizations where quite promising.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-08-14 07:28:33
Thanks David, great news!

Is the old -x6 switch still somehow accessible?

Also, any progress towards a single file encoder/decoder yet?
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-08-14 08:25:15
Wow, David this sounds good. I am going to play with this alpha during the week to test lossy and lossless modes.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DrazardX on 2006-08-14 09:34:08
Here's my results... (for a 40 min. album) (Original Size = 412,937 KBs)
Quote
Options - File Size - Encode Time

-hhmx3 - 284,519 KBs - 722.97 secs
-hhmx  - 284,673 KBs - 227.89 secs
-hhm  - 284,882 KBs - 132.14 secs

-hmx3  - 285,292 KBs - 478.22 secs
-hmx  - 285,473 KBs - 200.67 secs
-hm    - 285,677 KBs - 101.63 secs

-mx3  - 287,386 KBs - 315.95 secs
-mx    - 287,636 KBs - 110.19 secs
-m    - 288,851 KBs -  86.92 secs

-m is in there because I always use it.

-x does have a nice effect on the normal setting, but it's not good with either high settings.  I think it's better to have it off by default, unless using either high setting would automatically turn it off.  The new high setting does seem nice, but my PC handles the old high setting perfectly, so i'm probably gonna stick with that.  I can see how the new high setting would be a great option for some people though.

Instead of changing the old high setting to "very high", couldn't you name the new one something different such as "new high" or "high new" kinda like OptimFrog?  Then again, I really like how "very high" sounds.  Only thing is getting used to -hhm instead of -hm, but I wouldn't exactly call that a problem.

I'm looking forward to see how the newest version turns out in the end.  Thanks for all your work on WavPack.
Title: WavPack 4.4 alpha 2 for Windows
Post by: A_Man_Eating_Duck on 2006-08-14 09:38:46
i did a quick and dirty test of decoding speed with foobar

-h == 99.999x
-hh == 77.003x

CPU = AMD 64 3000+ @ 2.3

nice work
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-08-14 10:11:15
From brief testing its good to see that -x2 is doing something as opposed to the old -x2, and its not that slow. -x or -x1 is doing only auto-joint stereo like v4.31 from what I can tell. This -x1 should be speed optimised as I find it very useful in lossy mode. Currently I use -hx1 in v4.31 MMX. The new high is also working better than normal mode -x so one can use it instead of -hh and still get comptitive compression and lossy quality and quick decoding for devices.

I wil do more tests, but this looks like a step in the right direction. David you are the man !
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-08-14 11:43:24
What is the function of -x1 in 4.4 ? Looks like its doing some extra compression as well as auto-M/S ?

I compressed a file with 4.31 and 4.4

4.31 -h = 971 k

4.31 -hx1 = 971 k

4.4a -hh = 974 k

4.4a -hhx1 = 969 k

Ok, I get it: now -x1 is also doing extra compression and its quicker than the old -x1. I also think that defaulting an optimized -x1 would be a good idea.
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-14 15:58:47
Are the source code available for download?
That would be helpful for us non-Windows users!
Title: WavPack 4.4 alpha 2 for Windows
Post by: LaserSokrates on 2006-08-14 17:30:54
http://wavpack.com/downloads.html (http://wavpack.com/downloads.html)
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-08-14 18:47:15
I think he meant the sources for 4.4.
Title: WavPack 4.4 alpha 2 for Windows
Post by: LaserSokrates on 2006-08-14 19:14:41
I could have thought a bit before posting.... but the sources do not seem to be online.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Zurman on 2006-08-14 21:15:17
Always good to see that lossless codecs are still improving. Keep up the good work bryant, and I might give up ape in favor of wavpack
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-14 21:26:23
I just tried 4.4 hx3b288s0 and hx3b352s0 on my standard problem samples trumpet, herding_calls and harp40_1 in comparison to 4.31 hx6bxxxs0.

4.4 is great. With 4.31 I could abx trumpet with hx6b288s0 at 9/10 and with hx6b352s0 at 8/10.
With 4.4 hx3s0 and same nominal bitrates I could not reliably abx.

4.4 hx3 better than 4.31 hx6? Will try again tomorrow, I'm pretty tired now.

Great work, David. Thanks a lot .
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-14 21:38:39
I wonder how the existing decoders are working when fed with this new high mode. Decompression must be different to the old high mode cause otherwise decompression performance couldn't be better.

But I guess I don't have to worry about compatibility.
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-08-15 00:46:47
4.4 hx3 better than 4.31 hx6? Will try again tomorrow, I'm pretty tired now.

Great work, David. Thanks a lot .


You can get a rough idea by the lossless compression ratios - the one with the better compression should have the best s/n ratio. A small difference of say +- 5 kbit won't make much difference and you can also use -n switch to get the average noise report.
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-15 05:42:52
4.4 is great. With 4.31 I could abx trumpet with hx6b288s0 at 9/10 and with hx6b352s0 at 8/10.
With 4.4 hx3s0 and same nominal bitrates I could not reliably abx.

You perform an ABX test on a lossless audio codec? 
Title: WavPack 4.4 alpha 2 for Windows
Post by: pepoluan on 2006-08-15 05:51:24
4.4 is great. With 4.31 I could abx trumpet with hx6b288s0 at 9/10 and with hx6b352s0 at 8/10.
With 4.4 hx3s0 and same nominal bitrates I could not reliably abx.

You perform an ABX test on a lossless audio codec? 
remember that WavPack has a hybrid 'recoverable-lossy' mode. I think halb27 meant he ABX-ed the lossy WavPack stream/fork/whatever
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-15 06:58:01
Thanks everybody for your feedback; looks like no serious blow-ups yet... 

askoff:
I am definitely interested in the MMX enhancements! I really want to get 4.4 out quickly, so it won't be before then, but I want to get it into SVN right after that. I haven't looked into that code for a while, but the last time I looked there wasn't anything for decoding, which I'd also like.

DARcode:
The old -x mode is no longer available. However, I do plan on merging some of the features of the old -x mode into the new one for those weird (mostly artificial) samples that really benefit from generating filters from scratch (which is what the old version did). This will probably end up being -x4, or something. Sorry, the combined program is still rather low on the list...

DrazardX:
I thought about calling the new mode "new high", but I never liked names like that because they don't tell you where the mode fits with the other modes (e.g. is "new high" higher or lower than "high"?). And, fact is, I'd rather have people who don't any better simply use the new mode so I don't see them post that their WavPack files sometimes skip on the iPod! I haven't decided about the default, and even if I make some modes use "extra" by default you'll always be able to turn it off with -x0. BTW, welcome to HA! 

halb27:
I would be surprised if 4.4 hx3 is better than 4.31 hx6 on lossy mode! However, in the process of doing this I did discover a bug that would cause unbalanced noise when lossy mode switches on and off joint stereo (which the -x mode does). Perhaps you are hearing that. Oh, and I guess I forgot to mention that all this stuff is all backward compatible. I haven't used up all the WavPack 4.0 format functionality yet...

shadowking:
This new "extra" works very differently than the old version. For each mode I have 256 decorrelation filters which are organized into a binary search tree. So, I simply search the tree testing the current best filter against a new from the table. For -x1 I always compare against one other filter for each block, for -x2 I compare against 3 others and -x3 always searches through the whole tree (which takes 8 compares). So, unlike the old mode where different levels actually did different things, in this mode different levels simply so more of the same thing. Since joint stereo is one characteristic of the filter, that simply gets indirectly factored in. BTW, the -j[n] switch currently will not do anything in the new -x mode; haven't quite figured out what to do for that.

I am getting close to actually working on that smart noise shaping that we talked about a long time ago. In the meantime, do you think it would be worthwhile to include that prototype mode in a new release even considering that it can't work with correction files? Do you still use it when creating pure lossy files?

krmathis:
My next project is to merge all these changes into the new common source in SVN. As soon as I do that I will post so everyone can use it (I am actually becoming a pretty avid Ubuntu user myself).

BTW, if anyone is interested in reading a little more about how WavPack works internally, I finished up an article on it for an upcoming book. Here's the excerpt:

http://www.wavpack.com/WavPack.pdf (http://www.wavpack.com/WavPack.pdf)

Thanks again everyone!
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-15 07:59:38
4.4 is great. With 4.31 I could abx trumpet with hx6b288s0 at 9/10 and with hx6b352s0 at 8/10.
With 4.4 hx3s0 and same nominal bitrates I could not reliably abx.

You perform an ABX test on a lossless audio codec? 

As you can see from the parameters I use wavPack as a very high quality lossy encoder.

halb27:
I would be surprised if 4.4 hx3 is better than 4.31 hx6 on lossy mode! However, in the process of doing this I did discover a bug that would cause unbalanced noise when lossy mode switches on and off joint stereo (which the -x mode does). Perhaps you are hearing that. ...

As from former tests I expected herding_calls to be the most critical among the samples I tested. And I was very surprised that I could abx trumpet at high bitrate with 4.31hx6. There's a little 'blip' (hard to describe) within the critical passages. It's very subtle, but it's there. With 4.4hx3 I couldn't hear it. Maybe it's caused by the bug you mentioned.

As I will not use 4.4 in alpha state for productive purposes: can you do a correction for 4.31 concerning this bug?
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-08-15 15:27:45
Thanks everybody for your feedback; looks like no serious blow-ups yet... 


shadowking:

I am getting close to actually working on that smart noise shaping that we talked about a long time ago. In the meantime, do you think it would be worthwhile to include that prototype mode in a new release even considering that it can't work with correction files? Do you still use it when creating pure lossy files?


I used correction files for dvd archiving to this point, so I didn't use it much. Now I think I'll only keep them only for certain albums. I reckon it won't hurt to put in a switch for it if the real thing won't make it into 4.4 final .. that way people who don't use correction files can still use it. If you think that it will be ready by 4.4x or 4.5 then that's fine too.
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-15 16:11:22
krmathis:
My next project is to merge all these changes into the new common source in SVN. As soon as I do that I will post so everyone can use it (I am actually becoming a pretty avid Ubuntu user myself).

Nice to see you becoming a GNU/Linux user.
This hopefully mean you will be able to share alpha source code as well, so non-Windows users can take part of the testing as well. Which mean even more testers and feedback! 

Are the WavPack SVN server a public one? If so, where can I find it?
I don't see it mentioned on the website, and the SourceForge project site only list an old CVS repository (18 months since last checkin). https://sourceforge.net/cvs/?group_id=74831 (https://sourceforge.net/cvs/?group_id=74831)

Thanks in advance!

As you can see from the parameters I use wavPack as a very high quality lossy encoder.
I understand.
"hx6b352s0" did (does) not look like a parameter to me, so I did not catch it.
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-15 21:09:59
I redid my test with -b352hx3s0 on trumpet, herding_calls, harp40_1, and added Atemlied (which I could easily abx at 288kbps) and badvilbel (which should be abxable up to the 4xx kbps range).

Except for badvilbel I could not abx these samples, and even badvilbel was acceptable.

I will use 4.4 b352hx3s0 in the future when it's final.

Thanks again, David.
Title: WavPack 4.4 alpha 2 for Windows
Post by: dutch109 on 2006-08-17 18:01:00
Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-17 19:15:49
Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?

I have foobar engrave the replay gain info. No problems.
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-18 06:54:39
Are the WavPack SVN server a public one? If so, where can I find it?
I don't see it mentioned on the website, and the SourceForge project site only list an old CVS repository (18 months since last checkin). https://sourceforge.net/cvs/?group_id=74831 (https://sourceforge.net/cvs/?group_id=74831)

The SVN server is at http://svn.slomosnail.de/wavpack (http://svn.slomosnail.de/wavpack)

There, now it's public! 

The code in there should build on Linux and there are project files for MS Visual C++ 2005. It has been reorganized from the released sources with separate folders for the library and the cli modules, however the alpha code is not in there yet, nor are any of the other plugins (and the docs are not up-to-date either). I hope to have some of that done this weekend...




Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?

This would be possible for track gain, but I never figured out a good way to handle album gain this way unless wavpack was used to process a whole album at once (which EAC won't do). Having it a separate program allows easy processing of track and album gain. I'm not sure how FLAC handles album gain...
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-18 07:37:15
David, will there be a fix for the 4.31 stereo mode switching bug?
Title: WavPack 4.4 alpha 2 for Windows
Post by: dutch109 on 2006-08-18 10:18:44

Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?

This would be possible for track gain, but I never figured out a good way to handle album gain this way unless wavpack was used to process a whole album at once (which EAC won't do). Having it a separate program allows easy processing of track and album gain. I'm not sure how FLAC handles album gain...

Thanks for your answer.
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-18 15:46:19
Are the WavPack SVN server a public one? If so, where can I find it?
I don't see it mentioned on the website, and the SourceForge project site only list an old CVS repository (18 months since last checkin). https://sourceforge.net/cvs/?group_id=74831 (https://sourceforge.net/cvs/?group_id=74831)

The SVN server is at http://svn.slomosnail.de/wavpack (http://svn.slomosnail.de/wavpack)

There, now it's public! 
Excellent!
I will keep an eye on it for further updates.

Quote
...however the alpha code is not in there yet, nor are any of the other plugins (and the docs are not up-to-date either). I hope to have some of that done this weekend...
Lets hope so, cause I really want to give the alpha 2 a go...
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-21 05:52:46
Are the WavPack SVN server a public one? If so, where can I find it?
I don't see it mentioned on the website, and the SourceForge project site only list an old CVS repository (18 months since last checkin). https://sourceforge.net/cvs/?group_id=74831 (https://sourceforge.net/cvs/?group_id=74831)

The SVN server is at http://svn.slomosnail.de/wavpack (http://svn.slomosnail.de/wavpack)

There, now it's public! 
Excellent!
I will keep an eye on it for further updates.

Quote
...however the alpha code is not in there yet, nor are any of the other plugins (and the docs are not up-to-date either). I hope to have some of that done this weekend...
Lets hope so, cause I really want to give the alpha 2 a go...

Okay, I have updated SVN with the alpha sources and also fixed a bug that could cause a crash when combining -c and -i and -r on a wav file that has the incorrect number of samples (thanks Marius!). I have also switched to Visual C++ 2005 (from 6.0) and compiled a new Windows version here:

http://www.wavpack.com/wavpack44a3.zip (http://www.wavpack.com/wavpack44a3.zip)

I had been told that the new Microsoft compilers had better optimization than before. Well, this executable is much bigger and much slower than the previous (at least on my P4) so I'm not impressed yet. Perhaps I need to tweak the settings some... 


David, will there be a fix for the 4.31 stereo mode switching bug?

No, I won't be going back to fix old versions when I'm just a few weeks away from a new, much better release. I would actually trust the current alpha more than a patched 4.31 anyway, but if you want to stick with the old version for now I understand.

I am planning to try your command args on "trumpet" to see if it actually does trigger that bug (at this point I am not sure that the bug actually exists).
Title: WavPack 4.4 alpha 2 for Windows
Post by: NeXT on 2006-08-21 09:35:49
Very glad to see WavPack getting better.. =)

Bug: When executing (simple exec, without passing any parameters!) this command file (it's invaid, I know)
Code: [Select]
C:\Bin\WavPack\wavpack.exe %1 -w "CUESHEET=@%~dpn1.cue"

, WavPack 4.44a3 and 4.44a2 crashes.

WavPack 4.43 doesn't crash on it.
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-08-21 14:37:29
... No, I won't be going back to fix old versions when I'm just a few weeks away from a new, much better release. I would actually trust the current alpha more than a patched 4.31 anyway, but if you want to stick with the old version for now I understand. ...

I didn't realize you are just a few weeks away from a new release. Sure I can wait with my music which I want to encode at extremely high quality. Not so much at the moment anyway as I did encode most of my music collection.

Thanks for your great work.
Title: WavPack 4.4 alpha 2 for Windows
Post by: NeXT on 2006-08-21 14:41:21
Also I have a strange sample: a 546 MB wav file (CD with Russian and Soviet marches; very LQ, restorated)

Code: [Select]
title                    size (bytes)
----------------------------------------------
Original                 572 970 764
WavPack 4.44a3 -hhx3     121 386 354
WavPack 4.43   -hx5      116 903 311 (!)
Monkey's Audio -c4000     84 539 008 (!!)

   

(!) - an old x mode gives such significant improvement in some cases. Could you return it again, and change it's switch, f.e., to -xx ?

(!!) - SUCH a big difference between best mode of WavPack and very high mode of MAC (it also has -c5000 (insane) mode) -- is it normal, or it is unexpected WavPack's behaviuor?
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-21 15:58:02
Okay, I have updated SVN with the alpha sources and also fixed a bug that could cause a crash when combining -c and -i and -r on a wav file that has the incorrect number of samples (thanks Marius!).

Thanks!

It might be me, but I can't get rev. 7 to build.
Version 4.32 compiled with a plain "./configure && make", and the README file in SVN have the same instructions:
Code: [Select]
To build everything, type:
1. ./configure
2. make
3. make install
But there are no configure file in the source, so it fails.
Making autogen.sh executable and run it end with errors as well. Like this:
Quote
./configure
-bash: ./configure: No such file or directory
chmod +x autogen.sh
./autogen.sh
./autogen.sh: line 5: libtoolize: command not found
configure.ac: installing `./mkinstalldirs'
Use of uninitialized value in concatenation (.) or string at /usr/bin/automake line 8449.
: installing `./config.guess'
Use of uninitialized value in concatenation (.) or string at /usr/bin/automake line 8449.
: installing `./config.sub'
aclocal.m4:825: required file `./ltmain.sh' not found
This process creates the needed configure file, but running "./configure && make" errors out like this:
Quote
make
Making all in src
source='bits.c' object='libwavpack_la-bits.lo' libtool=yes \
depfile='.deps/libwavpack_la-bits.Plo' tmpdepfile='.deps/libwavpack_la-bits.TPlo' \
depmode=gcc3 /bin/sh ../depcomp \
/bin/sh ../libtool --mode=compile gcc -DPACKAGE_NAME=\"wavpack\" -DPACKAGE_TARNAME=\"wavpack\" -DPACKAGE_VERSION=\"4.33\" -DPACKAGE_STRING=\"wavpack\ 4.33\" -DPACKAGE_BUGREPORT=\"bryant@wavpack.com\" -DVERSION_OS=\"Darwin\" -DHIGHFIRST=1 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DHAVE_LIBM=1  -I. -I.    -g -O2 -c -o libwavpack_la-bits.lo `test -f 'bits.c' || echo './'`bits.c
../libtool: ../libtool: No such file or directory
make[1]: *** [libwavpack_la-bits.lo] Error 127
make: *** [all-recursive] Error 1

You provide two different source code archives for 4.32, so perhaps this is the MS Windows one only?
So do anyone know how to proceed from here?
Title: WavPack 4.4 alpha 2 for Windows
Post by: dutch109 on 2006-08-21 16:22:07

Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?

This would be possible for track gain, but I never figured out a good way to handle album gain this way unless wavpack was used to process a whole album at once (which EAC won't do). Having it a separate program allows easy processing of track and album gain. I'm not sure how FLAC handles album gain...

I only use Replay Gain to avoid clipping so I'd really enjoy an option to add track gain tag during the encoding process (like --replay-gain with FLAC), and also a modification in the Winamp plugin to enable clipping prevention without enabling Replay Gain (like in the Nullsoft Vorbis plugin).

Anyway, thanks for your great work on WavPack, I think I will transcode all my Monkey's Audio files to WavPack when the version 4.4 will be released.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-08-21 20:45:05
Quick test with my usual sample WAV.

Code: [Select]
02 - Faith No More - Epic.wav 51.849.884 bytes

AMD Sempron 3400+ (2 GHz) CPU
ASUS K8N-E-Deluxe Motherboard (NVIDIA nForce3 250Gb Chipset)
2 x Corsair CMX512-3200XL (1 GB) PC3200 RAM
2 x Samsung SpinPoint SP2504C 250GB 7200RPM 8MB Buffer SATA HDD into a RAID 1 set
Microsoft Windows XP Professional SP2 OS

4.31  -hx6m  35.127.920 bytes 880.22 secs
4.31  -hxm   35.131.392 bytes 170.67 secs
4.4a3 -hhx3m 35.134.062 bytes 52.55 secs
4.4a3 -hhxm  35.146.048 bytes 14.09 secs
4.4a3 -hx3m  35.192.996 bytes 35.33 secs
4.4a3 -hxm   35.213.174 bytes 10.72 secs

V4.4a3 -hhx gives better compression and is faster'n -hx3?
Also, v4.31 -hx compresses better'n v4.4a3 -hhx3?
The new version is pretty fast and -hhx3m is very efficient, tho it'd be very cool to be able to achieve v4.31 -hx6 compression ratio w/ it.

Re:http://www.hydrogenaudio.org/forums/index....st&p=422091 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=47474&view=findpost&p=422091)
Quote
rjamorim:"Maybe speed will increase a little if David or someone else works on assembly optimizations, but compression probably already reached its sweet spot. I believe that, from now on, David will focus more on format features and improving software and hardware support."
Is that so really?

Last but surely not least many many thanks for your brilliant work David, appreciated!

EDIT:Grammar + misc imprecisions.
Title: WavPack 4.4 alpha 2 for Windows
Post by: dv1989 on 2006-08-21 22:58:07
It's somewhat disappointing to see 4.31 tending to outperform 4.4a in terms of compression - but the differences aren't too great and I'm sure that the resulting benefits are worth it!
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-22 06:28:03
NeXT:
I'll try to look into that crashing problem you have there. 

Yes, the old -x mode does often do a little better than the new -x mode, but at a huge speed penalty. I am planning on re-introducing it integrated into the new scheme, probably as simply -x4.

As for your file there, this looks very much like the old "mono" problem that would be addressed with the --optimize-mono switch. Have you tried that?

halb27:
I can see why you might think that a new release was not imminent because the first alpha was so long ago. But, I didn't do a release for the first alpha because I didn't think the only added feature (--optimize-mono) was significant enough to justify a new release. There are certainly enough new now to warrant it, however, so it won't be long. 

krmathis:
Yes, the documentation is for the actual distribution. To build from SVN you will need libtool (I have version 1.5.22) and maybe autoconf. You should be able to easily find that for your distro. Hope that helps; I'm kinda new at this part myself... 

The source packages are definitely not going to be separate anymore.

dutch109:
What I try to do is get basic functionality in there and worry about extra convenience later. Someday I'd like to combine all three command-line programs into one that will do everything (like FLAC), but so far I've been concentrating on adding features and functions that are not available any other way.

It turns out that clipping prevention is not needed with WavPack. In lossless mode you can't get clipping without ReplayGain, and in lossy mode the clipping prevention is automatically done during decode.

DARcode:
Yes, on 4.4a3, -hhx will normally compress better and faster than -hx3, but keep in mind that -hx3 will decode faster than -hhx (and play better on Rockbox). You either pay once up front, or every time you decode. 

I don't think these very fast -x modes will ever quite match the old -x mode in ultimate compression, but I will be putting in a modified version of the original -x mode that should work as well (or slightly better).

What Roberto said is basically correct. I don't think the WavPack format will be able to get any better compression than 4.31 with -x (well, maybe a tiny bit more, or maybe better with certain files). What I have been trying to achieve is getting close to that level of compression without taking forever to get it. And of course there can always be speed improvements through optimization (like MMX).

dv1989:
That is just temporary situation until the old -x mode is re-integrated. The idea is that the usable compression is better.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-08-22 09:14:27
Thanks David, I appreciate very much all the time you put in the development of your superb audio compressor and answering questions here on HA.

I agree on the "usable" compression take, but please do re-integrate the "old" -x mode, I'm already salivating at the prospect of an MMX/SSEx optimized -hx4 (x4 = v4.31 -x mode) :] !

Finally, is Rockbox the only option for WV on DAP or have you been made aware of further projects?

EDIT: "Usable" comopression and Rockbox parts
Title: WavPack 4.4 alpha 2 for Windows
Post by: NeXT on 2006-08-22 11:06:57
NeXT:
I'll try to look into that crashing problem you have there. 


I should mention that that commandline is INVALID -- WavPack should do nothing and exit. So my example is not a PROBLEM, preventing from doing needed job, but just a bug, discovered acidently.

As for your file there, this looks very much like the old "mono" problem that would be addressed with the --optimize-mono switch. Have you tried that?


Oh, I never noticed this command, because it appears only in WavPack 4.4 internal help, but not in any html file (neither site, nor bundled User Manual), or internal help of WavPack 4.31. Yes, it helped, that wav compressed with --optimize-mono -hhx3 has 84 539 008 bytes now, but still diference with Monkey's Audio is 8 667 630 bytes. Waiting for returning old -x mode to test it with --optimize-mono..
Title: WavPack 4.4 alpha 2 for Windows
Post by: lantern_ on 2006-08-22 13:41:47
I seem to be having problems accessing the sourcecode from the svn server.  Here is the error message I receive:

Error * PROPFIND request failed on '/wavpack' PROPFIND of '/wavpack': Could not read status line: An existing connection was  forcibly closed by the remote host.  (http://svn.slomosnail.de)

I am using tortoisesvn.

Thanks
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-23 05:52:07
Finally, is Rockbox the only option for WV on DAP or have you been made aware of further projects?

Unfortunately, no, Rockbox is the only DAP option right now, at least for portables. There is the Roku PhotoBridge which has a third-party plugin available for WavPack:

http://www.rokulabs.com/products/photobridge/index.php (http://www.rokulabs.com/products/photobridge/index.php)



I seem to be having problems accessing the sourcecode from the svn server.  Here is the error message I receive:

Error * PROPFIND request failed on '/wavpack' PROPFIND of '/wavpack': Could not read status line: An existing connection was  forcibly closed by the remote host.  (http://svn.slomosnail.de)

I am using tortoisesvn.

I don't normally use tortoisesvn (I use svn under cygwin) but I do happen to have it installed now and just tried it and it worked fine. Not sure what could be up; I assume you tried it more than once...?



Oh, I never noticed this command, because it appears only in WavPack 4.4 internal help, but not in any html file (neither site, nor bundled User Manual), or internal help of WavPack 4.31. Yes, it helped, that wav compressed with --optimize-mono -hhx3 has 84 539 008 bytes now, but still diference with Monkey's Audio is 8 667 630 bytes. Waiting for returning old -x mode to test it with --optimize-mono..

The reason the command isn't in the manual is because it was only introduced with alpha 1. You might want to look at this thread:

http://www.hydrogenaudio.org/forums/index....showtopic=43866 (http://www.hydrogenaudio.org/forums/index.php?showtopic=43866)

I probably should have mentioned this in this thread also.
Title: WavPack 4.4 alpha 2 for Windows
Post by: speakermagnet on 2006-08-23 10:11:14

Finally, is Rockbox the only option for WV on DAP or have you been made aware of further projects?

Unfortunately, no, Rockbox is the only DAP option right now, at least for portables. There is the Roku PhotoBridge which has a third-party plugin available for WavPack:

http://www.rokulabs.com/products/photobridge/index.php (http://www.rokulabs.com/products/photobridge/index.php)

But wait!  I know there was an effort to get a wavpack codec supported in TCPMP (in fact I am testing it right now on my Treo 650).  Bryant are you aware of this development?

Support on my Treo is one of the main reasons I will be switching my entire collection over to wavpack: one set of files that can be used for both my portable/Treo (space conscious hence lossy) and my home (space and bandwidth o'plenty).  This new 4.4 release sounds good enough to wait for.

In the mean time I just have to get slimserver to support wavpack (.wv + wvc) including tagging to stream to my SliMP3 and my Roku Soundbridge M1001.  I figure since it can support FLAC then we should be able to get wavpack going.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Slo Mo Snail on 2006-08-23 11:24:49
./autogen.sh
./autogen.sh: line 5: libtoolize: command not found


You must have libtool installed before running autogen.sh
Title: WavPack 4.4 alpha 2 for Windows
Post by: lantern_ on 2006-08-23 11:44:07
I have tried several times.  Now it is asking for a username and password to access svn.slomosnail.de.
Title: WavPack 4.4 alpha 2 for Windows
Post by: krmathis on 2006-08-23 16:00:44
Edit: I have moved my "Compile WavPack from SVN" problems to a new thread:
http://www.hydrogenaudio.org/forums/index....showtopic=47682 (http://www.hydrogenaudio.org/forums/index.php?showtopic=47682)
Title: WavPack 4.4 alpha 2 for Windows
Post by: lantern_ on 2006-08-24 02:10:43
I'm not sure what I did, but SVN seems to be working.
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-08-24 06:51:58
But wait!  I know there was an effort to get a wavpack codec supported in TCPMP (in fact I am testing it right now on my Treo 650).  Bryant are you aware of this development?

Support on my Treo is one of the main reasons I will be switching my entire collection over to wavpack: one set of files that can be used for both my portable/Treo (space conscious hence lossy) and my home (space and bandwidth o'plenty).  This new 4.4 release sounds good enough to wait for.

In the mean time I just have to get slimserver to support wavpack (.wv + wvc) including tagging to stream to my SliMP3 and my Roku Soundbridge M1001.  I figure since it can support FLAC then we should be able to get wavpack going.

I was aware of the TCPMP development, but I seem to forget about it because it's not official and I don't have one of the devices. Thanks for reminding me and I'm glad to hear it's working good!

I also have a Squeezebox and tried to get Slimserver playing WavPack with it. Unfortunately, I didn't know Perl so I got a Perl book and went through that, but then ran out of time for that project. I'll get back around to that one of these days; if anyone wants to jump in before me I'd be happy to help... 
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-08-26 18:44:56
Recompressed a couple of test ablums (Faith No More - The Real Thing and Primus - Frizzle Fry) with 4.4a3 -hx3m and compared to 4.31  -hxm they are only ~200/250 Kb bigger (including full tags w/ RG), so the time save and better DAP support is definitely worth it, bravo David!

EDIT: Grammar.

EDIT2: Mo' gramma.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-09-05 10:06:22
24-7 Spyz - Gumbo Millennium is just ~220 kb bigger too  .

Any new alphas to play with yet ?
Can't wait to test -hx4 (x4 = old v4.31 -x mode), maybe MMX/SSEx optimized too.
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-06 07:36:27
24-7 Spyz - Gumbo Millennium is just ~220 kb bigger too  .

Any new alphas to play with yet ?
Can't wait to test -hx4 (x4 = old v4.31 -x mode), maybe MMX/SSEx optimized too.

Unfortunately I have not got a chance to work on -x4 yet (and I'm not even thinking about MMX). I have added commands to wvunpack to automate the extraction of embedded cuesheets, but that's just in SVN for now. Thanks for your (and everyone's) testing so far (and thanks for your patience)... 

I did finish up a first pass at a test suite that should be a useful resource for developers including WavPack support in their applications (and is important for acceptance of WavPack in the Linux community). It also has some samples at various lossy bitrates that provide a demo of WavPack's capabilies outside pure lossless. Anyway, if anyone would like to check it out, it can be found here:

http://www.rarewares.org/wavpack/ (http://www.rarewares.org/wavpack/)

BTW, many thanks to Roberto for the hosting! 
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-06 07:44:51
I have added commands to wvunpack to automate the extraction of embedded cuesheets, but that's just in SVN for now.
Yow beauwty!

Looks like I may have to get SVN setup to play...

Thanks for listening David.
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-06 08:04:39
Looks like I may have to get SVN setup to play...

Thanks for listening David.

Actually, it's not too fancy. It doesn't edit the cuesheet or anything like that, but it does get it out without having to jump through hoops.

I could easily post another alpha if you're not set up to compile it and would like to beat on it... 
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-09-06 08:41:41
Great to see you're able to allocate time to WV these days and things are progessing quite a bit, thank you David, being patient is the least we can do, such a brilliant product and free to boot, I'd like to help more (tho ain't no coder) or being able to donate at least.

P.S.
I'm a Kubuntu user too, albeit a total noob.

EDIT: Grammar grammar grammar.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-06 09:56:58
I could easily post another alpha if you're not set up to compile it and would like to beat on it... 
I'm not currently setup for it, but I wouldn't want you to waste your time; that was not my intention.

Thanks for the offer.
Title: WavPack 4.4 alpha 2 for Windows
Post by: kjoonlee on 2006-09-06 12:07:43
SVN works over HTTP, so any web-downloader will work if you're in a pinch; just do a recursive download and you'll get the current revision of the mainline trunk.

(The WavPack SVN server doesn't seem to use trunks or branches, though.)
Title: WavPack 4.4 alpha 2 for Windows
Post by: lantern_ on 2006-09-06 12:56:59
Here is a build from the source this morning:

http://www.4shared.com/file/3518984/727cc0...k_20060906.html (http://www.4shared.com/file/3518984/727cc0c8/wavpack_20060906.html)

It was built with MSVC++ 2006 Express Edition, with the default project settings.  If there are any problems, please let me know.

Thanks Bryant!  Great software!
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-07 05:34:31
I could easily post another alpha if you're not set up to compile it and would like to beat on it... 
I'm not currently setup for it, but I wouldn't want you to waste your time; that was not my intention.

Thanks for the offer.

Well, it certainly would not be a waste of time (and probably wouldn't have taken 5 minutes) but lantern did it while I was sleeping (thanks!) and I checked his version and it seems to be fine. I would be interested to know if this meets at least some of the desired utility, although I know you are busy...
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-07 09:54:28
I actually managed to beat lantern to it, but didn't think to post my builds.  I had TortoiseSVN here at work already, and installed Visual C++ 2005 Express with little trouble.  It's more fun with a little effort.

What can I say, it does exactly what it says on the tin!  I like the options to both output to STDOUT and to file on extraction.  It's great that WVUNPACK asks for confirmation before overwriting an existing cuesheet, as it does for the WAV file.

I tested with a self-extracting file which I believe I successfully added a cuesheet to on encoding, but on extraction the cuesheet was not extracted.  I may need to test again to be sure; it may be safer to encode to WV, check the cuesheet is there (-c switch ), and then prepend the wvselfx.exe file, before I test.

IMHO, if a self-extracting file has a cuesheet it will be required, and it would be great if it could be automatically extracted when the EXE is double-clicked.  Is that achievable?  There's something nice about the idea of giving someone an EXE and saying "double click that and you have everything you need to burn to CD in Nero". (If someone knew about EAC, foobar, or Burrrn they could handle the WV themselves).

I can't really think what else to test... the cuesheet is simply a piece of text (in this instance) so WVUNPACK has no concerns about compliancy, etc.  I can't think how I could trip it over... it either extracts or it doesn't.  Is there anything specific you would like tested?

Thanks again David.

Edit: OK, tested self-extract again using method above, and no cuesheet.
Title: WavPack 4.4 alpha 2 for Windows
Post by: dv1989 on 2006-09-07 11:57:18
I have added commands to wvunpack to automate the extraction of embedded cuesheets, but that's just in SVN for now.
Yow beauwty!

I second that; thanks, Bryant!
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-08 05:24:36
IMHO, if a self-extracting file has a cuesheet it will be required, and it would be great if it could be automatically extracted when the EXE is double-clicked.  Is that achievable?  There's something nice about the idea of giving someone an EXE and saying "double click that and you have everything you need to burn to CD in Nero". (If someone knew about EAC, foobar, or Burrrn they could handle the WV themselves).
The wvselfx header is due for an update and that's where the logic needs to go to self-extract a cuesheet. My plan was to simply have it extract the cuesheet if there's one there; I can't imagine a scenario where that would be a problem.


I can't really think what else to test... the cuesheet is simply a piece of text (in this instance) so WVUNPACK has no concerns about compliancy, etc.  I can't think how I could trip it over... it either extracts or it doesn't.  Is there anything specific you would like tested?
Actually, the fact that you ran it on a different machine with a different image and cuesheet is a good test. I was mostly concerned whether this was enough functionality and that the options made sense. It turns out that it is a little more complicated than it seems because the cuesheet is stored as UTF-8 in the tag and needs to be converted to ANSI, and I had a bug where newlines were not being handled correctly, but I think it's all sorted out now.

Thanks for the feedback! 
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-08 07:59:39
The wvselfx header is due for an update and that's where the logic needs to go to self-extract a cuesheet. My plan was to simply have it extract the cuesheet if there's one there; I can't imagine a scenario where that would be a problem.
Superb.  That would be ideal.

I was mostly concerned whether this was enough functionality and that the options made sense.
Well, we still have the idea of editing the cuesheet upon extraction to ensure that it points to the WAVE file, but I realise that this is a lot more effort, and possibly should simply be bounced back to the embedder's responsibility to ensure that the cuesheet points to a WAVE of the same name before embedding.

Unfortunately mine don't, and I guess EAC doesn't do that if you rip compressed.  However, again, I suppose you could say that the embedder should have the skills to edit the cuesheet.  It's all down to making it as easy as possible for the decrypter, not the encrypter.

Both would be good though.

It turns out that it is a little more complicated than it seems because the cuesheet is stored as UTF-8 in the tag and needs to be converted to ANSI, and I had a bug where newlines were not being handled correctly, but I think it's all sorted out now.
When I made my changes to Tag I had to mess about to get new lines coming out OK in STDOUT.  Hmmm... I wonder whether it barfs with non-ASCII characters.

That's why you do what I do, and I just stick my oar in.
Title: WavPack 4.4 alpha 2 for Windows
Post by: DARcode on 2006-09-20 16:20:53
David, I've been using your nice little proggy copytags a lot lately, it's definitely pretty handy, any intention of integrating its functionality into wavpack.exe?
If not in the near future what about when all executables will be merged into one?
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-22 18:58:58
David, I've been using your nice little proggy copytags a lot lately, it's definitely pretty handy, any intention of integrating its functionality into wavpack.exe?
If not in the near future what about when all executables will be merged into one?

Oops, sorry, for some reason I missed this post...

Glad you can use copytags. Just don't use it on anything important! 

Yes, once wavpack.exe can accept wv files as input (for transcoding) then it would also copy any tags found.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-24 11:29:19
As part of my Yalac testing I have just completed some tests with 4.4a3.  For the full results, please see this post on the Yalac thread (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=43494&view=findpost&p=434373) for info.

Here are the WavPack-related results though, settings in compression order.

REMOVED, please see post #68 below.
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-24 23:33:05
Thanks Synthetic Soul for adding the new WavPack version, I really appreciate all your work on this!

The results fit my expectations pretty much, however I think there's one error. I find it unlikely that "4.4a3" and "4.4a3 -f" would get exactly the same compression ratio. I think they must be both "-f". If that's the case then it's also interesting that the two decode speeds differ by almost 3% when they should be identical! I suspect that this indicates that your test method has a higher level of measurement error than your 4 significant digits suggest and that perhaps rounding to the nearest integer would be more appropriate.

Finally, a request. If I could add a single option combination it would be "-fx". I think this is significant because it offers faster decoding and better compression than "-f" alone, and with a fairly small increase in encode time. For those users who might consider WavPack as a FLAC replacement it is certainly the most logical choice. I would even be willing to give up the other -x2 options for this because they really don't offer too much over -x alone. But, of course, this is all up to your descretion. 

Thanks again for the results.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-25 07:37:55
Argh!  I have just checked the scripts and both runs were using -f.  I do try to check my results when I publish, as I am very concerned about spreading misinformation, but this obviously slipped my net.  Many apologies.

Personally, I'm not suprised that the speeds vary slightly.  My reports currently list TIMER's global speed which includes CPU and IO time.  I suspect that the IO process, which in my case slows the process down, introduces some variables.  Your point about using integer values only is extremely valid, and not something I had previously considered.  I had a brief discussion about significant bits a while back; not being a math wiz I don't really consider the fact that the decimal places shown indicate the precision.  I will change it ASAP.  FYI: My scripts are now recording both CPU+IO and CPU only times, but I need time to update my report system to be able to display both/either.  I would be a lot happier displaying CPU-only results.

Of course I will run -fx and add that in to the report.

Thanks for the feedback David, and again, apologies for the false info.

BTW, although it was posted in another thread, I have to say:

It's kind of like poking someone with a stick; most of the time you'd get them in the leg or the arm, but if you were really lucky you would get them in the eye and cause real trouble.
... has got to be the best analogy I have ever seen!
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-25 08:04:24
OK, I've updated the default figures immediately, as they were wrong.  I'll run the -fx (and probably -fx2 and -fx3) tests today and get them up as soon as I can.

Here's the correct data:

Code: [Select]
Encoder Setting    Compression    Encode    Decode
==================================================
WavPack -hx            64.337%        1x       46x
WavPack 4.4a3 -hhx3    64.382%        4x       45x
WavPack 4.4a3 -hhx2    64.396%        8x       45x
WavPack 4.4a3 -hhx     64.423%       13x       45x
WavPack -h             64.487%       28x       46x
WavPack 4.4a3 -hh      64.500%       28x       46x
WavPack 4.4a3 -hx3     64.679%        6x       53x
WavPack 4.4a3 -hx2     64.708%       11x       53x
WavPack 4.4a3 -hx      64.764%       18x       53x
WavPack 4.4a3 -h       64.877%       33x       53x
WavPack -x             65.144%        3x       66x
WavPack 4.4a3 -x3      65.285%        9x       65x
WavPack 4.4a3 -x2      65.312%       17x       65x
WavPack 4.4a3 -x       65.371%       25x       64x
WavPack 4.4a3          65.582%       43x       62x
WavPack                65.750%       46x       67x
WavPack 4.4a3 -f       66.741%       49x       66x
WavPack -f             67.095%       49x       69x
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-25 19:38:32
And now with -fx(2/3):

Code: [Select]
Encoder Setting    Compression    Encode    Decode
==================================================
WavPack -hx            64.337%        1x       46x
WavPack 4.4a3 -hhx3    64.382%        4x       45x
WavPack 4.4a3 -hhx2    64.396%        8x       45x
WavPack 4.4a3 -hhx     64.423%       13x       45x
WavPack -h             64.487%       28x       46x
WavPack 4.4a3 -hh      64.500%       28x       46x
WavPack 4.4a3 -hx3     64.679%        6x       53x
WavPack 4.4a3 -hx2     64.708%       11x       53x
WavPack 4.4a3 -hx      64.764%       18x       53x
WavPack 4.4a3 -h       64.877%       33x       53x
WavPack -x             65.144%        3x       66x
WavPack 4.4a3 -x3      65.285%        9x       65x
WavPack 4.4a3 -x2      65.312%       17x       65x
WavPack 4.4a3 -x       65.371%       25x       64x
WavPack 4.4a3          65.582%       43x       62x
WavPack                65.750%       46x       67x
WavPack 4.4a3 -fx3     66.386%       15x       67x
WavPack 4.4a3 -fx2     66.445%       25x       67x
WavPack 4.4a3 -fx      66.536%       33x       66x
WavPack 4.4a3 -f       66.741%       49x       66x
WavPack -f             67.095%       49x       69x
Code: [Select]
Encoder Setting    Compression    Encode    Decode
==================================================
WavPack 4.4a3          65.582%       43x       62x
FLAC (-5)              66.279%       38x       72x
WavPack 4.4a3 -fx      66.536%       33x       66x
Code: [Select]
Encoder Setting    Compression    Encode    Decode
==================================================
Flake -12              65.368%        7x       69x
WavPack 4.4a3 -x       65.371%       25x       64x

http://synthetic-soul.co.uk/comparison/lossless/?All=1 (http://synthetic-soul.co.uk/comparison/lossless/?All=1)

I made reference in the Yalac thread when I posted my initial results that I had never previously considered WavPack's- -f option, and was surprised at the speed and compression.  It's impressive that WavPack can compete with FLAC -5 in speed, and still surpass Flake in compression.

I also find it interesting that the changes in 4.4 are geared toward faster, less processor-intensive decoding, and that has come at a small cost in compression.  I must admit, I think I may be sticking with -h (-hh), but it's very possible that I may switch to the new -h for a little faster decoding at the cost of a few MiB, as it seems the difference to me would only be around 4MiB per GiB encoded, which really is negligable.

Edit: David, can you see the presets changing at all for the release version, or will I be able to rename to 4.4 and leave the results as is?
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-26 06:54:34
Ah yes, those results make more sense now! Thanks for updating them so quickly and adding the -f results also.

Yeah, I find that this speed measurement business can be a real nightmare trying to get results that are consistent and make sense. There are enough variations between CPU's and OS's and I/O devices and compilers to make your head spin around! Anyway, I think having those numbers rounded to ints makes sense and doesn't convey more accuracy than is actually there.

I don't know if you originally encoded your files with -h or -hx, but in my corpus the new -hx3 actually compresses better than the old -h (i.e. new -hh). I notice that for some reason the -x mode achieves a lot less extra compression on your corpus than on mine. Probably that's because my corpus was designed to have a wide and varying range of material, which is exactly what the -x mode is trying to take advantage of.

I'm not really sure what I'm going to finally do with the presets. I am going to try to add a -x4 setting to work more like the old -x mode, but this is really only for special cases and certainly wouldn't make sense for your corpus. I would also like to see if I can make some quick performance tweaks while still staying in pure C, but time will tell whether that happens. Basically, I reserve the right to improve everything dramatically!

Oh, BTW, glad you liked the analogy...   
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-26 07:24:35
Yeah, I find that this speed measurement business can be a real nightmare trying to get results that are consistent and make sense. There are enough variations between CPU's and OS's and I/O devices and compilers to make your head spin around! Anyway, I think having those numbers rounded to ints makes sense and doesn't convey more accuracy than is actually there.
I don't really like posting my "IO-infected" rates, but it's how I started reporting and I would need some time to switch totally to CPU-only.  I am also still undecided as to how irrelevant IO+CPU times are, as this is what I would see.  I've never meant my tests to be the definitive figures for each encoder, but it concerns me that people may think that is what I am trying to portray.  Thanks for the suggestion to report integer values; it makes a lot of sense.

I don't know if you originally encoded your files with -h or -hx, but in my corpus the new -hx3 actually compresses better than the old -h (i.e. new -hh). I notice that for some reason the -x mode achieves a lot less extra compression on your corpus than on mine. Probably that's because my corpus was designed to have a wide and varying range of material, which is exactly what the -x mode is trying to take advantage of.
I use -hm at the moment.  I assumed that -x must perform better with other corpuses than mine.  A lot of people seem to use -x.  The fact that it is not nearly such a performance hit as the old -x is great anyway, and does mean I may consider using it, in case my music taste gets a little more varied.

I'm not really sure what I'm going to finally do with the presets. I am going to try to add a -x4 setting to work more like the old -x mode, but this is really only for special cases and certainly wouldn't make sense for your corpus. I would also like to see if I can make some quick performance tweaks while still staying in pure C, but time will tell whether that happens. Basically, I reserve the right to improve everything dramatically!
Hey, fine by me.  I will re-test when you release 4.4; it may be interesting to compare the two results anyway, to ensure they match where they should.

Thanks David.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-26 09:50:09
Out of interest, here's the results for my "Yalac" corpus in a different format.

Firstly, the speeds are CPU-only, so they are not affected by my hard drive.  Bear in mind that, if you decode to file, you may not attain these speeds, even if using the same equipment, as writing to disc may introduce some further delay (refer to my results listed above to see the difference my hard drive makes, or take a look at the graph on this unfinished page (http://synthetic-soul.co.uk/comparison/lossless/rate.asp)).

Secondly, I have grouped the switches, and I think the result quite neatly demonstrates the effect of using -x in addition to a core setting, so I thought I would share.

Code: [Select]
Mode    Encoding    Decoding    Compression
===========================================
-hh          32x         53x        64.500%
  x          15x                    64.423%
  x2          8x                    64.396%
  x3          4x                    64.382%
  
-h           41x         64x        64.877%
  x          20x                    64.764%
  x2         12x                    64.708%
  x3          6x                    64.679%
  
Default      59x         78x        65.582%
  x          29x                    65.371%
  x2         18x                    65.312%
  x3         10x                    65.285%

-f           69x         91x        66.741%
  x          36x                    66.536%
  x2         27x                    66.445%
  x3         16x                    66.386%

 
I am currently running the same scripts on my "FLAC" corpus.  When I have the results (around a day's time) I'll post them in this same format.  I'm hoping the greater (although probably still not great) variety in my "FLAC" corpus may demonstrate -x's benefits more.
Title: WavPack 4.4 alpha 2 for Windows
Post by: ssjkakaroto on 2006-09-26 13:09:11
Synthetic Soul the build you're using doesn't have any MMX/SSEx optimizations right?
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-26 14:32:39
No, it doesn't.  It's the version from post #30 in this thread.

I'm not overly interested in optimised versions until David applies them himself (at which point I will become very interested).
Title: WavPack 4.4 alpha 2 for Windows
Post by: shadowking on 2006-09-26 14:57:22
I am very interested in optimised versions since my PC isn't the fastest.
Title: WavPack 4.4 alpha 2 for Windows
Post by: he-jo on 2006-09-26 16:32:03
I currently have to work abroad, but I'll probably investigate in those optimisations again, when I'm back home in November. Anyway, I think David should release a final version first, so that we have a stable base to do the testing on.

It's really nice to see the recent improvements in the field of lessless encoding. Thanks to all codec authors for sharing their work with us!


Kind regards,
Jo.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Synthetic Soul on 2006-09-27 12:40:54
I am currently running the same scripts on my "FLAC" corpus.  When I have the results (around a day's time) I'll post them in this same format.  I'm hoping the greater (although probably still not great) variety in my "FLAC" corpus may demonstrate -x's benefits more.
A little more benefit with this corpus.

Code: [Select]
Mode    Encoding    Decoding    Compression
===========================================
-hh          32x         53x        58.775%
  x          14x                    58.674%
  x2          8x                    58.637%
  x3          4x                    58.626%

-h           40x         65x        59.058%
  x          19x                    58.919%
  x2         12x                    58.867%
  x3          6x                    58.841%

Default      60x         80x        59.715%
  x          28x                    59.469%
  x2         18x                    59.398%
  x3         10x                    59.369%

-f           70x         93x        61.020%
  x          39x                    60.485%
  x2         28x                    60.369%
  x3         16x                    60.303%
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-09-28 07:03:13
I currently have to work abroad, but I'll probably investigate in those optimisations again, when I'm back home in November. Anyway, I think David should release a final version first, so that we have a stable base to do the testing on.

I agree. Currently, the function decorr_stereo_pass() is unchanged, so the previous patch could be dropped into the current code, but I don't think it's worth it for now because I plan on trying to do some more optimization on the C code.

Once the release is done I would be very interested in getting these MMX optimizations in, and I really appreciate your and wisodev's work on this. 

David
Title: WavPack 4.4 alpha 2 for Windows
Post by: Xenion on 2006-10-19 13:39:34
i did not read all pages of this thread so i don't know if this is the wrong position to post this.

i have a feature request for the new wavpack version:
it would be great if the wavpack encoder would accept wavpack files as input. then you could easily update your "old" wavepack files to the new version. of course all tags/etc have to be copied also.
thanks and keep up the great work.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Fandango on 2006-10-19 16:09:21
i have a feature request for the new wavpack version:
it would be great if the wavpack encoder would accept wavpack files as input. then you could easily update your "old" wavepack files to the new version. of course all tags/etc have to be copied also.


You can easily achive the audio conversion via pipes:

Code: [Select]
wvunpack "original track.wv" - | wavpack - "reencoded track.wv"


This has one disadvantage tho, the tags will not be preserved.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Xenion on 2006-10-19 23:39:30
i have a feature request for the new wavpack version:
it would be great if the wavpack encoder would accept wavpack files as input. then you could easily update your "old" wavepack files to the new version. of course all tags/etc have to be copied also.


You can easily achive the audio conversion via pipes:

Code: [Select]
wvunpack "original track.wv" - | wavpack - "reencoded track.wv"


This has one disadvantage tho, the tags will not be preserved.


not a solution for me as this doesn't copy my embedded cuesheets and logs but thanks anyway
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-10-20 06:30:00
i did not read all pages of this thread so i don't know if this is the wrong position to post this.

i have a feature request for the new wavpack version:
it would be great if the wavpack encoder would accept wavpack files as input. then you could easily update your "old" wavepack files to the new version. of course all tags/etc have to be copied also.
thanks and keep up the great work.

Unfortunately, this feature will not be able to make it into the new version, but it is something that is planned (especially now that FLAC can do it). 

However, in my testing I use a few short commands that basically achieves the same thing (and I'm sure a simple batch file would simplify it further). I describe it here:

http://www.hydrogenaudio.org/forums/index....mp;#entry423126 (http://www.hydrogenaudio.org/forums/index.php?showtopic=33773&st=0&p=423126&#entry423126)

And, of course, Foobar2000 or dBpowerAMP would both work fine (once they are updated with the new library).
Title: WavPack 4.4 alpha 2 for Windows
Post by: Lych on 2006-11-08 08:47:01
I'm curious, how much more can lossless audio compression be optimized?  Wavpack seems to have achieved its maximum compression ratio (while being statistically significant)  .
Title: WavPack 4.4 alpha 2 for Windows
Post by: halb27 on 2006-11-08 09:45:38
I guess we have to face the fact that we're close to what can be achieved. All the good codecs like wavPack, FLAC, Monkey and others yield a compression ratio which is more or less the same (not counting peas). Progress is still there, but it's not very essential any more. The developers have a done a great job already.

But other things count as well, like David's progress when using wavPack on mobile DAPs. Top quality lossy versions of lossless codecs are very interesting too. I'm very curious about 4.4 final and I'm looking forward to it.
Title: WavPack 4.4 alpha 2 for Windows
Post by: Wombat on 2006-12-01 12:57:34
bryant, one question. When i remember right you did some programming for Slimdevices to add wavpack support to Sllimserver.
The Transporter of them now has a Ubicom IP3K series Proceessor @325MHz. The Squeezeboxes only had 250MHz.
I can imagine a -h encoded file should now be possible to be decoded natively in the unit.
Do you have any infos if something like this is planned?
Title: WavPack 4.4 alpha 2 for Windows
Post by: bryant on 2006-12-06 06:50:18
bryant, one question. When i remember right you did some programming for Slimdevices to add wavpack support to Sllimserver.
The Transporter of them now has a Ubicom IP3K series Proceessor @325MHz. The Squeezeboxes only had 250MHz.
I can imagine a -h encoded file should now be possible to be decoded natively in the unit.
Do you have any infos if something like this is planned?

I did try to get WavPack working with SlimServer, but got sidetracked. I still plan to go back to it at some point.

There was never any plan to get WavPack decoding natively because that source is closed (and I guess that the tools are probably expensive, so it won't be hacked either). However, if it actually was done I think it would work pretty well even at 250 MHz. Even decoding the -h (now -hh) mode of WavPack lossy (which is worst for CPU usage) uses less than 75 MHz on an ARM core, and I suspect that their processor isn't that different. And I think WavPack lossy is a natural for the Transporter because high-res 24/96 material compresses very nicely to around 1024 kbps.

But it turns out I'm not "in the loop" with them right now... 
Title: WavPack 4.4 alpha 2 for Windows
Post by: Wombat on 2006-12-06 08:51:46
But it turns out I'm not "in the loop" with them right now... 

Thanks for the clarification. So lets hope it sometimes happens
Title: WavPack 4.4 alpha 2 for Windows
Post by: navin on 2007-01-03 06:12:43
Warning: I am a newbie.

Some time back (Oct 2005) I encoded all my CDs using WavPack 4.2. I have now about 180GB of Wavpack files.

the arguments used were
hm -w "Artist=%a" -w "Album=%g" -w "Track=%n" -w "Title=%t" -w "Year=%y" -w "Genre=%m" %s %d

the file path used was
F:\wav\%a\%g\%n - %t

1. Do I need to reencode them using the latest version of WavPack?

2. Can these files be played on any hard disk based player (espcially any protable). The idea is to stop using my CDs and used Hard disks (both at home and on the road) for playback. I presently use MP3s on an ipod on the road but there is a significant difference between my MP3 (--preset extreme) and my wavpack files and i'd love to use wavpack if possible.

I dont want to have a PC or Mac involved for playback as the boot time is a deterent.
Title: WavPack 4.4 alpha 2 for Windows
Post by: kanak on 2007-01-03 07:13:07
1. Do I need to reencode them using the latest version of WavPack?

2. Can these files be played on any hard disk based player (espcially any protable). The idea is to stop using my CDs and used Hard disks (both at home and on the road) for playback. I presently use MP3s on an ipod on the road but there is a significant difference between my MP3 (--preset extreme) and my wavpack files and i'd love to use wavpack if possible.

I dont want to have a PC or Mac involved for playback as the boot time is a deterent.


1. Reencoding is not necessary. You will get slightly better compression, but it's just not worth it.

2. Any? No. You can find a list of hardware support for wavpack here (http://www.wavpack.com/links.html). If you have an ipod, you can rockbox it and use wavpack on it.
Title: WavPack 4.4 alpha 2 for Windows
Post by: navin on 2007-09-11 09:52:33
2. Any? No. You can find a list of hardware support for wavpack here (http://www.wavpack.com/links.html). If you have an ipod, you can rockbox it and use wavpack on it.


Thanks Kanak, now that there is a 160GB ipod out storing wavpack files I am more serious about using my wavpack library.

Q1: If I use rockbox do I still need iTunes? If not how do I sync my ipod and create playlists?
Q2: My car CD player (Alpine) is designed to control my ipod. This works nicely using MP3s and an ipod. Will I use this compatibility if I switch to Rockbox?
Title: WavPack 4.4 alpha 2 for Windows
Post by: preglow on 2007-09-11 18:42:08
Q1: If I use rockbox do I still need iTunes? If not how do I sync my ipod and create playlists?

If the new Ipods use the same hardware and encryption schemes as the 2nd generation Nano, you might have to wait a long while before seeing Rockbox support. Rockbox does not need Itunes at all, you "sync" music by plain and simply copying files to the Ipod with your tool of choice, and create playlists the usual way, either on your computer (then copying them to the Ipod), or by creating them directly on the Ipod with Rockbox.

Q2: My car CD player (Alpine) is designed to control my ipod. This works nicely using MP3s and an ipod. Will I use this compatibility if I switch to Rockbox?

Rockbox does not support this, but anyone is free to figure out how this works and code it for us.
Title: WavPack 4.4 alpha 2 for Windows
Post by: navin on 2007-09-12 05:18:20
If the new Ipods use the same hardware and encryption schemes as the 2nd generation Nano, you might have to wait a long while before seeing Rockbox support.


how do we know what scheme is used by the new 80GB and 160GB Ipods? Is there any way to determine this before we buy the new ipods. Presently I have old 80GB ipods (Ipod Video).

Is there any other software (other than Rockbox) that will allow one to use wavpack files on an ipod?
Title: WavPack 4.4 alpha 2 for Windows
Post by: preglow on 2007-09-12 10:52:13
how do we know what scheme is used by the new 80GB and 160GB Ipods? Is there any way to determine this before we buy the new ipods. Presently I have old 80GB ipods (Ipod Video).

Is there any other software (other than Rockbox) that will allow one to use wavpack files on an ipod?

Only way to find out is wait until someone opens one up and has a good look. The hardware does indeed seem to be similar to the 2nd gen. Nano, which means we have no data sheets on it, and I sincerely doubt they have downgraded their efforts at locking us out with encryption.
And no, nothing (to my knowledge) plays Wavpack apart from Rockbox. On Ipods you pretty much only have Rockbox and Ipodlinux to choose from, and the latter does not seem to be developed much anymore, and does in any case not support Wavpack.
Your old Ipod _is_ supported by Rockbox, and nothing is keeping you from using your Wavpack files on that.
Title: WavPack 4.4 alpha 2 for Windows
Post by: navin on 2008-01-17 07:32:53
Your old Ipod _is_ supported by Rockbox, and nothing is keeping you from using your Wavpack files on that.


Found a solution. Use new 160GB ipod with wavpack files and rockbox for portable listening with Bithead and good headphone (AKG 701 or Beyer DT990) and old 80GB ipod with MP3 files (EAC/LAME) for the Alpine car deck.
Title: WavPack 4.4 alpha 2 for Windows
Post by: greynol on 2008-01-17 07:39:52
new 160GB ipod
+
rockbox
=
Not currently possible