Skip to main content

Topic: WavPack 4.41 beta available (Read 55915 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #50
I don't really understand the MMX, SSE stuff; however would I be right in saying that, due to the integer arithmetic nature of WavPack, SSE code is pointless?  Anything else up your sleeve?
Right. Because WavPack encoding is integer-only (and generally not parallelizable) there's little or nothing to be gained by SSE and his brothers. MMX should be able to make a nice improvement because the left and right channels can be done together, but because of some optimization available for 16-bit data that doesn't work with MMX, the MMX stuff only really helps with 24-bit (and float) data.

I did get an idea from one of the FLAC 1.1.4 improvements that might be applicable to WavPack and I'll give that a try. You never know when something's going to make a big difference...

Quote
Anyway, I'm sure all your users will agree, the extra benefit provided by this beta with no loss in compression is extremely welcome.  Many thanks for your continued efforts.
Ah, well it's my pleasure... 

  • DARcode
  • [*][*][*][*][*]
  • Members (Donating)
WavPack 4.41 beta available
Reply #51
[...]Anyway, I'm sure all your users will agree, the extra benefit provided by this beta with no loss in compression is extremely welcome.  Many thanks for your continued efforts.
Definitely extremely welcome and surely worth thanking whole heartedly for.
Haven't begun re-encoding from 4.3x to 4.4x for better DAP support yet, so the added speed is greatly appreciated.
WavPack 4.80.0 -b384hx6cmv / qaac 2.59 -V 100

  • DARcode
  • [*][*][*][*][*]
  • Members (Donating)
WavPack 4.41 beta available
Reply #52
[...]I did get an idea from one of the FLAC 1.1.4 improvements that might be applicable to WavPack and I'll give that a try. You never know when something's going to make a big difference...[...]
Are you going to try and include the above into the 4.41 changes or can we expect a separate release please?
Thanks a bunch once more for your time and brilliant work.
WavPack 4.80.0 -b384hx6cmv / qaac 2.59 -V 100

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #53
[...]I did get an idea from one of the FLAC 1.1.4 improvements that might be applicable to WavPack and I'll give that a try. You never know when something's going to make a big difference...[...]
Are you going to try and include the above into the 4.41 changes or can we expect a separate release please?
Thanks a bunch once more for your time and brilliant work.

I tried it the other day but wasn't able to make a big enough improvement for how ugly the change is. I'll work at a little more, but I'm not sure it's going to work out. 

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #54
I have created a second beta that I believe makes another worthwhile improvement in performance. This involves accessing the bitstream in 16-bit chunks instead of 8-bit chunks, so it should affect the faster modes more (at least on a percentage basis).

Like the first beta, it should be bit identical to 4.40.0 in all circumstances.

Thanks in advance for any testing... 

Here it is.

  • Madman2003
  • [*][*][*]
WavPack 4.41 beta available
Reply #55
Some sourcecode would be nice for us non-windows users.

  • Mangix
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #56
the svn server is at http://svn.slomosnail.de/wavpack

grab subversion(if you don't already have it) and get it.

WavPack 4.41 beta available
Reply #57
i'm just wondering if there is any harm having the --optimize-mono to be on by default?
  • Last Edit: 11 March, 2007, 09:57:01 PM by A_Man_Eating_Duck
Who are you and how did you get in here ?
I'm a locksmith, I'm a locksmith.

  • Mangix
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #58
IIRC, --optimize-mono only breaks compatability with any version of WavPack below 4.31

  • shadowking
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #59
--optimize-mono is also buggy in lossy mode - bitrate drops , though only on dual-mono content.
wavpack -b4x4s1c

WavPack 4.41 beta available
Reply #60
i did a quick comparison using the --optimize-mono switch and without it in lossless mode, and the file sizes are the same, this is on normal stereo audio.
Who are you and how did you get in here ?
I'm a locksmith, I'm a locksmith.

  • shadowking
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #61
It only kicks in on dual-mono content or similar, so its safe to use in lossless if your player doesn't need < 4.31 decoder. From what I understand is that there is a new decoder that fixes this without the hack and will be used at some point in the future.. or maybe the  --optimize hack will hardcoded instead ?
  • Last Edit: 12 March, 2007, 04:42:50 AM by shadowking
wavpack -b4x4s1c

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #62
--optimize-mono is also buggy in lossy mode - bitrate drops , though only on dual-mono content.

Have you actually seen this lately? I thought I fixed it before I released 4.40 but I never mentioned it in the changelog because it (the bug) never made it into a real release.

Now that the DirectShow filter is based on the latest library, the only decoder that can't handle the new mode is the XMMS plugin that I provide. Once I figure out how to build that with the latest library I could make the --optimize-mono the default, with perhaps a new switch to turn it off (for maximum compatibility).

Unless you're using the XMMS plugin (or are encoding files for a large audience), it's perfectly safe to use all the time.

  • Madman2003
  • [*][*][*]
WavPack 4.41 beta available
Reply #63
Encoding speeds are amazing compared to flac, wavpack -hh seems much faster than flac -8 (order of 4 difference), decoding is about half the speed (only 40% difference with mode -h compared to the 100% difference of -hh). Good work, wavpack has the potential to beat flac. Keep up the good work.
  • Last Edit: 12 March, 2007, 07:03:00 PM by Madman2003

  • shadowking
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #64

--optimize-mono is also buggy in lossy mode - bitrate drops , though only on dual-mono content.

Have you actually seen this lately? I thought I fixed it before I released 4.40 but I never mentioned it in the changelog because it (the bug) never made it into a real release.

Now that the DirectShow filter is based on the latest library, the only decoder that can't handle the new mode is the XMMS plugin that I provide. Once I figure out how to build that with the latest library I could make the --optimize-mono the default, with perhaps a new switch to turn it off (for maximum compatibility).

Unless you're using the XMMS plugin (or are encoding files for a large audience), it's perfectly safe to use all the time.


http://www.hydrogenaudio.org/forums/index....ost&id=2888

When encoding that sample with optimize-mono bitrate drops 60k or so. SNR is similar with or without the switch, but really should be much higher. If I add the missing 60k - i.e. change b320 to b380 I'll get what should be , I think..



WAVPACK  Hybrid Lossless Audio Compressor  Win32 Version 4.41.0-beta2
Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

ave noise = -59.18 dB, peak noise = -51.82 dB
created 001__08__Help_Me_Rhonda___Beach_Boys_cut___dual_mono.wv in 1.40 secs (lo
ssy, 301 kbps)


WAVPACK  Hybrid Lossless Audio Compressor  Win32 Version 4.41.0-beta2
Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

ave noise = -60.85 dB, peak noise = -53.20 dB
created 001__08__Help_Me_Rhonda___Beach_Boys_cut___dual_mono.wv in 1.40 secs (lo
ssy, 243 kbps)
wavpack -b4x4s1c

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #65
When encoding that sample with optimize-mono bitrate drops 60k or so. SNR is similar with or without the switch, but really should be much higher. If I add the missing 60k - i.e. change b320 to b380 I'll get what should be , I think..

That's actually the intended behavior. In the original release that supported --mono-optimize, the bitrate would drop to about half the specified bitrate and the noise would go up significantly compared to the stereo encoding, which was obviously wrong. In the newer versions the target was to keep the noise constant and let the bitrate drop (but not as much as before).

My thinking was that a worst-case scenario for this would be a track that was basically pure mono, but with an occasional sample different (in one channel) by one LSB. In this case, most blocks would be encoded in the new special mono mode, but the blocks with the single sample difference would have to be encoded in the regular stereo mode. In this case I think the ideal situation would be to have the noise remain about the same, which would mean that the bitrate would jump up in the stereo blocks because the "mono" mode is more efficient (the whole reason it exists). I think the bitrate jumping around would not be an issue, but the noise jumping around might be audible (and if it wasn't audible then why waste bits to make it even lower in the "mono" blocks?)

This way I can say that the --optimize-mono option can reduce the bitrate (in either lossless or lossy mode) with tracks that contain mono audio, but should never have any audible effect in lossy mode.

David

WavPack 4.41 beta available
Reply #66
I have created a second beta... Thanks in advance for any testing... 


Here is some evaluation of latest WavPack binaries conducted on my PIV Prescott 2.8ghz.
Ratio is calculated on my SetF.

Code: [Select]
|      | Ratio |    4.40.00   |   4.41.0b1   |   4.41.0b2   |

|  f   | 65,66 |  96,7  115,6 | 100,4  124,4 | 102,4  137,6 |
| fx1  | 65,24 |  59,9  116,1 |  64,2  123,2 |  65,0  137,6 |
| fx2  | 65,17 |  45,1  117,1 |  49,1  123,8 |  49,4  137,6 |
| fx3  | 65,13 |  27,8  117,1 |  31,1  125,6 |  31,2  138,3 |
| fx4  | 65,08 |  12,0  114,5 |  13,5  124,4 |  13,5  139,8 |
| fx5  | 65,07 |   9,7  115,0 |  10,9  123,8 |  10,9  139,0 |
| fx6  | 65,07 |   8,3  115,6 |   9,3  123,2 |   9,3  139,8 |

|      | 64,00 |  82,0   98,1 |  85,8  105,7 |  87,0  115,0 |
|  x1  | 63,56 |  44,4   95,9 |  49,5  102,0 |  50,1  110,6 |
|  x2  | 63,51 |  29,4   95,6 |  34,0  101,6 |  33,9  111,6 |
|  x3  | 63,52 |  15,9   95,6 |  19,0  102,0 |  18,7  112,6 |
|  x4  | 63,27 |   4,3   95,9 |   5,1  100,8 |   5,1  110,2 |
|  x5  | 63,25 |   3,0   95,6 |   3,5  100,0 |   3,5  109,7 |
|  x6  | 63,20 |   1,5   91,5 |   1,8  100,4 |   1,8  105,3 |

|  h   | 62,88 |  57,1   75,4 |  64,8   74,7 |  64,8   78,8 |
| hx1  | 62,78 |  30,0   76,5 |  34,7   78,3 |  34,4   83,1 |
| hx2  | 62,70 |  18,3   76,2 |  21,9   77,8 |  21,7   82,5 |
| hx3  | 62,66 |   9,2   76,2 |  11,4   77,8 |  11,3   82,3 |
| hx4  | 62,43 |   2,6   75,4 |   3,1   77,8 |   3,1   83,1 |
| hx5  | 62,39 |   1,9   76,2 |   2,3   78,3 |   2,3   82,8 |
| hx6  | 62,38 |   1,3   76,5 |   1,6   78,3 |   1,6   83,9 |

|  hh  | 62,47 |  49,7   60,9 |  52,5   62,7 |  53,0   65,8 |
| hhx1 | 62,28 |  22,2   61,2 |  26,6   61,8 |  26,7   64,8 |
| hhx2 | 62,22 |  12,9   61,8 |  15,8   61,9 |  15,8   65,0 |
| hhx3 | 62,17 |   6,3   60,9 |   7,9   61,8 |   7,9   65,0 |
| hhx4 | 62,01 |   1,7   62,2 |   2,0   61,5 |   2,0   65,3 |
| hhx5 | 61,96 |   1,0   62,1 |   1,2   61,8 |   1,2   65,5 |
| hhx6 | 61,96 |   0,7   62,4 |   0,9   61,8 |   0,9   66,0 |


A nice 16% encoding speed improvement and 12% decoding speed improvement, when going from 4.40 to 4.41b2.
Most encoding speed improvement is achieved with 4.41b1, most decoding speed improvement with 4.41b2.
Encoding speed improvement is slightly more focused on slower modes, while decoding speed improvement is the other way round.

  • shadowking
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #67

When encoding that sample with optimize-mono bitrate drops 60k or so. SNR is similar with or without the switch, but really should be much higher. If I add the missing 60k - i.e. change b320 to b380 I'll get what should be , I think..

That's actually the intended behavior. In the original release that supported --mono-optimize, the bitrate would drop to about half the specified bitrate and the noise would go up significantly compared to the stereo encoding, which was obviously wrong. In the newer versions the target was to keep the noise constant and let the bitrate drop (but not as much as before).


This way I can say that the --optimize-mono option can reduce the bitrate (in either lossless or lossy mode) with tracks that contain mono audio, but should never have any audible effect in lossy mode.

David


Thanks for the explanation.
wavpack -b4x4s1c

  • Martin H
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #68
1 image file ~ 11 tracks ~ 300MB | Intel Celeron 1.7GHz. :

Code: [Select]
D:\Temp>wavpack -h *.wav

 WAVPACK  Hybrid Lossless Audio Compressor  Win32 Version 4.40.0
 Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

created Evanescence - Fallen.wv in 129.54 secs (lossless, 32.76%)

D:\Temp>wavpack441b2 -h *.wav

 WAVPACK  Hybrid Lossless Audio Compressor  Win32 Version 4.41.0-beta2
 Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

created Evanescence - Fallen.wv in 118.27 secs (lossless, 32.76%)

D:\Temp>wvunpack *.wv

 WVUNPACK  Hybrid Lossless Audio Decompressor  Win32 Version 4.40.0
 Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

restored Evanescence - Fallen.wav in 91.87 secs (lossless, 32.76%)

D:\Temp>wvunpack441b2 *.wv

 WVUNPACK  Hybrid Lossless Audio Decompressor  Win32 Version 4.41.0-beta2
 Copyright © 1998 - 2006 Conifer Software.  All Rights Reserved.

restored Evanescence - Fallen.wav in 80.67 secs (lossless, 32.76%)
Result :

WavPack v4.41b2 encoded the image file 11.27 seconds faster than v4.40.

WvUnpack v4.41b2 decoded the image file 11.20 seconds faster than v4.40.


Nice work, David - and many thanks for your continued efforts, my friend

  • DARcode
  • [*][*][*][*][*]
  • Members (Donating)
WavPack 4.41 beta available
Reply #69

Nice improvement indeed!
Many thanks, David!
WavPack 4.80.0 -b384hx6cmv / qaac 2.59 -V 100

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #70
Thanks Josef, Martin, and DARcode for the tests! 

These numbers are very much in line with what I am seeing here and I think a new release is certainly justified at this point.

  • Mangix
  • [*][*][*][*][*]
WavPack 4.41 beta available
Reply #71
Right. Because WavPack encoding is integer-only (and generally not parallelizable) there's little or nothing to be gained by SSE and his brothers. MMX should be able to make a nice improvement because the left and right channels can be done together, but because of some optimization available for 16-bit data that doesn't work with MMX, the MMX stuff only really helps with 24-bit (and float) data.

what about for PCM Float? SSE should be able to make a nice improvement there, shouldn't it?

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #72

Right. Because WavPack encoding is integer-only (and generally not parallelizable) there's little or nothing to be gained by SSE and his brothers. MMX should be able to make a nice improvement because the left and right channels can be done together, but because of some optimization available for 16-bit data that doesn't work with MMX, the MMX stuff only really helps with 24-bit (and float) data.

what about for PCM Float? SSE should be able to make a nice improvement there, shouldn't it?

Well, no. WavPack actually uses no floating-point operations even when operating on IEEE float data. I did this to ensure lossless operation even with slight variations in rounding behavior and to allow integer-only platforms (like Rockbox) to play IEEE float files even without an FPU.

  • he-jo
  • [*]
WavPack 4.41 beta available
Reply #73
Hi David,

there has been an idea in my mind for a while, and now I'd like to ask you, if you think this would be possible

As I read it from your docs, each block in the encoded data stream is independent from the previous one. So, if two blocks have the same parameters (which should be almost always the case in your current implementation - except for –optimize-mono), we could decorrelate them at once with SSE2 or AltiVec, couldn't we?


Regards,
Jo.

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
WavPack 4.41 beta available
Reply #74
there has been an idea in my mind for a while, and now I'd like to ask you, if you think this would be possible

As I read it from your docs, each block in the encoded data stream is independent from the previous one. So, if two blocks have the same parameters (which should be almost always the case in your current implementation - except for –optimize-mono), we could decorrelate them at once with SSE2 or AltiVec, couldn't we?

I'm not completely up to speed (so to speak) on SSE2 or AltiVec, but what you suggest sounds reasonable. Unless the user specifies a -x mode, all blocks will have exactly the same sequence of terms and deltas.

I have also thought that the existing MMX code could also be used for 24-bit mono data by doing multiple blocks in parallel.

Of course, these changes would involve a lot more internal shuffling than the current stuff did. 

BTW, please feel free to e-mail me directly on your ideas. I really do appreciate your input (and work so far). 

David

edit: spelling
  • Last Edit: 22 March, 2007, 08:16:01 AM by bryant