Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: WavPack 5.5.0 Release Candidate (Read 21241 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: WavPack 5.5.0 Release Candidate

Reply #25 –
For wiki etc, and since trying to create a 257 channel .wav is a hassle and I'd anyway have to ask to get the "officially supported":

(1) I see the WavPack front page has changed the description of channel count to "up to at least 256 channels".
 * Does that mean you have removed the 256 channel cap to accommodate that BrainWavPack application
and if so:
 * Would a "reasonably accurate" description be that WavPack supports 256 channels but will accept higher channel count provided that other limitations are satisfied (not all tested) - up to a hard  definite maximum of 4096?

(2) "integer sampling rates up to 1 GHz" it says. I have gotten it to work to 4 Gi - 1.
 * Is it kinda the same as I suggested above, officially 1 GHz but won't object until 4 GiHz?
Just because "supports any .wav sampling rate (i.e. integers between 0 4 GiHz)" is easier to explain than "you see there is this thing called 'long integer' which is why you can cover signals you won't even call audio ...".

Re: WavPack 5.5.0 Release Candidate

Reply #26 –
Quote from: Porcus on
For wiki etc, and since trying to create a 257 channel .wav is a hassle and I'd anyway have to ask to get the "officially supported":

(1) I see the WavPack front page has changed the description of channel count to "up to at least 256 channels".
 * Does that mean you have removed the 256 channel cap to accommodate that BrainWavPack application
and if so:
 * Would a "reasonably accurate" description be that WavPack supports 256 channels but will accept higher channel count provided that other limitations are satisfied (not all tested) - up to a hard  definite maximum of 4096?
The official support remains 256 channels, which seems high enough to me for all but the most esoteric brain-wave type stuff.

The format and library have supported 4096 channels for a while, but has not been well-tested, and there's not been a way to even make them. With this release I have added an undocumented option --raw-pcm-ex that lifts the limitation for experimentation purposes (and the brain-wave guys).

But just one example of how this can manifest in weirdness, if you unpack one of these files to another uncompressed format (like WAV) then that resulting WAV file will not not be compatible with WavPack. So maybe in the future, after some more testing, I will lift the 256 channel limit everywhere. Or maybe have some sort of --chill-out option for that (the alternative always reminds me of the airport).

Quote
(2) "integer sampling rates up to 1 GHz" it says. I have gotten it to work to 4 Gi - 1.
 * Is it kinda the same as I suggested above, officially 1 GHz but won't object until 4 GiHz?
Just because "supports any .wav sampling rate (i.e. integers between 0 4 GiHz)" is easier to explain than "you see there is this thing called 'long integer' which is why you can cover signals you won't even call audio ...".
Here again the format and library limit is 2 Gi - 1, so I suspect that's what you got to work. And of course since the WAV header has another unsigned 32-bit field "bytes per second", its real limit varies with channel count and sample depth. Oof.

I've had at least one CVE around wacky sample rates and so have gotten a little shy in this area, which may explain why I set the --raw-pcm limit to 1 GHz. It certainly could have been 2 GHz, and you can get that with WAV (disregarding the other issue above), but 1 GHz is easy to remember and seems like a reasonable upper limit for most sampled physical phenomena.

The cool formats (AIFF and CAF) support floating-point values and I really should have done that, and maybe will in the future. Handy when specifying 0.01 Hz for barometric data, for example, but then I need to add months and years to the "duration" display. You can see how this can be complicating in unpredictable ways.

Sorry I couldn't just answer "yes" or "no" for these and thanks again for your valuable updates to the Wiki!   :)

Re: WavPack 5.5.0 Release Candidate

Reply #27 –
Edit: So with the last few wiki updates, I think I have removed some errors and not introduced new ones? https://wiki.hydrogenaud.io/index.php?title=WavPack&diff=35791&oldid=35777

Quote from: bryant on
Here again the format and library limit is 2 Gi - 1, so I suspect that's what you got to work.
Looks like my memory had gotten too much pro-WavPack bias after everything else working just as good with WavPack as with anything else. So here are the corner cases where some animals are outpacking WavPack, I think:
* 2Gi fails for WavPack. (Also ffmpeg cannot handle it.)
* 4Gi - 1 is accepted by Monkey's and OptimFROG and also by MPEG-4 ALS which with -v returns this fun output:
Code: [Select]
Audio format : int / 16 bit / -1 Hz / 1 ch
Bit rate     : -0.0 kbit/s
Playing time : -100000000.0 sec
PCM file size: 200000104 bytes
ALS file size: 11466700 bytes
Compr. ratio : 17.442 (5.73 %)
Average bps  : 0.917
Average rate : -0.0 kbit/s

Processing took 7.59 sec (-13168290.8 x real-time)

Quote from: bryant on
The cool formats (AIFF and CAF) support floating-point values
Quick and pigsty-dirty idea for those who use have allocated 4 bytes and declared it a long signed integer variable for something inherently positive: use the sign to indicate that it is float  :))

Re: WavPack 5.5.0 Release Candidate

Reply #28 –
Quote from: bryant on

Perhaps Peter will chime in with more context.


I'll have a look at this again to see if I can get it working with more supported/standard APIs

Re: WavPack 5.5.0 Release Candidate

Reply #29 –
Thanks. If it is not too complicated, it would be interesting to have a browser-based decoding speed benchmark with user-supplied WavPack files too.

Re: WavPack 5.5.0 Release Candidate

Reply #30 –
Quote from: bryant on
And of course since the WAV header has another unsigned 32-bit field "bytes per second", its real limit varies with channel count and sample depth.
My pathetic SATA SSD is not fast enough to handle this data rate, but then the file can only be 1 second long and therefore not so useful either. So WAV is basically useless for this purpose. Luckily some of the "best" audio formats like DSD2048 is still below 100MHz :))

Re: WavPack 5.5.0 Release Candidate

Reply #31 –
Quote from: Porcus on
Quote from: bryant on
The cool formats (AIFF and CAF) support floating-point values
Quick and pigsty-dirty idea for those who use have allocated 4 bytes and declared it a long signed integer variable for something inherently positive: use the sign to indicate that it is float  :))

Haha, yeah, that would work! At first I thought you would only get 31 bits for the float value, but it would actually be the negation of the sample rate. Clever! Unfortunately I can't add it to WavPack that way because it wouldn't be backward compatible. I would need to have the integer version remain (and be as close as possible to the actual value) and then have another field for the floating point that old decoders would never see.

And what's crazy is that in AIFF it's actually an 80-bit extended precision float! I wonder what language back then (1988) supported that (it's poorly supported now). They tried to cover all the possibilities but never considered anyone would need over 4 GB. Anyway, for CAF they went back to a more reasonable 64-bit double.

Re: WavPack 5.5.0 Release Candidate

Reply #32 –
Quote from: bennetng on
Quote from: bryant on
And of course since the WAV header has another unsigned 32-bit field "bytes per second", its real limit varies with channel count and sample depth.
My pathetic SATA SSD is not fast enough to handle this data rate, but then the file can only be 1 second long and therefore not so useful either. So WAV is basically useless for this purpose. Luckily some of the "best" audio formats like DSD2048 is still below 100MHz :))

4 GB per second is crazy fast. I'm not sure PC RAM can go that fast.

My Tektronix oscilloscope is up to 8 GB per second (2 GS/s * 4-ch * 8-bit) but only goes for 2.5K samples (i.e., just over 1 uS).

Re: WavPack 5.5.0 Release Candidate

Reply #33 –
RAM can go that fast - most storage, however, would find it rather challenging.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM)

Re: WavPack 5.5.0 Release Candidate

Reply #34 –
Quote from: bryant on
And what's crazy is that in AIFF it's actually an 80-bit extended precision float! I wonder what language back then (1988) supported that (it's poorly supported now).
Probably Pascal. It was a standard data type on the 68k FPU and in Apple's mathematics libraries.

Re: WavPack 5.5.0 Release Candidate

Reply #35 –
Concerning why put AIFF off until WavPack can import AIFF metadata:
WavPack does not universally have that capability with WAVE either. (E.g. here, Mp3tag sees a COMMENT tag in non-ID3 RIFF.)

So ... porcine ignorant question time again, but as far as WavPack cannot universally import WAVE RIFF metadata (that it stores perfectly) and WavPack cannot universally import metadata from DSD files (that it stores perfectly) - is there anything in particular about AIFF except getting the --import-id3 to work [which, anyway, it doesn't do for all ID3]?

... well there might be, of course, not related to metadata, but ...
Quote from: bryant on
And what's crazy is that in AIFF it's actually an 80-bit extended precision float!
... the choir demands 64-bit float "just in case" they would ever encounter such a file  ;D

Re: WavPack 5.5.0 Release Candidate

Reply #36 –
Quote from: Porcus on
Concerning why put AIFF off until WavPack can import AIFF metadata:
WavPack does not universally have that capability with WAVE either. (E.g. here, Mp3tag sees a COMMENT tag in non-ID3 RIFF.)

So ... porcine ignorant question time again, but as far as WavPack cannot universally import WAVE RIFF metadata (that it stores perfectly) and WavPack cannot universally import metadata from DSD files (that it stores perfectly) - is there anything in particular about AIFF except getting the --import-id3 to work [which, anyway, it doesn't do for all ID3]?
No, the two projects (AIFF and ID3v2.4) are completely orthogonal. For silly historical reasons I was doing them together, but I could certainly do AIFF first and it would just work with ID3v2.3 (like everything else). That may even be the most common ID3 variant...

Quote
Quote from: bryant on
And what's crazy is that in AIFF it's actually an 80-bit extended precision float!
... the choir demands 64-bit float "just in case" they would ever encounter such a file  ;D
Don't know if I've ever mentioned this story, but I got a 64-bit float WAV file from a guy who was asking about support for it. The very first thing I tried was to verify that the 64-bit float values were in fact not representable in 32-bit floats, because otherwise this would just be 100% bloat. Well, every sample survived the roundtrip to 32-bit losslessly! I got him to convert it to 32-bit float (with FFmpeg or Foobar2000, I don't remember) and he agreed it sounded the same. I've been doubly dubious of this ever since.

At one point I was thinking that it would be fairly east to implement this in a backward-compatible way by dividing each 64-bit float value into the sum of two 32-bit floats. One could be stored in the regular stream and the "correction" stream would be stored such that old decoders would ignore it (or omitted in lossy mode, obviously). Unfortunately when I looked a little deeper I realized that since the mantissa is more than twice as large on 64-bit floats (52 vs. 23 bits) this won't work.  :(

Re: WavPack 5.5.0 Release Candidate

Reply #37 –
Quote from: bennetng on
Thanks. If it is not too complicated, it would be interesting to have a browser-based decoding speed benchmark with user-supplied WavPack files too.

I've updated the code so it should now work with Firefox too.
https://www.wavpack.com/WebAssembly/index.htm

If you're interested in the source code etc, that can be found on my GitHub page:

https://github.com/soiaf/WebAssembly-WavPack

I also wrote a very quick (read 'hack') system to show how quick it can decode a file. David was kind enough to also host this on his site at
https://www.wavpack.com/WebAssembly/timer.htm

It's certainly interesting to see how fast it can decode WavPack files, from my very brief tests, Firefox seemed to be the fastest browser, but I don't know if this is better JavaScript or WebAssembly processing (or both!). However maybe others will get different results!

Re: WavPack 5.5.0 Release Candidate

Reply #38 –
Thank you very much! Here is what I got from a CDDA image I encoded many years ago using WavPack 4 with high setting, I am using i3-12100, 16GB DDR4 on Windows 10 and Firefox:
Code: [Select]
It took 27094 milliseconds to decode the file
We decoded 187598460 samples
Another CDDA image encoded with WavPack 5 fast settings:
Code: [Select]
It took 10299 milliseconds to decode the file
We decoded 133040880 samples
I repeated the test several times and got similar results.

Re: WavPack 5.5.0 Release Candidate

Reply #39 –
Thanks @soiaf ... very nice!

And thanks @bennetng for posting your timing results.

It's amazing to me that WavPack can decode in a browser at over 100x realtime! Just for fun I opened up 5 different tabs and had a WavPack file playing in each one and was still under 20% CPU load.
 

Re: WavPack 5.5.0 Release Candidate

Reply #40 –
Just for the braintrust ...
There is actually some scientific literature on lossless compression of EEG data - that Google Scholar search returened 1620 hits although not all are relevant. In particular, none of the hits on https://scholar.google.com/scholar?q="lossless+compression"+"eeg"+"wavpack" are ...
(Shorten was used here, as they found a 1999 paper that tested it: https://dr.ntu.edu.sg/bitstream/10356/101141/1/A%20two-dimensional%20approach%20for%20lossless%20EEG%20compression.pdf )