Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: WavPack 5.6.4 Has Multithreading Option (Read 4797 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: WavPack 5.6.4 Has Multithreading Option

Reply #25 –
Quote from: bryant on
Also, there's a preprint available of the "brainwave" compression paper.
The paper mentions the use of two different ADCs and the NP1 is "high-pass". Do you know the specs? For example, does the ADC use bandpass sampling or just typical sampling from DC to Nyquist but with a highpass filter applied?

[edit]OK, the high-pass is not very high so unlikely to use bandpass sampling.
Quote
NP1 devices include an analog 300-Hz high-pass filter [...] which is sampled at 30 kHz.

Also, the paper emphasizes "accurate reproduction of fast transients (i.e., spikes)" and due to the non-audio nature I guess the differences between x4 and x3 should be very big. Do you have some of their data to try out, or did you give some advice to them about how to encode the data efficiently in terms of size vs speed?

Re: WavPack 5.6.4 Has Multithreading Option

Reply #26 –
Quote from: bennetng on
Also, the paper emphasizes "accurate reproduction of fast transients (i.e., spikes)" and due to the non-audio nature I guess the differences between x4 and x3 should be very big. Do you have some of their data to try out, or did you give some advice to them about how to encode the data efficiently in terms of size vs speed?
They did give me data to experiment with, although not until just recently. What I was surprised about was how noisy the signal is and how sparse the spikes are. In fact, some of the 384 channels seemed to be, just looking in an editor and listening, completely free of spikes and consisted of just white noise, with a few constant low-level sines added in probably from power supplies (e.g., 60 Hz, 10,070 Hz, 14,570 Hz).

Needless to say, the higher modes and the extra modes made very little difference, like 1.5% from -f to -hhx6, and all of that improvement would have been from picking up those sines. So unless some other data samples look significantly different, the extra modes are not going to achieve much, especially considering their cost. In fact, I think they ended up deciding that even the "high" mode was unjustified.

Re: WavPack 5.6.4 Has Multithreading Option

Reply #27 –
Quote from: bryant on
Needless to say, the higher modes and the extra modes made very little difference, like 1.5% from -f to -hhx6, and all of that improvement would have been from picking up those sines. So unless some other data samples look significantly different, the extra modes are not going to achieve much, especially considering their cost. In fact, I think they ended up deciding that even the "high" mode was unjustified.

Hm, ... at the risk of making a fool out of myself by miscounting figures after the decimal point:
I look at lines 550 to 555 of the PDF version: blosc-zstd would be cheaper if decompressed 528 times blah blah. How they got to that:
If I understand the numbers right, 5 cents per CPU hour and a speed of 12.82 x RT, encoding this hour of data costs only 0.4 cent (and so does every decoding)? A one-time -x makes for ... how much, fifty percent more time? 0.2 cents? That is like 1/30th of a percent of a year's 615 cents storage cost, so if -x reduces the file by 0.1 percent it is regained after four months in the cloud?

The benefits of -h would depend on the number of decompressions of course.

Mildly surprising at first glance that they measure WavPack to decoding faster than FLAC - but then on the other hand, how did they even make FLAC decode 384 channels? Too lazy to dig into the details, and so should they be, when a format that supports it is put up against one that doesn't ...

Re: WavPack 5.6.4 Has Multithreading Option

Reply #28 –
The cloud cost calculations are a mystery to me, but I assume the authors know what they’re talking about (I certainly don’t). Specifically, it seems to me that CPU cost would depend on the number of threads consumed (i.e., the “total” CPU power cost), not just the user time that a process took, but again I really have no clue how that’s billed (and it probably varies from service to service).

As for the comparison of WavPack and FLAC, there is some other code in there that flattens the 384 channels into stereo pairs for FLAC, which of course is essentially what WavPack does. And, like I recently verified, “high” mode in WavPack makes essentially no difference (0.16% on the file they provided).

WavPack decodes faster than FLAC because of the multithreading I recently added (which works really well for multichannel). It originally was slower by the amount one would expect, but I noticed that some of the general-purpose compressors that were getting amazing decoding speeds were natively multithreaded, and I thought that a little unfair.
 

Re: WavPack 5.6.4 Has Multithreading Option

Reply #29 –
Quote from: bryant on
And, like I recently verified, “high” mode in WavPack makes essentially no difference (0.16% on the file they provided).
Essentially no difference is ... relative. 0.16 percent - not percentage points - saves 1 cent annual storage cost for that amount of data. Quite a lot compared to differences in encoding CPU costs (in their figures). If a normal-mode encode/decode costs 0.4 and -h is a factor of 1.3 (I think that is sane for CDDA, but ...) then the 1 cent breaks even at seven decodes a year. Not saying it is worth it, but my point is that if storage costs so much more than processing, the "essentially no difference" will be weighted up accordingly.

Anyway, if I were referee on that part of the paper, I would be very fine with not trying to tweak one of the compressors. Fine-tuning for whatever optimum would be subject to pricing plan (or even does encoding locally), blah blah you know all this of course.
(Still to a reader who don't know audio codecs at all, there is a point that there are settings available that incur only one-time costs upon encoding and do not cost decoding effort.)