-h as the new default mode

Topic: -h as the new default mode (Read 11087 times) previous topic - next topic

0 Members and 2 Guests are viewing this topic.

Re: -h as the new default mode

Reply #25 – 2023-08-13 11:34:52

-bn enable hybrid compression
n = 2.0 to 23.9 bits/sample, or
n = 24-9600 kbits/second (kbps) (Implies -c)

-c hybrid lossless mode (used with -b to create
correction file (.wvc) in hybrid mode)

--no-c Force hybrid lossy mode (use with -b to disable creation of
correction file (.wvc) in hybrid mode)

-------

Alternative:

-c<n> = shortcut for '-b<n> -c' to specify hybrid lossless mode
with a single option, n = 2.0 to 23.9 bits/sample, or
n = 24-9600 kbits/second (kbps)

Re: -h as the new default mode

Reply #26 – 2024-02-18 04:16:06

Indeed, WavPack is so flexible that it becomes hard to use.

Whenever this codec pops up on the radar, I have to re-read the manual and look through forum threads to wrap my head around the optimal settings. And, to be honest, every time I feel like I've missed something. Because there are too many xxx and hhh without an intuitive relationship between the encoding/decoding speed and the size. Also because there are poorly documented options in terms of their benefit such as -b (from 24 to, err, 9600?), -g (normal mode?), --cross-decorr (good/bad?), --merge-blocks (do/dont?), -m (why not by default, why md5 instead of xxh3-64 or blake3), -n (is that for my eyes or used internally as well?), -s (what is the default and what's the impact?), --use-dns (huh?), etc.

Aggrrhh!

For comparison: flac works optimally without settings, and as for lossy codecs I need to remember one flag only: qaac -V109 or even no setting at all because default -V91 is good enough, oggenc -q8, lame -V0, mpcenc --insane, etc.

For example, now I'd like to compress a few albums in hybrid mode, keeping transparent-sounding lossies on HDD and uploading correction files to the cloud. Believe it or not, I saw a suitable command not in the manual, but in the signature of @shadowking, namely wavpack -b320hhcs.5. However, I'm still lost, because he changed that command multiple times in favor of increasing the bitrate and adding xxx, let alone -s remains a mystery.

Re: -h as the new default mode

Reply #27 – 2024-02-18 10:02:24

Quote from: Kraeved on 2024-02-18 04:16:06

Indeed, WavPack is so flexible that it becomes hard to use.

I'm guilty of many misconceptions about the settings and impacts myself.
But as long as you are not interested in the hybrid, then you can rename the executable to get settings, and drag and drop. Like this: https://hydrogenaud.io/index.php/topic,122626.msg1012159.html#msg1012159

Quote from: Kraeved on 2024-02-18 04:16:06

Whenever this codec pops up on the radar, I have to re-read the manual and look through forum threads to wrap my head around the optimal settings. And, to be honest, every time I feel like I've missed something. Because there are too many xxx and hhh without an intuitive relationship between the encoding/decoding speed and the size.

Quick to remember:
Impact on both encoding and decoding speed: alphabetical, -f/-g/-h/-hh
x-tra processing during encoding does not affect decoding speed (well except if it finds something smart). That's the -x

I'm guilty of proposing the "-g", which made it easier to run FOR loops for testing ...

As goes -m:
* Why MD5? If you want a hash of the uncompressed PCM, there is no use in implementing anything but MD5, since that is so well established. FLAC has it by default, so it can be used to identify files - even across codecs (as long as sources are little-endian signed ...). Yes with today's knowledge we can tell it was a bad choice, but we are stuck with it. WavPack calls it a fingerprint, rather than a verification tool.
* Why off by default? The historical explanation is likely that WavPack is older ... The reason to keep it off as default, is likely how WavPack is used in audio editor plug-ins, and so -hxm would slow down every save. Besides, version 5 of the file format introduces a (much quicker) checksums for encoded blocks, so you don't need it to verify.

I think we would have to look back at the history here. FWhen FLAC was released, one was very cautious about ensuring that the losslessness was verifiable - also to the extent that you could ensure that some random decoder would output the right thing. Hence MD5 on by default, even if it would slow down things. FLAC was the only open alternative (well depending on how you viewed the Shorten license), and so the only codec that would reasonably expect to get 3rd party decoders.
For trust in the losslessness, you got a hash of the unencoded PCM stored into the format, to be compared to a hash of the decoded PCM. Of course you would then take a checksum algorithm that was implemented everywhere. True, cryptographical vulnerabilities were already known, but still. Also, the early FLAC encoding was slow, so the slowdown from computing MD5 didn't raise eyebrows. A decoder in a low-powered hardware player could always refrain from computing it.

Indeed, at the time, you would see filename.shn files come with filename.md5. Like described here: https://userpages.umbc.edu/~hamilton/shnfaq.html .

Now within a few years after FLAC was released, the status of MD5 was:
* FLAC had it on by default
* WavPack, OptimFROG and TAK offered it optionally - and no other checksum
* Monkey's being Monkey's would instead hash the encode, and produce a different checksum - but would use the MD5 algorithm too
* And then you got some un-checksummed formats that should die out if they haven't already. WMAL, ALAC - and TTA, which claims to have a checksum but which decodes to static without checking.

Bottom line is, yes you can argue against MD5 - not that it matters much with today's computing power - but you need to understand the history here. There was a need to assure the general public that yes this works. Still it took long to convince audiophools about losslessness - like, when speaker manufacturer B&W had a music store, they would offer AIFF because ... well just because.

Quote from: Kraeved on 2024-02-18 04:16:06

For comparison: flac works optimally without settings

The default is well tuned indeed. But the reference flac encoder gives you direct access to a whole lot of settings that could confuse you if you care. Like, flac --lax -8pb8192 -r7 -l14 -A "subdivide_tukey(4);flattop;gauss(3e-2)" , there you go.

Re: -h as the new default mode

Reply #28 – 2024-02-18 10:47:06

For lossless things are more obvious , For lossy HQ IMO what would help are preset mappings for the
most common rates as --hybrid (default =1 etc)

--hybrid 1 = -b256hx3c
--hybrid 2 = -b320hx4c

Notice my suggestions are similar and also different to the manual. I have gone for a more defensive route while taking into account playback
hardware of the last 10 years.

Re: -h as the new default mode

Reply #29 – 2024-02-19 12:57:06

@Porcus, thank you for the historical background, so well-written that it should be moved to the official wiki.

@shadowking, I don't think that -h is attractive enough to become a default. FLAC-5 wins in both encoding and decoding speed by 3-4 times, taking up only ~100-200 kilobytes more than WavPack -h, at least on my end.

Code: [Select]

$ (for %# in (*.*) do @echo %~z# = %#) | sort
14408062 = vangelis-b4xh.wv
14410736 = vangelis-b4x4h.wv
14462110 = vangelis-b4x4.wv
14489676 = vangelis-b4x4f.wv
14494230 = vangelis-b4x.wv
14548740 = vangelis-b4xf.wv
15940004 = vangelis-b384x3h.wv
15945376 = vangelis-b384xf.wv
15967604 = vangelis-b384x4f.wv
15968562 = vangelis-b384x3f.wv
15990734 = vangelis-b384x4.wv
15996612 = vangelis-b384x3.wv
16004762 = vangelis-b384x.wv
32090104 = vangelis-xh.wv
32227531 = vangelis.flac
32408458 = vangelis-x.wv
32777412 = vangelis-xf.wv
56064668 = vangelis.wav

Since I do not understand where the number 384 came from, but I guess that 256 and 320 will not suit me due to the official remark that lossy codecs do better there, I have settled so far on -b4x. How do I intend to remember this setting? Mnemonics: -x = eXcellent (lossless), -b4x = bee-four-ex/before excellent (lossy).

Re: -h as the new default mode

Reply #30 – 2024-02-19 14:13:10

I would use -x4 over -x .My rational is that if i wanted a speed compromise I'd rather do it at 256 / b2.9.
When using 352 / b4 I favour a more defensive approach.

The remark you mention is that WV won't compete with traditional (128-192) rates . It won't operate less than 196k in 44khz/16 and quality won't be good . I found that the lowest rate 196 / -b2 works well for spoken word cd's etc. For others I use 256 or more.

Re: -h as the new default mode

Reply #31 – 2024-02-19 14:35:12

In my limited testing WV -h often outperforms Flac -8 for cd-audio. Sometimes WV -x4 too.
The decoding well IMO not an issue for typical & even heavier use unless maybe 24/7 transcoding or benchmarking .
And why flac -5, My impression was that -0 or -1 decoded faster . I would use -0 or -1 theres plenty of HDD for small to medium collections.
Then I tried WAV+cue and its even faster and theres HDD space for that too.

Re: -h as the new default mode

Reply #32 – 2024-02-20 10:50:15

Here: https://hydrogenaud.io/index.php/topic,125248.0.html

Exactly as I remember that WV normal is superb often beating flac. Would be intereseting to
enable -x1 by default since some tests for non-cd show huge impact.

Re: -h as the new default mode

Reply #33 – 2024-02-20 14:06:09

-x is good on higher sampling rates, at least on the corpus I used in Reply 7 of this thread, that it should be recommended.
But I do see the point of not changing anything. It increases encoding time by like 50 percent, and for those who use WavPack plugins in audio processing, that is like "time to save a file", and you don't want to wait that much longer than uncompressed?

Even if David himself uses -hx IIRC. WavPack evidently has a release philosphy that if it ain't broken, don't ~~break~~ disturb it. (When was the last time anyone saw a WavPack 3 file?)
So that makes a case for keeping the defaults and rather tell users to rename the executable into one that applies the options.
For those who "need" drag and drop, it is a good thing to have that done once, without deal with command lines.

Now for how WavPack compares to FLAC, then reference flac.exe has had a couple of improvements - like calculating the predictor in double precision. But for what they are worth, look at ktf's compression studies: http://audiograaf.nl/downloads.html

* High resolution:
The most recent one has flac 1.4, and calculating the predictor with double precision makes a big impact on high resolution. Although WavPack does quite well on the overall graphs, the reason it beats flac so much is one generated synthetic file, the Pokemon, and that is likely to WavPack's treatment of wasted bits that captures "signals that shouldn't happen": https://hydrogenaud.io/index.php/topic,121770.msg1024715.html#msg1024715
In ktf's corpus, there is one such file. In a representative hi-rez corpus ... who knows?!
And so a competition between [TAK -p1 or FLAC] and WavPack -hx4 on 24-bit audio, would likely boil down to how many such weirdnesses there are around.

* CDDA:
Seems that WavPack -hx outcompresses flac -8 still.
ktf's revisions 5 and 6 test -hx4 (that hadn't been tested earlier), but not any -x1. But -h performs around flac.exe -6 to -7 on CDDA:
AMD CPU with slightly older flac.exe: http://audiograaf.nl/losslesstest/revision%205/Average%20of%20all%20CDDA%20sources.pdf
Intel CPU, with slightly newer flac.exe: http://audiograaf.nl/losslesstest/revision%206/Average%20of%20CDDA%20sources.pdf
In his revision 4, -hx over -h would make more impact than flac -8 over "-6 to -7". The WavPack curves are -f, default, -h, -hx, -hh, -hhx: http://audiograaf.nl/losslesstest/Lossless%20audio%20codec%20comparison%20-%20revision%204.pdf
Now why does FLAC improve so much from revision 4, relative to WavPack? Part of it is a much broader test corpus. It seems to benefit TAK when compared to Monkey's, and FLAC when compared to WavPack. That seems to be a bigger impact than the improvement of FLAC itself. (I should add, there is absolutely no reason to accuse ktf of cherry-picking corpus to benefit FLAC - indeed, revision 5 was when we knew FLAC could be improved further, and yet it was run with the old one. And the double-precision improvement on FLAC was decided upon before knowing the performance on that particular corpus too.)

Edit: Oh, CDDA sizes on my usual 38 CDs - a not-so-broad corpus compared to ktf's.
Here, WavPack -hx beats flac (1.4.3) by 0.375 percent (not percentage points) even when beefing up flac with -8pr7.   But the "-p" can make flac sneak past on classical music.
All 38, smallest first:
11913862544   wv -mhx
11958733454   flac -8pr7 --no-padding
11967171287   flac -8 --no-padding
11972043112   wv -mh
12032408675   flac --no-padding (that's default -5)
12039392684   wv -mx
12119277966   wv -m

The 14 CDs of classical music, also in size order:
3982936563   flac -8pr7 --no-padding
3983762402   wv -mhx
3985858502   flac -8 --no-padding
4013068344   wv -mh
4021994386   wv -mx
4057553192   wv -m

More edits:
* The WavPack 5.6.6 beta posted in the forums was used, and I see that there's -m there too - makes 56 bytes extra per CD I think?
* flac 1.4.3 32-bit from Xiph. Note, -r7 is unlikely to make much impact on classical music. Also note, keeping .flac with no padding isn't that smart ...

Re: -h as the new default mode

Reply #34 – 2024-02-24 19:10:29

When I compare wavpack -xh with any flac compression flag, including -8, the difference in decoding speed catches my eye: 85x vs 300-400x. How should this difference be understood? Little or no attention is paid to this detail in official manuals, although this probably has to do, at least, with resources consumption and seek speed, so in some cases this may outweigh the argument about the improving compression by one or two hundred kilobytes.

Re: -h as the new default mode

Reply #35 – 2024-02-24 22:06:49

Decoding speed yeah. Does to a large extent measure CPU load, but note that compiler optimizations / instruction sets matter, and might sometimes just mean that parts of the process will go faster and hotter.

FLAC was constructed for ultra-light decoding. Look here: https://hydrogenaud.io/index.php/topic,82125.0.html for numbers on Rockbox devices, which would - to save battery - clock down and not go faster than necessary. FLAC is so "simple" at decoding that you can decode some samples by hand with pen and paper, and indeed if you look at the IETF draft, the appendix carries out decoding of minimal .flac files in detail. Latest at https://github.com/ietf-wg-cellar/flac-specification/releases/tag/draft-ietf-cellar-flac-14

A bit of history with all reservations for accuracy; bear in mind that WavPack the algorithm is older than both FLAC and Monkey's. (Indeed it inspired the creation of the latter.)
Monkey's and TTA are "symmetric", and so was WavPack (I think?) and OptimFROG in old days: you would decode using the encoding algorithm in reverse, taking the same time. Heavier compression? You had to fire more effort at it, in both directions.
I guess we were many who had the gut feeling that if hard drive space was the limiting factor, then you would need that. So Monkey's users could save $$s on hard drives and that codec had a heyday some twenty years ago. But when Monkey's introduced the "Insane" level, there were portable players that could play in principle play it, but not in real time. Meanwhile, FLAC would play at longer battery life than MP3.
Then came TAK. Asymmetric, somewhat heavier on CPU than FLAC, but compressed like Monkey's High (nearly by the tests performed then). TAK changed our perception of what was possible with an asymmetric codec.

Josh Coalson, the creator of FLAC, had a comparison of the codecs available then. This is the oldest web.archive.org version that has decoding times - when FROG had just become OptimFROG: http://web.archive.org/web/20021017022735/http://flac.sourceforge.net:80/comparison.html
It should be noted that WavPack's fast mode was faster at version 3.97, both in encoding and decoding, than early FLAC. You can see that -5 didn't encode particularly fast at the time, but FLAC was to improve. Note, FLAC and Shorten (and in part some RKAU mode) did exhibit this big asymmetry between encoding time and decoding time.
Fast forward to 2007: http://web.archive.org/web/20070823070902/http://flac.sourceforge.net/comparison.html

Now, what has happened since 2002? Except a few new ones?
* FLAC has become more efficient. Most recently also in the fastest encodings. But FLAC works as follows: try many possibilities and pick the best. None of them are heavy at decoding. And by trial and error one has found ways where FLAC can try fewer without having to do so much brute-forcing. You can put the reference encoder at work trying out > 1000 different parameter combinations (per frame) if you seriously want to see how low bang for the buck you can get.
* Among the "symmetric" codecs, both WavPack (the -x) and OptimFROG has gotten "extra encoding processing" which does some of the same: spend more effort on encoding without hurting decoding. But Monkey's has frozen its bitstream: given the signal and the mode (say "high"), the encoded bitstream is determined. Encode a CD with the newest Monkey's - it will be bit-identical to a 3.99 encode. TTA also has a bitstream determined uniquely by the audio - the only thing that could improve would be code optimization.
* We know more about compilers; build matters! https://hydrogenaud.io/index.php/topic,123025.msg1029768.html#msg1029768 .
* And CPU matters. When ktf recently switched his comparison studies from AMD to Intel, WavPack performance numbers suffered compared to FLAC. But meanwhile there was a test on Celeron, where all for sudden the fastest TAK mode would decode faster than FLAC.
* High resolution audio has started messing up what we "knew" from typical CD audio. Especially since a lot of "high resolution" has very little content in the top octaves or the lowest bits.

Re: -h as the new default mode

Reply #36 – 2024-02-25 10:15:20

flac -8 doesn't give me anything over -7 but is way slower .. Anyone else find that ?

Re: -h as the new default mode

Reply #37 – 2024-02-25 12:02:19

Like, the difference between the rightmost two blue triangles in the upper diagram here? Quite a flat end of the graph.
http://audiograaf.nl/losslesstest/revision%206/Average%20of%20CDDA%20sources.pdf

The reason that -7 compresses better than -6 is that it allows for longer prediction. -7 and -8 are the same on that parameter, and that's why they decompress equally fast; see the virtually overlapping triangles in the lower chart.