-h as the new default mode

Topic: -h as the new default mode (Read 15392 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

-h as the new default mode

2021-01-16 12:38:38

Perception is everything to many. Ones that look at percent compressed vs flac vs ones that look
for the 'fastest' decoding even though both are meanigless in modern scenarios.

Still, It will stick that both have same compression but flac is 'fast' and wavpack is 'slow' due to
looking at some 'studies' and graphs.

If -h became the new default, Wavpack compression on paper would look much better than flac. The speed
would still be very fast both ways. This could help popularity. It would stick in peoples mind that wavpack
compress better than flac and most other codecs - except TAK & APE. Tak is closed source. APE -c2000 is fast but slow to decode.
APE -c1000 compression is similar to wavpack -h but still slower in decoding . WV has better license then APE.

I think win win for -h as default. -hh can be new -h, normal could be 'fast' and fast could become 'very fast'

Re: -h as the new default mode

Reply #1 – 2021-01-20 03:38:22

Hi shadowking, and thanks for the suggestion! I actually have thought about this and I do use the high mode the most, although sometimes I use -x6 as the default when I’m not in a hurry.

I think though that there are other reasons for people not using WavPack unrelated to this, like limited hardware support and confusion about all its additional features. If people are looking for a codec just to rip their CDs to I suggest FLAC.

If someone needs something unique that WavPack has, like 32-bit audio (integer or float), or DSD, or hybrid/lossy, or dozens of channels, etc, then obviously I recommend it.

And at this point I think it would only add confusion to either rename the modes I have, or force high mode on by default and create a new option for “normal” (which is probably the way I’d go).

BTW, do you really still use 4.80?

Re: -h as the new default mode

Reply #2 – 2021-01-20 08:20:10

Hi David,

Last night I had another thought, This time much simpler. The idea is to hide the complicated options
and emphasise 'presets' for the most common lossless / hybrid options. I took ideas from zip, lzop, flac, mac cli tools

Proposal;

lossless
-0   = -f
-1   = -x
-2   = -hx (recommended)

Hybrid
-hybrid0 = -b256hx3c
-hybrid1 = -b320hx3c
-hybrid2 = -b384hx4c

--
Yes still on 4.8. I really should try latest or just delete my dumb signature

Re: -h as the new default mode

Reply #3 – 2021-01-23 22:52:10

Yes, that's actually kind of a good idea that I had not thought of. In fact, FFmpeg works that way (although I don't agree with what they've chosen, for example you can only get the high -x modes with -hh).

Of course, it doesn't address the original issue unless we force one of them as default, but it's something to think about.

As for the 4.8 issue, you know you're missing out on the new quick verify feature by using that, right?

Re: -h as the new default mode

Reply #4 – 2021-01-27 23:02:27

Nice ideas by shadowking, course as long as the "complicated options" remain available (I guess that's what he means by hiding them).

P.S.
Stil on 4.80.0?! How can you live without wvtag

? Come on, man

Re: -h as the new default mode

Reply #5 – 2021-11-10 23:48:48

Has this gone anywhere since please? Thanks.

Re: -h as the new default mode

Reply #6 – 2021-11-12 00:08:27

What one could do, is to elevate the "-x" to be the preset. -x0 to -x6.
Then take the fast/normal/high/very high mode to be a modifier. Akin to how TAK has -p as the preset and then with modifiers, except that I would propose -x0 as default if -h is default. "-x" could still mean "-x1".

If one is to go for -h as new default mode, one could have a letter for current "normal". "g" works. Between f(ast) and h(igh), there is g(ood). (Heck, if others can call some preset "best", you can call that mode "good".) Or you can consider f and ff for the faster than high. Of f and ff, so that -x0ff means current --fast. And you can keep "--fast: synonymous to -x0ff".

-x1hh ... or, to highlight the warning against it for portables, require it to be given separately.

All this won't change much but the mindset; and it should, if you embrace -h (and still warn against -hh), for then the x is what the user would have to select unless diving into the detail.

"purely numerical" presets like FLAC have an issue with WavPack: as long as one has already decided that -hx shall be accepted as -h -x, then it would confuse a user if -3 -x is not the same as -x3.

Now whether going for -h is a good or a bad idea is another issue; it is a double-edged sword in that it puts the default to even slower decoding - but on the other hand, those obsessed with that might look elsewhere.

But I would think twice before letting new -hx be default, as it takes twice the time to encode as current default does. That is quite some change. -h is already smaller than any FLAC. And with ktf's freshly posted flac beta, it is about the speed of -7.

Oh, and I think the manual does something very wrong when speaking of (very) high "quality" for -h and -hh. Very high compression effort?

Re: -h as the new default mode

Reply #7 – 2021-11-12 10:20:45

... but if one wants to pick presets from a selection of reasonable settings, then ... I put my computer to a couple of overnight tasks with around 3 GB CDDA and 3 GB 96/24 (well a few 88.2/24), same corpus as here. Warning: Rock to metal, not much classical. But if one is to pick presets, I think the considerations below are a significant point (and then others can fill in with more testing).

The below table is interesting. The -x3 setting is frankly quite useless in this test. -x1 is so excellent on this hirez corpus that it is tempting to make it mandatory, despite whatever I said in the previous post about a new default taking so much more time.

Explanation: CDDA to the left, hi-rez to the right. Size and seconds are self-explanatory; times are taken from Igor Pavlov's timer64.exe ("process time", I think it ignores time to access files on the SDD.)
The -Δt/Δsize means the cost in seconds per megabyte saved compared to the setting immediately above it. Reasonably, this should be progressive within the mode (which means everything else equal): you would have to pay more seconds for another megabyte reduction if the file is already smaller.
-x3 violates that (except in fast mode): if you are willing to pay for going -x2 to -x3, you get better per second going -x4. Indeed, for the higher-rez material, -x3 is both bigger and slower than -x2, indicated by "BIGSLOW". (That means that a number immediately below "BIGSLOW" is low because it is in comparison to something bad.)
The other "bigger" indicated, are when mode is switched and file size goes up. Then the negatives are when higher mode yields both faster encoding and smaller file.

I also question the usefulness of -x5. It is already so slow that you would leave it in the background for long rather than watching it. If you are first going -x5, why not leave it overnight or for the week-end for -x6?

To get a reference to FLAC: ktf's newly posted beta, which improves quite a bit, beats WavPack on hirez unless you go up to -h -x4. (This edited after a big mistake.)
But for CDDA: -7 compresses to both time&size between normal and -x1, and -8 to both time&size between -x1 and -x2. -x2 is likely smaller than any FLAC ever - for this CDDA corpus.

setting	bytes CDDA	sec	-Δt/Δsize	-Δt/Δsize	sec	bytes hirez
-f	2210757614	68			58	2354239318
-f -x1	2202072762	100	4	1	83	2323061806
-f -x2	2198609618	131	9	16	106	2321563994
-f -x3	2197522750	204	67	56	166	2320481204
-f -x4	2196193122	461	193	65	356	2317536024
-f -x5	2195912276	571	393	106	420	2316936660
-f -x6	2195820750	657	938	7132	491	2316926664
[n]	2172500794	82	−25	−8	61	2265273710
[n] -x1	2159246612	133	4	1	96	2236071090
[n] -x2	2156648886	194	23	35	138	2234880022
[n] -x3	2155729900	342	161	BIGSLOW	242	2238618428
[n] -x4	2148852240	1220	128	20	964	2201779442
[n] -x5	2148336890	1717	963	145	1440	2198486522
[n] -x6	2146702532	3329	986	208	2967	2191150966
-h	2143903450	104	−1152	bigger	81	2242607186
-h -x1	2138172066	176	13	2	131	2217805534
-h -x2	2136565630	279	64	116	208	2217136902
-h -x3	2136430268	535	1889	BIGSLOW	400	2219912878
-h -x4	2131282798	1964	278	15	1672	2137653818
-h -x5	2130429436	2676	835	398	2249	2136206612
-h -x6	2130069132	3647	2695	626	3285	2134552628
-hh	2133201720	134	bigger	bigger	100	2233102796
-hh -x1	2128773398	234	23	2	172	2203717666
-hh -x2	2127751110	392	155	125	291	2202768210
-hh -x3	2127514768	806	1752	BIGSLOW	585	2203496882
-hh -x4	2124631636	2833	703	23	2477	2121330764
-hh -x5	2123744660	4566	1954	1358	3876	2120301594
-hh -x6	2123586950	6255	10708	2921	5483	2119751366

Re: -h as the new default mode

Reply #8 – 2021-11-12 13:39:10

Quote from: Porcus on 2021-11-12 10:20:45

-x3 is both bigger and slower than -x2, indicated by "BIGSLOW". (That means that a number immediately below "BIGSLOW" is low because it is in comparison to something bad.)

Oh, but for hirez the numbers indicate something to scratch one's head over. Disregarding -f:

-x2 to -x4 (normal mode): Shaving 1 MB costs 25 seconds. That is cheaper than going -x1 to -x2 (35 sec per MB s(h)aved).
-x2 to -x4 (high mode): Shaving 1 MB costs 18 seconds. That is much cheaper than -x1 to -x2 (116 sec per MB).
-x2 to -x4 (hh mode): Shaving 1 MB costs 27 seconds. That is much cheaper than -x1 to -x2 (127 sec per MB).

So a reasonable interpretation (well the official documentation pretty much says the same) is that going for high x is a speculative investment of time; one that could yield good return if and when lower x (including no x) might not be well optimized for the particular signal in question.
Now this single test (no classical music, yeah ...) indicates that x2 might not be well-optimized for high-frequency content; x3 might not be well optimized at all?

Then also to think over: getting a MB for half a minute is pretty good bang for buck if you are in the market for high compression - but that number is because WavPack performs quite bad on these files without -x4: ktf's new FLAC beta at -8 saves two percent (45 megabytes) over -hhx and at faster encoding time.

This would provoke the question: should presets be "same for all signals" or should the encoder read sample rate and shift settings accordingly? But no, not in this case: a default setting cannot run fifteen times as long on a GB of 96/24, compared to a GB of CDDA.
(But @bryant : Is there maybe something you should look at about selecting block size from sample rate? Also I noticed from timer64 output that WavPack spends only a fraction of the memory that flac.exe does, but that might be due to flac.exe trying to buffer before writing to limit the number of i/o operations and the file fragmenting that would follow?)

Quote from: Porcus on 2021-11-12 00:08:27

-h is already smaller than any FLAC.

That was CDDA.

Re: -h as the new default mode

Reply #9 – 2022-07-15 00:57:38

If you would consider changing defaults... -x (-x4?) might also be worth considering, given encoding is a one-time task, and decoding performance is not affected and compression is comparable to -h alone.

Re: -h as the new default mode

Reply #10 – 2022-07-15 16:03:25

Quote from: evan123 on 2022-07-15 00:57:38

If you would consider changing defaults... -x (-x4?) might also be worth considering, given encoding is a one-time task, and decoding performance is not affected and compression is comparable to -h alone.

Your right, -x4 does a lot of work without being very slow.
-hx4 takes it even further while still retaining OK speed.
The effect isn't as pronounced with -hhx4. The -hh doesn't benefit as much.
Plain -h and -hh work great for red book CD audio. pretty fast.
I guess looking at porcus chart, The normal mode is well balanced and very fast.
normal + x1 is also reasonable for compression vs speed.

Re: -h as the new default mode

Reply #11 – 2022-07-15 16:57:38

David can answer for himself, but as far as I understand, WavPack has found a niche as a .wav substitute in CoolEdit and Audition (since it restores the .wav file bit-exactly with file structure and metadata).
That is not an "encode once" situation - you will wait for an encode every time you save. To slow down the default will be a nasty surprise to those users, and for all that I know they might make up a significant amount of the WavPack userbase.

Also, the new 5.5.0 lets you set your own options to the executable for drag and drop (yeah, touting my own idea here): https://hydrogenaud.io/index.php/topic,122626.msg1012159.html#msg1012159
Combine that with WavPack's nice recompression feature: it is now useful even to those who aren't happy about the command line. Copy David's suggested filename, and you have a WavPack exe that encodes to -hx with MD5 and verification - and drop all your default-saved .wv files onto it. Put a "4" or a "6" after the "x" for overnight jobs or when you are going on holiday.

As for the -x4 size s(h)avings:
ktf's comparison (most recent edition) confirms what the table above suggested, namely that -x4 makes a difference on high resolution material (costly yes, but those compression figures for an asymmetric fast-decoding format would have looked awesome hadn't TAK moved the bar up). But also it is very expensive for a percent at CDDA. You will find the multi-channel material around the middle of the two.
Even if the figures on bonkers-resolution files are impressive, it looks like a very bad idea to change the default to target the scarcer material.

I'd say, teach WavPack users the joys of recompression.

Re: -h as the new default mode

Reply #12 – 2022-07-15 18:45:01

Even for DAWs using the standard file formats, it would also be handy to use a fast WavPack setting for the freeze files. Reaper for example can freeze to WavPack. Modern and advanced DAWs are often non-destructive. For example, in a complex orchestral project which uses dozens of Gigabytes of samples and hundreds of polyphony:
https://youtu.be/skAxKtDXnpU

Let's say you have the piano partially finished or want to deal with it later, you can let the DAW render that part to audio temporary to save CPU/RAM. Since the MIDI data for the piano is still there, at some point after you modified the strings section and find that the piano arrangement needed to be modified, you can unfreeze the piano, which means delete the rendered piano audio and let the plugin generating the piano note in real time again when you play the MIDI keyboard or edit the notes using mouse. Depends on the DAW and plugin it is even possible to selectively freeze a musical phrase instead of the whole track.
https://youtu.be/FUOLnz2cvB4

Also, with WavPack's float support, there is no need to worry about the frozen parts are clipped or not, as the rendering may also take some time and floating point will ensure there is no need to retry due to clipping. Some instruments, especially the electronic synthesizers, can have some dramatic and time-varying filters to induce clipping if rendered in fixed point.

Re: -h as the new default mode

Reply #13 – 2022-07-17 19:41:14

The two fastest encoding modes of WavPack are the non-extra “fast” mode and then the non-extra “normal” mode (the current default). Looking at ktf’s comparisons (for example, here) it’s obvious that the first compression gain (from “fast” to “normal”) is pretty much the biggest jump. Except for hi-res, going all the way to -hhx6 doesn’t duplicate that improvement, so I think that makes the current default the “sweet spot” for encoding performance. And, by coincidence, it’s also the closest match to FLAC -5 (which is also, even more obviously, its “sweet spot” in ktf’s graphs).

My personal default setting is -hx because it gives another nice compression bump, but is over twice as slow encoding as the default, and also decodes slower. If someone else likes that they can rename their executable to include it (as Porcus suggests) or can even modify the source and build a custom executable (now that -g and -x0 are available to cancel those selections). But to require the user to specify additional (new!) options to get to the encoding sweet spot doesn't seem right.

Re: -h as the new default mode

Reply #14 – 2022-07-17 20:47:45

My solution is to just write a simple script with the CLI options I want, and then do all encoding by invoking the script, rather than running the program directly. These days computers are so fast I don't see much benefit in using anything but the most aggressive settings, as you only have to spend time encoding the file once, but the space savings are permanent. I can wait a little while longer.

Re: -h as the new default mode

Reply #15 – 2022-07-17 23:36:18

Quote from: Aleron Ives on 2022-07-17 20:47:45

My solution is to just write a simple script with the CLI options I want, and then do all encoding by invoking the script, rather than running the program directly. These days computers are so fast I don't see much benefit in using anything but the most aggressive settings, as you only have to spend time encoding the file once, but the space savings are permanent. I can wait a little while longer.

If anyone has a .bat that would do the following, then post it.
For each file (possibly traversing directory, possibly allowing drag and drop?)
- wvunpack -ss to find the mode, and reject file if it isn't a valid WavPack file or if it is high/extra high, but if it is fast or normal (or none given, isn't that what ffmpeg does?):
- recompress using -hx4 [or suitable setting]

As long as user is disciplined enough to stick to default or -f or ffmpeg (ffmpeg's default is like -fj0!) for initial encode, then this .bat can be run periodically.

Re: -h as the new default mode

Reply #16 – 2022-07-18 00:08:48

Quote from: bryant on 2022-07-17 19:41:14

t -g and -x0 are available

... though not in the release notes.

Re: -h as the new default mode

Reply #17 – 2022-07-18 03:32:35

Quote from: Porcus on 2022-07-17 23:36:18

If anyone has a .bat that would do the following, then post it.

Although this is possible, it's much simpler to just blindly recompress all .wv files using your new preferred settings, as you only need to run the script once. After that, you can just be sure to use those settings when adding new material to your collection, so you can be sure everything was compressed with the same settings.

Re: -h as the new default mode

Reply #18 – 2022-07-18 11:12:32

bat/cmd can have strange quirks when dealing with foreign unicode characters, for example if Windows' default codepage is Chinese but you are dealing with files with Japanese file name. Also some reserved characters like brackets and such can screw up things. One needs to be a guru of the bat/cmd language to debug these issues.

The changing exe filename approach if done correctly (both developers and users) can somewhat simplify things without using a GUI frontend.

.ini files alongside with the .exe file could be another approach as well.

Re: -h as the new default mode

Reply #19 – 2022-07-18 20:00:25

Yeah, I try to use only ASCII characters in filenames and reserve non-ASCII characters for tags so that all programs will be able to parse the filenames correctly.

Re: -h as the new default mode

Reply #20 – 2023-08-12 16:18:07

I thought again recently about this. I now think nearly everything is fine as it is.
The normal mode is fine as default. The -h should be 'high mode' (vs quality) just like -f is fast mode.
That sounds good in regards to compression (default is in the middle).
In the --help section it can also be mentioned as having quality related effect.
-c should -not- create the .wvc , Instead wvc should be created automatically whenever -b is
invoked. This way all output is lossless and those objecting to using Wavpack on grounds that they might
end up with lossy only can step back and dont have to run to flac etc..

Re: -h as the new default mode

Reply #21 – 2023-08-12 16:23:49

Maybe 'complex' options can be moved to --help. typing just wavpack can look very KISS ;

WAVPACK Hybrid Lossless Audio Compressor

Usage:   WAVPACK [-options] infile[.wav]|infile.wv|- [outfile[.wv]|outpath|-]
   (infile may contain wildcards: ?,*)

Options:
-f = fast mode (fast, but some compromise in compression ratio)
-h = high mode (better compression ratio, but slower)
-v = verify output file integrity after write (no pipes)
-x = extra encode processing (no decoding speed penalty)
--help = complete help

Re: -h as the new default mode

Reply #22 – 2023-08-12 16:31:32

Or without -x and -h appearing first ;

WAVPACK Hybrid Lossless Audio Compressor

Usage: WAVPACK [-options] infile[.wav]|infile.wv|- [outfile[.wv]|outpath|-]
(infile may contain wildcards: ?,*)

Options:
-h = high mode (better compression ratio, but slower)
-f = fast mode (fast, but some compromise in compression ratio)
-v = verify output file integrity after write (no pipes)
--help = complete help

Re: -h as the new default mode

Reply #23 – 2023-08-12 18:38:43

Quote from: shadowking on 2023-08-12 16:18:07

-c should -not- create the .wvc , Instead wvc should be created automatically whenever -b is
invoked. This way all output is lossless and those objecting to using Wavpack on grounds that they might
end up with lossy only can step back and dont have to run to flac etc..

The 5.6.4 beta takes a different approach where lossless-users don't have to touch the "-b". From the short help:

Code: [Select]

          -c<n> = shortcut for '-b<n> -c' to specify hybrid lossless mode
                  with a single option, n = 2.0 to 23.9 bits/sample, or
                                        n = 24-9600 kbits/second (kbps)

This way you don't have to "invoke lossy and then invoke correction" at the risk of forgetting the latter. You can just skip -b entirely.
Also it doesn't break old behaviour for anyone who used it correctly.

Quote from: shadowking on 2023-08-12 16:23:49

Maybe 'complex' options can be moved to --help. typing just wavpack can look very KISS ;

You keep getting more of a point here as the short help grows longer: by 5.6.4 now it exceeds the default cmd window's 25 lines. Surely there is a setting to alter those "25". But now the "Usage" falls out of the screen.
If it is to be shortened down, who knows what is to go. (By the way, I do not like the phrase "quality" for lossless ... though the text should be valid for lossy too.)

Anyway, if formats stay in the short help, here is a tweak to get slightly information without altering layout - it becomes one character wider in the top-right column, but note the AIFF column width matches the one immediately above. Also the Apple formats end up on one line, just as the DSD formats do.

Code: [Select]

 Formats: .wav (default, bwf/rf64 okay)  .w64 (Sony Wave64)
          .aif/.aiff/.aifc (Apple AIFF)  .caf (Core Audio Format)
          .dff (Philips DSDIFF)          .dsf (Sony DSD stream)
          .wv (transcode from existing WavPack file, with tags)

Re: -h as the new default mode

Reply #24 – 2023-08-13 11:13:24

WAVPACK Hybrid Lossless Audio Compressor Win64 Version 5.6.0
Copyright (c) 1998 - 2022 David Bryant. All Rights Reserved.

Usage:   WAVPACK [-options] infile[.wav]|infile.ext|- [outfile[.wv]|outpath|-]
WAVPACK --drop [-options] infile[.wav]|infile.ext [...]
   (infile may contain wildcards: ?,*)

Formats: .wav (default, bwf/rf64 okay) .aif (Apple AIFF)
.w64 (Sony Wave64)    .caf (Core Audio Format)
.dff (Philips DSDIFF) .dsf (Sony DSD stream)
.wv (transcode from existing WavPack file, with tags)

Options:
-h = high mode (better compression ratio, but slower)
-f = fast mode (fast, but some compromise in compression ratio)
-m = compute & store MD5 signature of raw audio data
--pause = pause before exiting (if console window disappears)
-v = verify output file integrity after write (no pipes)
-x = extra encode processing (no decoding speed penalty)
--help = complete help

Web:    Visit www.wavpack.com for latest version and info

Notice