Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lossyWAV 1.4.2 Development (was 1.5.0) (Read 162333 times) previous topic - next topic
0 Members and 3 Guests are viewing this topic.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #175 –
@Nick.C :
I don't know what is going on here, but when trying the setting in your signature, and compressing with MP4ALS (encoder here), it just becomes unresponsive. Which is a flaw in that software, but:

It works fine with 1.4.2-processed files!

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #176 –
It could be another manifestation of the padding bugs that have recently been bothersome, or it could be that the MP4ALS program does not deal with odd length chunks properly (and that the 'fact' chunk length is odd).

Are you processing to WAV then using the codec or are you piping direct? If the former then some issue with the 'fact' chunk is more likely to be the culprit.

[edit] After processing 55 sample test set to WAV with Beta 1.4.3e I can confirm that the length of the 'fact' chunk with the settings in my signature is odd, and correctly zero padded with one extra byte (that does not form part of the stated chunk length), and encoding with FLAC works perfectly - so I strongly suspect that the issue lies with MP4ALS not dealing with zero padded odd length chunks. [/edit]

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #177 –
Please find attached a new beta release of lossyWAV. Removed as superseded.

lossyWAV beta 1.4.3f, 26/05/2024 (expires 31/12/2024)
  • Due to two reported cases of apparent issues with encoders not dealing with odd length chunks properly a padding byte is now added to the end of the settings string contained in the 'fact' chunk of lossyWAV processed files, if the length of the string is initially odd, and the size of the 'fact' chunk will now always be even.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #178 –
Lossywav support was added to HALAC. In version 0.2.9, it will be shared with other arrangements. The Lossywav format is detected from the "fact" keyword in Header. And in this case a different mode is switched. If the "-fast" parameter is used, it is disabled.
However, I am currently sharing Encoder and Decoder, which is available for test. The results are not bad for starting in terms of speed/compression ratio. Encode speed is considered the same as HALAC Normal. However, the decode speed is much more (prediction process was not used)! Rice coding has not been used yet and the block size is 2048.
All other codecs are at the highest compression level in the default settings. And single thread results...
Code: [Select]
İskender Paydaş - Zamansız Şarkılar (2011)
10 Tracks, 469.385.056 bytes, i7 3770k, 16 gb ram, 240 gb ssd.
Wma      :  19.06,  11.26, 174,956,882
Flac     :  18.76,   4.73, 175,805,836
Halac    :   2.37,   1.81, 191,318,610
Tak      :  20.27,   7.18, 193,906,253
Wavpack  :  29.40,  16.38, 230,017,269
OptimFrog: 184.12, 111.69, 254,001,399
Monkeys  :  59.42,  63.83, 325,501,439
Alac     :  18.83,   8.87, 340,827,308

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #179 –
Some bitrates. Here I took the LossyWAV settings in Nick.C's signature. Lossy part only.
For those with a "+", then the first number is with block size set to 512, and the "+25" etc is the extra when using 2048 like HALAC does.
275   ...   ALS, smallest
287   +25   ALS -l
292    +9   TAK -p4m
318   +11   FLAC -5
327   +13   WavPack -hhx4
353   +27   FLAC -0r0
387     -   HALAC (2048 blocksize)
725         HALAC but without the identifying FACT chunk
1411        wav


Speeds then?

Comparing to FLAC encoded with -0r0 --no-md5sum --keep-foreign-metadata and block size set:
6.6 resp 7.9   FLAC encode to 2048 resp 512
9.0 HALAC encode

But HALAC decodes faster:
7.3 HALAC decode
11 FLAC decode these MD5-less files with --keep-foreign-metadata, maybe differing some tenths over block size used.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #180 –
According to the compression results of me with the music in me, the situation is similar to the majority of the tests I have performed. Of course this is the first trial. Better can be done. Lossywav is activated as a special mode for now.
They also perform low performance as I don't understand the reason for Alac, Optimfrog and Ape.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #181 –
The idea behind LossyWAV was to exploit FLAC's wasted bits feature. And IIRC the reason for 512 samples blocks, is that then the number of wasted bits can change rapidly enough as the signal does.
TAK handles wasted bits. MPEG-4 ALS does (but you need to invoke the "-l" switch). WavPack too - and it can also handle other "redundant LSBs patterns" ( https://hydrogenaud.io/index.php/topic,121770.msg1024689.html#msg1024689 , and WavPack's handling is also likely the reason why it does so well on the high resolution lossless corpus in ktf's test: see the Pokemon on page 7).

ALAC does not handle wasted bits. Either because ALAC generally sucks, or because Apple was worried about some patent.
Monkey's does not handle wasted bits. Unlike ALAC, which never was intended to be a heavy compressor, it is more of an oversight for Monkey's, I say: take a 16 bit signal, pad it to 24, and the ape compresses baaad.
OptimFROG does handle wasted bits, but it does not operate on such low block sizes. Indeed, both the animals seem to think that longer is better, and then there is this odd signal where they get outcompressed by FLAC's way to handle ultra-short changes in the signal. Here is one where OptimFROG does awesome, but only in its lightest mode (shortest blocks I guess): https://hydrogenaud.io/index.php/topic,124862.msg1038820.html#msg1038820
More: https://hydrogenaud.io/index.php/topic,122413.0.html

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #182 –
Many thanks for the illuminating information Porcus. According to the sample music in the link you show, the test results are as follows. There is no difference between the normal state of music and the lossy of the state.
Code: [Select]
00 - Dan Worrall - I Won The Loudness War
Halac (2048) :    466,531
Tak (512)    :    685,202
Flac -8 (512):    904,162
Wma          :    917,077
Flac -0 (512):  1,121,225
Wavpack (512):  1,321,353
Ofr          :  1,991,918
Alac         :  3,215,927
Ape          :  7,597,601
Wav          : 15,211,284

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #183 –
Yeah, so that is one where LossyWav doesn't zero out anything that wasn't already zero (you find F800 and 0800 all over, except some 0000 at the beginning) - so in one way it is off topic to a LossyWAV thread (my bad for bringing it up if so), and on the other it shows why there is LossyFLAC and LossyTAK, but not LossyAPE.

I do get the frog down to 224 399, attached. And WavPack down to 348 022.

HALAC doesn't have very expensive block headers?

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #184 –
I can get 339,316 when I increase the block size to 32k. However, we cannot generalize this to everything. As far as I understand, the optimfrog arithmetic coder should be using something. Because only Rice coding cannot go down from a certain limit.

Yes, HALAC uses tANS as an entropy encoder and contains considerable side information for each block. Therefore, very small blocks are not suitable in terms of both processing speed and compression ratio. In other words, this sample is created for the file for the 1900 blocks and there are probability distributions, etc. for each.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #185 –
I had the opportunity to study the LossyWAV data structure a bit. Now you can get slightly better results at the same speeds. FSE(tANS) - block size 2048.

Code: [Select]
Sean Paul     525,066,836 bytes
HALAC 0.2.9 : 215,403,561 bytes
HALAC 0.3.0a: 198,858,446 bytes

Busta Rhymes  829,964,360 bytes
HALAC 0.2.9 : 350,671,533 bytes
HALAC 0.3.0a: 335,745,475 bytes

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #186 –
Since lossyWAV is technically a DSP effect (that reduces the bitrate when encoded to certain lossless codecs like FLAC, but it is moot for uncompressed audio files), I wonder this could be implemented as foobar2000 DSP component to perhaps, simplify the conversion process just by putting lossyWAV into the "Active DSPs" part on the converter side and what would lossyWAV-processed FLACs sound like when put on the playback DSP chain

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #187 –
lossyWAV is a DSP - in that the output format is the same as the input format and sample values may change.

If lossyWAV was implemented as a playback chain DSP then it would *have* to be the last in the chain if the preservation of zeroed LSBs was to be guaranteed.

Although I would question the need to insert lossyWAV as a DSP in the playback chain in the first place - why waste the CPU cycles processing the source to a lossy/lossier form while listening when one could simply listen to the unprocessed version?

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #188 –
Hello

Is it a good idea to use LossyWav in Bluetooth audio (SBC)?
Plan is to convert my CD.flac's to 48k, then LossyWav and re-pack wav's again.
Code: [Select]
flac.exe --decode --output-name=out.wav in.flac
ssrc_hp.exe --rate 48000 --twopass --profile long in.wav out.wav
lossyWAV.exe in.wav --quality extraportable --outdir %cd%
flac.exe --no-utf8-convert --output-name=out.flac --verify --blocksize=512 --compression-level-5 in.wav
Packages:
flac-1.3.4-win.zip\flac-1.3.4-win\win32\flac.exe
ssrc-1.33.tar.gz\ssrc-1.33.tar\ssrc-1.33\bin\ssrc_hp.exe
lossyWAV_v1.4.2.zip\bin\32-Bit\lossyWAV.exe

AFAIK, phones and DAP's are internally 48k, so it might be a good idea to convert audio files before use.
Bluetooth audio is lossy, so source material has to be lossless.
But CD.flac's are too good and too large to casual listening.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #189 –
Quote from: Nick.C on
lossyWAV is a DSP - in that the output format is the same as the input format and sample values may change.

If lossyWAV was implemented as a playback chain DSP then it would *have* to be the last in the chain if the preservation of zeroed LSBs was to be guaranteed.

Although I would question the need to insert lossyWAV as a DSP in the playback chain in the first place - why waste the CPU cycles processing the source to a lossy/lossier form while listening when one could simply listen to the unprocessed version?
Yep, it would be a kind of DSP component intended for use within foobar2000's converter (to reduce the filesize of lossless compression codecs in a lossy way, though it only works well if the lossyWAV DSP effect is placed in the last DSP chain), rather than just a playback DSP effect (though it can be used as a mean for previewing what would lossyWAV-processed files sounds like) if implemented as a foobar2000 component

BTW, do anyone have any interests for making lossyWAV as a foobar2000 component (a DSP component intended for use in converter as I said before) happen?

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #190 –
Progress made on improving perceived quality at the lowest preset, focused on improving the weighted adaptive noise shaping option, in what will be Beta 1.4.3g. Using a calculation limit frequency of c.13.8kHz (my ears are not getting any younger after all) among other settings yields an album bitrate range (so far in my full collection trranscode) of 142kbps (The Rolling Stones [1964] The Rolling Stones EP) to 442kbps (Jean-Michel Jarre [2008] Oxygéne Live in Brussels) with an average of 272kbps down from 889kbps.

I also tripped over another problem sample (for lossyWAV anyway) - the first fifteen seconds of "For You" by Rae Morris on the 2016 compilation release "Originals - Power Ballads". Can't work out how to extract a permissible length sample to share, so if anyone can please advise me how to achieve this then I'll happily do so. Found a guide on how to use FFMPEG to extract the first fifteen seconds, attached.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #191 –
Please find attached a new beta release of lossyWAV.

lossyWAV beta 1.4.3g, 13/10/2024 (expires 31/03/2025)

N.B.: the executable is 64-bit only and requires Windows 8 or later.

  • Weighted adaptive shaping method modified. Previous optional parameters have been removed and the option now takes a single optional parameter to determine the "width" of curve averaging to smooth the desired noise shaping curve over up to <n> FFT bins (dependent on the distance from each end of the FFT the bin being calculated is located). Valid in the range 0 to 64; default = 0, i.e. no smoothing.
  • Low quality preset range extended down to -10.0, with presets "U, unadvised" at q=-10, "D, dubious" at q=-8.75, "Q, questionable" at q=-7.5, "A, aggressive" at q=-6.25. Quality presets at and above "X, extraportable", q=-5, remain unchanged

Recent low quality testing has been with the following lossyWAV command line options:
Code: [Select]
-q U -s W 8 taps 20 -A 0.14 -a 4 --limit 13782 --feedback 10

For clarity quality settings below -5.0 and presets below "extraportable", i.e. from "aggressive" to "unadvised" inclusive, are expected to be non-transparent in many cases, increasingly so further down the quality scale. Their inclusion is part of an exploration into the low quality depths of the method. It should be noted that the above testing command line employs two mitigations, i.e. the significant reduction in the "-A, --altspread" parameter from the default (0.14 vs 0.768544) and setting the feedback check to the max.

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #192 –
@Nick.C
lossyWAV Beta 1.4.3g.7z
Not working in my offline Windows 7 x64 computer.
Quote
lossyWAV.exe - Entry Point Not Found
The procedure entry point GetSystemTimePreciseAsFileTime could not be located in the dynamic link library KERNEL32.dll.
Don't waste too much time on this, because Windows 7 is EOL...

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #193 –
Quote from: Jetro on
@Nick.C
lossyWAV Beta 1.4.3g.7z
Not working in my offline Windows 7 x64 computer.
Quote
lossyWAV.exe - Entry Point Not Found
The procedure entry point GetSystemTimePreciseAsFileTime could not be located in the dynamic link library KERNEL32.dll.
Don't waste too much time on this, because Windows 7 is EOL...
That's a consequence of the executable using a Windows 8+ feature. I'll add a notification to the post with the executable attached.

Thanks for raising the issue.

If you want a version that does not use this later Windows feature, please let me know and I can compile one.
 

Re: lossyWAV 1.4.2 Development (was 1.5.0)

Reply #194 –
Thanks Nick.C

No need to compile W7 executables in beta, I can use lossyWAV_v1.4.2.zip.
But if you release stable/final/eol, etc, then I might need W7 x64 -version.