HydrogenAudio

Hydrogenaudio Forum => Validated News => Topic started by: Nick.C on 2011-08-06 21:50:15

Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-06 21:50:15
lossyWAV 1.3.0 is released.

lossyWAV is a near lossless audio processor which dynamically reduces the bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise to the processed output. The added noise is adaptively shaped by default and can alternatively be fixed noise shaped or white noise depending on command line parameters. When lossyWAV processed output is compressed with certain lossless codecs (FLAC, Wavpack, Tak, LPAC, MPEG-4 ALS and WMA-Lossless) the bitrate of the output file is significantly* reduced compared to the lossless original.

Changes from 1.2.0b:Download* on average, depending on content.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-06 21:53:04
Code: [Select]
lossyWAV 1.3.0, Copyright (C) 2007-2011 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV is a near lossless audio processor which dynamically reduces the
bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise
to the processed output. The amount of permissible added noise is based on
analysis of the signal levels in the default frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a different --limit (in the range 10kHz to 20kHz).

For many audio signals there is little content at very high frequencies and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

The noise added by the process is shaped using an adaptive method provided by
Sebastian Gesemann. This method, as implemented in lossyWAV, aims to use the
signal itself as the basis of the filter used for noise shaping. Adaptive noise
shaping is enabled by default.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-q, --quality <t>    where t is one of the following (default = standard):
    I, insane        highest quality output, suitable for transcoding;
    E, extreme       higher quality output, suitable for transcoding;
    H, high          high quality output, suitable for transcoding;
    S, standard      default quality output, considered to be transparent;
    C, economic      intermediate quality output, likely to be transparent;
    P, portable      good quality output for DAP use, may not be transparent;
    X, extraportable lowest quality output, not fully transparent.

Standard Options:

-C, --correction     write correction file for processed WAV file; default=off.
-f, --force          forcibly over-write output file if it exists; default=off.
-h, --help           display help.
-L, --longhelp       display extended help.
-M, --merge          merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>     destination directory for the output file(s).
-v, --version        display the lossyWAV version number.
-w, --writetolog     create (or add to) lossyWAV.log in the output directory.

Advanced Options:

-                    take WAV input from STDIN.
-c, --check          check if WAV file has already been processed; default=off.
                     errorlevel=16 if already processed, 0 if not.
-q, --quality <n>    quality preset (-5.0<=n<=10.0); (-5=lowest, 10=highest;
                     default=2.5; I=10; E=7.5; H=5; S=2.5; C=0; P=-2.5; X=-5).
--, --stdout         write WAV output to STDOUT.
    --stdinname <t>  pseudo filename to use when input from STDIN.

Advanced Quality Options:

-A, --adaptive <n/t> modify settings for Sebastian Gesemann's adaptive noise
                     shaping method. takes a parameter to set the order of the
                     FIR filter, (32<=n<=96; default=64; multiple of 8 only);
                     "OFF" to disable adaptive shaping; "NOWARP" to disable
                     default frequency warping;
-a, --analyses <n>   set number of FFT analysis lengths, (2<=n<=6; default=3,
                     i.e. 32, 64 & 1024 samples. n=2, remove 32 sample FFT;
                     n>3 add 512; n>4, add 256; n>6, add 128) nb. FFT lengths.
                     stated are for 44.1/48kHz audio, higher sample rates will
                     automatically increase all FFT lengths as required.
-l, --limit <n>      set upper frequency limit to be used in analyses to n Hz;
                     (10000<=n<=20000; default=16000).
    --linkchannels   revert to original single bits-to-remove value for all
                     channels rather than channel dependent bits-to-remove.
    --maxclips <n>   set max. number of acceptable clips per channel per block;
                     (0<=n<=16; default=3,3,3,3,3,2,2,2,2,2,1,1,1,0,0,0).
-m, --midside        analyse 2 channel audio for mid/side content.
    --nodccorrect    disable DC correction of audio data prior to FFT analysis,
                     default=on; (DC offset calculated per FFT data set).
    --scale <n>      factor to scale audio by; (0.0625<n<=8.0; default=1).
-s, --shaping [n]    enable fixed noise shaping, takes optional parameter [n]
                     to allow user defined shaping proportion (0.0<=n<=1.0),
                     otherwise default to quality setting dependent value.
                     Disables adaptive noise shaping.
    --static <n>     set minimum-bits-to-keep-static to n bits (default=6;
                     7<=n<=28, limited to bits-per-sample - 4).
-U, --underlap <n>   enable underlap mode to increase number of FFT analyses
                     performed at each FFT length, (n = 2, 4 or 8, default=2).

Output Options:

    --bitdist        show distrubution of bits to remove.
    --blockdist      show distribution of lowest / highest significant bit of
                     input codec-blocks and bit-removed codec-blocks.
-d, --detail         enable per block per channel bits-to-remove data display.
-F, --freqdist       enable frequency analysis display of input data.
-H, --histogram      show sample value histogram (input, lossy and correction).
    --longdist       show long frequency distribution data (input/lossy/lwcdf).
    --perchannel     show selected distribution data per channel.
-p, --postanalyse    enable frequency analysis display of output and
                     correction data in addition to input data.
    --sampledist     show distribution of lowest / highest significant bit of
                     input samples and bit-removed samples.
    --spread [full]  show detailed [more detailed] results from the spreading/
                     averaging algorithm.
-W, --width <n>      select width of output options (79<=n<=255).

System Options:

-B, --below          set process priority to below normal.
    --low            set process priority to low.
-N, --nowarnings     suppress lossyWAV warnings.
-Q, --quiet          significantly reduce screen output.
-S, --silent         no screen output.

Special thanks go to:

David Robinson       for the publication of his lossyFLAC method, guidance, and
                     the motivation to implement his method as lossyWAV.

Horst Albrecht       for ABX testing, valuable support in tuning the internal
                     presets, constructive criticism and all the feedback.

Sebastian Gesemann   for the adaptive noise shaping method and the amount of
                     help received in implementing it and also for the basis of
                     the fixed noise shaping method.

Matteo Frigo and     for libfftw3-3.dll contained in the FFTW distribution
Steven G Johnson     (v3.2.1 or v3.2.2).

Mark G Beckett       for the Delphi unit that provides an interface to the
(Univ. of Edinburgh) relevant fftw routines in libfftw3-3.dll.

Don Cross            for the Complex-FFT algorithm originally used.
[/size]
Link to the hydrogenaudio wiki article (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV)

Suggested foobar2000 converter setup:

lossyFLAC:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyTAK:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyWV:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyWMALSL*:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wma
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wmaencode.exe - %d --codec lsl --ignorelength
Format is: lossless or hybrid
Highest BPS mode supported: 24

Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.

*: Uses the wmaencoder executable written by lvqcl. Found here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=90519&view=findpost&p=767754).
Title: lossyWAV 1.3.0 released
Post by: Steve Forte Rio on 2011-08-07 08:35:31
BRAVO!
Title: lossyWAV 1.3.0 released
Post by: halb27 on 2011-08-07 09:37:09
Great work! Thank you, Nick.
Title: lossyWAV 1.3.0 released
Post by: TBeck on 2011-08-11 01:11:57
Really cool! Thank you!
Title: lossyWAV 1.3.0 released
Post by: punkrockdude on 2011-08-11 16:11:15
Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.
Title: lossyWAV 1.3.0 released
Post by: vvneagleone on 2011-08-11 16:20:01
Nice work! For what purposes do you think this will be most commonly used?
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-11 17:44:52
Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.
I use deadbeef - it's a nice simple player.

Nice work! For what purposes do you think this will be most commonly used?
Not really got a firm opinion on that one. Some might use it as a space saving alternative to lossless (say, extreme or insane). Some certainly use it for digitized vinyl. I use it on my DAP of choice (extraportable). lossyWAV sits in its niche - it's not all things to all users - it's just another possible way for users to enjoy audio.
Title: lossyWAV 1.3.0 released
Post by: punkrockdude on 2011-08-11 20:21:28
Nick.C: Do you play FLAC/LossyWAV with your Android HD2? If so, what application do you use to play them? Regards.
I use deadbeef - it's a nice simple player.
Thank you. Regards.
Title: lossyWAV 1.3.0 released
Post by: larryfine on 2011-08-11 22:35:37
Great! Thank you very much!
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-14 11:44:49
I've just completed a full music collection transcode FLAC (-5) to lossyWAV (--quality extraportable --maxclips 0|FLAC -5 -b 512 -p 3072) resulting in 307 kbit/s (from 882 kbit/s).
Title: lossyWAV 1.3.0 released
Post by: zerowalker on 2011-08-17 04:16:14
Is this a Almost Lossless compression?
As that is what i get it to be after reading this.
I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.
Is there a GUI for this, as i am probably failing with commands, as i just tried to drag and drop randomly;P

Other than my stupidity, it seems very interesting, and if it is what i think, i would very likely experiment with it:)!
Title: lossyWAV 1.3.0 released
Post by: bryant on 2011-08-17 06:40:08
I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.

This is the expected behavior. However, what's not obvious is that the resulting WAV file has redundancies (lots of zeros in the less significant bits) that lossless compressors can take advantage of to get much better compression than they would on the original WAV file.

The sizes that you should be comparing are the original WAV files compressed with FLAC and the lossy.wav files compressed with FLAC.
Title: lossyWAV 1.3.0 released
Post by: zerowalker on 2011-08-17 19:29:55
I tried it out, but didn´t really get it to work, it just made lossy.wav that was the same size (or a bit larger i think) as the original wave.

This is the expected behavior. However, what's not obvious is that the resulting WAV file has redundancies (lots of zeros in the less significant bits) that lossless compressors can take advantage of to get much better compression than they would on the original WAV file.

The sizes that you should be comparing are the original WAV files compressed with FLAC and the lossy.wav files compressed with FLAC.


Okay, what settings or command is recommended?
Thank you:)
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-17 19:38:15
The settings for foobar2000 conversion to lossyFLAC, lossyWV and lossyTak are in post #2.
Title: lossyWAV 1.3.0 released
Post by: punkrockdude on 2011-08-18 00:37:59
The settings for foobar2000 conversion to lossyFLAC, lossyWV and lossyTak are in post #2.
I have had a problem with the LossyFLAC command line and had to seperate "-o%d" into "-o %d". Regards.
Title: lossyWAV 1.3.0 released
Post by: Nowings69 on 2011-08-18 08:29:51
Congratulations
Title: lossyWAV 1.3.0 released
Post by: punkrockdude on 2011-08-21 21:15:32
As an intrigued fan of LossyWAV may I ask what you are working on for the next major update or beta version? Regards.
Title: lossyWAV 1.3.0 released
Post by: Anakunda on 2011-08-21 21:17:08
HI! I'm curious what listening configuration is loosywaw -q standard targetted. I have achieved average bitrate at 398kbps. It's higher than lame preset insane, is the quality also higher? I guess this is suitable for lower class home hifi audio equipment. If I'm wrong correct me 
The auCDtect is pretty fooled  saying CDDA-100% for most tracks.
Anyway I'd like to see a public listening tst between loosless format, loosyWAV and MP3@320k. Hopefully someone gets the idea too 
Title: lossyWAV 1.3.0 released
Post by: drumliner on 2011-08-21 23:41:08
just tried lossywav to create some lossy.flac files and i'm having a strange issue with it, so some help would be most welcome. i'm using it through foobar, having copied the settings directly from this thread. while the encoding takes place as i would expect it to and the size of the resulting files seems appropriate, there's something wrong with the files as they're all reported by various players as being 405'47'' long, no matter what their real length is (except for a 1'' sample that came out supposedly 270' long). players are also unable to seek the files, reported bitrates are obviously abnormally low and audiotester reports the lossyflacs as being truncated at their real length.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-08-22 07:48:19
@punkrockdude: I'm not active developing at the moment - having a bit of time off. Identification and implementation of a psychoacoustic model may take some time....

@Anakunda: I would love to see the results too - but it takes a group with the time and inclination to carry out a listening test....

@drumliner: Apologies - settings updated. There was a change in foobar2000 that required the use of the "ignore-chunk-sizes" parameter in FLAC. I omitted to include the latest settings in post #2.
Title: lossyWAV 1.3.0 released
Post by: drumliner on 2011-08-22 13:05:56
thanks Nick. working properly now
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-09-04 21:22:44
Added: foobar2000 settings for producing lossyWMALSL. Thanks for developing wmaencoder, lvqcl!
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 14:43:01
I updated from 1.1.0c (which the wiki (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV) currently points to) to 1.3.0
and detected that it processes my test file 6 times slower than 1.1.0c (5x instead of 33x). Is this known? What is the reason? Thank you.
Title: lossyWAV 1.3.0 released
Post by: Northpack on 2011-09-19 15:13:00
I updated from 1.1.0c (which the wiki (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV) currently points to) to 1.3.0
and detected that it processes my test file 6 times slower than 1.1.0c (5x instead of 33x). Is this known? What is the reason? Thank you.


That's because of the adaptive noise shaping introduced in version 1.3.0. It gives a higher quality/bitrate ratio but slows down processing. You can disable it with "--adaptive OFF".
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 16:53:29
thank you Northpack. This increases speed to 9x, but still isn't close to the 33x I had with 1.1.0c

This is the command line I used, anything wrong?

Code: [Select]
lossywav1.3.0.exe input.wav -o outputdir --correction --quality portable --adaptive OFF
result: 9x speed

lossywav1.1.0c.exe input.wav -o outputdir --correction --portable
result: 33x speed


input file is a standard 16 bit stereo 44.1 KHz PCM wav
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 17:50:12
another issue I came across:

I always thought that there is unsuitable real-world wave input for lossyWAV (i.e. an input which won't result in significantly higher compression when prepocessed with lossyWAV). I just came across an example which seems to defy that rule. I am not sure why. Maybe user error?

The source file is the mp3 audio track of a documentary (so it's mostly human voice with occasional background music and f/x), converted to wav (16 bit stereo 44.1 KHz PCM).
I used --portable for lossyWAV and highest compression (8) for FLAC.

source.wav: 925 178 232 bytes
source.flac: 362 984 714 bytes (554 kbit/s)
source.1.1.0c.lossy.flac: 362 182 771 bytes (552 kbit/s) (-0,2%)
source.1.1.0c.lwcdf.flac: 10 507 100 bytes


The compressibility of the correction file suggests that the source wave file was not changed much. I inspected the correction file to confirm that. There are really very little changes and even those are hard to spot (it looked like a flatline and even when pointed to the peak values by Wavelab. Only after maximizing the vertical scale AND increasing gain by 400% I could barely see some not-zero-values).

The point is, preprocessing with lossyWAV saved only 0,2% in filesize (YES, I used 512 blocksize for flac),
while wiki (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV#Indicative_bitrate_reduction) gives these indicative expected results:

Quote
lossless flac: 854 kbit/s
--portable lossywav flac: 376 kbit/s (-56%)


This is quite different from my results. Both in terms of absolute kbit/s and relative savings (-56% vs -0,2%).


So, after updating to 1.3.0 I repeated this test:

source.wav: 925 178 232 bytes
source.flac: 362 984 714 bytes (554 kbit/s)
source.1.3.0.lossy.flac: 322 170 992 bytes (491 kbit/s) (-11%)


The -11% are of course a step forward from the next-to-nothing -0,2%,
but still it does not match the expectations for --portable settings, as indicated in wiki (-56%).

Sure, the wiki numbers are rough reference points only,
but these results are still way off.

Your thoughts?

(It was a documentary about paranormal phenomenons, maybe that's why  )
Title: lossyWAV 1.3.0 released
Post by: Northpack on 2011-09-19 18:30:37
thank you Northpack. This increases speed to 9x, but still isn't close to the 33x I had with 1.1.0c

I guess that's still due to the new (fixed) noise shaping algorithms. Maybe Nic can comment on that. If you want it fast, I suppose you should stay with 1.1.0c

I always thought that there is unsuitable real-world wave input for lossyWAV (i.e. an input which won't result in significantly higher compression when prepocessed with lossyWAV). I just came across an example which seems to defy that rule.

No surprise here. It's known that there are some kind of signals which FLAC itself compresses very efficiently (i.e. solo piano music). Those signals can actually have a worse compression ratio when preprocessed with lossyWAV, because it forces FLAC to use a very small block size (http://flac.sourceforge.net/format.html#blocking) of 512 samples, thus decreasing it's efficiency (while lossyWAV can't always make up for the loss in those cases, see the small correction file).
Title: lossyWAV 1.3.0 released
Post by: lvqcl on 2011-09-19 18:38:46
thank you Northpack. This increases speed to 9x, but still isn't close to the 33x I had with 1.1.0c

Make sure that lossyWAV uses fftw library (libfftw3-3.dll).
Title: lossyWAV 1.3.0 released
Post by: b66pak on 2011-09-19 20:09:33
indeed is faster with libfftw3-3.dll in lossyWAV folder...

about output file size:

i get -48% for a 43min 16bit 48kHz .wav with wma lossless...
_
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 20:11:55
edit:
Quote
I always thought that there is NO unsuitable real-world wave input for lossyWAV


If you want it fast, I suppose you should stay with 1.1.0c

supposing 1.3.0 is more efficient (and generally better quality thanks to noise shaping), I stay with 1.3.0,
I just thought I report that, maybe there is a problem Nic could look into
or some error on my behalf
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 20:30:48
Make sure that lossyWAV uses fftw library (libfftw3-3.dll).


results:

Code: [Select]
lossywav1.1.0c.exe input.wav -o outputdir --correction --portable
33x speed

lossywav1.3.0.exe input.wav -o outputdir --correction --quality portable
5x speed --> with libfftw3-3.dll: 7x speed

lossywav1.3.0.exe input.wav -o outputdir --correction --quality portable --adaptive OFF
9x speed --> with libfftw3-3.dll: 17x speed


still only ~half the 1.1.0c speed,
but much better than without dll, so thanks for the tip.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-09-19 20:40:17
Regarding processing speed, there are three things to note:

1) lossyWAV up to v1.2.0 used an approximation for SQRT and LOG2 functions. These routines were faster than the FSQRT and FYL2X x87 instructions at a cost of accuracy. These approximations were removed in the run-up to the release of 1.3.0 as I realised that the development problems with adaptive-noise-shaping that I was experiencing were due the use of approximations rather than the best accuracy available. This change increases processing time.

2) When the latest quality presets were developed I decided to enable the shortest FFT (32 samples, circa 0.73 msec for 44.1kHz output) for all quality presets. This change also increases processing time.

3) If either adaptive noise shaping or fixed noise shaping are enabled then the remove-bits procedure uses an FIR filter (default 96 tap) as part of the bit-removal process. This change also increases processing time.

Regarding content where processing using lossyWAV and compression using FLAC results in a larger file-size - this is not a new discovery - content was found quite early in the original development of lossyWAV where there was no advantage gained.

[edit] Two updates to the wiki article - indicative bitrate table updated and lossyWMALSL settings added. [/edit]
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 20:43:30
Regarding processing speed, there are 3 things to note: [...]

@Nick.C: many thanks for the explanation for the speed impact. In a nutshell it is due to better quality/higher accuracy. Knowing this, the user is certainly more comfortable with the slower speed :-)

Regarding content where processing using lossyWAV and compression using FLAC results in a larger file-size - this is not a new discovery - content was found quite early in the original development of lossyWAV where there was no advantage gained.

which type of content is that?

As said above, I would understand that with audio where flac alone compresses extremely well (piano solo, as indicated by Northpack),
but in my case ... ?


No surprise here. It's known that there are some kind of signals which FLAC itself compresses very efficiently (i.e. solo piano music).

@Northpack: From my posting, you know that the audio is nothing of that kind. It's a TV documentary, with voice, music, effects, etc. If that's not an input where lossyWAV can play out its strengths, then what is !(?) I think there is more to my findings than you want to accept. If - despite the nature of the audio input - the source.flac file were already almost "perfectly" compressed (leaving only a tiny margin of reduction of 0,2%), then why all of a sudden the new version yields 11% ? See. This suggests, that there was some problem with 1.1.0c. Once that is established, the question whether the 11% is too low compared to the average file size savings of 56% is legitimate.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-09-19 20:47:26
@ chrizoo: Please post a 30 second clip to allow corroboration of findings.

which type of content is that?
Content with reduced signal strength at some point in the range 20Hz to c.16kHz. Number of bits to remove is calculated based on the average signal strength and also minimum signal strength over the calculation range.
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-09-19 22:19:01
@Nick.C:
sounds like a rare and very exceptional case to me.
I PMed you concerning my test files.

i get -48% for a 43min 16bit 48kHz .wav with wma lossless...

@b66pak: I'd say that would be within the expected range around -56% (assuming you also used --portable).
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-09-21 20:23:43
Chris,

Thanks for taking the time to provide me with the sample. Below are a sox spectrogram of the lossless original and the post-analysis frequency "plots" (short and long) contained in the lossyWAV.log file. From these it can be seen that there is next to no signal above bins 24 of 32 and 178 of 512 (c. 16.5kHz and 15.5kHz respectively). This explains why you were getting no reduction in size - no bits were being removed due to no signal at the high end of the calculation range (varies upwards with quality setting from about 15.3kHz). As you can see, I have used --limit 14000 for this processing - the resultant lossy.flac file is 196MiB (315 kbit/s), the lwcdf.flac file is 260MiB.

So, yes this is a sample that will not benefit from lossyWAV processing using default settings. However, simply setting the upper calculation frequency limit to below the lowpass that seems to have been applied to the sample fixes this. It almost looks as is this sample has been resampled from 32kHz (NICAM frequency?).

[attachment=6708:source.lossless.png]

Code: [Select]
lossyWAV 1.3.0, Copyright © 2007-2011 Nick Currie. Copyleft.
Processed : 21/09/2011 19:59:22
Settings  : --quality extraportable --limit 14000
Filename  : source.wav
File Info : 44.10kHz; 2 channel; 16 bit
Results  : 3.8594 bits

Frequency Analysis of audio data.
+----+---------------------+---------------------+---------------------+------+
| Bin| Input Average (dBFS)| Lossy Average (dBFS)| LWCDF Average (dBFS)|Change|
+----+------+--------------+------+--------------+------+--------------+------+
| DC | -30.6|OOOOOOOOOO....| -30.6|OOOOOOOOOO....| -71.3|OOOO..........|  0.00|
|L  1| -26.5|OOOOOOOOOO....| -26.5|OOOOOOOOOO....| -67.1|OOOO..........|L 0.00|
|  2| -30.5|OOOOOOOOOO....| -30.5|OOOOOOOOOO....| -68.7|OOOO..........|  0.00|
|  3| -35.8|OOOOOOOOO.....| -35.8|OOOOOOOOO.....| -70.9|OOOO..........|  0.00|
|  4| -40.4|OOOOOOOO......| -40.4|OOOOOOOO......| -72.5|OOOO..........|  0.00|
|  5| -43.2|OOOOOOOO......| -43.2|OOOOOOOO......| -73.3|OOOO..........|  0.00|
|  6| -45.0|OOOOOOOO......| -45.0|OOOOOOOO......| -73.8|OOO...........|  0.01|
|  7| -46.2|OOOOOOO.......| -46.2|OOOOOOO.......| -74.1|OOO...........|  0.01|
|  8| -47.3|OOOOOOO.......| -47.2|OOOOOOO.......| -74.6|OOO...........|  0.01|
|  9| -48.1|OOOOOOO.......| -48.1|OOOOOOO.......| -74.9|OOO...........|  0.01|
|  10| -48.6|OOOOOOO.......| -48.6|OOOOOOO.......| -75.2|OOO...........|  0.01|
|  11| -49.7|OOOOOOO.......| -49.7|OOOOOOO.......| -75.7|OOO...........|  0.01|
|  12| -51.4|OOOOOOO.......| -51.4|OOOOOOO.......| -76.4|OOO...........|  0.01|
|  13| -53.1|OOOOOO........| -53.0|OOOOOO........| -77.2|OOO...........|  0.02|
|  14| -55.4|OOOOOO........| -55.4|OOOOOO........| -78.3|OOO...........|  0.02|
|  15| -56.6|OOOOOO........| -56.6|OOOOOO........| -79.1|OOO...........|  0.02|
|  16| -57.2|OOOOOO........| -57.2|OOOOOO........| -79.8|OOO...........|  0.02|
|  17| -60.6|OOOOO.........| -60.5|OOOOO.........| -80.6|OO............|  0.04|
|  18| -62.6|OOOOO.........| -62.5|OOOOO.........| -81.4|OO............|  0.06|
|  19| -64.2|OOOOO.........| -64.1|OOOOO.........| -82.1|OO............|  0.07|
|U 20| -65.8|OOOOO.........| -65.7|OOOOO.........| -82.9|OO............|U 0.09|
|  21| -67.5|OOOO..........| -67.4|OOOO..........| -83.9|OO............|  0.10|
|  22| -72.0|OOOO..........| -71.8|OOOO..........| -84.9|OO............|  0.22|
|  23| -84.2|OO............| -81.8|OO............| -85.4|OO............|  2.47|
|  24| -98.1|..............| -83.9|OO............| -83.9|OO............| 14.23|
|  25| -98.1|..............| -82.2|OO............| -82.2|OO............| 15.92|
|  26| -98.1|..............| -80.4|OOO...........| -80.4|OOO...........| 17.66|
|  27| -98.1|..............| -79.2|OOO...........| -79.2|OOO...........| 18.90|
|  28| -98.1|..............| -77.9|OOO...........| -77.9|OOO...........| 20.24|
|* 29| -98.1|..............| -77.2|OOO...........| -77.2|OOO...........|*20.87|
|  30| -98.1|..............| -77.1|OOO...........| -77.1|OOO...........| 21.03|
|  31| -98.1|..............| -79.9|OOO...........| -79.9|OOO...........| 18.17|
|FS/2| -98.1|..............| -83.7|OO............| -83.7|OO............| 14.44|
+----+------+--------------+------+--------------+------+--------------+------+
Legend: L = Lower Frequency Calculation Limit ( 0.020kHz);
        U = Upper Frequency Calculation Limit (14.000kHz);
        * = 20.000kHz

Frequency Analysis of audio data.
+----+---------------------+---------------------+---------------------+------+
| Bin| Input Average (dBFS)| Lossy Average (dBFS)| LWCDF Average (dBFS)|Change|
+----+------+--------------+------+--------------+------+--------------+------+
| DC | -33.0|OOOOOOOOO.....| -33.0|OOOOOOOOO.....| -75.9|OOO...........|  0.00|
|L  1| -28.9|OOOOOOOOOO....| -28.9|OOOOOOOOOO....| -71.8|OOOO..........|L 0.00|
|  2| -28.1|OOOOOOOOOO....| -28.1|OOOOOOOOOO....| -70.8|OOOO..........|  0.00|
|  3| -29.1|OOOOOOOOOO....| -29.1|OOOOOOOOOO....| -71.2|OOOO..........|  0.00|
|  4| -29.7|OOOOOOOOOO....| -29.7|OOOOOOOOOO....| -71.9|OOOO..........|  0.00|
|  5| -29.8|OOOOOOOOOO....| -29.8|OOOOOOOOOO....| -72.2|OOOO..........|  0.00|
|  6| -30.9|OOOOOOOOOO....| -30.9|OOOOOOOOOO....| -72.9|OOOO..........|  0.00|
|  7| -32.2|OOOOOOOOO.....| -32.2|OOOOOOOOO.....| -74.0|OOO...........|  0.00|
|  8| -34.7|OOOOOOOOO.....| -34.7|OOOOOOOOO.....| -75.1|OOO...........|  0.00|
|  9| -37.2|OOOOOOOOO.....| -37.2|OOOOOOOOO.....| -75.9|OOO...........|  0.00|
|  10| -38.1|OOOOOOOOO.....| -38.1|OOOOOOOOO.....| -76.5|OOO...........|  0.00|
|  11| -38.6|OOOOOOOO......| -38.6|OOOOOOOO......| -77.0|OOO...........|  0.00|
|  12| -38.9|OOOOOOOO......| -38.9|OOOOOOOO......| -77.3|OOO...........|  0.00|
|  13| -39.5|OOOOOOOO......| -39.5|OOOOOOOO......| -77.6|OOO...........|  0.00|
|  14| -40.2|OOOOOOOO......| -40.2|OOOOOOOO......| -77.8|OOO...........|  0.00|
|  15| -40.2|OOOOOOOO......| -40.2|OOOOOOOO......| -77.9|OOO...........|  0.00|
|  16| -40.7|OOOOOOOO......| -40.7|OOOOOOOO......| -78.1|OOO...........|  0.00|
|  17| -41.6|OOOOOOOO......| -41.6|OOOOOOOO......| -78.4|OOO...........|  0.00|
|  18| -41.6|OOOOOOOO......| -41.6|OOOOOOOO......| -78.6|OOO...........|  0.00|
|  19| -41.8|OOOOOOOO......| -41.8|OOOOOOOO......| -78.7|OOO...........|  0.00|
|  20| -42.4|OOOOOOOO......| -42.4|OOOOOOOO......| -79.0|OOO...........|  0.00|
|  21| -43.5|OOOOOOOO......| -43.5|OOOOOOOO......| -79.4|OOO...........|  0.00|
|  22| -44.9|OOOOOOOO......| -44.9|OOOOOOOO......| -79.7|OOO...........|  0.00|
|  23| -45.8|OOOOOOO.......| -45.8|OOOOOOO.......| -80.0|OOO...........|  0.00|
|  24| -46.5|OOOOOOO.......| -46.5|OOOOOOO.......| -80.2|OOO...........|  0.00|
|  25| -47.0|OOOOOOO.......| -47.0|OOOOOOO.......| -80.4|OOO...........|  0.00|
|  26| -47.3|OOOOOOO.......| -47.3|OOOOOOO.......| -80.6|OO............|  0.00|
|  27| -47.8|OOOOOOO.......| -47.8|OOOOOOO.......| -80.8|OO............|  0.00|
|  28| -48.4|OOOOOOO.......| -48.4|OOOOOOO.......| -80.9|OO............|  0.00|
|  29| -48.7|OOOOOOO.......| -48.7|OOOOOOO.......| -81.1|OO............|  0.00|
|  30| -48.9|OOOOOOO.......| -48.9|OOOOOOO.......| -81.2|OO............|  0.00|
|  31| -49.5|OOOOOOO.......| -49.5|OOOOOOO.......| -81.5|OO............|  0.00|
|  32| -50.0|OOOOOOO.......| -50.0|OOOOOOO.......| -81.9|OO............|  0.00|
|  33| -50.8|OOOOOOO.......| -50.8|OOOOOOO.......| -82.1|OO............|  0.00|
|  34| -51.9|OOOOOOO.......| -51.9|OOOOOOO.......| -82.2|OO............|  0.01|
|  35| -51.9|OOOOOOO.......| -51.9|OOOOOOO.......| -82.2|OO............|  0.01|
|  36| -51.8|OOOOOOO.......| -51.8|OOOOOOO.......| -82.3|OO............|  0.00|
|  37| -52.0|OOOOOOO.......| -52.0|OOOOOOO.......| -82.4|OO............|  0.00|
|  38| -52.5|OOOOOOO.......| -52.5|OOOOOOO.......| -82.5|OO............|  0.00|
|  39| -52.5|OOOOOOO.......| -52.5|OOOOOOO.......| -82.4|OO............|  0.00|
|  40| -52.2|OOOOOOO.......| -52.2|OOOOOOO.......| -82.4|OO............|  0.00|
|  41| -51.9|OOOOOOO.......| -51.9|OOOOOOO.......| -82.3|OO............|  0.00|
|  42| -52.3|OOOOOOO.......| -52.3|OOOOOOO.......| -82.3|OO............|  0.00|
|  43| -52.6|OOOOOO........| -52.6|OOOOOO........| -82.3|OO............|  0.01|
|  44| -52.6|OOOOOO........| -52.6|OOOOOO........| -82.3|OO............|  0.01|
|  45| -52.9|OOOOOO........| -52.9|OOOOOO........| -82.5|OO............|  0.01|
|  46| -54.0|OOOOOO........| -54.0|OOOOOO........| -82.7|OO............|  0.01|
|  47| -54.7|OOOOOO........| -54.7|OOOOOO........| -82.9|OO............|  0.01|
|  48| -55.0|OOOOOO........| -55.0|OOOOOO........| -83.0|OO............|  0.01|
|  49| -55.3|OOOOOO........| -55.3|OOOOOO........| -83.1|OO............|  0.01|
|  50| -55.5|OOOOOO........| -55.5|OOOOOO........| -83.1|OO............|  0.01|
|  51| -55.3|OOOOOO........| -55.3|OOOOOO........| -83.1|OO............|  0.01|
|  52| -55.2|OOOOOO........| -55.2|OOOOOO........| -83.0|OO............|  0.01|
|  53| -55.1|OOOOOO........| -55.1|OOOOOO........| -83.0|OO............|  0.01|
|  54| -54.8|OOOOOO........| -54.8|OOOOOO........| -82.9|OO............|  0.01|
|  55| -54.6|OOOOOO........| -54.6|OOOOOO........| -82.9|OO............|  0.01|
|  56| -54.9|OOOOOO........| -54.8|OOOOOO........| -83.0|OO............|  0.01|
|  57| -55.1|OOOOOO........| -55.1|OOOOOO........| -83.1|OO............|  0.01|
|  58| -55.3|OOOOOO........| -55.3|OOOOOO........| -83.1|OO............|  0.01|
|  59| -55.5|OOOOOO........| -55.5|OOOOOO........| -83.3|OO............|  0.01|
|  60| -55.7|OOOOOO........| -55.7|OOOOOO........| -83.5|OO............|  0.01|
|  61| -55.9|OOOOOO........| -55.9|OOOOOO........| -83.6|OO............|  0.01|
|  62| -56.3|OOOOOO........| -56.3|OOOOOO........| -83.6|OO............|  0.01|
|  63| -56.5|OOOOOO........| -56.5|OOOOOO........| -83.6|OO............|  0.01|
|  64| -56.5|OOOOOO........| -56.5|OOOOOO........| -83.6|OO............|  0.01|
|  65| -56.3|OOOOOO........| -56.3|OOOOOO........| -83.7|OO............|  0.01|
|  66| -56.6|OOOOOO........| -56.6|OOOOOO........| -83.8|OO............|  0.01|
|  67| -56.9|OOOOOO........| -56.9|OOOOOO........| -83.9|OO............|  0.01|
|  68| -56.9|OOOOOO........| -56.9|OOOOOO........| -83.9|OO............|  0.01|
|  69| -56.9|OOOOOO........| -56.9|OOOOOO........| -83.9|OO............|  0.01|
|  70| -57.2|OOOOOO........| -57.2|OOOOOO........| -83.9|OO............|  0.01|
|  71| -57.3|OOOOOO........| -57.3|OOOOOO........| -84.0|OO............|  0.01|
|  72| -57.3|OOOOOO........| -57.3|OOOOOO........| -84.0|OO............|  0.01|
|  73| -57.2|OOOOOO........| -57.2|OOOOOO........| -84.1|OO............|  0.01|
|  74| -57.3|OOOOOO........| -57.3|OOOOOO........| -84.1|OO............|  0.01|
|  75| -57.4|OOOOOO........| -57.4|OOOOOO........| -84.0|OO............|  0.01|
|  76| -57.3|OOOOOO........| -57.3|OOOOOO........| -84.0|OO............|  0.01|
|  77| -57.2|OOOOOO........| -57.1|OOOOOO........| -84.0|OO............|  0.01|
|  78| -57.4|OOOOOO........| -57.4|OOOOOO........| -84.1|OO............|  0.01|
|  79| -57.7|OOOOOO........| -57.7|OOOOOO........| -84.3|OO............|  0.01|
|  80| -57.8|OOOOOO........| -57.8|OOOOOO........| -84.3|OO............|  0.01|
|  81| -57.8|OOOOOO........| -57.8|OOOOOO........| -84.3|OO............|  0.01|
|  82| -57.8|OOOOOO........| -57.8|OOOOOO........| -84.3|OO............|  0.01|
|  83| -57.8|OOOOOO........| -57.8|OOOOOO........| -84.3|OO............|  0.01|
|  84| -57.8|OOOOOO........| -57.8|OOOOOO........| -84.3|OO............|  0.01|
|  85| -58.0|OOOOOO........| -58.0|OOOOOO........| -84.4|OO............|  0.01|
|  86| -58.4|OOOOOO........| -58.3|OOOOOO........| -84.6|OO............|  0.01|
|  87| -58.5|OOOOOO........| -58.5|OOOOOO........| -84.7|OO............|  0.01|
|  88| -58.8|OOOOOO........| -58.7|OOOOOO........| -84.8|OO............|  0.01|
|  89| -58.9|OOOOOO........| -58.9|OOOOOO........| -84.9|OO............|  0.01|
|  90| -59.1|OOOOOO........| -59.1|OOOOOO........| -84.9|OO............|  0.01|
|  91| -59.3|OOOOOO........| -59.3|OOOOOO........| -85.0|OO............|  0.01|
|  92| -59.6|OOOOO.........| -59.6|OOOOO.........| -85.1|OO............|  0.01|
|  93| -60.1|OOOOO.........| -60.1|OOOOO.........| -85.3|OO............|  0.01|
|  94| -60.3|OOOOO.........| -60.3|OOOOO.........| -85.4|OO............|  0.01|
|  95| -60.5|OOOOO.........| -60.5|OOOOO.........| -85.4|OO............|  0.01|
|  96| -60.6|OOOOO.........| -60.6|OOOOO.........| -85.5|OO............|  0.01|
|  97| -60.7|OOOOO.........| -60.7|OOOOO.........| -85.5|OO............|  0.01|
|  98| -61.0|OOOOO.........| -60.9|OOOOO.........| -85.6|OO............|  0.01|
|  99| -61.3|OOOOO.........| -61.2|OOOOO.........| -85.7|OO............|  0.02|
| 100| -61.5|OOOOO.........| -61.4|OOOOO.........| -85.9|OO............|  0.02|
| 101| -61.3|OOOOO.........| -61.3|OOOOO.........| -86.0|OO............|  0.02|
| 102| -61.1|OOOOO.........| -61.1|OOOOO.........| -86.0|OO............|  0.02|
| 103| -61.4|OOOOO.........| -61.4|OOOOO.........| -86.1|OO............|  0.01|
| 104| -61.9|OOOOO.........| -61.9|OOOOO.........| -86.2|OO............|  0.02|
| 105| -62.4|OOOOO.........| -62.4|OOOOO.........| -86.3|OO............|  0.02|
| 106| -62.8|OOOOO.........| -62.8|OOOOO.........| -86.4|OO............|  0.02|
| 107| -63.1|OOOOO.........| -63.1|OOOOO.........| -86.6|OO............|  0.02|
| 108| -63.6|OOOOO.........| -63.5|OOOOO.........| -86.8|OO............|  0.02|
| 109| -63.8|OOOOO.........| -63.8|OOOOO.........| -87.0|OO............|  0.02|
| 110| -64.1|OOOOO.........| -64.1|OOOOO.........| -87.2|OO............|  0.02|
| 111| -64.6|OOOOO.........| -64.6|OOOOO.........| -87.3|OO............|  0.02|
| 112| -64.9|OOOOO.........| -64.9|OOOOO.........| -87.4|OO............|  0.03|
| 113| -65.2|OOOOO.........| -65.1|OOOOO.........| -87.5|OO............|  0.03|
| 114| -65.1|OOOOO.........| -65.1|OOOOO.........| -87.6|OO............|  0.03|
| 115| -65.3|OOOOO.........| -65.3|OOOOO.........| -87.7|O.............|  0.02|
| 116| -65.8|OOOOO.........| -65.8|OOOOO.........| -87.8|O.............|  0.03|
| 117| -66.1|OOOOO.........| -66.1|OOOOO.........| -88.0|O.............|  0.03|
| 118| -66.5|OOOOO.........| -66.5|OOOOO.........| -88.1|O.............|  0.03|
| 119| -66.5|OOOOO.........| -66.5|OOOOO.........| -88.2|O.............|  0.03|
| 120| -66.5|OOOOO.........| -66.5|OOOOO.........| -88.2|O.............|  0.03|
| 121| -66.7|OOOO..........| -66.7|OOOO..........| -88.3|O.............|  0.03|
| 122| -67.0|OOOO..........| -67.0|OOOO..........| -88.3|O.............|  0.03|
| 123| -67.0|OOOO..........| -67.0|OOOO..........| -88.3|O.............|  0.03|
| 124| -66.9|OOOO..........| -66.8|OOOO..........| -88.4|O.............|  0.03|
| 125| -62.5|OOOOO.........| -62.5|OOOOO.........| -88.4|O.............|  0.01|
| 126| -61.3|OOOOO.........| -61.3|OOOOO.........| -88.6|O.............|  0.01|
| 127| -66.0|OOOOO.........| -66.0|OOOOO.........| -88.7|O.............|  0.02|
| 128| -67.9|OOOO..........| -67.8|OOOO..........| -88.9|O.............|  0.04|
| 129| -68.4|OOOO..........| -68.3|OOOO..........| -89.0|O.............|  0.04|
| 130| -68.5|OOOO..........| -68.5|OOOO..........| -89.1|O.............|  0.03|
| 131| -68.7|OOOO..........| -68.7|OOOO..........| -89.2|O.............|  0.04|
| 132| -69.1|OOOO..........| -69.1|OOOO..........| -89.3|O.............|  0.04|
| 133| -69.3|OOOO..........| -69.2|OOOO..........| -89.3|O.............|  0.04|
| 134| -69.5|OOOO..........| -69.5|OOOO..........| -89.4|O.............|  0.05|
| 135| -69.8|OOOO..........| -69.8|OOOO..........| -89.4|O.............|  0.05|
| 136| -70.1|OOOO..........| -70.1|OOOO..........| -89.6|O.............|  0.05|
| 137| -70.4|OOOO..........| -70.3|OOOO..........| -89.7|O.............|  0.05|
| 138| -70.7|OOOO..........| -70.7|OOOO..........| -89.9|O.............|  0.05|
| 139| -71.1|OOOO..........| -71.1|OOOO..........| -90.0|O.............|  0.06|
| 140| -71.3|OOOO..........| -71.3|OOOO..........| -90.2|O.............|  0.06|
| 141| -70.8|OOOO..........| -70.8|OOOO..........| -90.3|O.............|  0.05|
| 142| -71.2|OOOO..........| -71.2|OOOO..........| -90.4|O.............|  0.05|
| 143| -71.6|OOOO..........| -71.5|OOOO..........| -90.4|O.............|  0.05|
| 144| -71.8|OOOO..........| -71.7|OOOO..........| -90.5|O.............|  0.06|
| 145| -71.8|OOOO..........| -71.8|OOOO..........| -90.5|O.............|  0.06|
| 146| -71.5|OOOO..........| -71.5|OOOO..........| -90.5|O.............|  0.06|
| 147| -71.8|OOOO..........| -71.8|OOOO..........| -90.6|O.............|  0.06|
| 148| -72.5|OOOO..........| -72.4|OOOO..........| -90.7|O.............|  0.06|
| 149| -72.8|OOOO..........| -72.8|OOOO..........| -90.8|O.............|  0.07|
| 150| -73.0|OOOO..........| -73.0|OOOO..........| -91.0|O.............|  0.07|
| 151| -73.2|OOOO..........| -73.2|OOOO..........| -91.1|O.............|  0.08|
| 152| -73.3|OOOO..........| -73.3|OOOO..........| -91.3|O.............|  0.07|
| 153| -73.5|OOOO..........| -73.5|OOOO..........| -91.4|O.............|  0.08|
| 154| -73.6|OOO...........| -73.5|OOOO..........| -91.5|O.............|  0.08|
| 155| -73.9|OOO...........| -73.8|OOO...........| -91.5|O.............|  0.07|
| 156| -74.1|OOO...........| -74.1|OOO...........| -91.6|O.............|  0.08|
| 157| -74.5|OOO...........| -74.5|OOO...........| -91.7|O.............|  0.09|
| 158| -74.9|OOO...........| -74.8|OOO...........| -91.7|O.............|  0.09|
| 159| -74.7|OOO...........| -74.6|OOO...........| -91.8|O.............|  0.08|
| 160| -74.9|OOO...........| -74.8|OOO...........| -91.9|O.............|  0.09|
| 161| -75.4|OOO...........| -75.3|OOO...........| -92.0|O.............|  0.10|
| 162| -75.4|OOO...........| -75.3|OOO...........| -92.1|O.............|  0.09|
|U163| -75.1|OOO...........| -75.0|OOO...........| -92.3|O.............|U 0.08|
| 164| -75.3|OOO...........| -75.2|OOO...........| -92.5|O.............|  0.08|
| 165| -75.8|OOO...........| -75.7|OOO...........| -92.6|O.............|  0.08|
| 166| -76.0|OOO...........| -75.9|OOO...........| -92.7|O.............|  0.09|
| 167| -76.0|OOO...........| -75.9|OOO...........| -92.8|O.............|  0.09|
| 168| -76.1|OOO...........| -76.0|OOO...........| -92.9|O.............|  0.09|
| 169| -76.7|OOO...........| -76.6|OOO...........| -93.0|O.............|  0.10|
| 170| -77.1|OOO...........| -76.9|OOO...........| -93.1|O.............|  0.12|
| 171| -77.2|OOO...........| -77.0|OOO...........| -93.2|O.............|  0.11|
| 172| -77.6|OOO...........| -77.4|OOO...........| -93.3|O.............|  0.12|
| 173| -78.3|OOO...........| -78.1|OOO...........| -93.4|O.............|  0.13|
| 174| -78.9|OOO...........| -78.8|OOO...........| -93.6|O.............|  0.15|
| 175| -80.6|OOO...........| -80.4|OOO...........| -93.8|O.............|  0.20|
| 176| -84.5|OO............| -84.1|OO............| -94.0|O.............|  0.45|
| 177| -90.2|O.............| -88.8|O.............| -94.3|O.............|  1.44|
| 178| -98.1|..............| -93.0|O.............| -94.5|O.............|  5.08|
| 179| -98.1|..............| -94.6|O.............| -94.7|..............|  3.54|
| 180| -98.1|..............| -94.8|..............| -94.8|..............|  3.31|
| 181| -98.1|..............| -94.9|..............| -94.9|..............|  3.21|
| 182| -98.1|..............| -94.9|..............| -95.0|..............|  3.15|
| 183| -98.1|..............| -95.0|..............| -95.0|..............|  3.14|
| 184| -98.1|..............| -94.9|..............| -94.9|..............|  3.18|
| 185| -98.1|..............| -94.8|..............| -94.8|..............|  3.28|
| 186| -98.1|..............| -94.7|..............| -94.7|..............|  3.40|
| 187| -98.1|..............| -94.5|O.............| -94.5|O.............|  3.60|
| 188| -98.1|..............| -94.3|O.............| -94.3|O.............|  3.83|
| 189| -98.1|..............| -94.0|O.............| -94.0|O.............|  4.11|
| 190| -98.1|..............| -93.7|O.............| -93.7|O.............|  4.40|
| 191| -98.1|..............| -93.4|O.............| -93.4|O.............|  4.74|
| 192| -98.1|..............| -93.0|O.............| -93.0|O.............|  5.08|
| 193| -98.1|..............| -92.7|O.............| -92.7|O.............|  5.39|
| 194| -98.1|..............| -92.4|O.............| -92.4|O.............|  5.68|
| 195| -98.1|..............| -92.2|O.............| -92.2|O.............|  5.92|
| 196| -98.1|..............| -92.0|O.............| -92.0|O.............|  6.14|
| 197| -98.1|..............| -91.8|O.............| -91.8|O.............|  6.31|
| 198| -98.1|..............| -91.6|O.............| -91.6|O.............|  6.45|
| 199| -98.1|..............| -91.5|O.............| -91.5|O.............|  6.57|
| 200| -98.1|..............| -91.4|O.............| -91.4|O.............|  6.70|
| 201| -98.1|..............| -91.2|O.............| -91.3|O.............|  6.84|
| 202| -98.1|..............| -91.1|O.............| -91.1|O.............|  7.00|
| 203| -98.1|..............| -90.9|O.............| -90.9|O.............|  7.21|
| 204| -98.1|..............| -90.6|O.............| -90.6|O.............|  7.46|
| 205| -98.1|..............| -90.3|O.............| -90.3|O.............|  7.75|
| 206| -98.1|..............| -90.0|O.............| -90.0|O.............|  8.06|
| 207| -98.1|..............| -89.7|O.............| -89.7|O.............|  8.41|
| 208| -98.1|..............| -89.4|O.............| -89.4|O.............|  8.72|
| 209| -98.1|..............| -89.1|O.............| -89.1|O.............|  9.01|
| 210| -98.1|..............| -88.8|O.............| -88.8|O.............|  9.25|
| 211| -98.1|..............| -88.7|O.............| -88.7|O.............|  9.43|
| 212| -98.1|..............| -88.6|O.............| -88.6|O.............|  9.53|
| 213| -98.1|..............| -88.5|O.............| -88.5|O.............|  9.60|
| 214| -98.1|..............| -88.5|O.............| -88.5|O.............|  9.63|
| 215| -98.1|..............| -88.5|O.............| -88.5|O.............|  9.64|
| 216| -98.1|..............| -88.4|O.............| -88.4|O.............|  9.66|
| 217| -98.1|..............| -88.4|O.............| -88.4|O.............|  9.70|
| 218| -98.1|..............| -88.3|O.............| -88.3|O.............|  9.78|
| 219| -98.1|..............| -88.2|O.............| -88.2|O.............|  9.93|
| 220| -98.1|..............| -87.9|O.............| -87.9|O.............| 10.14|
| 221| -98.1|..............| -87.7|O.............| -87.7|O.............| 10.41|
| 222| -98.1|..............| -87.4|OO............| -87.4|OO............| 10.74|
| 223| -98.1|..............| -87.0|OO............| -87.0|OO............| 11.09|
| 224| -98.1|..............| -86.7|OO............| -86.7|OO............| 11.42|
| 225| -98.1|..............| -86.4|OO............| -86.4|OO............| 11.73|
| 226| -98.1|..............| -86.1|OO............| -86.1|OO............| 11.96|
| 227| -98.1|..............| -86.0|OO............| -86.0|OO............| 12.08|
| 228| -98.1|..............| -86.0|OO............| -86.0|OO............| 12.11|
| 229| -98.1|..............| -86.1|OO............| -86.1|OO............| 12.01|
| 230| -98.1|..............| -86.2|OO............| -86.2|OO............| 11.85|
| 231| -98.1|..............| -86.4|OO............| -86.4|OO............| 11.66|
|*232| -98.1|..............| -86.6|OO............| -86.6|OO............|*11.49|
| 233| -98.1|..............| -86.7|OO............| -86.7|OO............| 11.41|
| 234| -98.1|..............| -86.7|OO............| -86.7|OO............| 11.39|
| 235| -98.1|..............| -86.6|OO............| -86.6|OO............| 11.48|
| 236| -98.1|..............| -86.4|OO............| -86.4|OO............| 11.70|
| 237| -98.1|..............| -86.1|OO............| -86.1|OO............| 12.02|
| 238| -98.1|..............| -85.7|OO............| -85.7|OO............| 12.41|
| 239| -98.1|..............| -85.3|OO............| -85.3|OO............| 12.82|
| 240| -98.1|..............| -84.9|OO............| -84.9|OO............| 13.15|
| 241| -98.1|..............| -84.8|OO............| -84.9|OO............| 13.24|
| 242| -98.1|..............| -85.0|OO............| -85.0|OO............| 13.06|
| 243| -98.1|..............| -85.6|OO............| -85.6|OO............| 12.49|
| 244| -98.1|..............| -86.5|OO............| -86.5|OO............| 11.61|
| 245| -98.1|..............| -87.5|OO............| -87.5|OO............| 10.58|
| 246| -98.1|..............| -88.6|O.............| -88.6|O.............|  9.49|
| 247| -98.1|..............| -89.6|O.............| -89.7|O.............|  8.44|
| 248| -98.1|..............| -90.6|O.............| -90.6|O.............|  7.47|
| 249| -98.1|..............| -91.5|O.............| -91.5|O.............|  6.61|
| 250| -98.1|..............| -92.2|O.............| -92.2|O.............|  5.90|
| 251| -98.1|..............| -92.8|O.............| -92.8|O.............|  5.29|
| 252| -98.1|..............| -93.3|O.............| -93.3|O.............|  4.79|
| 253| -98.1|..............| -93.7|O.............| -93.7|O.............|  4.41|
| 254| -98.1|..............| -94.0|O.............| -94.0|O.............|  4.14|
| 255| -98.1|..............| -94.1|O.............| -94.1|O.............|  3.98|
|FS/2| -98.1|..............| -94.2|O.............| -94.2|O.............|  3.93|
+----+------+--------------+------+--------------+------+--------------+------+
Legend: L = Lower Frequency Calculation Limit ( 0.020kHz);
        U = Upper Frequency Calculation Limit (14.000kHz);
        * = 20.000kHz

[edit]Added bitrate of lossy.flac file.[/edit]
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-10-05 19:55:08
sorry for the late reply, I was gone.
Thank you for having looked into that issue.

How can we make generalizations based on this specific example? What conclusions can be drawn from it on a generic/general level?

I do not understand why a lowpass would affect the zeroing of LSBs and thus lossyWAVs inability to make a wav more compressible.
There is also no mention anywhere (lossyWAV wiki, cmdln help, etc.), that lowpassed audio does not work with LossyWAV.

Also: how can the sudden jump from 0,2% to 11% (see above) be explained?

thank you
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2011-10-05 21:40:42
From lossyWAV long help:
"Process Description:

lossyWAV is a near lossless audio processor which dynamically reduces the
bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise
to the processed output. The amount of permissible added noise is based on
analysis of the signal levels in the default frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a different --limit (in the range 10kHz to 20kHz).

For many audio signals there is little content at very high frequencies and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit."


So, in essence, processing content using lossyWAV with little or no content at one or more points in the calculation range (20Hz to c.16kHz) will result in less bits removed. This is not a lowpass - this is an outcome of the frequency range in which lossyWAV searches for the lowest frequency bin result. The lowest frequency bin result influences the number of bits that lossyWAV will remove from a particular block of audio. This is mentioned both in the long help and in the wiki (under "Quality Presets").

Lowpassed audio by definition has little or no content above a set frequency. If that frequency is below the upper calculation for lossyWAV then fewer (if any) bits will be removed.

I believe that your recent change from v1.1.0c to v1.3.0 explains the "sudden jump between 0.2% and 11%" as the quality presets were amended during the development of v1.3.0.

[edit] clarity [/edit]
Title: lossyWAV 1.3.0 released
Post by: chrizoo on 2011-10-05 22:32:45
first of all I'm sorry that I raised the issue when the answers seem to have been there already,
but I simply failed to understand them.

Thanks for explaining. Let me see if I understood:

In essence, for audio input with low levels above 16 kHz there won't be much LSB-zeroing (at least if --limit parameter is not specified) because there is a greater risk that the added noise might be audible. Correct?

What frequency range is the added noise in?
Or does this vary a lot depending on the audio input?

Quote
with little or no content at one or more points in the calculation range (20Hz to c.16kHz) will result in less bits removed.

... but not because it can't be done, BUT for "security reasons", right (as mentioned above)?

Quote
I believe that your recent change from v1.1.0c to v1.3.0 explains the "sudden jump between 0.2% and 11%" as the quality presets were amended during the development of v1.3.0.

oh, that explains it then. thanks.

This is not a lowpass - this is an outcome of the frequency range in which lossyWAV searches for the lowest frequency bin result.

I wasn't implying that lossyWAV does a lowpass, but that the audio input HAD been lowpassed previously (before parsing with lossyWAV).


What conclusions can non-experts users such as myself can draw from all this? Do we have to (or should we?) create a spectogram each time before processing audio with lossyWAV to see (A) if lossyWAV preparation will lead to a noteworthy file size reduction and (B) if we need to use a non-default value for the --limit parameter ?


(PS: This also means, that instead of sending you 90 minutes of lossless audio, I could just have sent you a jpeg image file of the spectogram? lol)
Title: lossyWAV 1.3.0 released
Post by: botface on 2011-10-06 09:49:08
What conclusions can non-experts users such as myself can draw from all this? Do we have to (or should we?) create a spectogram each time before processing audio with lossyWAV to see (A) if lossyWAV preparation will lead to a noteworthy file size reduction and (B) if we need to use a non-default value for the --limit parameter ?



What I do is process the file using Lossywav in the normal way. If the result doesn't give me the file size reduction I'm looking for I make a choice between a) Live with it, b) use a lossy encoder instead, c) stick with FLAC
Title: lossyWAV 1.3.0 released
Post by: 2Bdecided on 2011-10-06 12:54:29
You'll also get zero bits removed if there are deep+wide notch filter(s) anywhere in the audible band.

If there's a chunk of spectrum that you allow lossyWAV to analyse that's noise-free (or content-free), and it's wide enough not to get hidden during the spreading functions, then lossyWAV won't throw any bits away at all. It can't, because doing so would add noise above the noise floor - which is exactly what lossyWAV is designed to avoid.

I'm not sure if any release from Nick has ever taken the Minimum Audio Field / Absolute Threshold of Hearing into account, which would let you add more noise at higher frequencies even if they were included in the analysis. It doesn't help very often, and requires a (potentially very wrong) assumption about the replay level, so it's not really worth it IMO. It could have helped a little here though. Not as much as changing the limit or resampling to 32kHz though. If you were designing lossyWAV into a lossless codec, you'd put hidden automatic internal resampling in there for situations like this - i.e. any audio where downsampling is basically lossless gets downsampled when encoding, and then upsampled again when decoding.

My advice is just to live with it. If you ever find a lot of content like this (e.g. discs sourced from 128kbps mp3s) you could resample the lot to 32kHz yourself if you want to use lossyWAV and care about efficiency. Or specify a lower analysis frequency limit.

It's interesting that VBR mp3 can bloat when there's lots of information at higher frequencies, while lossyWAV can bloat when there's very little information at the highest analysed frequency. It would have been far easier to design mp3 to avoid this than it would been to design lossyWAV to avoid it.

Cheers,
David.
Title: lossyWAV 1.3.0 released
Post by: uncajesse on 2011-11-19 10:55:27
I fully updated this to work with v1.3, if anyone was interested in this mess.
lFLCDrop.v1.3.0.0.zip (http://dl.dropbox.com/u/16327919/hosted/lFLCDrop.v1.3.0.0.zip)
Title: lossyWAV 1.3.0 released
Post by: FreaqyFrequency on 2012-02-17 01:01:58
Hey Nick/all involved parties,

This is an absolutely fantastic tool, many thanks for implementing it.

I just ran a test with Absolution by Muse, and it introduced significant audible clipping on a few tracks.  My settings were -q X -l 10000.  These settings worked fine and proved to be transparent for a jazz sextet CD I tested a few days ago, but the results here are rather nasty.  Any suggestions for improving results, aside from upping the q parameter?

Thanks,

-Sean
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2012-02-17 20:23:41
Hi Sean,

You could try using scaling to reduce the amplitude of the signal (pre-processing). This should help to avoid clipping.

Can you please post a clip of an affected area of the sample (less than 30 seconds in length)?

[edit] .... The "-l 10000" is ignoring any signal dips over 10kHz. This will allow signal over 10kHz to potentially be swamped by the added noise (adaptively shaped using your command line). This could be your issue.

In this type of situation, a spectrogram can be useful to determine whether there is any signal above the selected limit (upper frequency limit used in calculations when determining lowest signal power FFT bin). [See the example at post #37]

I suggest that you try removing "-l 10000" from the command line and determining whether that solves the issue.  [/edit]

Nick.
Title: lossyWAV 1.3.0 released
Post by: softrunner on 2012-07-27 02:12:50
Is it possible to get lower quaity (and bitrate) than Extra Portable? I just want to hear what kind of distortions it brings into the sound. It is said about Extra Portable - "not fully transparent", but was is proved? Are there any problematic samples for lossyWAV 1.3.0?
Title: lossyWAV 1.3.0 released
Post by: halb27 on 2012-07-28 09:15:14
IMO the 'not fully transparent' statement is to be understood from lossyWAV history and from the high quality claims lossyWAV has.
lossyWAV is meant to use a lossless codec in a lossy bit saving way without making the simplifications audible. And it worked more or less from the start at the quality level 'standard' at an average bitrate of roughly 500 kbps. Nick.C then did a restless job in improving things even before applying adaptive noise shaping, and transparency was achieved at a bitrate of roughly 400 kbps.
Unfortunately there were not many people who did listening tests with resultant audible issues (easy to understand as lossyWAV quality was great from the very start), so nothing is known for sure.
After Nick.C introduced adaptive noise shaping the transparency border probably is much lower than 400 kbps, perhaps transparency is achieved even with 'extra portable' (I can't remember anybody having reported problems), but because of the lack of listening tests and because of lossyWAV's quality claims the description of the various quality levels is rather a bit conservative I guess.

Nick.C can report on this better, but as he didn't reply yet may be he's on holidays.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2012-07-29 19:30:07
As Horst has said, the aim during lossyWAV development was to produce a perceived quality (even at the lowest quality setting) that was likely* to be transparent.

I say likely as, as has also been mentioned, there were few (but very much appreciated) ABX testers during the development phase.

Problematic samples in lossyWAV tend to manifest themselves as samples that the process does not manage to remove many bits from - these generally have dips in the frequency range inside the calculation range where the lowest frequency response is determined.
Title: lossyWAV 1.3.0 released
Post by: Dynamic on 2012-07-30 10:03:19
Problematic samples in lossyWAV tend to manifest themselves as samples that the process does not manage to remove many bits from - these generally have dips in the frequency range inside the calculation range where the lowest frequency response is determined.


Do I take it that you mean the only 'problem' in these samples was only that the bitrate didn't get reduced compared to lossless? I don't recall reading of any non-transparency in those cases.



As an aside - I had until recently lost touch with Hydrogen Audio and lossyWAV for quite a few months. I've caught up with the 1.3 development thread (link available via lossyWAV Wiki (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV)), which was fascinating reading.

May I offer my belated congratulations and thanks to you, Nick, and also to David, Horst and Sebastian for their valuable contributions. I have to say that my ears, equipment and listening environment seem inadequate (from the time or two I tried) to detect the subtle non-transparencies that Horst reported on eig and furious killer samples at the lowest quality settings.

I remember the original idea was exciting - essentially smartly counteracting the bitrate bloat caused by excessive mastering loudness by exploiting the raised noise floor - something which had previously been partially achievable using wavgain to re-scale the audio, albeit with a permanent volume normalization, though that didn't bother me personally.

The compressed bitrates you've now achieved with a pre-processor that only zeroes some of the LSBs is quite remarkable. All this achieved with no filtering of the audio spectrum and no temporal smearing either. It's an awesome piece of work.

And yet there's probably still scope to go a little further and calculate psychoacoustic masking curves under which the shaped noise spectrum could be fitted, which would presumably also calculate bits-to-remove instead based on the area under the masking curve and thus overcome the 'problem' with lowpassed or (rarely) bandpassed material automatically.

Great stuff, anyway.
Title: lossyWAV 1.3.0 released
Post by: Dynamic on 2012-08-19 16:35:15
I had a read of the original thread on the lossyFLAC (http://www.hydrogenaudio.org/forums/index.php?showtopic=55522) idea (via link from foot of LossyWAV Wiki page (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV)) and it got me thinking, though I admit it's around the limits of my knowledge, so I could do with a sanity-check.

The crux of the issue is whether or not the adaptive noise shaping algorithm's influence on the rounding decision takes account of the actual error signal from neighbouring samples (including the enlarged negative-going error signal caused by lossyWAV's form of clipping and might therefore compensate for much of that error and its spectral and DC-bias effects in the rounding decisions made over a number of subsequent samples). My supposition is that it does use the actual error, but if it instead assumes, say, a typical statistical distribution of rounding errors, for example, my thoughts might be completely misguided. I suspect Nick.C or SebastianG will be best placed to put me right on this issue.

Early on, 2Bdecided spotted the issue with clipping of positive values near full scale that cannot be rounded up to +32768 because that number cannot be represented in 16-bit signed binary and we can't use +32767 and still keep the zeroed LSBs across the block to provide coding efficiency to the lossless encoder that recognises unused-bits.

There's a Max Clips Per Block variable in Nick.C's lossyWAV to limit the number of clippings allowed in any single codec block. If there are too many clips, the Bits To Remove value for that block of 512 samples is reduced and the block is re-processed to see if Max Clips Per Block is no longer exceeded or if Bits To Remove needs to be lowered again. The choice of Max Clips Per Block is decided by the quality preset or by user over-ride.

This helps statistically limit the amount of net DC offset that might be introduced into the added noise within the 512-sample block and helps ensure the energy of the additional error spikes must remain spread over a relatively broad-spectrum (i.e. short duration provides little power per spectral bin) and contribute only a small amount of total noise energy over the 512-sample block compared to the noise that should have been added by the lossyWAV rounding procedure when operating without clipping.

I'm wondering (and not sure if SebastianG or Nick.C is best placed to answer) whether the Adaptive Noise Shaping (ANS) enabled in version 1.3, involving the carrying-forward of rounding/truncation errors in such a way that they are spectrally-shaped, actually might have mitigated a lot of the original "clipping" concern (so long as there aren't an enormous number of clips, occuring at very periodic intervals to force a tonal error signal that might be unmasked).

In other words the fact that the round up or round down decision is biased by the errors in other samples in the 512-sample block and the spectral shape of the signal that the residual noise can be masked behind, may help prevent clipping errors from accumulating across the block as a whole and might even offset any DC bias by the end of the block.

My interpretation might be naive, in thinking of it a little like graphical bit-depth reduction, specifically Error Diffusion image dithering where rounding error is carried over to bias the rounding decision in subsequent pixels. Clearly in the lossyWAV ANS case, the error signal is also spectrally shaped by influencing the decision to round up rather than down or down rather than up according to the shaping filter.

Is it the case that the actual rounding error introduced is considered over the duration of the shaping filter so that for a sample where clipping occurs, the resulting negative DC offset and the additional error magnitude would tend to be counteracted by contributing to the rounding and noise-shaping decisions of numerous other samples in the same codec block rather than resulting in a net DC offset for the block and a spectrally-spread additional noise contribution (assuming each clipping error in brief, so resembles a Dirac Delta Function (negative-going one-sample spike with white spectrum or low power spectral density).

Then I thought to search on the issue and found this, so maybe I'm wrong:
In the 1.3 Development Thread, NickC wrote (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=81002&view=findpost&p=752505):
Thinking of reducing maximum clips-per-channel-per-codec-block to zero when adaptive shaping is active to reduce the possibility of the current method which allows a certain number of clips from "overloading" the noise-shaping filter(s).


If my interpretation is right, however, I wonder if the Maximum Clips Per Block value should actually be allowed to increase further when ANS is enabled or should a different measure of the residual (lwcdf) spectrum be used instead when forced down-rounding of high positive values has been detected in a block, for example to ensure that it's a reasonable match to the target noise spectrum without too much of a DC offset or any large spectral peaks above that targetted noise spectrum?

If I'm being hopelessly naive, I don't insist you explain it so that I understand. You're welcome to tell me that it just doesn't work like that (e.g. it doesn't take into account the actual error signal, just a statistical assumption). You probably have more important things to do with your time. My serious knowledge of digital signal processing stuff is limited to what I needed in work on very high rate single-bit discriminated instrumentation, rather than audio file manipulation / post-processing and didn't include noise shaping filters.
Title: lossyWAV 1.3.0 released
Post by: tycho on 2012-08-21 22:53:44
FYI. I've started translating the lossyWAV 1.3.0 code to C++, in the hope that it could be better optimized (including 64 bit) and supported on all platforms. All files are roughly translated, but it's still a bit left before they compile. If anyone wants to help out, please say so. (Nick, its a straight port, so you will immediately recognize the code, even if you don't know C/C++ much).

By the way, I found an interesting article from January/April called The faster-than-fast Fourier transform (http://web.mit.edu/newsoffice/2012/faster-fourier-transforms-0118.html), which may or may not be relevant.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2012-08-22 19:15:04
Hi Tycho,

Many thanks for tackling the conversion - I have been thinking of trying to learn C++ to allow me to translate lossyWAV, but you've beaten me to it!

I will have to start learning now so that I can contribute in some way to the translation process....

Not too sure about the new transform - it seems to work for sparse signals.

Nick.

[edit] Also, I suggest that we use this thread (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=96635&view=findpost&p=806153) for the development. [/edit]
Title: lossyWAV 1.3.0 released
Post by: LoRd_MuldeR on 2012-08-29 22:37:13
What about Unicode support for lossyWAV?

http://www.mediafire.com/file/cwyb3wf9dm23...eSupport-v1.zip (http://www.mediafire.com/file/cwyb3wf9dm2332p/lossyWAV-v1.3.0-UnicodeSupport-v1.zip)
Title: lossyWAV 1.3.0 released
Post by: tycho on 2012-08-30 09:07:02
What about Unicode support for lossyWAV?

http://www.mediafire.com/file/cwyb3wf9dm23...eSupport-v1.zip (http://www.mediafire.com/file/cwyb3wf9dm2332p/lossyWAV-v1.3.0-UnicodeSupport-v1.zip)

Thanks. The C++ version should have unicode support as well.
Title: lossyWAV 1.3.0 released
Post by: LoRd_MuldeR on 2012-08-30 14:29:59
Thanks. The C++ version should have unicode support as well.


Haven't looked at the C++ port yet. Is it "ready for production" yet?

Also: With "should have" you mean it does have Unicode support (for Win32) or adding Unicode support (for Win32) is on the TODO list?

In the latter case, I could add Unicode support to the C++ port, if that is desired...
Title: lossyWAV 1.3.0 released
Post by: tycho on 2012-08-30 15:16:51
Thanks. The C++ version should have unicode support as well.


Haven't looked at the C++ port yet. Is it "ready for production" yet?

Also: With "should have" you mean it does have Unicode support (for Win32) or adding Unicode support (for Win32) is on the TODO list?

In the latter case, I could add Unicode support to the C++ port, if that is desired...

No, there's still some delphi I/O calls not translated to their fstream counterparts, and when it finally links it probably won't run at first. You'll find the discussions about this in the another thread and in the uploads section.  When it comes to Unicode, it's not done so thanks for the offer. I thought maybe http://utfcpp.sourceforge.net/ (http://utfcpp.sourceforge.net/) could be used - seems small and nice.
Title: lossyWAV 1.3.0 released
Post by: LoRd_MuldeR on 2012-08-30 15:39:04
I have hacked a Unicode support into a large number of tools that obviosuly were written for Linux only or the author didn't care about Unicode.

Since I usually don't want to change char to wchar_t everywhere (although that would be the "native" way to support Unicode on Win32), because that requires various other changes and breaks Linux compatibility (yes, you can fix that by using Preprocessor/Macro magic, but it doesn't exacatly make the code nicer), I usually go the UTF-8 route.

As an example, you may look at my UTF-8 patch that got committed into Opus-Tools:
http://git.xiph.org/?p=opus-tools.git;a=co...cf1b7c1b0e0c934 (http://git.xiph.org/?p=opus-tools.git;a=commitdiff;h=58347273ab4c9e9d30ab8ffd62acbac348f7e0c8;hp=4cc06a7f0fe7cd93196e3078bcf1b7c1b0e0c934)

So it basically requires one extra .cpp/.h file containing a few support routines to hide the lack of UTF8 support in the Win32 API. You also need to replace fopen() with fopen_utf8().
Title: lossyWAV 1.3.0 released
Post by: DonP on 2012-12-16 16:40:31
No surprise here. It's known that there are some kind of signals which FLAC itself compresses very efficiently (i.e. solo piano music). Those signals can actually have a worse compression ratio when preprocessed with lossyWAV, because it forces FLAC to use a very small block size (http://flac.sourceforge.net/format.html#blocking) of 512 samples, thus decreasing it's efficiency (while lossyWAV can't always make up for the loss in those cases, see the small correction file).


I found this from a search after a piano track got bigger with lossywav (standard level for lossywav, and 5 for flac).  Maybe there's some place to mention this caveat in the wiki article? 

A similar piece on harpsichord (I guess due to more harmonics and less initial compression by flac) did shrink from 824 to 538 kb/s
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2012-12-16 19:15:40
Maybe there's some place to mention this caveat in the wiki article?

Sounds like a plan - it did come up in one of the development threads, but it bears repeating in the significantly more concise wiki article.
Title: lossyWAV 1.3.0 released
Post by: Adagio on 2013-02-22 23:48:47
A question more related to the foobar2000 converter.

Is there a way to also have the lossyWAV correction files written during the conversion to FLAC?

Simply adding --correction to the command line produces an error:

Encoder: C:\Windows\System32\cmd.exe
Extension : lossy.flac
Parameters: /d /c C:\"Program Files"\bin\lossywav - --correction --quality standard --silent --stdout|
            C:\"Program Files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes

Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-02-23 12:53:22
When creating correction files, --stdout cannot be used.

So, remove the
Code: [Select]
--stdout|C:\"Program Files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes
portion and replace it with
Code: [Select]
-C --stdinname "%d"
This will create WAV files rather than FLAC files. To compress to FLAC, import to foobar2000 and create a custom FLAC converter with "--keep-foreign-metadata" in the command line. If "--keep-foreign-metadata" is omitted then the files will not be able to be reinstated back to lossless as the 'FACT' chunks will be lost.
Title: lossyWAV 1.3.0 released
Post by: ozok on 2013-06-19 12:49:52
I've been trying to add lossyWAV support to TAudioConverter. I noticed something odd about how lossyWAV handles command line parameters. First I tried to use this simple command line parameters:
Code: [Select]
"input.wav" -q H


However I got an error "No input file specified". Then I remembered that some of the command line tools I wrote with Delphi had some strange behavior like this when I tried to use them within TAC. It looked as though lossyWAV ignored first parameter so I used this which works fine:
Code: [Select]
" " "input.wav" -q H


I wonder if this has something to do with Delphi or am I doing something wrong? I use a custom class derived from JVCL TJvCreateProcess class to run backends.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-06-19 20:52:45
Hi ozok,

I'm not sure. From memory, the Delphi version does not use paramstr(0) [usually points to the executable]. Depending how you call lossyWAV, the program may not be interpreting the parameters passed to it absolutely correctly. I'm glad that you found a fix though.

If you're working with correction files, please avoid lossyWAV beta 1.3.1a until I confirm that the correction file / merge system is working again.

Nick.
Title: lossyWAV 1.3.0 released
Post by: ozok on 2013-06-19 22:01:00
Thanks for the quick reply Nick.C.

I don't use correction files, I just use lossyWAV's output to feed to encoders. Everything else works fine at the moment (with 1.3.0), I'll try latest version now.

Regards.
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-10 23:37:06
What's wrong with this?

(Foobar2000 v1.2.9)

Quote
1 out of 1 tracks converted with major problems.

Source: "E:\Zenek\Metal\Epica - Discography\Albums\2003 - The Phantom Agony\05. Illusive Consensus.flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters) : "C:\Users\Bull\Desktop\05. Illusive Consensus.lossy.flac"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "C:\Windows\System32\cmd.exe" /d /c  F:\Dropbox\fb2k\lossyWAV\lossywav - -q X --silent --stdout|F:\Dropbox\fb2k\Flac\flac - -b 512 -5 -f -o"05. Illusive Consensus.lossy.flac" --ignore-chunk-sizes
  Working folder: C:\Users\Bull\Desktop\
 
  Conversion failed: The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters
Title: lossyWAV 1.3.0 released
Post by: db1989 on 2013-09-11 01:08:45
Just the first thing I saw, but try adding the missing space between “-o” and the output filename.
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-11 20:35:23
Just the first thing I saw, but try adding the missing space between “-o” and the output filename.

It doesn't worked.
But in the second post (http://www.hydrogenaudio.org/forums/index.php?showtopic=90104&st=0&p=765347&#entry765347) the code is same.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-09-11 20:57:55
Hi Mardel,

Could Drive C: be full? foobar2000 sometimes (especially when tagging a lot of data) creates temporary files which effectively double the disk space required to create the output file.
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-11 22:47:12
Hi Mardel,

Could Drive C: be full? foobar2000 sometimes (especially when tagging a lot of data) creates temporary files which effectively double the disk space required to create the output file.


Hi Nick!

27 GB Free on C.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-09-12 13:20:21
If you are using a beta version, please download the latest beta as I introduced an expiry date (to avoid potentially bad experimental parameters hanging around....).

The current beta expiry is the end of September. I will be releasing another beta in the next few days with another experimental parameter (bit-removal noise feedback checking in addition to clipping detection).
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-12 18:58:46
I used this 1.3.0 stable version.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-09-13 18:54:52
Hmmm.... Could you please try removing the FLAC portion of the command so that a ".lossy.wav" file is produced?

If that fails, could you please add "--adaptive off" to the lossyWAV command?
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-13 21:18:02
Hmmm.... Could you please try removing the FLAC portion of the command so that a ".lossy.wav" file is produced?

If that fails, could you please add "--adaptive off" to the lossyWAV command?

Thx!
But I'v got another problem (WASAPI component doesn't worked after install).
I reinstalled foobar and now works both (WASAPI, Lossywav encode).
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-09-13 21:49:16
Thanks Mardel, glad to hear that you have solved the problem.
Title: lossyWAV 1.3.0 released
Post by: darkbyte on 2013-09-14 14:57:43
I wonder if anyone thought about doing an ~320kbps listening test with LossyWAV extraportable involved. I have currently encoded my whole organized lossless music collection into lossyFLAC with LossyWAV 1.3.0 Extraportable (all source files which were not 44.1kHz 16 bit files were converted by foobar before passing them to LossyWAV. I have very few high resolution tracks though) and i have to say that i'm surprised to not notice artifacts or at least they are well hidden from me and the average bitrate is just 314kbps. I haven't done side-by-side comparison on tracks just saying this based on the fact that i have many tracks that i've listened to many times in both format so i should have notice if they differ. Although i like to do a correct ABX test (or should i use ABC/HR testing tool for this? i've never done something like this but i guess this will be my jump on the testing bandwagon, so i have to read a correct test setup tutorial first).

Is there any killer samples for LossyWAV to test? I've noticed that bitrate during playback for some spots may reach even the original one. Also there are some tracks which doesn't yield so much bitrate saving (like Break 'N Sweat from Skrillex which is over 564kbps even with extraportable setting, or some game music with lower than 16kHz lowpassing) but they still high quality so i don't mind the higher bitrate if it's really required.
Title: lossyWAV 1.3.0 released
Post by: sundance on 2013-09-14 18:51:23
Hello Nick,
seems I've got a similar issue like Mardel had (foobar 1.2.9 w/lossywav 1.3.0), although re-installation of fb2k didn't help here.

Setting up with the wiki recommended configuration gives the same error message Mardel had (Error 0x00000001).

Then I tried to simplify things by running this converter setup:
Code: [Select]
Encoder file: C:\Program Files\CmdlineTools\lossyWAV.exe
Parameters: - --outdir %d

Converter Status Report:
Code: [Select]
1 out of 1 tracks converted with major problems.

Source: "Z:\Robert Plant - Ship of Fools.wav"
  An error occurred while writing to file (The encoder has terminated prematurely with code 49 (0x00000031); please re-check parameters) : "Z:\TEMP\Robert Plant - Ship of Fools.lossy.wav"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "C:\Program Files\CmdlineTools\lossyWAV.exe" - --outdir "Robert Plant - Ship of Fools.lossy.wav"
  Working folder: Z:\TEMP\
  
  Conversion failed: The encoder has terminated prematurely with code 49 (0x00000031); please re-check parameters


Then w/o piping (and using a different target dir):
Code: [Select]
Encoder file: C:\Program Files\CmdlineTools\lossyWAV.exe
Parameters: %s --outdir %d

Status:
Code: [Select]
1 out of 1 tracks converted with major problems.

Source: "Z:\Robert Plant - Ship of Fools.wav"
  An error occurred while finalizing the encoding process (Object not found) : "D:\Robert Plant - Ship of Fools.lossy.wav"
  Conversion failed: Object not found


Any idea what I could try?

.sundance.

P.S. I tried to run it directly in a cmd.exe window:
Code: [Select]
"C:\Program Files\CmdlineTools\lossyWAV.exe" "Robert Plant - Ship of Fools.wav" --outdir "z:\temp\Robert Plant - Ship of Fools.lossy.wav"
lossyWAV 1.3.0, Copyright (C) 2007-2011 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
%lossyWAV Error% : Directory : z:\temp\Robert Plant - Ship of Fools.lossy.wav\ cannot be accessed.

Maybe there's something wrong with supplying a full path with --outdir
because this works:
Code: [Select]
"C:\Program Files\CmdlineTools\lossyWAV.exe" "Robert Plant - Ship of Fools.wav" --outdir "z:\temp"
lossyWAV 1.3.0, Copyright (C) 2007-2011 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
Filename  : Robert Plant - Ship of Fools.wav
File Info : 44.10kHz; 2 channel; 16 bit; 05:02.57, 50,90MiB
Progress  : 11.79%; 2.7503 bits; 11.40x; 00:03.12/00:26.54 ...
Title: lossyWAV 1.3.0 released
Post by: lvqcl on 2013-09-14 18:56:35
Any idea what I could try?

Don't use --outdir
Title: lossyWAV 1.3.0 released
Post by: sundance on 2013-09-14 19:04:17
@lvqcl:
Tried that, but fb2k complains about invalid encoder parameter without specifying %d
Title: lossyWAV 1.3.0 released
Post by: Mardel on 2013-09-14 19:05:07
...

My problem is quite unique.
If I reboot my pc I have to reinstall foobar agian because the problems come forward ASIO and WASAPI and pipe encode doesn't work.
But after reboot ASIO only works with an older compenent (1.2.6).
I don't know is windows (win 7 64 bit) or creative soundcard (xfi xtrememusic) related problem but pretty annoying.
Title: lossyWAV 1.3.0 released
Post by: lvqcl on 2013-09-14 19:15:33
Tried that, but fb2k complains about invalid encoder parameter without specifying %d

I forgot that LossyWAV doesn't have an option to specify just output file name.
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2013-09-14 19:31:44
There is....

Try "--stdinname %d".

[edit] This is then used to create the ".lossy.wav" output filename. [/edit]

[edit2] Re-reading the question, "--outdir" should be used to specify the path to the directory where the output file should be created.

Also, why not just use the destination dialog in the foobar2000 converter to specify your output directory? [/edit2]
Title: lossyWAV 1.3.0 released
Post by: sundance on 2013-09-15 07:31:34
Also, why not just use the destination dialog in the foobar2000 converter to specify your output directory?

That's what I did anyway, but still %d has to be supplied somewhere in the fb2k converter setup to be valid (and, sadly, something like "$directory(%d)" doesn't work either...)
But honestly, I don't need a "lossyWAV" setup; this issue just surfaced when I tried to create a working "lossyFLAC" converter setup (during my investigations), because I failed to make the wiki suggestion work here:
Code: [Select]
Encoder: C:\Windows\System32\cmd.exe
Extension : lossy.flac
Parameters: /d /c C:\"Program Files"\bin\lossywav - --quality standard --silent --stdout|
            C:\"Program Files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes
Title: lossyWAV 1.3.0 released
Post by: weepy on 2014-01-08 10:24:06
Hi there - wonderful project,

I was wondering if it was ever going to be possible to get a LINUX/MAC compilable version of lossyWAV ?

Perhaps this is something to do with the port to C from Delphi ?

Cheers
Title: lossyWAV 1.3.0 released
Post by: sidewalking on 2014-03-18 19:58:59
Also, why not just use the destination dialog in the foobar2000 converter to specify your output directory?

That's what I did anyway, but still %d has to be supplied somewhere in the fb2k converter setup to be valid (and, sadly, something like "$directory(%d)" doesn't work either...)
But honestly, I don't need a "lossyWAV" setup; this issue just surfaced when I tried to create a working "lossyFLAC" converter setup (during my investigations), because I failed to make the wiki suggestion work here:
Code: [Select]
Encoder: C:\Windows\System32\cmd.exe
Extension : lossy.flac
Parameters: /d /c C:\"Program Files"\bin\lossywav - --quality standard --silent --stdout|
            C:\"Program Files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes



Sounds like some of the issue I'm having with my foobar string.  I tried lossyFLAC first and then just modified the last part to try WavPack for lossyWv to troubleshoot if my syntax was wrong on the lossyWAV side or the output encoder side.  Same issue both ways, it's like it's looking for a concatenated filename first and now it's just not finding the output file (but this is all within one second...no encoding/decoding is taking place that I can tell.


Quote
1 out of 1 tracks converted with major problems.

Source: "C:\Users\scott\Documents\{APPS}\13 - Hallowed Ground.flac"
  An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters) : "C:\Users\scott\Music\How to Destroy Angels - (2013) Welcome Oblivion\13 - Hallowed Ground.lossy.wv"
  Additional information:
  Encoder stream format: 44100Hz / 2ch / 16bps
  Command line: "C:\Windows\System32\cmd.exe" /d /c C:\Users\scott\Documents\{APPS}\foobar2000\lossywav - -quality high --silent --stdout|C:\Users\scott\Documents\{APPS}\foobar2000\wavpack -hm --blocksize=512 --merge-blocks -i - "13 - Hallowed Ground.lossy.wv"
  Working folder: C:\Users\scott\Music\How to Destroy Angels - (2013) Welcome Oblivion\
 
  Conversion failed: The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters


Here's my command line and parameters:

Code: [Select]
/d /c C:\Users\scott\Documents\{APPS}\foobar2000\lossywav - -quality high --silent --stdout|C:\Users\scott\Documents\{APPS}\foobar2000\wavpack -hm --blocksize=512 --merge-blocks -i - %d


I'm sure it's a small thing like a space or missing something...just cannot get it right.  Any feedback or help I can get on my command here?  I use Foobar commands all the time, putting my binaries all in the root foobar directory (as you see up there).

Thanks for any advice!!!

Scott
Title: lossyWAV 1.3.0 released
Post by: akin0780 on 2014-03-18 20:20:09
This should work:

Place lossyWAV.exe and wavpack.exe in C:\foobar2000\ and use the following settings:

/d /c C:\foobar2000\lossywav.exe - -q 5 --silent --stdout|C:\foobar2000\wavpack.exe --blocksize=512 --merge-blocks -i - %d
Title: lossyWAV 1.3.0 released
Post by: Nick.C on 2014-03-18 20:58:02
Here's my command line and parameters:

Code: [Select]
/d /c C:\Users\scott\Documents\{APPS}\foobar2000\lossywav - -quality high --silent --stdout|C:\Users\scott\Documents\{APPS}\foobar2000\wavpack -hm --blocksize=512 --merge-blocks -i - %d


I'm sure it's a small thing like a space or missing something...just cannot get it right.  Any feedback or help I can get on my command here?  I use Foobar commands all the time, putting my binaries all in the root foobar directory (as you see up there).


It was indeed a small issue: you had "-quality" rather than "--quality" in the command line.
Title: lossyWAV 1.3.0 released
Post by: sidewalking on 2014-03-20 01:11:39
Here's my command line and parameters:

Code: [Select]
/d /c C:\Users\scott\Documents\{APPS}\foobar2000\lossywav - -quality high --silent --stdout|C:\Users\scott\Documents\{APPS}\foobar2000\wavpack -hm --blocksize=512 --merge-blocks -i - %d


I'm sure it's a small thing like a space or missing something...just cannot get it right.  Any feedback or help I can get on my command here?  I use Foobar commands all the time, putting my binaries all in the root foobar directory (as you see up there).


It was indeed a small issue: you had "-quality" rather than "--quality" in the command line.


Thanks to you both! I knew it would be something as small as that, easily overlooked and whoops you with the unwanted results! 
Title: lossyWAV 1.3.0 released
Post by: halb27 on 2014-06-22 22:08:53
I did a personal listening test a while ago with various problem samples (all tonal, with the exception of castanets) using mp3, aac, mpc at various bitrates. Out of interest I also included lossyWAV though the samples were targeting at mp3 and alike.
Out of the samples tested herding_calls (http://dl.getdropbox.com/u/2681777/ProblemSamples/herding_calls.flac) was the outstanding problem for lossyWAV. It took the economic quality (~400 kbps on average) to make it transparent for me.