Skip to main content
Topic: lossyWAV 1.4.2 released. (Read 14405 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lossyWAV 1.4.2 released.

lossyWAV 1.4.2 is released.

lossyWAV is a near lossless audio processor which dynamically reduces the bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise to the processed output. The added noise is adaptively shaped by default and can alternatively be fixed noise shaped or white noise depending on command line parameters. When lossyWAV processed output is compressed with certain lossless codecs (FLAC, Wavpack, Tak, LPAC, MPEG-4 ALS and WMA-Lossless) the bitrate of the output file is significantly* reduced compared to the lossless original.

*: on average, depending on content.

Changes from 1.4.0:

lossyWAV v1.4.2 15/09/2016
  • Modification to "-l, --limit" parameter. The upper bound for the upper calculation limit is now 45.35% of the sample-rate of audio processed. Entering a value that exceeds this will no longer give an error, however the selected limit will be reduced to the upper bound for the upper calculation limit appropriate to the sample-rate of the audio being processed. The lower bound for the upper calculation limit remains at 12.5kHz.
  • Introduction of a "hybrid" adaptive noise shaping "-s h, --shaping hybrid" that takes into account the results of all analyses performed and not just those for c.1.5ms and c.20ms.
  • The order in which additional analyses are introduced using the "-a <n>, --analyses <n>" parameter is changed for the "-A, --altspread" and "-s h, --shaping hybrid" options, The order of addition is broadly the reverse of the default, i.e. 512, 256, 128, 16 samples rather than 16, 128, 256, 512 samples.
  • The "-f, --freqdist" parameter now takes an optional "all" parameter and will display the frequency distribution for each of the calculated analyses. Replaces "--longdist" parameter.
  • Modifications to "-A, --altspread [n]" parameter. Now disables "old" integer bin spreading (still used in default spreading) and permits the user to reduce the effect that adjacent bins have on the calculated minimum result for each bin. Takes an optional parameter in the range 0<=n<=1 to six decimal places, default is 1.0.
  • Bugfix to "-c, --checking" parameter.
  • Reintroduction of the "--static" parameter to modify the minimum absolute number of bits to keep during processing;
  • Reintroduction of the "-D, --dynamic" parameter to modify the minimum dynamic number of bits to keep during processing.
  • Cosmetic issue regarding display of time corrected (thanks Atak_Snajpera for bringing this to my attention);
  • Bug in input scaling corrected (not an issue if scaling was not used previously while processing).
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

lossyWAV 1.4.2 released.

Reply #1
Code: [Select]
lossyWAV 1.4.2, Copyright (C) 2007-2016 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV is a near lossless audio processor which dynamically reduces the
bitdepth of the signal on a block-by-block basis. Bitdepth reduction adds noise
to the processed output. The amount of permissible added noise is based on
analysis of the signal levels in the default frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a different --limit (in the range 10kHz to 20kHz).

For many audio signals there is little content at very high frequencies and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate of the losslessly compressed output
dramatically for no perceptible benefit.

The noise added by the process is shaped using an adaptive method provided by
Sebastian Gesemann. This method, as implemented in lossyWAV, aims to use the
signal itself as the basis of the filter used for noise shaping. Adaptive noise
shaping is enabled by default.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-q, --quality <t>    where t is one of the following (default = standard):
    I, insane        highest quality output, suitable for transcoding;
    E, extreme       higher quality output, suitable for transcoding;
    H, high          high quality output, suitable for transcoding;
    S, standard      default quality output, considered to be transparent;
    C, economic      intermediate quality output, likely to be transparent;
    P, portable      good quality output for DAP use, may not be transparent;
    X, extraportable lowest quality output, probably not transparent.

Standard Options:

-C, --correction     write correction file for processed WAV file; default=off.
-f, --force          forcibly over-write output file if it exists; default=off.
-h, --help           display help.
-L, --longhelp       display extended help.
-M, --merge          merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>     destination directory for the output file(s).
-v, --version        display the lossyWAV version number.
-w, --writetolog     create (or add to) lossyWAV.log in the output directory.

Advanced Options:

-                    take WAV input from STDIN.
-c, --check          check if WAV file has already been processed; default=off.
                     errorlevel=16 if already processed, 0 if not.
-q, --quality <n>    quality preset (-5.0<=n<=10.0); (-5=lowest, 10=highest;
                     default=2.5; I=10.0; E=7.5; H=5.0; S=2.5; C=0.0; P=-2.5;
                     X=-5.0.
--, --stdout         write WAV output to STDOUT.
    --stdinname <t>  pseudo filename to use when input from STDIN.

Advanced Quality Options:

-A, --altspread [n]  disables 'old' sperading mechanism in favour of 'new'
                     mechanism (default spreading uses both 'old' and 'new'
                     mechanisms). Takes an optional parameter, n, which relates
                     to the proportion of adjacent bins taken into account when
                     calculating spread average for a particular bin (0<=n<=1;
                     default = 0.768544).
-a, --analyses <n>   set number of FFT analysis lengths, (2<=n<=7; default=3,
                     i.e. 32, 64 & 1024 samples. n = 2, remove 32 sample FFT;
                     n > 3 add 16; n > 4, add 128; n > 5, add 256, n > 6, add
                     512) n.b. FFT lengths stated are for 44.1/48kHz audio,
                     higher sample rates will automatically increase all FFT
                     lengths as required.
-D, --dynamic <n>    select minimum_bits_to_keep_dynamic to n bits (default
                     2.71 at -q X and 5.00 at -q I, 1.0 <= n <= 7.0.
    --feedback [n]   enable experimental bit removal / adaptive noise shaping
                     noise limiter. Tuning has been carried out at -q X and
                     should have a negligible effect at -q S. Optional setting
                     (0.0 <= n <= 10.0, default = 0.0) automatically selects
                     the following parameters (0 = least effect, 10 = most):
       r, round <n>  limit deviation from expected added noise due to rounding
                     (-2.0 <= n <= 2.0, default = 0.0).
       n, noise <n>  limit added noise due to adaptive noise shaping
                     (-2.5 <= n <= 7.5, default = 0.0).
       a, aclips <n> number of permissible exceedences of adaptive noise
                     shaping level limit (0 <= n <= 64, default = 32).
       A, alevel <n> adaptive noise shaping level limit (-2.0 <= n <= 2.5,
                     default = 0.0).
       V, verbose    enable more detailed feedback information in output.
-I, --ignore-chunk-sizes.
                     ignore 'RIFF' and 'data' chunk sizes in input.
-l, --limit <n>      set upper frequency limit to be used in analyses to n Hz;
                     (12500 <= n <= 20000*; default=16000).
                     *: for 44.1/48 kHz audio. Upper limit for audio of
                     other sampling rates is limited to sample-rate x 45.35%
    --linkchannels   revert to original single bits-to-remove value for all
                     channels rather than channel dependent bits-to-remove.
    --maxclips <n>   set max. number of acceptable clips per channel per block;
                     (0 <= n <= 16; default = 3,3,3,3,3,2,2,2,2,2,1,1,1,0,0,0).
-m, --midside        analyse 2 channel audio for mid/side content.
    --nodccorrect    disable DC correction of audio data prior to FFT analysis,
                     default=on; (DC offset calculated per FFT data set).
-n, --noskew         disable application of low frequency level reduction prior
                     to determination of bits-to-remove.
    --scale <n>      factor to scale audio by; (0.03125 < n <= 8.0; default=1).
-s, --shaping        modify settings for noise shaping used in bit-removal:
       a, altfilter  enable alternative adaptive shaping filter method.
       A, average    set factor of shape modification above upper calculation
                     frequency limit (0.00000 <= n <= 1.00000)
       c, cubic      enable cubic interpolation when defining filter shape
       e, extra      additional white noise to add during creation of filter
       f, fixed      disable adaptive noise shaping (use fixed shaping)
       h, hybrid     enable hybrid alternative to default adaptive noise shaping
                     method. Uses all available calculated analyses to create
                     the desired noise filter shape rather than only those for
                     1.5ms and 20ms FFT analyses.
       n, nowarp     disable warped noise shaping (use linear frequency shaping)
       o, off        disable noise shaping altogether (use simple rounding)
       s, scale <n>  change effectiveness of noise shaping (0 < n <= 2; default
                     = 1.0)
       t, taps <n>   select number of taps to use in FIR filter (8 <= n <= 256;
                     default = 64)
       w, warp       enable cubic interpolation when creating warped filter
    --static <n>     set minimum-bits-to-keep-static to n bits (default=6;
                     3<=n<=28, limited to bits-per-sample - 3).
-U, --underlap <n>   enable underlap mode to increase number of FFT analyses
                     performed at each FFT length, (n = 2, 4 or 8, default=2).

Output Options:

    --bitdist        show distrubution of bits to remove.
    --blockdist      show distribution of lowest / highest significant bit of
                     input codec-blocks and bit-removed codec-blocks.
-d, --detail         enable per block per channel bits-to-remove data display.
-F, --freqdist [all] enable frequency analysis display of input data. Use of
                     'all' parameter displays all calculated analyses.
-H, --histogram      show sample value histogram (input, lossy and correction).
    --perchannel     show selected distribution data per channel.
-p, --postanalyse    enable frequency analysis display of output and
                     correction data in addition to input data.
    --sampledist     show distribution of lowest / highest significant bit of
                     input samples and bit-removed samples.
    --spread [full]  show detailed [more detailed] results from the spreading/
                     averaging algorithm.
-W, --width <n>      select width of output options (79<=n<=255).

System Options:

-B, --below          set process priority to below normal.
    --low            set process priority to low.
-N, --nowarnings     suppress lossyWAV warnings.
-Q, --quiet          significantly reduce screen output.
-S, --silent         no screen output.

Special thanks go to:

David Robinson       for the publication of his lossyFLAC method, guidance, and
                     the motivation to implement his method as lossyWAV.

Horst Albrecht       for ABX testing, valuable support in tuning the internal
                     presets, constructive criticism and all the feedback.

Sebastian Gesemann   for the adaptive noise shaping method and the amount of
                     help received in implementing it and also for the basis of
                     the fixed noise shaping method.

Tyge Lovset          for the C++ translation initiative.

Matteo Frigo and     for libfftw3-3.dll contained in the FFTW distribution
Steven G Johnson     (v3.2.1 or v3.2.2).

Mark G Beckett       for the Delphi unit that provides an interface to the
(Univ. of Edinburgh) relevant fftw routines in libfftw3-3.dll.

Don Cross            for the Complex-FFT algorithm originally used.

Link to the hydrogenaudio wiki article

Suggested foobar2000 converter setup:

lossyFLAC:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d --ignore-chunk-sizes
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyTAK:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyWV:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24
lossyWMALSL*:
Code: [Select]
Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wma
Parameters: /d /c c:\"program files"\bin\lossywav - --quality standard --silent --stdout|c:\"program files"\bin\wmaencode.exe - %d --codec lsl --ignorelength
Format is: lossless or hybrid
Highest BPS mode supported: 24
Enclose the element of the path containing spaces within double quotation marks ("), e.g. C:\"Program Files"\directory_where_executable_is\executable_name. This is a Windows limitation.

*: Uses the wmaencoder executable written by lvqcl. Found here.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-



Re: lossyWAV 1.4.2 released.

Reply #4
Congrats with the release. First one in two years, I believe? This one seems to have gone under the radar.

I posted a new lossyflac command line tool here too, which may simplify the creation of lossy.flac and reconstruction of flac files from lossy and correction files in flac format.

Re: lossyWAV 1.4.2 released.

Reply #5
I have been looking into the correction files produced, and done some comparisons between correction files from lossywav v1.4.2, with both noise shaping on and off. I also compared with the ancient but very fast version 1.1.0 which does not have noise shaping at all.

Since I have not much education in signal processing, my questions is if it is possible to read anything useful out of the rest-signal in correction files that may influence the quality of the lossy.wav files?
E.g.
- the amount and strength of the signal in the correction file?
- the perceived sound and volume of the correction file?

I am aware that the rest signal in the correction file is apparent only where the signal in the original source is loud (high s/n ratio), so it should complete overshadow the missing bits, so to speak, which is the whole purpose. But even if the noise is better shaped in the newer versions, could the increased total amount of audible rest-signal be a problem (when lossy.flacs are about equally sized), or is it just a natural effect from the noise shaping?

I used Kalimba.mp3 converted to wav (included with windows 7) as an example (not necessarily representative, and yes I should have used a lossless source), but got results that were a bit surprising to me:

    61 394 920 Kalimba.wav
    36 318 060 Kalimba.flac
lossywav-1.4.2 -C -q C (default noise shaping on). Speed: 28.2x
    21 756 051 Kalimba.lossy.flac
    22 559 917 Kalimba.lwcdf.flac  --> rather loud and good amount of signal
    44 315 968 total                       --> 22% larger than the lossless flac.
lossywav-1.4.2 -C -q C -s o (noise shaping off). Speed: 71.4x
    21 831 125 Kalimba.lossy.flac
    21 986 755 Kalimba.lwcdf.flac  --> much lesser signal/sound, more high pitched.
    43 817 880 total                       --> 20.6% larger than the lossless flac.
lossywav-1.1.0 -C -P (no noise shaping). Speed: 115x
    20 520 340 Kalimba.lossy.flac
    19 974 629 Kalimba.lwcdf.flac --> least amount of sound/signals.
    40 494 969 total.                     --> 11.5% larger than the lossless flac.

Note the large difference between compression of the correction file between v1.1.0 and v1.4.2 even with noise shaping off.

Re: lossyWAV 1.4.2 released.

Reply #6
Please ignore my previous post, as the the numbers were wrong or not representative. The questions asked are therefore irrelevant. Latest lossyWAV works nicely as expected!

Re: lossyWAV 1.4.2 released.

Reply #7
Following lossyWAV 1.4.2 to here just to say: thank you, Nick.C.

Re: lossyWAV 1.4.2 released.

Reply #8
lossywav-for-posix updated.

I've got this to compile on OSX's clang compiler (El Capitan) - three changes required:
* remove all the 'd' suffixes from doubles when expression type is double
* change 'd' to 'f' suffix when expression type is float
* sincos needs to be changed to __sincos
* remove all link make parameters apart from -Wl

I've tested it and get bite-identical wav files after merging back.

I was considering forking the posix fork for a Mac specific version, but it struck me that the changes would probably work on posix systems too?


Re: lossyWAV 1.4.2 released.

Reply #9
lossywav-for-posix updated.

I've got this to compile on OSX's clang compiler (El Capitan) - three changes required:
* remove all the 'd' suffixes from doubles when expression type is double
* change 'd' to 'f' suffix when expression type is float
* sincos needs to be changed to __sincos
* remove all link make parameters apart from -Wl

I've tested it and get bite-identical wav files after merging back.

I was considering forking the posix fork for a Mac specific version, but it struck me that the changes would probably work on posix systems too?


OSX Version here for now:
https://github.com/falloutphil/lossywav-for-osx

Re: lossyWAV 1.4.2 released.

Reply #10
lossywav-for-posix updated.

I've got this to compile on OSX's clang compiler (El Capitan) - three changes required:

Thank you. And I apologize for not testing clang++ compatibility, at least in GNU/Linux.

Quote
* remove all the 'd' suffixes from doubles when expression type is double
Good catch.
Can you make a commit with this change alone? So I can give proper attribution to you?

Quote
* change 'd' to 'f' suffix when expression type is float
I don't think that's necessary. Removing the 'd' suffix should suffice.

Quote
* sincos needs to be changed to __sincos

Apparently, sincos() is a GNU extension. sin()/cos() can be used
instead (already fixed).

Quote
* remove all link make parameters apart from -Wl

Can you try the latest HEAD (after removing d suffixes, and report the exact errors you are getting?

Quote
I was considering forking the posix fork for a Mac specific version, but it struck me that the changes would probably work on posix systems too?

Yeah. There is no need for yet another branch (I wouldn't call my changes a fork).

Ideally, I would use a good build system (I like waf myself). And get rid of all those clang++ warnings. But I want to
keep the diff with upstream minimal. So I can keep merging new releases trivially.

Re: lossyWAV 1.4.2 released.

Reply #11
Quote from: 2012
Ideally, I would use a good build system (I like waf myself). And get rid of all those clang++ warnings. But I want to keep the diff with upstream minimal. So I can keep merging new releases trivially.

So, I went ahead and migrated to waf. Compiler/linker flags are all checked before being enabled. And the APIs I used in my patches are also checked.

The build process has been successfully tested in:
* GNU/Linux (g++, clang++).
* Mac OSX (clang++).
* FreeBSD (clang++).
* NetBSD (g++).
* OpenBSD (eg++).

Also, Homebrew users can install lossywav with a single command now.

As I already mentioned, I wouldn't fix clang++ warnings, or check for API support outside my patches, to keep the diff with upstream simple.

Re: lossyWAV 1.4.2 released.

Reply #12
If it is wanted, this spec file works in CentOS 7 and probably Fedora

Code: [Select]
%define revision .r1
# increment majrel when version or revision changes
%define majrel 0
# increment minrel when just tweaking spec file
%define minrel 1

Name:           lossywav
Version:        1.4.2p
Release:        %{majrel}%{?revision}%{?dist}.%{minrel}
Summary:        LossyWAV for POSIX

Group:          Applications/Multimedia
License:        GPLv3
URL:            https://github.com/MoSal/lossywav-for-posix
Source0:        lossywav-for-posix-%{version}%{?revision}.tar.gz

BuildRequires:  fftw-devel >= 3.3.0
BuildRequires:  python

%description
This is an attempt to build lossyWAV natively in POSIX systems.


%prep
%setup -q -n lossywav-for-posix-%{version}%{?revision}


%build
./waf configure --prefix=%{_prefix} --enable-fftw3
./waf build


%install
./waf install --destdir=%{buildroot}


%files
%doc README.lossyWAV.md README.md lossyWAV.png
%license gpl.txt
%{_bindir}/lossywav


%changelog
* Thu Dec 08 2016 Alice Wonder <buildmaster@librelamp.com> - 1.4.2p-0.r1.1
- Initial spec file, builds on CentOS 7

This seems like a really useful project, thank you to both the original developer and the POSIX port!

Re: lossyWAV 1.4.2 released.

Reply #13
I have a question. Are default quality presets affected in new version? For example  --quality high provides better output in 1.4.2 in comparison with 1.4.0?

Re: lossyWAV 1.4.2 released.

Reply #14
I have a question. Are default quality presets affected in new version? For example  --quality high provides better output in 1.4.2 in comparison with 1.4.0?
The vanilla quality presets did not change between 1.4.0 and 1.4.2.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

Re: lossyWAV 1.4.2 released.

Reply #15
Thank you Nick. But as far as I understand those presents are still "the most recommended setting" for average user who just want to have stable quality without experiments, or....you might propose some better alternative --quality high for me? :))
Again thank you.

P.S. I see suggested converter setup above, but I see suggested not as 100% recommend. At least in my English knowledge that is it. :-)

Re: lossyWAV 1.4.2 released.

Reply #16
....you might propose some better alternative --quality high for me? :))
Only you can determine which settings are appropriate to your needs / ability to discern the noise added by the lossyWAV processing.

--standard remains as the suggested setting.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

Re: lossyWAV 1.4.2 released.

Reply #17
I think w64 output is broken. lossywav itself can parse it. But not ffmpeg or sox (using libsndfile backend, I think).

I thought it was my fault. But then I tested the official binaries (well, using wine). And got
the same result.

On another note, this might interest you Nick.

Re: lossyWAV 1.4.2 released.

Reply #18
Thanks for letting me know - I'll take a look.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

Re: lossyWAV 1.4.2 released.

Reply #19
Results of investigation into ffmpeg / sox and W64, so far:

ffmpeg (613f789, 2017-12-25) processes lossyWAV piped output but does not process lossyWAV processed files.

sox 14.4.2 processes neither lossyWAV processed files nor piped input.

I also looked at WavPack and FLAC:

WavPack 5.1.0 will compress lossyWAV processed files but will not process piped output from lossyWAV.

FLAC 1.3.2 will compress both lossyWAV processed files and piped output.

Wondering if this has something to do with the "fact" chunk - for ffmpeg, at least.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

Re: lossyWAV 1.4.2 released.

Reply #20
The w64 bug have been fixed in FFmpeg.

Re: lossyWAV 1.4.2 released.

Reply #21
Quote
Results of investigation into ffmpeg / sox and W64, so far:

I didn't see this earlier. Thank you for investigating.

Quote
The w64 bug have been fixed in FFmpeg.

Confirmed.

 

Re: lossyWAV 1.4.2 released.

Reply #22
The w64 bug have been fixed in FFmpeg.

Thanks for that information - much appreciated. :)
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-

 
SimplePortal 1.0.0 RC1 © 2008-2018