HydrogenAudio

Following the release of lossyWAV 1.0.0b, I feel it is time to kick off development of the next minor release.

Items currently on the list for inclusion in 1.x.0:

[blockquote]1.1.0: STDIN input;
1.1.0: STDOUT output;
1.1.0: Channel independent bit removal;
1.1.0: Reversion to same bits-to-remove for all channels;
1.1.0: Noise shaping;
1.2.0: Checking of S (=L-R) channel for matrix surround content;[/blockquote]
If you have any ideas, suggestions, code optimisations, etc, please post them here.
[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]

lossyWAV 1.1.0b, Copyright (C) 2007,2008 Nick Currie. Copyleft.

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.

Process Description:

lossyWAV adds white noise to the processed output. The amount of added noise is
based on analysis of the signal levels in the frequency range 20Hz to 16kHz.

If signals above the upper limiting frequency are at an even lower level, they
can be swamped by the added noise. This is usually inaudible, but the behaviour
can be changed by specifying a higher --limit (in the range 16kHz to 20kHz).

For many audio signals, there is little content at very high frequencies, and
forcing lossyWAV to keep the added noise level lower than the content at these
frequencies can increase the bitrate dramatically for no perceptible benefit.

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-I, --insane        highest quality output, suitable for transcoding;
-E, --extreme       high quality output, also suitable for transcoding;
-S, --standard      default quality output, considered to be transparent;
-P, --portable      good quality output for DAP use. Not considered to be fully
                    transparent, but considered fit for its intended purpose.

Standard Options:

-c, --check         check if WAV file has already been processed; default=off.
                    errorlevel=16 if already processed, 0 if not.
-C, --correction    write correction file for processed WAV file; default=off.
-f, --force         forcibly over-write output file if it exists; default=off.
-h, --help          display help.
-L, --longhelp      display extended help.
-M, --merge         merge existing lossy.wav and lwcdf.wav files.
-o, --outdir <t>    destination directory for the output file(s).
-v, --version       display the lossyWAV version number.

Advanced Options:

-                   if filename="-" then WAV input is taken from STDIN.
    --blockdist     show distribution of lowest significant bit of input
                    codec-blocks and bit-removed codec-blocks.
-D, --dither <n>    enable variable PDF dither of output; default=off;
                    0 = rectangular; 1 = triangular; 0.5 = half way between.
-l, --limit <n>     set upper frequency limit to be used in analyses to n Hz;
                    (16000<=n<=20000), default = 16000.
    --linkchannels  Revert to original single bits-to-remove value for all
                    channels rather than channel dependent bits-to-remove.
-q, --quality <n>   quality preset (10=highest quality, 0=lowest bitrate;
                    default = --standard = 5; --insane = 10; --extreme = 7.5;
                    --portable = 2.5)
    --sampledist    show distribution of lowest significant bit of input
                    samples and bit-removed samples.
    --scale <n>     scaling factor from WaveGain, etc; (0.0<n<=8.0),default=1.0
-s, --shaping <n>   enable fixed noise shaping; (0.00<=n<=1.00); default=q/10;
                    0.00 = off, 1.00 = 100% effectiveness, 0.50 = 50%, etc.
    --stdinname <t> pseudo filename to use when input from STDIN.
    --stdout        write processed WAV output to STDOUT.
-w, --writetolog    create (or append to) lossyWAV.log in the output directory.

System Options:

-B, --below         set process priority to below normal.
-d, --detail        enable detailed bits-to-remove information output mode
    --low           set process priority to low.
-n, --nowarnings    suppress lossyWAV warnings.
-Q, --quiet         significantly reduce screen output.
    --silent        no screen output.

Special thanks:

David Robinson      for the publication of his lossyFLAC method, guidance, and
                    the motivation to implement the method as lossyWAV.
Horst Albrecht      for ABX testing, valuable support in tuning the internal
                    presets, constructive criticism and all the feedback.
Sebastian Gesemann  for the noise shaping coefficients and help in using them
                    in the lossyWAV noise shaping implementation.
Don Cross           for the Complex-FFT algorithm used.

[/size]
Link to the hydrogenaudio wiki article (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV)

Suggested foobar2000 converter setup:

lossyFLAC:

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\flac - -b 512 -5 -f -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyTAK:

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.tak
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\takc -e -p2m -fsl512 -ihs - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

lossyWV:

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.wv
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --stdout|c:\"program files"\bin\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is: lossless or hybrid
Highest BPS mode supported: 24

There is a known problem within foobar2000 (although more likely to do with cmd.exe itself) when running an executable within the cmd.exe command line from a path which includes spaces. The suggested fix for this is to enclose the element of the path which contains spaces within double quotation marks ("), e.g. c:\"program files"\directory_where_executable_is\executable_name

Change log 1.1.0c: 30/04/2009
Exactly as 1.1.0b except that the WINE incompatibility issue has been fixed.

Executable here (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5066).
Source here (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5067).

[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]Change log 1.1.0b: 03/08/08
FFT lengths will now increase for higher bitrate audio, i.e. 88.2/96kHz, 176.4/192kHz and 352.8/384kHz;
improved logfile output and --detail output;
reference threshold constants for rectangular dither and triangular dither have been calculated so added noise should be the same for dither off and any dither level between 0 and 1 - the number of bits-to-remove will however reduce with "increasing" dither.

Change log 1.1.0: 12/07/08
Certain advanced parameters removed for final release.

Change log 1.0.1x RC4: 12/07/08
Final release candidate prior to release of 1.1.0

Change log 1.0.1w RC3: 02/07/08
Code tidied up a bit more (yet again....);
--wine parameter modified to stop the program using Windows API function calls when using piped input (should hopefully stop crashing under Wine).

Change log 1.0.1v RC2: 30/06/08
Code tidied up a bit more (again....);
--wine parameter implemented to stop the program using the GetLastError Windows API call when using piped input (should stop crashing under Wine).

Change log 1.0.1u RC1: 20/06/08
Code tidied up a bit more;
--bitdist parameter introduced to allow user to "examine" the distribution of lowest set bit on a codec-block by codec-block basis, channels treated separately.

Change log beta 1.0.1t: 11/06/08
Revision to STDIN handling - bug found where last codec-block read from foobar2000 using STDIN input was not being written to the output file.

Change log beta 1.0.1s: 09/06/08
Revision to STDIN handling. Now (fingers crossed) should work successfully inside Foobar2000;
Code and help tidied up;
Dither function fixed and augmented. Taking on board a statement by SG with respect to using a dither function somewhere between rectangular (rand - 0.5) and triangular (rand-0.5)+(rand-0.5), i.e. (rand-0.5)+s*(rand-0.5) {0<=s<=1}. s=0 = rectangular dither; s=1 = triangular dither. -D, --dither now requires a supplementary <n> in the range 0<=n<=1.

Change log beta 1.0.1r: 03/06/08
Implementation of fast square root function using lookup tables for fxtract(ed) exponent and mantissa of input value;
--scale parameter corrected to accepted values in the range 0<n<=8.

Change log beta 1.0.1q: 30/05/08
Codec-block overflow bug (when codec-block-size=4096) corrected;

Change log beta 1.0.1p: 29/05/08
Quality synonym automatic noise shaping bug corrected;

Change log beta 1.0.1o: 29/05/08
Spreading function spread-zones and spreading-function string modified to allow finer control of high frequency zones;
Code "recovered" from 1.0.1e after a minor hardware failure

Change log beta 1.0.1n: 26/05/08
Implementation of -H, --highskew <n> parameter. Functionally identical to the internal skewing applied to the FFT results (-36dB @ 20Hz to 0dB at 3.45kHz) except applied from 3.45kHz upwards. Valid in the range 0 to 36 (0=default=no high skew applied).

Change log beta 1.0.1m: 25/05/08
reintroduction of max-inter-block-change implementation limits increase in bits-to-remove between codec-blocks to 1 bit.

Change log beta 1.0.1k: 23/05/08
static maximum_bits_to_remove limitation re-applied in serial with dynamic maximum_bits_to_remove limitation;
Automatic noise shaping now applied using a shaping-factor of quality-level / 10.

Change log beta 1.0.1j: 23/05/08
-q <n> quality selection moved to advanced settings;
-E, --excessive changed to --extreme; -I, --insane added, equivalent to -q 10;
--lowpass changed to -l, --limit in keeping with discussion;
Process Description text added to --longhelp.

Change log beta 1.0.1i: 23/05/08
-q <n> quality selection moved to advanced settings;
-E, --excessive; -N, --normal; -P, --portable quality "names" introduced following discussion in the development thread. These equate to -q 7.5; -q 5.0 and -q 2.5 respectively.

Change log beta 1.0.1h: 20/05/08
minimum bits to keep values changed for -q 0 and -q 1 to 2.333 and 2.667 respectively.

Change log beta 1.0.1g: 22/05/08
Reference_threshold > threshold_index > bits_to_remove calculation refined;
spreading function string modified;
minimum bits to keep values changed for -q 0 and -q 1;
--writetolog (-w) parameter implemented to write minimal output to "lossyWAV.log". Appends to existing file if already exists;
--lowpass <n> parameter re-implemented to allow users to set upper frequency limit of the range that lossyWAV uses in its analyses (16000<=n<=24000).

Change log beta 1.0.1f: 20/05/08
Filenaming logic "improved" when STDIN and STDOUT used together.

Change log beta 1.0.1e: 19/05/08
STDIN / STDOUT mode tidied up. Use the following as a flossy.bat file for foobar conversion:

@echo off
z:\bin\lossyWAV %1 --low --nowarnings --quiet %3 %4 %5 %6 %7 %8 %9 --stdout|z:\bin\flac - -5 -f -b 512 -o%2

Unfortunately, due to the nature of piped input to FLAC, the lossyWAV 'fact' chunk is lost. This means no record is kept within the file that is has been processed with lossyWAV (however, the lower the quality setting of the processing, the more likely the bitrate will be an obvious indicator that the file has indeed been processed with lossyWAV);
Minor error found and amended in revised remove_bits procedure, no minimum_bits_to_keep value was being applied, although this has little impact at -q >= 2;
New parameter --linkchannels implemented to revert to old remove_bits method whereby all channels share the same bits_to_remove. Implementing this, I found an error in the original which was forcing more bits to be lost to clipping prevention than should have been (i.e. output was more conservative).

Change log beta 1.0.1d: 18/05/08
STDIN / STDOUT mode modified again (use '-' as a filename to enable STDIN input, --stdout to enable STDOUT output).
Console output has been redirected to 'con', rather than STDOUT.

Change log beta 1.0.1c: 16/05/08
STDIN / STDOUT mode modified again (use '-' as a filename to enable STDIN input).

Change log beta 1.0.1b: 15/05/08
Channel independent bit-removal implemented;
STDIN / STDOUT mode modified - still very much a work in progress.

Change log beta 1.0.1: 14/05/08
STDIN / STDOUT mode commenced.[/size]

I've been playing with STDIN / STDOUT. Setting input-file-name and output-file-name, using --silent and the following command line in a DOS box:

for %a in (..\_swav16\*.wav) do lossywav - -q 0 -S 0 --silent <"%a" |flac - -b 512 -5 --sign signed --bps 16 --sample-rate 44100 --channels 2 --endian little -o"%~na.lossy.flac" -f

I get a set of lossyFLAC files. However this method does not allow retention of the 'fact' chunk as the --keep-foreign-metadata FLAC parameter is incompatible with the - parameter to indicate STDIN input to FLAC.

Speed is better due to much less HDD access.

c:\data_nic\bin\lossywav - -q 0 -S 0 --silent |c:\data_nic\bin\flac - -b 512 -5 --sign signed --bps 16 --sample-rate 44100 --channels 2 --endian little -o"%d" -f

....does not work in foobar2000. Does anyone have any ideas?

[edit] lossywav beta 1.0.1 removed as being obsolete. [/edit]

I'm currently toying around with "frequency-warped all-pole lattice filters". I think they are the perfect fit for your case once I get them to work as noise shaping filters. These are the kinds of filters Edler et al used for their "new paradigm codec" (better known as Fraunhofer's "ultra low delay codec"). I'm confident that it's possible to turn these filters into noise shaping filters as required in the lossyWAV case.

If you're interested in this we should talk about how collaboration might look like.

Buzz words explained:

frequency warping = A technique that can be used in filter design to achieve nonuniform frequency resolution. In the context of lossy coding and masking thresholds this technique helps find filters that match the masking threshold well.

all-pole filter = A kind of digital filter. The transfer function's nominator is constant (ie the filter has no zeros, only poles). These are often used in speech codecs as synthesis filters but they also seem appropriate for matching masking thresholds (see Edler et al).

lattice filter = A special implementation that allows easy filter interpolation.

Cheers,
SG

I'm currently toying around with "frequency-warped all-pole lattice filters". I think they are the perfect fit for your case once I get them to work as noise shaping filters. These are the kinds of filters Edler et al used for their "new paradigm codec" (better known as Fraunhofer's "ultra low delay codec"). I'm confident that it's possible to turn these filters into noise shaping filters as required in the lossyWAV case.

If you're interested in this we should talk about how collaboration might look like.

Buzz words explained:

frequency warping = A technique that can be used in filter design to achieve nonuniform frequency resolution. In the context of lossy coding and masking thresholds this technique helps find filters that match the masking threshold well.

all-pole filter = A kind of digital filter. The transfer function's nominator is constant (ie the filter has no zeros, only poles). These are often used in speech codecs as synthesis filters but they also seem appropriate for matching masking thresholds (see Edler et al).

lattice filter = A special implementation that allows easy filter interpolation.

Cheers,
SG

I would be delighted to use your proposed noise shaping method. ygpm.

If you have any ideas, suggestions, code optimisations, etc, please post them here.

What about channel coupling (http://wiki.hydrogenaudio.org/index.php?title=Channel_coupling)/Joint stereo (http://en.wikipedia.org/wiki/Joint_%28audio_engineering%29) ? Can it applied (i mean: can it reduce bits into lossless area) ?

What about channel coupling (http://wiki.hydrogenaudio.org/index.php?title=Channel_coupling)/Joint stereo (http://en.wikipedia.org/wiki/Joint_%28audio_engineering%29) ? Can it applied (i mean: can it reduce bits into lossless area) ?

The lossless codec used after lossyWAV does take care of that.

I've had another look at the FLAC format specification and it appears that the wasted bits parameter is channel dependent rather than block dependent.

This raises the interesting possibility of removing different numbers of bits for each channel.... This will require quite a bit of rework in the remove_bits procedure, however I think that it will be worth it in the end as it can only increase the number of bits removed.

Also, the quantization/dithering part could be done jointly on the channels using a generalized metric in the spirit of M/S coding. You usually don't want the quantization noise's coherence (comparing left versus right) to differ greatly from the coherence between the left and right of your original signal, I suppose.

edit: This is probably overkill at the moment.

Also, the quantization/dithering part could be done jointly on the channels using a generalized metric in the spirit of M/S coding. You usually don't want the quantization noise's coherence (comparing left versus right) to differ greatly from the coherence between the left and right of your original signal, I suppose.

edit: This is probably overkill at the moment.

....but, the bit-removal related noise takes into account the RMS value of each channel with respect to maximum bits-to-remove and also each channel is treated separately for FFT analysis purposes. I have a working beta now and below are the resultant bitrates for my 53 problem sample set:

+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|Version| FLAC  | -q 10 | -q 9  | -q 8  | -q 7  | -q 6  | -q 5  | -q 4  | -q 3  | -q 2  | -q 1  | -q 0  |
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|1.0.0b |784kbps|637kbps|607kbps|577kbps|545kbps|513kbps|480kbps|449kbps|427kbps|390kbps|349kbps|306kbps|
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|1.0.1b |784kbps|654kbps|626kbps|596kbps|565kbps|534kbps|501kbps|470kbps|447kbps|408kbps|366kbps|329kbps|
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+

[edit] lossywav beta 1.0.1b removed as being obsolete. [/edit]

... I have a working beta now and below are the resultant bitrates for my 53 problem sample set: ...

Sorry: what has changed? I don't understand it. Especially I expected bitrate to go down.

Quote from: Nick.C on 2008-05-15 19:26:44

... I have a working beta now and below are the resultant bitrates for my 53 problem sample set: ...

Sorry: what has changed? I don't understand it. Especially I expected bitrate to go down.

Hehehe.... You spotted the mistake, I transposed the bitrates. I'll amend now.

+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|Version| FLAC  | -q 10 | -q 9  | -q 8  | -q 7  | -q 6  | -q 5  | -q 4  | -q 3  | -q 2  | -q 1  | -q 0  |
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|1.0.0b |784kbps|654kbps|626kbps|596kbps|565kbps|534kbps|501kbps|470kbps|447kbps|408kbps|366kbps|329kbps|
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
|1.0.1b |784kbps|637kbps|607kbps|577kbps|545kbps|513kbps|480kbps|449kbps|427kbps|390kbps|349kbps|306kbps|
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+

... I transposed the bitrates. ...

Very interesting results. I tried my regular track set using -q 4, -q 6.5, and -q 1.5.
With -q 6.5 and -q 4 my results are close to yours: a saving of nearly 20 kbps on average.
With -q 1.5 it's less than that however: a saving of a bit less than 10 kbps.

It's a welcome decrease in  bitrate.
I did a short listening test at -q 1.5 for badvilbel, triangle and Under the Boardwalk, and quality is fine to me.

I call it another step forward.

Very interesting results. I tried my regular track set using -q 4, -q 6.5, and -q 1.5.
With -q 6.5 and -q 4 my results are close to yours: a saving of nearly 20 kbps on average.
With -q 1.5 it's less than that however: a saving of a bit less than 10 kbps.

It's a welcome decrease in  bitrate.
I did a short listening test at -q 1.5 for badvilbel, triangle and Under the Boardwalk, and quality is fine to me.

I call it another step forward.

I think that it is. I transcoded my Mike Oldfield collection (38 discs as single files, 33.5 hours) FLAC: 797kbps; lossyFLAC -q 0 1.0.1b: 264kbps (232kbps to 290kbps album range) and still palatable to the ears.

[edit] I'd be very interested if anyone with some poly-channel WAV files could use 1.0.0b to process them and encode to FLAC then do the same with 1.0.1b. I feel that there may be a marked difference in the resultant bitrates.

The separation of the channels in terms of calculating the bits to remove and removing the bits has two effects: firstly, each separate channel RMS value is used (rather than the minimum of all channels) and bits-to-remove determined from that channel's FFT analyses; secondly, when removing the bits, if too many clips occur in one channel, only that channel's bits-to-remove is reduced until an acceptable number of clips is achieved (not all channels). [/edit]

This is brilliant. If you look at the early MATLAB code, you'll know I had plans to check for near-silent channels and take them out of the calculation - now that Nick has spotted that wasted_bits is channel dependent, there's no need, and you have this great increase in efficiency. For signals where most of the content is in one channel, this makes a huge difference.

I wonder what other lossless codecs do? If any do wasted_bits per block, not per channel, then you're adding more noise than you can get any benefit from, which will reduce the lossless encoding efficiency slightly. Not a big deal, but it would make sense to have the option to turn it off. Leave it on by default though.


While you're playing in this area, it might be worth adding an option (it should not be there by default) to check the stereo difference channel S=(L-R) and run the analysis on this too, allowing it to drag the bits_to_remove down on the L and R channels if the value for S is lower than that for L or R. This will prevent unexpected things turning up in the surround channel on matrix surround systems (e.g. Dolby ProLogic). Add an optional offset too - e.g. bits_to_remove in L and R should never be more than x above the bits_to_remove calculated for S. Obviously calculate and compare in the threshold domain, not the bits_to_remove domain, because that's too coarse. I just explained it this way for simplicity.


For a change like these, I think it's probably worth putting out another stable release before going for the radical change of noise shaping. There's little chance of "breaking" anything at this point, whereas tuning of the next stage could take a long time. Might as well give the benefit of the improvement to the masses!

Cheers,
David.

Nick, beta v1.0.1b is the first release that doesn't start in my win98. "Program has performed an illegal action bla bla"  Unknown error in unknown module and all zeros. So something has changed.

This is brilliant. If you look at the early MATLAB code, you'll know I had plans to check for near-silent channels and take them out of the calculation - now that Nick has spotted that wasted_bits is channel dependent, there's no need, and you have this great increase in efficiency. For signals where most of the content is in one channel, this makes a huge difference.

I wonder what other lossless codecs do? If any do wasted_bits per block, not per channel, then you're adding more noise than you can get any benefit from, which will reduce the lossless encoding efficiency slightly. Not a big deal, but it would make sense to have the option to turn it off. Leave it on by default though.


While you're playing in this area, it might be worth adding an option (it should not be there by default) to check the stereo difference channel S=(L-R) and run the analysis on this too, allowing it to drag the bits_to_remove down on the L and R channels if the value for S is lower than that for L or R. This will prevent unexpected things turning up in the surround channel on matrix surround systems (e.g. Dolby ProLogic). Add an optional offset too - e.g. bits_to_remove in L and R should never be more than x above the bits_to_remove calculated for S. Obviously calculate and compare in the threshold domain, not the bits_to_remove domain, because that's too coarse. I just explained it this way for simplicity.


For a change like these, I think it's probably worth putting out another stable release before going for the radical change of noise shaping. There's little chance of "breaking" anything at this point, whereas tuning of the next stage could take a long time. Might as well give the benefit of the improvement to the masses!

Cheers,
David.

It took me a bit by surprise how much the bitrate came down (on some tracks, not all). I will work out how to implement the parameter required to revert to the old method for codecs which cannot treat each channel separately for the purpose of wasted-bits.

I'll get this working properly and try to firm up the STDIN / STDOUT mechanisms before going further. Properly processing the S channel may have to wait until 1.2.0.

I would agree that the resultant bitrate reduction associated with this finding is important enough to warrant a 1.1.0 release earlier than expected and defer noise shaping to 1.2.0.

Nick, beta v1.0.1b is the first release that doesn't start in my win98. "Program has performed an illegal action bla bla"  Unknown error in unknown module and all zeros. So something has changed.

How are you running beta 1.0.1b? If using the STDIN / STDOUT option, it may still be a bit flakey - at 1.0.1b I was assuming both STDIN input and STDOUT output when using the '-' parameter. beta 1.0.1c changes this a bit by allowing '-' to indicate STDIN input in isolation and I am working on a '--nochunksin <bps> <channels> <rate>' parameter to allow "proper" STDIN input from FLAC, etc. The corresponding '--stdout' parameter is also in place and I am working on the '--nochunksout' parameter to disable any WAV information other than a stream of samples going to STDOUT. I will post beta 1.0.1c soon.

How are you running beta 1.0.1b?

Like any other first starts of new releases I started it from the command line without any parameters/options.  Normally one gets 'use proper syntax / type -help'.  This time it failed.

While you're playing in this area, it might be worth adding an option (it should not be there by default) to check the stereo difference channel S=(L-R) and run the analysis on this too [..]
David.

This would be very complicated with multi channel (>2) files and I somehow doubt the usefulness.

This will prevent unexpected things turning up in the surround channel on matrix surround systems (e.g. Dolby ProLogic).

I think this is new territory that has not (extensively) been tested with the current version as well. What should a matrix decoder do with white noise?

Quote from: 2Bdecided on 2008-05-16 11:38:40

While you're playing in this area, it might be worth adding an option (it should not be there by default) to check the stereo difference channel S=(L-R) and run the analysis on this too [..]

This would be very complicated with multi channel (>2) files and I somehow doubt the usefulness.

Quote from: 2Bdecided on 2008-05-16 11:38:40

This will prevent unexpected things turning up in the surround channel on matrix surround systems (e.g. Dolby ProLogic).

I think this is new territory that has not (extensively) been tested with the current version as well. What should a matrix decoder do with white noise?

You would not do it with "multi-channel" files - it is only relevant to stereo files.

I'm not sure what you mean by "What should a matrix decoder do with white noise?". It will decode it the same way it would any other signal, though if it's in the rear channels then some noise reduction might kick in, and if it's an active decoder (like Dolby Pro Logic) the "steering" might attenuate quieter sounds.


It is new territory.

Cheers,
David.

I've attached an example. a..._MS_done.flac is a critical sample for this issue, that you might choose to encode with lossyWAV.

If you take the .lossy version, decode the result through a matrix surround decoder, you will get noise in the rear channel. In the file a..._MS_done.lossy.MS.flac I've put the "centre" channel into the left channel, and the "rear" channel into the right channel, so you can hear the result easily. Note that it would be very difficult to hear in a real surround sound system, unless you put your ear right up to the rear speakers (some people do though!).

Even this critical sample isn't bad, and I can't imagine how it could ever get any worse than this, but it would be useful to have a switch to check the S channel to keep it as "clean" as the other (real!) "channels" - if not now, please add it to the list of features for the future! When the switch is activated, you should probably check M=L+R in the same way.

Cheers,
David.

Looking for a very high quality substitute for lossless archiving I ended up with v1.0.1b -q 7.0 --shaping 1.0.
Yields a bitrate of 528 kbps on average with my regular track set, which is 34 kbps more than when not using --shaping. But listening to the correction file noise is so much less audible when using noise shaping that it's worth spending this extra bitrate.
Bitrate difference is higher for lower quality settings as I noticed before: with v1.0.1b -q 5.5 using --shaping 1.0 or not makes up for a difference of 46 kbps with my regular track set.

Looking for a very high quality substitute for lossless archiving I ended up with v1.0.1b -q 7.0 --shaping 1.0.
Yields a bitrate of 528 kbps on average with my regular track set, which is 34 kbps more than when not using --shaping. But listening to the correction file noise is so much less audible when using noise shaping that it's worth spending this extra bitrate.
Bitrate difference is higher for lower quality settings as I noticed before: with v1.0.1b -q 5.5 using --shaping 1.0 or not makes up for a difference of 46 kbps with my regular track set.

I take it from that that you are content with the bit-removal process being channel dependent rather than the lowest of all channel bits-to-remove? In my listening to the results of the revised bit-removal, I am content with the results, also with the improved efficiency when losslessly encoded.

I am still working on the STDIN and STDOUT processes. At present lossyWAV beta 1.0.1c can output raw audio to FLAC and have it correctly encoded (using lossywav wavfilename.wav --stdout | flac - -5 -b 512 --bps 16 --channels 2 --sample-rate 44100 --sign signed --endian little -f -o"wavfilename.lossy.flac"). It can take input through STDIN, (i.e. lossywav - <wavfilename.wav) and will output "lossyWAV.lossy.wav".

I am having difficulty piping FLAC --stdout output or foobar2000 converter output into lossywav - I cannot find any documentation which details the transfer format for foobar2000. [edit] Using "flac -d wavfilename.flac --stdout|lossywav - -q 0" I got a lossywav processed file lossywav.lossy.wav - when encoded with FLAC it seems to have worked. However, a double pipe will not (yet, if ever) work. [/edit]

lossyWAV beta 1.0.1c attached to post #1 in this thread.

NB: using STDIN (filename='-') is only working if the --nochunksin parameter is NOT specified. At present using both in combination will cause the program to crash. This release specifically made to see of collector's crash issue has been resolved....

I take it from that that you are content with the bit-removal process being channel dependent rather than the lowest of all channel bits-to-remove? ....

Yes, absolutely. Honestly speaking my imagination has always been that each channel was processed independently. I was rather surprised to learn that bits-to-remove was identical to all channels.

I am having difficulty piping FLAC --stdout output or foobar2000 converter output into lossywav ....

Maybe the wavPack documentation for the -i option helps:

-i = ignore length in wav header (no pipe output allowed)

Some programs that pipe data to encoders do not always give the correct length in the wav headers that they provide (foobar's clienc and CDex are examples). In these cases use this option to force WavPack to ignore the header and accept the actual length. Because WavPack must seek to the beginning of the file to write the correct length, this option cannot be used with piped output.

As you have to use a lossless codec afterwards it looks like we can have only piping in the input or piping in the output of LossyWAV.
Guess you're done: you use a temp wav file as lossyWAV input and piping to FLAC, and you don't benefit from having a piped input to lossyWAV and a temp wav file interface to the lossless codec.

Yes, absolutely. Honestly speaking my imagination has always been that each channel was processed independently. I was rather surprised to learn that bits-to-remove was identical to all channels.

Up until 1.0.0, the processing was carired out separately then the minimum value used, however the maximum bits-to-remove was dependent on the averae RMS over all channels. At 1.0.0, the minimum RMS of all channels was used.

Maybe the wavPack documentation for the -i option helps:

-i = ignore length in wav header (no pipe output allowed)

Some programs that pipe data to encoders do not always give the correct length in the wav headers that they provide (foobar's clienc and CDex are examples). In these cases use this option to force WavPack to ignore the header and accept the actual length. Because WavPack must seek to the beginning of the file to write the correct length, this option cannot be used with piped output.

As you have to use a lossless codec afterwards it looks like we can have only piping in the input or piping in the output of LossyWAV.
Guess you're done: you use a temp wav file as lossyWAV input and piping to FLAC, and you don't benefit from having a piped input to lossyWAV and a temp wav file interface to the lossless codec.

I would *really* like to implement piped input and output in foobar as it is the major processing bottleneck now.

This release specifically made to see of collector's crash issue has been resolved....

Sorry, no changes. It still doesn't run.

Nothing new, just an observation for those who like to use lossyWAV in extremely high quality mode like me:
I tried v1.0.1b -q 7.0 --shaping 0.5 (instead of --shaping 1.0 which I did before).
This yields a bitrate of 503 kbps on average with my regular track set which is only 9 kbps more than when not using --shaping. That's more or less for free, and noise is still so much in the HF region that the most important frequency range of the fundamentals is more or less free of noise, and the overall noise perception when listening to the correction file is very low usually.
So I think this rather simple noise shaping which we have already is very favorable when using high quality settings.
As is known with low quality settings things are different: average bitrate when using -q 1.5 goes up from 312 kbps to 342 kbps when using --shaping 0.5.

Quote from: Nick.C on 2008-05-16 21:42:48

This release specifically made to see of collector's crash issue has been resolved....

Sorry, no changes. It still doesn't run.

I'll get round to tracking the issue this evening - sorry for the delay!

Nothing new, just an observation for those who like to use lossyWAV in extremely high quality mode like me:
I tried v1.0.1b -q 7.0 --shaping 0.5 (instead of --shaping 1.0 which I did before).
This yields a bitrate of 503 kbps on average with my regular track set which is only 9 kbps more than when not using --shaping. That's more or less for free, and noise is still so much in the HF region that the most important frequency range of the fundamentals is more or less free of noise, and the overall noise perception when listening to the correction file is very low usually.
So I think this rather simple noise shaping which we have already is very favorable when using high quality settings.
As is known with low quality settings things are different: average bitrate when using -q 1.5 goes up from 312 kbps to 342 kbps when using --shaping 0.5.

Sounds good - it could even be a standard part of the preset, i.e. quality_noise_shaping_factor : array[0..Quality_Presets] of double = (0,0,0,0.1,0.3,0.5,0.6,0.7,0.8,0.9,1);

Sounds good - it could even be a standard part of the preset, i.e. quality_noise_shaping_factor : array[0..Quality_Presets] of double = (0,0,0,0.1,0.3,0.5,0.6,0.7,0.8,0.9,1);

I thank you very much for having implemented noiseshaping as I really like it at something like -q 7.0.
I'm not sure however with quality settings that aren't so high whether it's safe to use and which way to use.
One sorrow for instance: with a weak noise shift like 0.1: isn't there a risk that noise level is increased in the area around 6 kHz where we're very sensitive towards noise? Another one: roughly speaking the quality assuring machinery controls SNR in various ways, but isn't the SNR of certain frequency regions made worse by shaping the noise?
With -q 7.0 --shaping 0.5 I feel pretty safe as I think a)  --shaping 0.5 shifts noise for the most part pretty much beyond 6 kHz, and b) with -q 7.0 there's a security margin that I expect to cover a certain decrease in HF SNR due to noise shifting.

With this understanding - hope it's correct - I would prefer not to default to current noise shifting other than with high quality settings.
With high quality settings >= 7.0 however it does make sense to me: something like --shaping 0.5 for -q 7.0, --shaping 0.6 for -q 8.0, --shaping 0.7 for -q 9.0, --shaping 0.8 for -q 10.0 (the exact details being a matter of taste).
The current noiseshaping may be favorable also for low bitrate (I listened to -q 1.5 --shaping 0.5 and was very content), but may be it's wise to leave it up to the user and not default to it.

Quote from: Nick.C on 2008-05-17 15:28:59

Sounds good - it could even be a standard part of the preset, i.e. quality_noise_shaping_factor : array[0..Quality_Presets] of double = (0,0,0,0.1,0.3,0.5,0.6,0.7,0.8,0.9,1);

I thank you very much for having implemented noiseshaping as I really like it at something like -q 7.0.
I'm not sure however with quality settings that aren't so high whether it's safe to use and which way to use.
One sorrow for instance: with a weak noise shift like 0.1: isn't there a risk that noise level is increased in the area around 6 kHz where we're very sensitive towards noise? Another one: roughly speaking the quality assuring machinery controls SNR in various ways, but isn't the SNR of certain frequency regions made worse by shaping the noise?
With -q 7.0 --shaping 0.5 I feel pretty safe as I think a)  --shaping 0.5 shifts noise for the most part pretty much beyond 6 kHz, and b) with -q 7.0 there's a security margin that I expect to cover a certain decrease in HF SNR due to noise shifting.

With this understanding - hope it's correct - I would prefer not to default to current noise shifting other than with high quality settings.
With high quality settings >= 7.0 however it does make sense to me: something like --shaping 0.5 for -q 7.0, --shaping 0.6 for -q 8.0, --shaping 0.7 for -q 9.0, --shaping 0.8 for -q 10.0 (the exact details being a matter of taste).
The current noiseshaping may be favorable also for low bitrate (I listened to -q 1.5 --shaping 0.5 and was very content), but may be it's wise to leave it up to the user and not default to it.

I have no knowledge of how the noise shaping is done in LossyWAV but if you have any control over it surely it should be possible to ensure that any noise is always shifted well out of harm's way

I have no knowledge of how the noise shaping is done in LossyWAV but if you have any control over it surely it should be possible to ensure that any noise is always shifted well out of harm's way

lossyWAV uses SebastianG's noise shaping method (http://wiki.hydrogenaudio.org/index.php?title=Noise_shaping) for 44.1kHz and 48kHz with thanks.

Speaking about it with SG, using any "factor" applied to the coefficients (factor to the power of (the coefficient index -1)) will work for any value of factor in the range 0.0 to 1.0. In this way I presume that even using 0.1 will tend to move some of the white noise added by the lossyWAV bit reduction method into the high frequency area.

[...] using any "factor" applied to the coefficients (factor to the power of (the coefficient index -1)) will work for any value of factor in the range 0.0 to 1.0. In this way I presume that even using 0.1 will tend to move some of the white noise added by the lossyWAV bit reduction method into the high frequency area.

Yes. This is a simple trick you can do with minimum phase filters. The "factor" actually scales the poles and zeros of the filter's transfer function. As they move closer to the origin (factor going from 1.0 down to 0.0) the filter's response becomes more and more flat. Setting this parameter to 0.0 is equivalent to disabling noise shaping.

I'm currently trying to get something fancier to work: Adaptive filters that quickly respond well to what the psychoacoustic model "decides" to be irrelevant.

Cheers,
SG

There has been a request for a DLL of lossyWAV. I have no experience of how this may be achieved, let alone for a DSP like lossyWAV.

Any thoughts, hints, tips, standard interfaces, etc would be extremely well received.

Nick.

lossyWAV beta 1.0.1d attached to post #1 in this thread.

lossyWAV beta 1.0.1d attached to post #1 in this thread.

Up and running again in win98    Thanks Nick.

Quote from: Nick.C on 2008-05-18 22:17:21

lossyWAV beta 1.0.1d attached to post #1 in this thread.

Up and running again in win98    Thanks Nick.

About time too - I was getting worried about that one....

lossyWAV beta 1.0.1f attached to post #1 in this thread.

[edit] problem with file-naming logic when stdin & stdout used in conjunction.... [/edit]

lossyWAV beta 1.0.1f attached to post #1 in this thread.
[edit] problem with file-naming logic when stdin & stdout used in conjunction.... [/edit]

Can't the flossy.bat-command line not end in something like -o"%2 -q3" ? I'm not into Foobar. From the start I rename my files like <musicfile -lq3.wav> since only one file in a testphase can be called musicfile.lossy.wav. That's an indication of lossy but no quality label. And to compare them I need to know the q that was used.
The --stdout works fine though

Quote from: Nick.C on 2008-05-19 21:51:07

lossyWAV beta 1.0.1f attached to post #1 in this thread.
[edit] problem with file-naming logic when stdin & stdout used in conjunction.... [/edit]

Can't the flossy.bat-command line not end in something like -o"%2 -q3" ? I'm not into Foobar. From the start I rename my files like <musicfile -lq3.wav> since only one file in a testphase can be called musicfile.lossy.wav. That's an indication of lossy but no quality label. And to compare them I need to know the q that was used.
The --stdout works fine though

Glad to hear that the STDOUT output is working for you.

Why not use:

lossywav %1 -q %2 %3 %4 %5 %6 %7 %8 %9 --stdout|flac - -5 -b 512 -f -o"%~n1.-q%2.lossy.flac"

It should work in the way you intend.

...I've attached an example. a..._MS_done.flac is a critical sample for this issue, that you might choose to encode with lossyWAV.

David.

I don't have the right setup to check these files ATM. You may want to check Dualstream or even wavpack.

Ghido said this regarding Optimfrog DS:

"- independent quantization levels for each channel (some TC use
  joined channel quantization, reducing spatial sound imaging)"

Lossywav why cant work with *.wav??? (lossywav *.wav -q2 in command line)  %lossyWAV Error% : Input file: *.wav does not exist. 

I prefer batch files, but i cant do this and i wont like to convert it one bye one.

Lossywav why cant work with *.wav??? 

I prefer batch files, but i cant do this and i wont like to convert it one bye one.

for %a in (*.wav) do lossywav "%a" -q 2

would work perfectly well from the command line or

@for %%a in (*.wav) do lossywav "%%a" -q 2

in a batch file.

Code: [Select]

@for %%a in (*.wav) do lossywav "%%a" -q 2

in a batch file.

Thx. This work correctly now. 

Quote from: Nick.C on 2008-05-20 17:48:14

Code: [Select]

@for %%a in (*.wav) do lossywav "%%a" -q 2

in a batch file.

Thx. This work correctly now. 

I would use:

@for %%a in (*.wav) do lossywav "%a" %1 %2 %3 %4 %5 %6 %7 %8 %9 --stdout|flac - -5 -b 512 -o"%~na.lossy.flac" --tag="LOSSYWAV"="lossyWAV 1.0.1f"

for creating lossyFLAC files quickly....

Change log 1.0.1d: 18/05/08
Console output has been redirected to 'con', rather than STDOUT.

Apart from loosing the --keep-foreign-metadata, this has the side effect that logging made like
lossyWAV %1 -q 5  >>mylogfile.txt
is no longer there.    (My script is (still) based on one from the original dev thread).

Is there an easy way to redirect from CON to file again?

I am having difficulty piping foobar2000 converter output into lossywav - I cannot find any documentation which details the transfer format for foobar2000.

The same here. It should be something like a wav file, although with an incorrect length in the header. Number of bits as specified in foobar2000 encoder setting (for lossless files usually the same as input file).

You could look into code from another codec, that reads stdin and works with foobar, or (after a search ) ask in the foobar2000 development forum (http://www.hydrogenaudio.org/forums/index.php?act=SF&s=&f=34) next door.

Quote

Change log 1.0.1d: 18/05/08Console output has been redirected to 'con', rather than STDOUT.

Apart from loosing the --keep-foreign-metadata, this has the side effect that logging made like
lossyWAV %1 -q 5  >>mylogfile.txt
is no longer there.    (My script is (still) based on one from the original dev thread).

Is there an easy way to redirect from CON to file again?

Quote from: Nick.C on 2008-05-16 21:42:48

I am having difficulty piping foobar2000 converter output into lossywav - I cannot find any documentation which details the transfer format for foobar2000.

The same here. It should be something like a wav file, although with an incorrect length in the header. Number of bits as specified in foobar2000 encoder setting (for lossless files usually the same as input file).

You could look into code from another codec, that reads stdin and works with foobar, or (after a search ) ask in the foobar2000 development forum (http://www.hydrogenaudio.org/forums/index.php?act=SF&s=&f=34) next door.

I'm going to implement a --writetolog parameter which will append output to lossywav.log in the same directory as the output file (where for stdout, I don't yet know) - this will write pertinent values from the process to this file.

Thanks for the pointer to the foobar2000 forums. I'll have a look there....

I would use:

Code: [Select]

@for %%a in (*.wav) do lossywav "%a" %1 %2 %3 %4 %5 %6 %7 %8 %9 --stdout|flac - -5 -b 512 -o"%~na.lossy.flac" --tag="LOSSYWAV"="lossyWAV 1.0.1f"

for creating lossyFLAC files quickly....

I like tak (-e -fsl 512), cause smaller than flac. I tested. 

Quote from: Nick.C on 2008-05-20 18:09:41

I would use:

Code: [Select]

@for %%a in (*.wav) do lossywav "%a" %1 %2 %3 %4 %5 %6 %7 %8 %9 --stdout|flac - -5 -b 512 -o"%~na.lossy.flac" --tag="LOSSYWAV"="lossyWAV 1.0.1f"

for creating lossyFLAC files quickly....

I like tak (-e -fsl 512), cause smaller than flac. I tested. 

Whatever you're happier with - I'm just glad it works with more than one codec....

After a short session, it looks that TAK, FLAC, LPAC and ALS are all compatible with the new channel independent bit depth reduction feature, while WV it's not. When someone else could confirm, and due to the (of course, very well deserved) high popularity of WV, we probably should take care of that. I didn't bother to test WMA so far.

      1.00,q8 1.01c,q8 Diff.
      6,7r.b. 7,0r.b.

FLAC  40,20   38,55   -1,65
TAK   39,02   37,36   -1,66
WV    40,53   40,57   +0,04
ALS   38,86   37,23   -1,63
LPAC  39,14   37,57   -1,57

After a short session, it looks that TAK, FLAC, LPAC and ALS are all compatible with the new channel independent bit depth reduction feature, while WV it's not. When someone else could confirm, and due to the (of course, very well deserved) high popularity of WV, we probably should take care of that. I didn't bother to test WMA so far.

Thanks Josef, that's encouraging (again!). What it does indicate to me is that maybe the --linkchannels parameter is not really required, although it doesn't really add that much to the overall processing time.

I've been thinking that the conversion from reference_threshold array (as calculated by the [unreleased] calculate-white-noise-level-from-rounding) to threshold_index array is too coarse (something I think David alluded to a short time ago). I have "widened" it by a factor of 48 (seems to be a good compromise between memory requirements and additional bits removed) and it has allowed a few kbps to be shaved off the FLAC'ed processed output.

I will post beta 1.0.1g today.

It just occured to me that in case of varying "wasted_bits" counts over the channels FLAC's the channel mode "M/S" is one of the two modes out of 4 possible that fail to exploit this. Consider L has 5 zeroed LSBs and R as 4 zeroed LSBs. Then computing S:=L-R and M:=R+S/2 (or L-S/2) results in channels which both have only 4 zeroed LSBs. Actually M has only 3 zeroed LSBs due to the division by two. In this case only "L/R" and "L/S" exploit the 5th zero bit in L.

Since there're no other option besides "-M" to turn adaptive M/S coding on I'm presuming that it also considers "L/S" and "R/S" which is a good idea when used on current lossyWAVs results.

IIRC, WavPack doesn't do M/S but rather interchannel prediction. So, it probably can't exploit different "wasted_bits" counts over the channels in general.

Cheers,
SG

I've been thinking about the difference between the replaygain / wavegain values calculated for the .lwcdf.wav file with --shaping 0 and --shaping 1 (spreadsheet attached).

There seems to be an average 12dB difference between the two (--shaping 1 = higher gain), although how this relates to bits to remove, I still need to determine.

I think that' s very natural and shows that your kind of noise shaping is effective.
The replaygain technique computes the perceived loudness. So replaygain says the perceived loudness of the error file is considerably lower when using noise shaping. So it confirms in an objective way what we can hear subjectively: noise shaping reduces the perceived noise - when listening to the noise.
Unfortunately it doesn't tell about masking, and so a noise shaped version isn't necessarily better when listening to the music.
With a high quality setting I feel pretty safe to use noise shaping, that's why I do it. The lower the quality setting however the more the risk IMO that noise concentrated in certain frequency regions can hurt (though it may well be that the benefits outweigh the risk also with lower quality settings).

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

Thanks for the sample, at what position in the audio are you hearing the distortion?

Also, what is the significance of the images?

I confirm,
I can hear it too sec 01, sec 07 & sec 12,
between sec 01 & sec 02 it is very very noisy in the back. I can ABX it 100%.

Quote from: Mardel on 2008-05-21 18:27:49

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

Thanks for the sample, at what position in the audio are you hearing the distortion? ...

I just abxed sec. 6.4-8.6 8/10 (and I'm sure those who know this track will do better). I wouldn't call this spot heavily distorted though, but that's a matter of taste.
We know -q 0 isn't a real quality mode (though usually it's quite okay). My personal favorite for low bitrate is -q 1.5. It's not significantly higher in bitrate (312 kbps on average for my regular music test set) but to me quality does a little jump there. The spot I mentioned with your track, Mardel, is ok to me using -q 1.5.

Mardel and sauvage78, do you mind trying -q 1.5?

Quote from: Mardel on 2008-05-21 18:27:49

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

Thanks for the sample, at what position in the audio are you hearing the distortion?

Also, what is the significance of the images?

I hear some sizzle in the high pitch range.
The essence of the picture, that 12 khz full is with sounds like that what they were not originally there.

... The essence of the picture, that 12 khz full is with sounds like that what they were not originally there.

Sure. lossyWAV does add noise. The essential question is: is it audible? We can't read that from a picture.
But with your sample we can hear it when using -q 0.

tried quickly v1.0.1f -q 1.5, the artefact is much reduced but still easyly ABXable IMHO
I wouldn't have thought only a 1.5 increase would have reduced it so much, so I think it should go away on higher setting ... futher test is required.

I was using -q 0 too, all I can say is that I don't want to use -q 0 anymore, for me it was as easy to ABX as MP3 64Kbps on castanets sample, a nightmare

Typo: I typed -q 1 intead of -q 0 several times, it's fixed now

Quote from: Mardel on 2008-05-21 19:43:01

... The essence of the picture, that 12 khz full is with sounds like that what they were not originally there.

Sure. lossyWAV does add noise. The essential question is: is it audible? We can't read that from a picture.
But with your sample we can hear it when using -q 0.

Maybe now is the time to revisit the -spf string (again) and maybe tighten up its response in the 12kHz to 16kHz region. I'll get to work re-introducing the -spf parameter (as --spf <string>) and post v1.0.1g this evening.

Thanks for the sample and listening input sauvage78 and Mardel (and, of course halb27).

tried quickly v1.0.1f -q 1.5, the artefact is much reduced but still easyly ABXable IMHO ...

Thanks for your test. Glad to hear that -q 1.5 is better for you though still easily ABXable. Your ears are better than mine.
We should keep in mind that general transparency is expected at -q 5, at least for people with good hearing. Due to a series of internal precautions it's not necessarily -q 5 which is to use, and we can stay a bit lower.

It would be interesting to learn at what quality level this sample becomes transparent to everybody who tries.

Maybe now is the time to revisit the -spf string (again) and maybe tighten up its response in the 12kHz to 16kHz region. ....

Please keep in mind that all the tests so far have been performed with very low quality settings.
When introducing a more restrictive -spf string for the highest frequency zone (you did that already with your latest spf change) bitrate will go up, and we must compare the results to the results of the current spf setting at a correspondingly higher quality setting.

Perhaps it would be wise to find out first what current quality setting is necessary for generally accepted transparency of this sample.
In case transparency can be achieved with a rather low current quality setting I think we shouldn't mind and just call it a problem sample we should be willing to accept with very low quality settings.
In case transparency is achieved not before ~ -q 5 I think it's worth investigating whether a change in the machinery can bring a progress.

I agree with halb27, I have spend the last 20 min doing serious ABX & it happened that -q1.5 was much harder to ABX than what I first thought ... I can ABX -q 0 100% without serious listening, I was forced to listen carefully  to ABX -q 1.5, it was placebo when I said it was easy ... I was finally able to ABX with near 100% success but it was not easy at all in the end ... I can ABX -q 0 with only the 2 first seconds ... I was forced to concentrate & listen up to second 7 to ABX -q1.5 ... & in the end it was very little added noise to an already noisy original sample for -q 1.5 while it's LOTS of noise for -q 0 ... so I was not self-confident even if I finally succeeded.

I think it is very specific to -q 0 & already very noisy source. -q 0 is not safe anymore for me but everything is not wasted. I am 100% sure that this sample is transparent for me far before -q 5 now as I doubt I can ABX even -q 2 ... but I won't test I am bored now

I agree with halb27, I have spend the last 20 min doing serious ABX & it happened that -q1.5 was much harder to ABX than what I first thought ... I can ABX -q 0 100% without serious listening, I was forced to listen carefully  to ABX -q 1.5, it was placebo when I said it was easy ... I was finally able to ABX with near 100% success but it was not easy at all in the end ... I can ABX -q 0 with only the 2 first seconds ... I was forced to concentrate & listen up to second 7 to ABX -q1.5 ... & in the end it was very little added noise to an already noisy original sample for -q 1 while it's LOTS of noise for -q 0 ... so I was not self-confident even if I finally succeeded.

I think it is very specific to -q 0 & already very noisy source. -q 0 is not safe anymore for me but everything is not wasted.

Glad to hear this. Do you mind going up a bit in quality level to learn when this track is transparent for you?

Sorry, I just edited that I wouldn't test -q 2 now ... maybe I will try tomorow cause it's bedtime for me now.

Don't expect miracle, I am french but I am no Guruboolez

Sorry, I just edited that I wouldn't test -q 2 now ... maybe I will try tomorow cause it's bedtime for me now.

Don't expect miracle, I am french but I am no Guruboolez

Sure, but your testing is very welcome. Thank you, and have a good night.

Sorry, I just edited that I wouldn't test -q 2 now ... maybe I will try tomorow cause it's bedtime for me now.

Don't expect miracle, I am french but I am no Guruboolez

Thanks for all of the input - I am very interested in your -q 2 ABX results.

Nick.

I've been thinking about -q 0 and 1 ("my" quality levels as my ears are a bit shot....). As I have been refining the reference_threshold > threshold_index > bits_to_remove calculations, I had a few kbit/s to play with which I have recycled into the quality of the output of the lower quality presets.

I have re-implemented the --lowpass parameter to allow user with better ears to raise the upper limit of the range of frequencies lossyWAV uses in its analyses.

I have tweaked the --spf and minimum_bits_to_keep:

  spreading_function_string         : string[precalc_analyses*(spread_zones+2)-1]='22222-22222-22223-12224-12234-12345';
  quality_minimum_bits_to_keep      : array[0..Quality_Presets] of double = (3.000,3.000,3.000,3.125,3.250,3.375,3.500,3.625,3.750,3.875,4.000);

and get the following:

|-----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| lossyWAV  | FLAC  | -q 10 | -q 9  | -q 8  | -q 7  | -q 6  | -q 5  | -q 4  | -q 3  | -q 2  | -q 1  | -q 0  |
|-----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
|1.0.1b     |784kbps|637kbps|607kbps|577kbps|545kbps|513kbps|480kbps|449kbps|427kbps|390kbps|349kbps|306kbps|
|-----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
|1.0.1g     |784kbps|639kbps|610kbps|580kbps|548kbps|516kbps|484kbps|453kbps|430kbps|396kbps|354kbps|320kbps|
|-----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
|1.0.1g --LC|784kbps|656kbps|628kbps|600kbps|569kbps|538kbps|506kbps|474kbps|450kbps|414kbps|369kbps|334kbps|
|-----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|

lossyWAV beta 1.0.1g attached to post #1 in this thread.

This morning I tested the problem sample at V1.0.1f -q 0/-q 0.5/-q 1/-q 1.5 with my speaker (Logitech Z3-I) instead of my headphone (Philips SHP1900) it was easier to ABX but I dunno if that is because I was tired yesterday or because of the cheap headphone (Mainboard is Asus A7N8X-Deluxe) this time I have keep my log to backup my claims as I done it very seriously.

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 09:56:09

File A: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossless.flac
File B: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossyQ0.0.flac

09:56:09 : Test started.
09:56:37 : 01/01  50.0%
09:56:50 : 02/02  25.0%
09:57:02 : 03/03  12.5%
09:57:18 : 04/04  6.3%
09:57:31 : 05/05  3.1%
09:57:45 : 06/06  1.6%
09:57:58 : 07/07  0.8%
09:58:09 : 08/08  0.4%
09:58:20 : 09/09  0.2%
09:58:33 : 10/10  0.1%
09:58:47 : 11/11  0.0%
09:58:58 : 12/12  0.0%
09:59:30 : Test finished.

----------
Total: 12/12 (0.0%)

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 10:01:16

File A: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossless.flac
File B: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossyQ0.5.flac

10:01:16 : Test started.
10:01:38 : 01/01  50.0%
10:01:53 : 02/02  25.0%
10:02:06 : 03/03  12.5%
10:02:19 : 04/04  6.3%
10:02:38 : 05/05  3.1%
10:02:51 : 06/06  1.6%
10:03:50 : 07/07  0.8%
10:04:01 : 08/08  0.4%
10:04:23 : 09/09  0.2%
10:04:36 : 10/10  0.1%
10:04:48 : 11/11  0.0%
10:04:59 : 12/12  0.0%
10:05:02 : Test finished.

----------
Total: 12/12 (0.0%)

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 10:06:22

File A: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossless.flac
File B: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossyQ1.0.flac

10:06:22 : Test started.
10:06:50 : 01/01  50.0%
10:07:20 : 02/02  25.0%
10:07:36 : 03/03  12.5%
10:07:47 : 04/04  6.3%
10:08:00 : 05/05  3.1%
10:08:24 : 06/06  1.6%
10:08:45 : 07/07  0.8%
10:09:06 : 08/08  0.4%
10:09:21 : 09/09  0.2%
10:09:52 : 10/10  0.1%
10:10:17 : 11/11  0.0%
10:10:36 : 12/12  0.0%
10:10:46 : Test finished.

----------
Total: 12/12 (0.0%)

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 10:20:41

File A: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossless.flac
File B: C:\Documents and Settings\Mes documents\02- Test\Nouveau dossier\b.lossyQ1.5.flac

10:20:41 : Test started.
10:21:01 : 01/01  50.0%
10:21:24 : 02/02  25.0%
10:21:54 : 03/03  12.5%
10:22:34 : 04/04  6.3%
10:24:09 : 05/05  3.1%
10:25:48 : 06/06  1.6%
10:26:20 : 07/07  0.8%
10:27:01 : 08/08  0.4%
10:27:21 : 09/09  0.2%
10:28:15 : 10/10  0.1%
10:29:33 : 11/11  0.0%
10:30:18 : 12/12  0.0%
10:30:22 : Test finished.

----------
Total: 12/12 (0.0%)

Will test -q 2 later as it took me 10min just for -q 1.5 but it becomes hard. -q 0 & -q 0.5 were easy (no hesitation), -q 1 medium (a few hesitations) & -q 1.5 hard (several hesitations), I was forced to re-listen several times at -q 1.5 but unlike yesterday I was able to ABX all with only the first 2 second & was not forced to listen up to second 7.

I can ABX -q 2 too but it takes 23min with a pause in the middle ... I will not try to ABX further it's too hard.

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 11:08:32

File A: C:\Documents and Settings\Mes documents\02- Test\b.lossless.wav
File B: C:\Documents and Settings\Mes documents\02- Test\b.lossyQ2.0.wav

11:08:32 : Test started.
11:10:25 : 01/01  50.0%
11:12:44 : 02/02  25.0%
11:13:49 : 03/03  12.5%
11:14:53 : 04/04  6.3%
11:16:12 : 05/05  3.1%
11:19:18 : 06/06  1.6%
11:22:21 : 07/07  0.8%
11:23:00 : 08/08  0.4%
11:23:57 : 09/09  0.2%
11:24:57 : 10/10  0.1%
11:28:47 : 11/11  0.0%
11:31:24 : 12/12  0.0%
11:31:28 : Test finished.

----------
Total: 12/12 (0.0%)

In the end I would describe it like this:
-q 0.0 & -q0.5 added noise/distortion, clear artefact/flaw
-q 1.5 & -q2.0 tearing in the original distortion, I don't ABX an added artefact, I ABX either the lenght of the distortion or the depht of the tearing (it's slightly longer/deeper)
-q1.0 is in the middle: slightly added noise, slighly longer distortion.

I have re-implemented the --lowpass parameter to allow user with better ears to raise the upper limit of the range of frequencies lossyWAV uses in its analyses.

THX. -q 2.5 --lowpass 17000 is a very good sound quality for my sample. I tried q 1; q 1.5;  q 2; q 2 lowpass 16k, 17k, 18, 19, 20k; q 2.5; q2.5 lowpass16k and 17k. I haven't heard any sound artifacts at -q 2.5 --lowpass 17000.

I can ABX -q 2 too but it takes 23min with a pause in the middle ... I will not try to ABX further it's too hard.

In the end I would describe it like this:
-q 0.0 & -q0.5 added noise/distortion, clear artefact/flaw
-q 1.5 & -q2.0 tearing in the original distortion, I don't ABX an added artefact, I ABX either the lenght of the distortion or the depht of the tearing (it's slightly longer/deeper)
-q1.0 is in the middle: slightly added noise, slighly longer distortion.

Many thanks for your efforts. As you can see above, I have modified the HF end of the spreading function in 1.0.1g. Also, you might be interested in using the --lowpass parameter to increase the upper limit of the range of frequencies lossyWAV uses in its analyses.

When your ears have recovered, I would appreciate it if you could listen to 1.0.1g -q 0 (it probably will still not require ABX, however I would be interested to hear if there is any improvement). Maybe slowly increasing --lowpass from 16000 to, say, 18000 will improve this particular sample (at the expense of FLAC bitrate).

Thanks again,

Nick.

Would it be rude to request a reality check here? No lossy codec should be transparent at the very bottom of its quality scale.

I think it's very useful to have at least one non-transparent setting at the bottom of the range - it lets people hear what kind of artefacts lossyWAV is adding. It also confirm to the average user that the quality scale works, and stops people from saying "even q0 is transparent - what happens if you lower the bitrate still further".

Cheers,
David.

-s, --scale <n>  scaling factor from WaveGain, etc; default=1.000000; n<>0!

LossyWav doesn't permit n to be a positive scale factor either, but many of my classical and easy listening piano albums require it.
My question: can it be changed in a future version ? My Wavegain.exe can do it but that's another run in the process.

Would it be rude to request a reality check here? No lossy codec should be transparent at the very bottom of its quality scale.

I think it's very useful to have at least one non-transparent setting at the bottom of the range - it lets people hear what kind of artefacts lossyWAV is adding. It also confirm to the average user that the quality scale works, and stops people from saying "even q0 is transparent - what happens if you lower the bitrate still further".

Cheers,
David.

Not at all - it's a requirement of development.... I agree with what you're saying, though maybe there is some scope for improving the performance at -q 0 a bit....

Code: [Select]

-s, --scale <n>  scaling factor from WaveGain, etc; default=1.000000; n<>0!

LossyWav doesn't permit n to be a positive scale factor either, but many of my classical and easy listening piano albums require it.
My question: can it be changed in a future version ? My Wavegain.exe can do it but that's another run in the process.

At present you would have to wavegain the file before processing with lossyWAV. What upper limit do you think would be reasonable, as it's an easy fix (1 value changed in the nparameters unit)?

If you wavegain at more than unity, you can introduce clipping, but more importantly the efficiency could be reduced (probably not by much in practice).

V1.0.1g  -q 0 Vs. Original

foo_abx 1.3.3 report
foobar2000 v0.9.5.2
2008/05/22 13:00:52

File A: C:\Documents and Settings\b.lossy.wav
File B: C:\Documents and Settings\b.wav

13:00:52 : Test started.
13:01:09 : 01/01  50.0%
13:01:21 : 02/02  25.0%
13:01:32 : 03/03  12.5%
13:01:44 : 04/04  6.3%
13:02:02 : 05/05  3.1%
13:02:17 : 06/06  1.6%
13:02:32 : 07/07  0.8%
13:02:52 : 08/08  0.4%
13:03:04 : 09/09  0.2%
13:03:15 : 10/10  0.1%
13:03:28 : 11/11  0.0%
13:03:42 : 12/12  0.0%
13:04:01 : Test finished.

----------
Total: 12/12 (0.0%)

For me V1.0.1g  -q 0 = V1.0.1f -q 0.5 it's better but the artefact is  still obvious & easy to spot. (3 min to ABX)

For me V1.0.1g  -q 0 = V1.0.1f -q 0.5 it's better but the artefact is  still obvious & easy to spot. (3 min to ABX)

Thanks for that - taking into account what David said about reality checking and that the lowest preset *should* introduce noticable artifacts, I think that I will step back a bit from the settings I put in place for 1.0.1g.

If I keep the spreading_function string at '22222-22222-22223-12224-12234-12345' but reduce the minimum_bits_to_keep to 2.333 and 2.667 respectively for -q 0 and -q 1, the resultant bitrate for -q 0 is 312kbps/325kbps and 352kbps/367kbps (default / --linkchannels) for my 54 problem sample set (to which I must add Mardel's latest problem sample).

I feel that this is a reasonable compromise as the HF regions are given a little more influence in the result at a small change in bitrate.

lossyWAV beta 1.0.1h attached to post #1 in this thread.

With this sample and -q 0 setting, it looks like level of added noise changes very frequently, and this modulation is more noticeable than the noise itself.

At present you would have to wavegain the file before processing with lossyWAV. What upper limit do you think would be reasonable, as it's an easy fix (1 value changed in the nparameters unit)?

Up till now, with dozens of tracks to go, the biggest album difference is +6.23 dB. Don't know what scale factor that is.

Quote from: Nick.C on 2008-05-22 12:04:03

At present you would have to wavegain the file before processing with lossyWAV. What upper limit do you think would be reasonable, as it's an easy fix (1 value changed in the nparameters unit)?

Up till now, with dozens of tracks to go, the biggest album difference is +6.23 dB. Don't know what scale factor that is.

About 2.0488, I reckon.

... I think it's very useful to have at least one non-transparent setting at the bottom of the range - it lets people hear what kind of artefacts lossyWAV is adding. It also confirm to the average user that the quality scale works, and stops people from saying "even q0 is transparent - what happens if you lower the bitrate still further". ...

This is a great argument IMO. We should really stop to expect -q 0 being transparent. Good quality as a rule should be enough.

I can ABX -q 2 too but it takes 23min with a pause in the middle ...

Thank you, sauvage 78. Your results show that -q 2 isn't transparent with this sample, but (please correct me if I'm wrong) seems to be a setting which is 'close enough to transparency' to be useful for very high quality DAP use. To a slightly minor degree this can be said also of -q 1.5.
Remains the question: What quality setting brings full transparency?

...  haven't heard any sound artifacts at -q 2.5 --lowpass 17000.

Thank you, Mardel.

If I keep the spreading_function string at '22222-22222-22223-12224-12234-12345' but reduce the minimum_bits_to_keep to 2.333 and 2.667 respectively for -q 0 and -q 1, ...

I am having trouble to keep up with the changes.
What's exactly the difference between 1.0.1h and 1.0.1f?
1.0.1h spf string = '22222-22222-22223-12224-12234-12345'? What was it with 1.0.1f?
1.0.1h minimum_bits_to_keep to 2.333 and 2.667 respectively for -q 0 and -q 1? What was it with 1.0.1f?
Any other changes?

Anyway thank you for your restless struggle for improvement.

Quote from: Nick.C on 2008-05-22 12:27:28

If I keep the spreading_function string at '22222-22222-22223-12224-12234-12345' but reduce the minimum_bits_to_keep to 2.333 and 2.667 respectively for -q 0 and -q 1, ...

I am having trouble to keep up with the changes.
What's exactly the difference between 1.0.1h and 1.0.1f?
1.0.1h spf string = '22222-22222-22223-12224-12234-12345'? What was it with 1.0.1f?
1.0.1h minimum_bits_to_keep to 2.333 and 2.667 respectively for -q 0 and -q 1? What was it with 1.0.1f?
Any other changes?

Anyway thank you for your restless struggle for improvement.

1.0.1f had the existing --spf string, 1.0.1g & h have the modified one, although the minimum_bits_to_keep was:

2.000 and 2.500 for -q 0 and -q 1 respectively in 1.0.1f.
3.000 and 3.000 for -q 0 and -q 1 respectively in 1.0.1g;
2.333 and 2.667 for -q 0 and -q 1 respectively in 1.0.1h;

I increased the "resolution" of the threshold index so that fewer bits are potentially lost due to the requirement to floor the spreading_function result to read the corresponding bits_to_remove value from the threshold_index array. By widening the threshold_index array by a factor (now 64) the flooring of  (spreading_function result * 64) has less of an impact.

I will leave --lowpass implemented as it will give those with more acute hearing some scope to include frequencies in the analyses above the existing 16kHz limit.

1.0.1f had the existing --spf string, 1.0.1g & h have the modified one, although the minimum_bits_to_keep was:

2.000 and 2.500 for -q 0 and -q 1 respectively in 1.0.1f.
3.000 and 3.000 for -q 0 and -q 1 respectively in 1.0.1g;
2.333 and 2.667 for -q 0 and -q 1 respectively in 1.0.1h;

I increased the "resolution" of the threshold index ...

Thank you, Nick. So we have 1.0.1f's spf string with 1.0.1h - I appreciate this very much.
The other things make lossyWAV more defensive without a remarkable bad impact on bitrate as I understand it. So that's fine.

With this sample and -q 0 setting, it looks like level of added noise changes very frequently, and this modulation is more noticeable than the noise itself.

I'll have a look at the bits-to-remove detail for the sample at -q 0 and see if it is large stepping in the bits-to-remove (i.e. added noise) - from what you say, it would seem to be.

halb27:
you're right -q 2 was near transparent with 1.0.1f & it is even more with 1.0.1h.
I tried to quickly ABX 1.0.1h -q2 vs original but I gave up after 5min as it was a boring task, I think it is still ABXable but it will take me 40 min just to prove to others something I am already sure for myself & even if -q2 is ABXable the fact that I can hear a difference doesn't mean that this difference sounds ugly.

So yes -q2 was listenable for DAP with this specific sample, specially in a noisy environnement, but my personal interest for lossywav is not in DAP even if I like the flexibility of using any lossless format.

Nick.C:
I was thinking lossywav was just removing unearable random noise in the back that takes much space to losslessly encode, but obviously it also add its own noise. I was thinking lossywav was as a kind of "lossless audio cleaner" by optimizing wav before encoding to lossless, I was thinking lossywav was not introducing "quantization like noise", but I was wrong it seems that even without quantization it add lossy-like additionnal noise. So I haven't followed all the lossywav developments, but now I am interested in knowing if lossywav applies some encoding strategy at some settings & not at some other settings. Even  without understanding in deep the used techniques, just to understand the logical step within the scale.

Why is Q0 to Q4 for DAP, Q5 magically called "transparent", Q6 & Q7 as good as lossless & Q8 to Q10 for transcoding ??? My technical knowledge is very limited but I would like to know if these claims are rationnal or just arbitrary. Is it just me or lossywav have become more lossy than it was supposed to be in the beginning ? If there is no different techniques at different settings isn't the scale a little too wide ? Wouldn't it be logic to have only 2 recommended settings within the scale covering each specific use:

"-near lossy" for DAP, covering Q0 to Q5 use, targetting transparency.
"-near lossless" for archiving, covering Q6 to Q10 use, targetting transcodable archiving.

I don't mean that the actual scale is bad, I like it, freedom is always good. But it is missleading for newbies as I ended using Q0 for archive thinking lossywav was overkill anyway no matter the setting. I dunno but

"-near lossy" could be the same as let's say -q 3 (ABXing can help here)
"-near lossless" could be the same as ... well I don't know but the least agressive possible while still optimizing the lossless encoding. (ABXing is useless here, only technical knowledge so I let the codecs developpers decide together, not only you Nick.C ... Monty, Ivan, Gabriel may have an hintfull opinion here)

I know this is still very arbitrary but it would help newbies using lossywav as it should, & not lowering its reputation ...

for me for there is only 4 uses for hybrid:
100% lossless for Archive + "-near lossy" for DAP.
"-near lossless" for Archive + 100% lossy for DAP.
"-near lossless" for Archive + "-near lossy" for DAP.
hybrid lossless part for Archive + hybrid lossy part for listening.

so in the end two well tuned settings can cover it all.

halb27:
you're right -q 2 was near transparent with 1.0.1f & it is even more with 1.0.1h.
I tried to quickly ABX 1.0.1h -q2 vs original but I gave up after 5min as it was a boring task, I think it is still ABXable but it will take me 40 min just to prove to others something I am already sure for myself & even if -q2 is ABXable the fact that I can hear a difference doesn't mean that this difference sounds ugly.

So yes -q2 was listenable for DAP with this specific sample, specially in a noisy environnement, but my personal interest for lossywav is not in DAP even if I like the flexibility of using any lossless format.

Nick.C:
I was thinking lossywav was just removing unearable random noise in the back that takes much space to losslessly encode, but obviously it also add its own noise. I was thinking lossywav was as a kind of "lossless audio cleaner" by optimizing wav before encoding to lossless, I was thinking lossywav was not introducing "quantization like noise", but I was wrong it seems that even without quantization it add lossy-like additionnal noise. So I haven't followed all the lossywav developments, but now I am interested in knowing if lossywav applies some encoding strategy at some settings & not at some other settings. Even  without understanding in deep the used techniques, just to understand the logical step within the scale.

Why is Q0 to Q4 for DAP, Q5 magically called "transparent", Q6 & Q7 as good as lossless & Q8 to Q10 for transcoding ??? My technical knowledge is very limited but I would like to know if these claims are rationnal or just arbitrary. Is it just me or lossywav have become more lossy than it was supposed to be in the beginning ? If there is no different techniques at different settings isn't the scale is little too wide ? Wouldn't it be logic to have only 2 recommended settings covering each specific use:

-near lossy for DAP, covering Q0 to Q5, targetting transparency.
-near lossless for archiving, covering Q6 to Q10, targetting transcoding.

I don't mean that the actual scale is bad, I like it, freedom is always good. But it is missleading for newbies as I ended using Q0 for archive thinking lossywav was overkill anyway no matter the setting. I dunno but

-near lossy could be the same as -q 3 (ABXing can help here)
-near lossless could be the same as ... well I don't know but the least agressive possible while still optimizing the lossless encoding. (ABXing is useless here, only technical knowledge so let the codecs developpers decide together, not only Nick.C)

I know this is very arbitrary but it would help newbies using lossywav as it should, & not lowering its reputation ...

lossyWAV has always added noise to the output, see the original lossyFLAC post started by David (2Bdecided) - the method used is his.

The Delphi transcode of the Matlab original started as just that - an attempt to duplicate David's method in a form that more users could make use of - Matlab is after all rather expensive....

Once agreement between both versions was achieved (about 0.3.0 roughly from memory), other ideas blossomed and added to the method.

The preset scale started as -1, -2 and -3. -1 = best, -2 = transparent, -3 = more aggressive. This evolved over time into -1 to -5, -1 to -7 and finally -q 10 to -q 0. As it states in the help:

Quality Options:

-q, --quality <n>   quality preset (10=highest quality, 0=lowest bitrate;
                    -q 5 is generally accepted to be transparent)
                    default=-q 5.

On the subject of transparency, as many problem samples as have been pointed out to halb27 and me have been included in the refinement of the quality presets. v0.6.4 RC1 was thought to be good until the day after release Alex B found a problem sample that halb27 also ABX'ed.

Transparency at -q 5 is not an arbitrary claim - although not every possible sample has been tested, therefore it is still "only" a claim (it should be noted that a very small group of users has been involved in the developmental ABX'ing process).

However, unless the optional (and still beta) noise shaping is selected, the actual bit removal process only rounds off the least significant <bits-to-remove> bits of each sample in a codec-block, thus adding white noise. If noise shaping is selected, the added noise is modified and a lot of it is pushed much higher in the spectrum (thanks to SebastianG's noise shaping coefficients).

Another, more advanced, method of noise shaping is in the early stages of development at present - but don't hold your breath.

Personally, I only use lossyWAV & FLAC as a DAP transcode - my lossless FLAC archive will remain intact.

The bottom line on the quality presets: 10 = best, 0 = worst - use one which you are comfortable with.

Thanks for taking the time to post.

Nick.

Quote from: lvqcl on 2008-05-22 18:56:22

With this sample and -q 0 setting, it looks like level of added noise changes very frequently, and this modulation is more noticeable than the noise itself.

I'll have a look at the bits-to-remove detail for the sample at -q 0 and see if it is large stepping in the bits-to-remove (i.e. added noise) - from what you say, it would seem to be.

I had a look:

Z:\WAV\tmp>lossywav "..\_swav\01 - Ginnungagap - The Black Hole.original.wav" -q 0 --detail -f
lossyWAV beta 1.0.1h, Copyright © 2007,2008 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
%lossyWAV Warning% : Detailled output mode enabled
Processing : 01 - Ginnungagap - The Black Hole.original.wav
Format     : 44.10kHz; 2 ch.; 16 bit.
Progress   :
Block    Time   00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 Tot.
====================================================================
    0    0.00s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   16    0.19s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   32    0.37s.  5  7 10 10 10  9  9 10  9  9  9  8  7  7  9  8 136
   48    0.56s.  9 11 10 10 10 10  9 10  9  9  9  9  9  9  8  9 150
   64    0.74s.  7  9  8  8  6  7  6  8  7  7  7  7  7  7  8 10 119
   80    0.93s. 10 10 10  9  9  9  9  8  8  8  8  9  8  8  9 11 143
   96    1.11s. 10 11 11 11 11 11 11 11 11 10 11 10 10 10 10 10 169
  112    1.30s. 10 10 10  9 10 10  9  9 10 10 10 10  9  9  9 10 154
  128    1.49s.  9 10  9  9 10 10 10  9  9 10 10  9  8  9  9  8 148
  144    1.67s.  9  9  9  9  9  8  9  9  9  9  9  9  9  9  8  8 141
  160    1.86s.  8  9  8  9  9  9  9  9  9  8  9  9  9  9  9  8 140
  176    2.04s.  8  8  9  8  8  8  9  9  8  9  8  8  9  8  8  8 133
  192    2.23s.  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8  8 127
  208    2.41s.  7  8  7  7  8  8  7  8  8  7  8  8  7  7  7  8 120
  224    2.60s.  7  8  7  7  6  8  7  8  7  7  7  7  7  7  7  6 113
  240    2.79s.  7  7  8  6  7  6  7  6  7  7  6  7  7  7  7  7 109
  256    2.97s.  7  7  7  7  7  7  6  7  7  7  7  6  6  7  7  7 109
  272    3.16s.  6  6  7  7  7  7  7  8  8  7  8  8  8  8  8  8 118
  288    3.34s.  7  8  7  8  7  7  8  7  7  7  7  7  7  7  7  7 115
  304    3.53s.  7  7  7  8  8  8  7  7  7  7  7  8  7  7  8  6 116
  320    3.72s.  7  7  7  7  7  7  7  7  7  7  7  7  7  6  6  7 110
  336    3.90s.  7  7  5  7  7  6  7  6  6  6  7  6  6  7  7  7 104
  352    4.09s.  7  7  7  7  7  7  6  7  6  6  7  7  6  7  7  7 108
  368    4.27s.  6  7  6  6  6  6  6  7  7  7  7  6  7  6  7  6 103
  384    4.46s.  6  7  6  6  6  7  6  6  6  6  6  6  6  6  7  6  99
  400    4.64s.  6  6  7  7  6  6  7  6  6  7  7  7  7  6  6  6 103
  416    4.83s.  6  6  5  6  6  6  6  6  6  6  6  6  6  5  6  5  93
  432    5.02s.  6  6  5  6  6  6  6  6  6  6  6  5  6  5  6  6  93
  448    5.20s.  5  6  6  6  6  6  6  6  5  5  5  5  6  5  5  6  89
  464    5.39s.  6  6  6  5  5  6  6  6  6  6  5  6  6  5  5  5  90
  480    5.57s.  5  6  5  5  6  6  6  6  6  6  5  5  6  6  6  6  91
  496    5.76s.  6  5  5  5  6  5  5  6  6  6  6  6  5  6  5  6  89
  512    5.94s.  6  5  5  5  5  5  6  6 10 10 10 10  9  9  9 10 120
  528    6.13s.  9  8  8  8  9  9  8 10 10 10  9 10 10  9  9  9 145
  544    6.32s.  9  9  8  8  8  7  8  7  8  8  8  7  7  7  7  7 123
  560    6.50s.  7  6  7  7  7 10 10 10 10 10  9  8  9  9  9  9 137
  576    6.69s.  8  8  8  8  8 11 11 11 11 11 10 11 10 11 10 11 158
  592    6.87s. 10 11 11 10 10 10 10 10 10 10 10  9 10 10  9  9 159
  608    7.06s.  9  9  9 10 10 10  9  9  9 10 10 10 10 10 10  9 153
  624    7.24s. 10  9  9  9  9  9  9  9 10 10  9  9 10  9  9  9 148
  640    7.43s.  9  9 10  9  9  9  7  9  9  9  9  9  8  9  9  8 141
  656    7.62s.  9  9  9  8  9  9  9  8  9  8  9  8  8  8  8  8 136
  672    7.80s.  9  8  8  8  8  8  7  7  8  9  9  8  8  9  8  7 129
  688    7.99s.  7  7  8  8  8  8  8  8  9  7  7  8  7  7  8  7 122
  704    8.17s.  8  8  8  8  7  7  7  7  8  6  7  7  6  7  8  6 115
  720    8.36s.  7  7  8  8  7  7  7  7  7  8  7  7  6  6  7  6 112
  736    8.54s.  7  7  6  7  5  6  7  7  6  6  5  7  6  7  7  7 103
  752    8.73s.  7  7  6  6  6  7  7  7  7  7  7  8  8  7  8  8 113
  768    8.92s.  8  8  8  8  7  7  7  7  7  7  7  6  6  7  8  7 115
  784    9.10s.  8  7  7  7  7  6  8  7  7  7  7  7  7  8  8  7 115
  800    9.29s.  7  7  7  7  7  7  7  8  8  7  7  7  7  7  7  7 114
  816    9.47s.  7  7  6  6  7  7  7  7  8  8  7  7  8  7  7  7 113
  832    9.66s.  7  7  7  7  7  7  6  7  7  8  7  7  7  6  6  7 110
  848    9.85s.  7  6  7  5  7  7  6  6  7  7  7  7  7  7  7  7 107
  864   10.03s.  6  6  6  7  6  7  7  6  7  5  6  6  4  6  7  7  99
  880   10.22s.  7  6  6  7  8  7  8  7  8  7  8  8  8  6  8  8 117
  896   10.40s.  8  8  8  7  7  7  8  7  8  7  7  7  7  8  7  7 118
  912   10.59s.  7  7  6  7  7  7  7  7  5  7  6  6  7  6  6  7 105
  928   10.77s.  7  7  7  6  6  7  7  7  7  7  7  7  7  7  8  7 111
  944   10.96s.  7  8  7  7  7  7  8  8  7  7  8  8  7  7  7  7 117
  960   11.15s.  7  7  7  7  7  7  7  7  7  7  7  7  8  7  7  7 113
  976   11.33s.  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 112
  992   11.52s.  7  7  7  6  6  7  7  7  7  6  7  7  7  6  8 11 113
 1008   11.70s. 10 10 10 10 10  9  9  9  8  8  8  8  8  9 10  9 145
 1024   11.89s. 10  9  9 10 10  9  9  9  9  8  8  9  8  8  9  7 141
 1040   12.07s.  7  7  8  8  7  8  7  7  7  6  7  8 10 10 10  9 126
 1056   12.26s. 10  9  9  9  9  8  8  8  8  8  8  8 11 11 11 11 146
 1072   12.45s. 11 11 10 11 10 11 11 10 11 10 11 10 10 10 10 10 167
 1088   12.63s. 10 10 10  9  9  9  7  8  9 10  9 10  9  9  9 10 147
 1104   12.82s. 10  8 10 10  9 10 10 10  9 10  9  9  9 10  9 10 152
 1120   13.00s.  7  9  9  8  9  9  8  9  9  9  9  9  9  8  9  9 139
 1136   13.19s.  8  9  9  9  9  9  9  9  7  9  9  9  9  9  8  7 138
 1152   13.37s.  9  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8 128
 1168   13.56s.  7  8  8  7  8  5  8  8  8  8  8  7  7  7  8  8 120
 1184   13.75s.  7  8  8  8  8  8  8  8  8  8  7  8  7  7  8  7 123
 1200   13.93s.  7  7  8  7  7  7  7  7  6  8  8  7  7  7  7  6 113
 1216   14.12s.  6  7  6  7  7  7  7  6  7  8  6  6  7  7  7  7 108
 1232   14.30s.  7  7  7  7  7  7  6  7  7  7  7  7  7  7  7  7 111
 1248   14.49s.  7  7  7  8  8  8  8  8  7  8  6  8  7  6  6  7 116
 1264   14.68s.  8  7  8  7  6  8  7  8  8  8  7  8  6  7  8  7 118
 1280   14.86s.  7  8  8  7  7  7  7  7  7  7  7  7  6  6  7  6 111
 1296   15.05s.  7  7  7  7  7  6  7  7  6  6  7  7  7  7  7  7 109
 1312   15.23s.  7  6  7  7  7  7  7  7  7  7  6  6  7  6  7  6 107
 1328   15.42s.  7  7  6  6  7  7  6  7  7  7  7  7  7  7  7  7 109
 1344   15.60s.  7  5  6  6  6  7  7  6  6  6  6  7  7  6  6  5  99
 1360   15.79s.  6  7  6  7  6  6  6  6  6  6  6  6  6  7  7  7 101
 1376   15.98s.  7  6  7  5  5  6  6  6  6  6  6  6  6  6  6  6  96
 1392   16.16s.  7  6  7  5  6  6  5  7  6  5  5  6  6  5  6  6  94
 1408   16.35s.  6  6  6  6  6  6  6  5  6  6  6  6  6  6  5  5
Average    : 2.78MB; 7.4112 bits; 32.93x; 0.50s;
%lossyWAV Warning% : 0.0112 bits not removed due to clipping.
%lossyWAV Warning% : 17 sample(s) clipped to limiting amplitude.

Z:\WAV\tmp>lossywav "..\_swav\01 - Ginnungagap - The Black Hole.original.wav" -q 0 --detail -f --minbits 3
lossyWAV beta 1.0.1h, Copyright © 2007,2008 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
%lossyWAV Warning% : Detailled output mode enabled
Processing : 01 - Ginnungagap - The Black Hole.original.wav
Format     : 44.10kHz; 2 ch.; 16 bit.
Progress   :
Block    Time   00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 Tot.
====================================================================
    0    0.00s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   16    0.19s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   32    0.37s.  5  7 10 10 10  9  9 10  9  9  9  8  7  7  9  8 136
   48    0.56s.  9 10 10 10 10 10  9 10  9  9  9  9  9  9  8  9 149
   64    0.74s.  7  9  8  8  6  7  6  8  7  7  7  7  7  7  8 10 119
   80    0.93s. 10 10 10  9  9  9  9  8  8  8  8  9  8  8  9 10 142
   96    1.11s. 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 160
  112    1.30s.  9 10 10  9 10  9  9  9 10 10 10 10  9  9  9 10 152
  128    1.49s.  9 10  9  9  9 10 10  9  9 10 10  9  8  9  9  8 147
  144    1.67s.  9  9  9  9  9  8  9  9  9  9  9  9  9  9  8  8 141
  160    1.86s.  8  9  8  9  9  9  9  9  9  8  9  9  9  9  9  8 140
  176    2.04s.  8  8  9  8  8  8  9  9  8  9  8  8  9  8  8  8 133
  192    2.23s.  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8  8 127
  208    2.41s.  7  8  7  7  8  8  7  8  8  7  8  8  7  7  7  8 120
  224    2.60s.  7  8  7  7  6  8  7  8  7  7  7  7  7  7  7  6 113
  240    2.79s.  7  7  8  6  7  6  7  6  7  7  6  7  7  7  7  7 109
  256    2.97s.  7  7  7  7  7  7  6  7  7  7  7  6  6  7  7  7 109
  272    3.16s.  6  6  7  7  7  7  7  8  8  7  8  8  8  8  8  8 118
  288    3.34s.  7  8  7  8  7  7  8  7  7  7  7  7  7  7  7  7 115
  304    3.53s.  7  7  7  8  8  8  7  7  7  7  7  8  7  7  8  6 116
  320    3.72s.  7  7  7  7  7  7  7  7  7  7  7  7  7  6  6  7 110
  336    3.90s.  7  7  5  7  7  6  7  6  6  6  7  6  6  7  7  7 104
  352    4.09s.  7  7  7  7  7  7  6  7  6  6  7  7  6  7  7  7 108
  368    4.27s.  6  7  6  6  6  6  6  7  7  7  7  6  7  6  7  6 103
  384    4.46s.  6  7  6  6  6  7  6  6  6  6  6  6  6  6  7  6  99
  400    4.64s.  6  6  7  7  6  6  7  6  6  7  7  7  7  6  6  6 103
  416    4.83s.  6  6  5  6  6  6  6  6  6  6  6  6  6  5  6  5  93
  432    5.02s.  6  6  5  6  6  6  6  6  6  6  6  5  6  5  6  6  93
  448    5.20s.  5  6  6  6  6  6  6  6  5  5  5  5  6  5  5  6  89
  464    5.39s.  6  6  6  5  5  6  6  6  6  6  5  6  6  5  5  5  90
  480    5.57s.  5  6  5  5  6  6  6  6  6  6  5  5  6  6  6  6  91
  496    5.76s.  6  5  5  5  6  5  5  6  6  6  6  6  5  6  5  6  89
  512    5.94s.  6  5  5  5  5  5  6  6 10 10 10 10  9  9  9 10 120
  528    6.13s.  9  8  8  8  9  9  8 10 10 10  9 10 10  9  9  9 145
  544    6.32s.  9  9  8  8  8  7  8  7  8  8  8  7  7  7  7  7 123
  560    6.50s.  7  6  7  7  7 10 10 10 10 10  9  8  9  9  9  9 137
  576    6.69s.  8  8  8  8  8 11 11 11 10 10 10 10 10 10 10 10 153
  592    6.87s. 10 10 10 10 10 10 10 10 10 10 10  9 10 10  9  9 157
  608    7.06s.  9  9  9 10 10 10  9  9  9 10 10 10 10 10 10  9 153
  624    7.24s. 10  9  9  9  9  9  9  9 10  9  9  9  9  9  9  9 146
  640    7.43s.  9  9 10  9  9  9  7  9  9  9  9  9  8  9  9  8 141
  656    7.62s.  9  9  9  8  9  9  9  8  9  8  9  8  8  8  8  8 136
  672    7.80s.  9  8  8  8  8  8  7  7  8  9  9  8  8  9  8  7 129
  688    7.99s.  7  7  8  8  8  8  8  8  9  7  7  8  7  7  8  7 122
  704    8.17s.  8  8  8  8  7  7  7  7  8  6  7  7  6  7  8  6 115
  720    8.36s.  7  7  8  8  7  7  7  7  7  8  7  7  6  6  7  6 112
  736    8.54s.  7  7  6  7  5  6  7  7  6  6  5  7  6  7  7  7 103
  752    8.73s.  7  7  6  6  6  7  7  7  7  7  7  8  8  7  8  8 113
  768    8.92s.  8  8  8  8  7  7  7  7  7  7  7  6  6  7  8  7 115
  784    9.10s.  8  7  7  7  7  6  8  7  7  7  7  7  7  8  8  7 115
  800    9.29s.  7  7  7  7  7  7  7  8  8  7  7  7  7  7  7  7 114
  816    9.47s.  7  7  6  6  7  7  7  7  8  8  7  7  8  7  7  7 113
  832    9.66s.  7  7  7  7  7  7  6  7  7  8  7  7  7  6  6  7 110
  848    9.85s.  7  6  7  5  7  7  6  6  7  7  7  7  7  7  7  7 107
  864   10.03s.  6  6  6  7  6  7  7  6  7  5  6  6  4  6  7  7  99
  880   10.22s.  7  6  6  7  8  7  8  7  8  7  8  8  8  6  8  8 117
  896   10.40s.  8  8  8  7  7  7  8  7  8  7  7  7  7  8  7  7 118
  912   10.59s.  7  7  6  7  7  7  7  7  5  7  6  6  7  6  6  7 105
  928   10.77s.  7  7  7  6  6  7  7  7  7  7  7  7  7  7  8  7 111
  944   10.96s.  7  8  7  7  7  7  8  8  7  7  8  8  7  7  7  7 117
  960   11.15s.  7  7  7  7  7  7  7  7  7  7  7  7  8  7  7  7 113
  976   11.33s.  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 112
  992   11.52s.  7  7  7  6  6  7  7  7  7  6  7  7  7  6  8 11 113
 1008   11.70s. 10 10 10 10 10  9  9  9  8  8  8  8  8  9 10  9 145
 1024   11.89s. 10  9  9 10  9  9  9  9  9  8  8  9  8  8  9  7 140
 1040   12.07s.  7  7  8  8  7  8  7  7  7  6  7  8 10 10 10  9 126
 1056   12.26s. 10  9  9  9  9  8  8  8  8  8  8  8 11 11 11 10 145
 1072   12.45s. 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 160
 1088   12.63s. 10 10 10  9  9  9  7  8  9 10  9 10  9  9  9 10 147
 1104   12.82s. 10  8  9 10  9 10 10 10  9 10  9  9  9 10  9 10 151
 1120   13.00s.  7  9  9  8  9  9  8  9  9  9  9  9  9  8  9  9 139
 1136   13.19s.  8  9  9  9  9  9  9  9  7  9  9  9  9  9  8  7 138
 1152   13.37s.  9  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8 128
 1168   13.56s.  7  8  8  7  8  5  8  8  8  8  8  7  7  7  8  8 120
 1184   13.75s.  7  8  8  8  8  8  8  8  8  8  7  8  7  7  8  7 123
 1200   13.93s.  7  7  8  7  7  7  7  7  6  8  8  7  7  7  7  6 113
 1216   14.12s.  6  7  6  7  7  7  7  6  7  8  6  6  7  7  7  7 108
 1232   14.30s.  7  7  7  7  7  7  6  7  7  7  7  7  7  7  7  7 111
 1248   14.49s.  7  7  7  8  8  8  8  8  7  8  6  8  7  6  6  7 116
 1264   14.68s.  8  7  8  7  6  8  7  8  8  8  7  8  6  7  8  7 118
 1280   14.86s.  7  8  8  7  7  7  7  7  7  7  7  7  6  6  7  6 111
 1296   15.05s.  7  7  7  7  7  6  7  7  6  6  7  7  7  7  7  7 109
 1312   15.23s.  7  6  7  7  7  7  7  7  7  7  6  6  7  6  7  6 107
 1328   15.42s.  7  7  6  6  7  7  6  7  7  7  7  7  7  7  7  7 109
 1344   15.60s.  7  5  6  6  6  7  7  6  6  6  6  7  7  6  6  5  99
 1360   15.79s.  6  7  6  7  6  6  6  6  6  6  6  6  6  7  7  7 101
 1376   15.98s.  7  6  7  5  5  6  6  6  6  6  6  6  6  6  6  6  96
 1392   16.16s.  7  6  7  5  6  6  5  7  6  5  5  6  6  5  6  6  94
 1408   16.35s.  6  6  6  6  6  6  6  5  6  6  6  6  6  6  5  5
Average    : 2.78MB; 7.3919 bits; 38.17x; 0.43s;
%lossyWAV Warning% : 0.0056 bits not removed due to clipping.
%lossyWAV Warning% : 3 sample(s) clipped to limiting amplitude.

Z:\WAV\tmp>lossywav "..\_swav\01 - Ginnungagap - The Black Hole.original.wav" -q 0 --detail -f --minbits 3.5
lossyWAV beta 1.0.1h, Copyright © 2007,2008 Nick Currie. Copyleft.
This is free software under the GNU GPLv3+ license; There is NO WARRANTY, to
the extent permitted by law. <http://www.gnu.org/licenses/> for details.
%lossyWAV Warning% : Detailled output mode enabled
Processing : 01 - Ginnungagap - The Black Hole.original.wav
Format     : 44.10kHz; 2 ch.; 16 bit.
Progress   :
Block    Time   00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 Tot.
====================================================================
    0    0.00s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   16    0.19s.  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0   0
   32    0.37s.  5  7 10 10 10  9  9  9  9  9  9  8  7  7  9  8 135
   48    0.56s.  9  9 10 10 10  9  9  9  9  9  9  9  9  9  8  9 146
   64    0.74s.  7  9  8  8  6  7  6  8  7  7  7  7  7  7  8 10 119
   80    0.93s. 10 10 10  9  9  9  9  8  8  8  8  9  8  8  9 10 142
   96    1.11s. 10 10 10  9 10  9  9 10  9 10  9  9  9  9 10  9 151
  112    1.30s.  9  9  9  9  9  9  9  9  9  9  9 10  9  9  9  9 145
  128    1.49s.  9  9  9  9  9  9  9  9  9  9  9  9  8  9  9  8 142
  144    1.67s.  9  9  9  9  9  8  9  9  9  9  9  9  9  9  8  8 141
  160    1.86s.  8  9  8  9  9  9  9  9  9  8  9  9  9  9  9  8 140
  176    2.04s.  8  8  9  8  8  8  9  9  8  9  8  8  9  8  8  8 133
  192    2.23s.  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8  8 127
  208    2.41s.  7  8  7  7  8  8  7  8  8  7  8  8  7  7  7  8 120
  224    2.60s.  7  8  7  7  6  8  7  8  7  7  7  7  7  7  7  6 113
  240    2.79s.  7  7  8  6  7  6  7  6  7  7  6  7  7  7  7  7 109
  256    2.97s.  7  7  7  7  7  7  6  7  7  7  7  6  6  7  7  7 109
  272    3.16s.  6  6  7  7  7  7  7  8  8  7  8  8  8  8  8  8 118
  288    3.34s.  7  8  7  8  7  7  8  7  7  7  7  7  7  7  7  7 115
  304    3.53s.  7  7  7  8  8  8  7  7  7  7  7  8  7  7  8  6 116
  320    3.72s.  7  7  7  7  7  7  7  7  7  7  7  7  7  6  6  7 110
  336    3.90s.  7  7  5  7  7  6  7  6  6  6  7  6  6  7  7  7 104
  352    4.09s.  7  7  7  7  7  7  6  7  6  6  7  7  6  7  7  7 108
  368    4.27s.  6  7  6  6  6  6  6  7  7  7  7  6  7  6  7  6 103
  384    4.46s.  6  7  6  6  6  7  6  6  6  6  6  6  6  6  7  6  99
  400    4.64s.  6  6  7  7  6  6  7  6  6  7  7  7  7  6  6  6 103
  416    4.83s.  6  6  5  6  6  6  6  6  6  6  6  6  6  5  6  5  93
  432    5.02s.  6  6  5  6  6  6  6  6  6  6  6  5  6  5  6  6  93
  448    5.20s.  5  6  6  6  6  6  6  6  5  5  5  5  6  5  5  6  89
  464    5.39s.  6  6  6  5  5  6  6  6  6  6  5  6  6  5  5  5  90
  480    5.57s.  5  6  5  5  6  6  6  6  6  6  5  5  6  6  6  6  91
  496    5.76s.  6  5  5  5  6  5  5  6  6  6  6  6  5  6  5  6  89
  512    5.94s.  6  5  5  5  5  5  6  6 10 10 10 10  9  9  9  9 119
  528    6.13s.  9  8  8  8  9  9  8 10 10 10  9 10  9  9  9  9 144
  544    6.32s.  9  9  8  8  8  7  8  7  8  8  8  7  7  7  7  7 123
  560    6.50s.  7  6  7  7  7 10 10 10 10 10  9  8  9  9  9  9 137
  576    6.69s.  8  8  8  8  8 10 10 11 10 10  9 10  9 10 10 10 149
  592    6.87s. 10 10 10  9  9 10  9 10  9  9  9  9  9  9  9  9 149
  608    7.06s.  9  9  9  9 10  9  9  9  9  9  9  9  9  9  9  9 145
  624    7.24s.  9  9  9  9  9  9  9  9  9  9  9  9  9  9  9  9 144
  640    7.43s.  9  9  9  9  9  9  7  9  9  9  9  9  8  9  9  8 140
  656    7.62s.  9  9  9  8  9  9  9  8  9  8  9  8  8  8  8  8 136
  672    7.80s.  9  8  8  8  8  8  7  7  8  9  9  8  8  9  8  7 129
  688    7.99s.  7  7  8  8  8  8  8  8  8  7  7  8  7  7  8  7 121
  704    8.17s.  8  8  8  8  7  7  7  7  8  6  7  7  6  7  8  6 115
  720    8.36s.  7  7  8  8  7  7  7  7  7  8  7  7  6  6  7  6 112
  736    8.54s.  7  7  6  7  5  6  7  7  6  6  5  7  6  7  7  7 103
  752    8.73s.  7  7  6  6  6  7  7  7  7  7  7  8  8  7  8  8 113
  768    8.92s.  8  8  8  8  7  7  7  7  7  7  7  6  6  7  8  7 115
  784    9.10s.  8  7  7  7  7  6  8  7  7  7  7  7  7  8  8  7 115
  800    9.29s.  7  7  7  7  7  7  7  8  8  7  7  7  7  7  7  7 114
  816    9.47s.  7  7  6  6  7  7  7  7  8  8  7  7  8  7  7  7 113
  832    9.66s.  7  7  7  7  7  7  6  7  7  8  7  7  7  6  6  7 110
  848    9.85s.  7  6  7  5  7  7  6  6  7  7  7  7  7  7  7  7 107
  864   10.03s.  6  6  6  7  6  7  7  6  7  5  6  6  4  6  7  7  99
  880   10.22s.  7  6  6  7  8  7  8  7  8  7  8  8  8  6  8  8 117
  896   10.40s.  8  8  8  7  7  7  8  7  8  7  7  7  7  8  7  7 118
  912   10.59s.  7  7  6  7  7  7  7  7  5  7  6  6  7  6  6  7 105
  928   10.77s.  7  7  7  6  6  7  7  7  7  7  7  7  7  7  8  7 111
  944   10.96s.  7  8  7  7  7  7  8  8  7  7  8  8  7  7  7  7 117
  960   11.15s.  7  7  7  7  7  7  7  7  7  7  7  7  8  7  7  7 113
  976   11.33s.  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 112
  992   11.52s.  7  7  7  6  6  7  7  7  7  6  7  7  7  6  8 10 112
 1008   11.70s. 10 10 10  9 10  9  9  9  8  8  8  8  8  9 10  9 144
 1024   11.89s. 10  9  9  9  9  9  9  9  9  8  8  9  8  8  9  7 139
 1040   12.07s.  7  7  8  8  7  8  7  7  7  6  7  8 10 10 10  9 126
 1056   12.26s.  9  9  9  9  9  8  8  8  8  8  8  8 10 10 10 10 141
 1072   12.45s.  9 10  9 10  9 10 10  9  9  9  9  9  9  9  9  9 148
 1088   12.63s.  9 10  9  9  9  9  7  8  9  9  9  9  9  9  9  9 142
 1104   12.82s.  9  8  9  9  9  9  9  9  9  9  9  9  9  9  9  9 143
 1120   13.00s.  7  9  9  8  9  9  8  9  9  9  9  9  9  8  9  9 139
 1136   13.19s.  8  9  9  9  9  9  9  9  7  9  9  9  9  9  8  7 138
 1152   13.37s.  9  7  8  8  8  8  8  8  8  8  8  8  8  8  8  8 128
 1168   13.56s.  7  8  8  7  8  5  8  8  8  8  8  7  7  7  8  8 120
 1184   13.75s.  7  8  8  8  8  8  8  8  8  8  7  8  7  7  8  7 123
 1200   13.93s.  7  7  8  7  7  7  7  7  6  8  8  7  7  7  7  6 113
 1216   14.12s.  6  7  6  7  7  7  7  6  7  8  6  6  7  7  7  7 108
 1232   14.30s.  7  7  7  7  7  7  6  7  7  7  7  7  7  7  7  7 111
 1248   14.49s.  7  7  7  8  8  8  8  8  7  8  6  8  7  6  6  7 116
 1264   14.68s.  8  7  8  7  6  8  7  8  8  8  7  8  6  7  8  7 118
 1280   14.86s.  7  8  8  7  7  7  7  7  7  7  7  7  6  6  7  6 111
 1296   15.05s.  7  7  7  7  7  6  7  7  6  6  7  7  7  7  7  7 109
 1312   15.23s.  7  6  7  7  7  7  7  7  7  7  6  6  7  6  7  6 107
 1328   15.42s.  7  7  6  6  7  7  6  7  7  7  7  7  7  7  7  7 109
 1344   15.60s.  7  5  6  6  6  7  7  6  6  6  6  7  7  6  6  5  99
 1360   15.79s.  6  7  6  7  6  6  6  6  6  6  6  6  6  7  7  7 101
 1376   15.98s.  7  6  7  5  5  6  6  6  6  6  6  6  6  6  6  6  96
 1392   16.16s.  7  6  7  5  6  6  5  7  6  5  5  6  6  5  6  6  94
 1408   16.35s.  6  6  6  6  6  6  6  5  6  6  6  6  6  6  5  5
Average    : 2.78MB; 7.3322 bits; 32.50x; 0.51s;
%lossyWAV Warning% : 0.0014 bits not removed due to clipping.

The issue might be the 11 bits removed around 1.1 seconds. They are removed because the signal is considered by the algorithm to be strong enough to mask the noise - in this instance obviously not. More analysis of the spectral makeup of the sample around the 1.1 second mark is required.

...So yes -q2 was listenable for DAP with this specific sample, specially in a noisy environnement, but my personal interest for lossywav is not in DAP even if I like the flexibility of using any lossless format. ...

If keeping bitrate very low isn't your primary concern but quality is (like for me) you're better off using a higher quality setting like -q 4 or higher.

Nick C. may go into more detail, but I think I can answer some questions you addressed at Nick (hope you don't mind, Nick - oh, you were faster anyway).
lossyWAV definitely adds noise by rounding the wave samples in such a way that the least significant bits become zero, and a series of lossless codecs can make good use of this. Roughly speaking this is not done with quiet spots in the music but it is done when the music is loud. The added noise is controlled - it should be rarely audible with the lowest quality settings, it should be very rarely audible and - if it's audible - not annoying with the low quality settings, it should never be audible with the medium quality settings (though the focus here is also on bitrate and we don't allow for a safety margin), and it should be always inaudible with an ever increasing safety margin at the high quality settings.
Nick's description addresses this with emphasis on lossyWAV applications like DAP use or archiving/transcoding.
-q 5 is special as it takes exact care of the original lossyWAV idea originating from 2Bdecided: make some frequency analyses and use the number of bits to remove (zero out) depending on the signal's frequency region with lowest volume.
From experience so far the idea is correct and -q 5 is really transparent.
Below -q 5 2Bdecided's requirements are more and more softened, but NickC.'s other methods of controlling noise obviously do a very good job in keeping noise low. They can't work miracles however: the noise control of -q 0 has so low requirements that it's more remarkable that quality usually is still good than that problem samples exist.
Above -q 5 all quality requirements are fulfilled with an ever increasing added safety margin.
Originally there were just 3 quality settings: apart from the -q 5 related quality setting a setting with an additional safety margin for archiving quality, and another one with a softened quality requirement for DAP use.
This is clearer than the current settings. Movement came in when a certain demand came up for very low bitrate usage. It's all a matter of taste - I think it's a good thing to have a fine grained quality scale. Below -q 5 we're exactly in the same situation as the usual lossy codecs like mp3 for which everybody likes a quality scale to use according to personal quality and bitrate demands. Above -q 5 we have no analogue to the usual lossy codecs, but for the safety margin it's pretty much the same thing as for the lowering of the  quality demands: I think it's good to be able to choose the degree of added security.
What has happened with these many quality settings is that the 'standard quality' has gotten a bit out of focus, though for good reason as I think -q 5 is overkill for the most of us, but obviously the feeling has gone that the more we lower the quality requirements compared to -q 5, the more natural is an increasing chance for audible noise.

I may look dumb, but with my little knowledge it sounds like lossywav is cleverly removing hard to compress inaudible noise & then cleverly dithering easy to compress inaudible noise ? no.
so the problem I heard was not a "quantization like distortion" but more a audible "bad dithering" ?

also where does low, medium & high setting starts & ends ... that's a big problem for a noob, with the old 3 settings best/transparent/more agressive it was easier not to make mistake, I think both a scale for freedoom & recommended settings for noobs are usefull & complementary.

not everyone is willing to annoy the dev with stupid questions or to read plenty of technical threads just to know how to use correctly the codec. I know where to find the original thread & wiki help, the wiki were too general, the original discussion too technical. (& I was too lazy  )

finally I don't get the use of any quality setting above Q5, if the original idea was proven correct so far what will  a "more transparent than transparent" bring ? if it doesn't bring anything for transparency then it must bring at last some added safety for transcoding, is it technically the case at last on paper ? Is there 11 settings just to have 5 in the middle ? -q0 & -q1 were proven ABXable, you could as well remove -q9 & -q10 & have a 7 settings scale with -q3 in the middle being the actual -q5 ... it would be clearer. Anyway it's just my thoughts, I know I don't understand all the logic behind & I am not the dev afterall, in the end hybrid audio is a real headhache

Thks for taking the time to answer.

I may look dumb, but with my little knowledge it sounds like lossywav is cleverly removing hard to compress inaudible noise & then cleverly dithering easy to compress inaudible noise ? no.
so the problem I heard was not a "quantization like distortion" but more a audible "bad dithering" ?

No question is dumb - ignoring the answer would be .  By default lossyWAV does not dither at all and does not know if any noise is easily compressed or not.

also where does low, medium & high setting starts & ends ... that's a big problem for a noob, with the old 3 settings best/transparent/more agressive it was easier not to make mistake, I think both a scale for freedoom & recommended settings for noobs are usefull & complementary.

Why not just use the default value - it is included just this reason.

not everyone is willing to annoy the dev with stupid questions or to read plenty of technical threads just to know how to use correctly the codec. I know where to find the original thread & wiki help, the wiki were too general, the original discussion too technical. (& I was too lazy  )

Maybe the wiki needs more work, however I thought it was clear enough....

Thks for taking the time to answer.

No problem.

Nick.

[edit]

finally I don't get the use of any quality setting above Q5, if the original idea was proven correct so far what will a "more transparent than transparent" bring ? if it doesn't bring anything for transparency then it must bring at last some added safety for transcoding, is it technically the case at last on paper ? Is there 11 settings just to have 5 in the middle ? -q0 & -q1 were proven ABXable, you could as well remove -q9 & -q10 & have a 6 settings scale with -q3 being the actual -q5 ... it would be clearer. Anyway it's just my thoughts, I know I don't understand all the logic behind & I am not the dev afterall, in the end hybrid audio is a real headhache

You are correct, the higher the quality settings give less likelihood of problems in transcoding. I explained already the evolution of the numerous quality presets - different people have commented on the project at different times with with different aims - this has pulled the quality presets wider and wider. I also wanted to have a highest quality level which still gave some bitrate reduction compared to fully lossless (-q 10 gives at least a 20% bitrate saving on average). I cannot tell you which quality preset to use - that's up to your own evaluation and compromise between perceived quality and resulting bitrate when subsequently encoded.
[/edit]

The issue might be the 11 bits removed around 1.1 seconds. They are removed because the signal is considered by the algorithm to be strong enough to mask the noise - in this instance obviously not. More analysis of the spectral makeup of the sample around the 1.1 second mark is required.

Well, the issue I hear is (I think) quick changes in noise intensity. They're highlighted in this picture:
[a href="http://img149.imageshack.us/my.php?image=32639078hq9.png" target="_blank"] )

Quote from: Nick.C on 2008-05-22 21:32:17

The issue might be the 11 bits removed around 1.1 seconds. They are removed because the signal is considered by the algorithm to be strong enough to mask the noise - in this instance obviously not. More analysis of the spectral makeup of the sample around the 1.1 second mark is required.

Well, the issue I hear is (I think) quick changes in noise intensity. They're highlighted in this picture:
[a href="http://img149.imageshack.us/my.php?image=32639078hq9.png" target="_blank"] )

How long are the steps in the signal - if it's a multiple of 11.6msec (512 samples) then the steps are caused by different bits-to-remove being applied to the output and adding different intensities of white noise.

How long are the steps in the signal - if it's a multiple of 11.6msec (512 samples) then the steps are caused by different bits-to-remove being applied to the output and adding different intensities of white noise.

Yes, they are.
But I can't see direct relationship between amplitude of noise and numbers that lossyWAV prints with --detail switch.  I can't find pattern

...
   80    0.93s. 10 10 10  9  9  9  9  8  8  8  8  9  8  8  9 11 143
   96    1.11s. 10 11 11 11 11 11 11 11 11 10 11 10 10 10 10 10 169
  112    1.30s. 10 10 10  9 10 10  9  9 10 10 10 10  9  9  9 10 154
...

Maye we should add a sticky somewhere. Inexperienced users should not touch any quality setting and will use the defaulted Q5 ??

Maye we should add a sticky somewhere. Inexperienced users should not touch any quality setting and will use the defaulted Q5 ??

That sounds like an eminently sensible suggestion - it's now on my to do list for the wiki / --help text.

I had another idea: create a few 'presets' that are mapped to common Q settings. Maybe even hide the Q scale from the normal screen and document it with --longhelp or similar.

-normal [Default] = Q5
-medium = Q3
-Portable = Q2

- Extreme / Archiving / transcoding = Q6..10

I had another idea: create a few 'presets' that are mapped to common Q settings;

-normal [Default] = Q5
-medium = Q3
-Portable = Q2

- Archiving / transcoding = Q7

Good suggestion - also added to to-do-list. Suggested names and associated -q value:

--highest := -q 10;
--high := -q 7.5
--archive := -q 7.5
--default := -q 5;
--normal := -q 5;
--portable := -q 2.5.
--nasty := -q 0

When I make this modification, I will de-emphasize the -q <n> settings in the --help text.

This is a sensible compromise position which retains the current flexibility for advanced users while giving the less experienced user a big hint as to what setting to use. In light of recent conversation on the topic it is a required modification and will be posted today.

Quote from: shadowking on 2008-05-23 08:21:40

I had another idea: create a few 'presets' that are mapped to common Q settings;

-normal [Default] = Q5
-medium = Q3
-Portable = Q2

-extreme = Q6..10

Good suggestion - also added to to-do-list. Suggested names and associated -q value:

--highest := -q 10;
--high := -q 7.5
--default := -q 5;
--normal := -q 5;
--portable := -q 2.5.
--nasty := -q 0

....

Yeah, I like this very much: shadowking's approach and your incarnation cause it looks like -q 2.5/5/7.5/10 are really those settings which IMO are of most practical use (7.5 or 10 for the main part as a substitute for lossless archivíng).
I'd just prefer one '-', so '-normal' instead of '--normal', I would drop '-default' as a synonym for '-normal' (of course '-normal' should be defaulted), and we shouldn't use a nasty mode IMO.
The '-q' parameter can remain as an advanced '--q' setting for the lovers of quality fine tuning.

The solution is so wonderful to me as we have seen a growing discussion on quality levels since we started having a finer quality granularity. This is covered by this new self-explaining quality granularity while still keeping up fine tuning abilities for those who like them.

That's exactly what I was suggesting, except that as I think -q 0 & -q 10 are not really recommandable I would not add them as recommended setting, & now that I understund the symbolic importance of -q 5 I would add it.

so ideally it would look like this for me

-q 2.5 = -portable (my "-near lossy")
-q 5 = -default
-q 7.5 = -archive (my "-near lossless")

-q0 & -q10 looks too much like flac -fast & -best which makes sense for flac but are sub-obtimal here.

I think that after some transcoding tests I would then either use the -default or -archive preset ...

Edit:
I agree with halb27 too, one - is enough, the easier the better.

If you really can't do without adding -q 0 & -q 10 to the presets then I would use maybe:
-q 0 = -noisy or -this-is-trash-metal
-q 10 = -overkill or -music-for-the-deaf
names that naturally sounds bad, but my english is too weak here.

...
-q 2.5 = -portable (my "-near lossy")
-q 5 = -default
-q 7.5 = -archive (my "-near lossless")
...

This brings us back to the 3 quality levels approach, but with the advantage of a proper naming.
Having thougt about it I also think it's better to stay with three levels.

I personally have dropped my lossless archive and encoded everything to -q 7 (for the most part), but on more or less rare occasion - depending on the meaning the track under consideration has to me - also used a higher -q setting up to -q 10 (and in very rare cases I encoded to lossless wavPack which I can use on my DAP). But it is really enough to use -archive as a rule, and I can still use a higher -q setting if available as an advanced option on those rare occasion where I want it.

So I suppoort this IMO better differentiation standard quality options / advanced quality options.

As a detail I suggest to use --shaping 0.5 with the -archive quality (more clearly in terms of the advanced options: with every -q setting from -q 7 up.
It makes the noise so much less audible (often totally inaudible when listening only to the noise containing correction file with the same volume setting as with the real music), and IMO is safe to use with these very high quality settings.

...
If you really can't do without adding -q 0 & -q 10 to the presets then I would use maybe:
-q 0 = -noisy or -this-is-trash-metal
-q 10 = -overkill or -music-for-the-deaf
names that naturally sounds bad, but my english is too weak here.

I really wouldn't like -q 0 among the standard options.
-q 7.5 should be overkill already, so it's hard to find a simple name for -q 10 (maybe -paranoid hits it best - but it's a pretty negative word. However if most members would appreciate a -nasty mode among the standard options a -paranoid mode would be a good counterweight).
Another argument for having just three standard quality options.

so if we mix, first we have something like this :

-q 00.0 = -nasty
-q 02.5 = -portable
-q 05.0 = -archive or -default
-q 07.5 = -overkill
-q 10.0 = -paranoid

it's not my ideal, but as I am not alone to use the codec ... it's better than nothing & I can live with it
& now if we kill -nasty & -paranoid, but agree that -paranoid is a better word than -overkill, we have.

-q 02.5 = -portable
-q 05.0 = -archive or -default
-q 07.5 = -paranoid

I like it like this.

Edit:
As -portable & -archive describe the use rather than the quality (unlike -nasty, -overkill & -paranoid)
maybe it would be better to use 3 words that all describe the final use, so:

-q 02.5 = -portable
-q 05.0 = -archive or -default
-q 07.5 = -transcoding

is possible too & even more self-speaking for noobs.

I had another idea: create a few 'presets' that are mapped to common Q settings. Maybe even hide the Q scale from the normal screen and document it with --longhelp or similar.

-normal [Default] = Q5
-medium = Q3
-Portable = Q2

- Extreme / Archiving / transcoding = Q6..10

That's how I do it for the moment. My batchfile exists of three settings to test. Q9 for near-lossless archives, Q6 for playable flac images and Q3 for DAP. I mentioned earlier that imho the eleven step scale is too wide. Let alone the numerous steps in between. 5.5678 or 0.1234    To hear the differences quickly I take these bigger steps.
There's also the various combo's in shaping, detail, lowpass. A half-deaf noob would be going mad.

[...], so it's hard to find a simple name for -q 10 (maybe -paranoid hits it best - but it's a pretty negative word. However if most members would appreciate a -nasty mode among the standard options a -paranoid mode would be a good counterweight).
Another argument for having just three standard quality options.

For -paranoid (or -riskless) I use flac.    And yes, maybe shaping and such should be an embedded part of the presets. so an inexperienced user won't have to bother about these settings. Like -e -r -b 4096 in flac; just -best or -fast.

Combining some previous suggestions posted here, these are the presets I'd like to see:

--q 2.5 = -portable
--q 5.0 = -normal
--q 7.5 = -archive

Of course, the presets would be mutually-exclusive with the "--q" advanced setting.

I wonder if -archive is a good term afterall as it really depends on what people intend to do with their "archive", personnaly I would keep -archive out of the game & keep it for lossless. Some people use lossy for archive because it's transparent to them, some people use lossless for archive as it is perfection to them. There is no consensus on it. So, I slowly tend toward:

-q 2.5 = -portable
-q 5.0 = -default
-q 7.5 = -transcoding

you could as well use -q 5 or -q7.5 for archival purpose as there is no evidence that transcoding -q 7.5 to lossy for DAP would be more transparent than transcoding -q 5.0 to lossy for DAP even if -q 7.5 is less agressive on paper.

So the main problem is whether the naming should address the application (like 'portable', 'archive', 'transcoding') or the quality.

Targeting at the application has its problems as we see in the discussion. I for instance wouldn't like -q 5 to be associated with archiving (archiving in the sense of a substitute for a lossless archive). I also don't like a specific 'transcoding' quality as I'd expect even from -q 2.5 that it should be transcodable to, say mp3, without really sacrifying quality.
These things are all a matter of taste and personal context, and as we see, an application oriented naming scheme leads to these subjective ambiguities.

So I think we're better off with the quality targeting approach, but here it's harder to find names.
My suggestion in clear but rather bad sounding words:

-goodEnough (for -q 2.5)
-transparent (for -q 5)
-safetyMargin (for -q 7.5)

or in short word form:

-fine (for -q 2.5)
-transparent (for -q 5)
-overkill (for -q 7.5)

with 'overkill' back as a rough equivalent to 'safetyMargin'. The added emotionality of 'overkill' isn't bad in this context IMO. It conflicts less with 'transparent' than a 'safetyMargin'. With this quality scale I wouldn't mind using 'overkill' though 'overkill' for -q 7.5 has the disadvantage that -q 5 already is expected to be overkill for most of us.

IMO this describes well what the quality settings are about.

Sure -transparent is a claim (after all we're in a world of lossy codecs), but IMO not a bad one. In case -transparent should be found not to be transparent on a sample it is a challenge to change the -transparent internals in order to reestablish transparency.

I think:

-medium (2.5)
-normal (5)
-insanity / crazy (7+)


This way things are simple and we are down to 3 real world scenarios - modest needs, normal or don't want to be bothered knowing.. then Q5 caters for the clueless while keeping true to the goals of lossywav real transparent sound at a much reduced bitrate compared to lossless. 'Insanity' gives a real hint to people that we are in overkill bitwaste territory but nonetheless it is there as an 'end to all' setting and can be considered a true replacement for lossless coding. -Medium attempts to compromise between quality and size while protectign the user from unfavourable situation where noise will be audiable and annoying - in other words -medium is probably enough for 98 % + cases but we don't want to risk ugly situations by going too low like Q0.

snip..

or in short word form:

-fine (for -q 2.5)
-transparent (for -q 5)
-overkill (for -q 7.5)

I like it . Its quite clean and concise.

What happened to this thread?

Can I "me too" give my opinion

-low = -q 1
-medium = -q 2.5
-normal = -q 5 (default)
-high = -q 7.5
-insane = -q 10

OK?
The doc should not give any quality expectation for the different -q values, instead the documentation of presets should tell the corresponding -q option.

It really seems to be boiling back down to an equivalent of -1, -2 and -3 per the original development....

How about:

-P, --portable = -q 2.5;
-N, --normal = -q 5.0 (default);
-E, --excessive = -q 7.5?

you could as well still use word describing targeted use & include quality expectation as a clue in the help:

-q 2.5 = -portable (good enough)
-q 5.0 = -default (transparent)
-q 7.5 = -transcoding (safety margin)

quote jido:
-low = -q 1

... ??? sorry but it starts at -q 0, you'd rather not give your opinion if you don't use the codec    unless it's a typo indeed  just teasing I already made the same typo

lossyWAV beta 1.0.1i attached to post #1 in this thread.

Probably too late, but

medium/portable
standard
extreme
insane

has a good tradition on these boards!

Cheers,
David.

Probably too late, but

medium/portable
standard
extreme
insane

has a good tradition on these boards!

Cheers,
David.

Never too late for you, Sir! Give me 5 mins....

No Nick - calm down!

I wasn't demanding all of these - just suggesting "standard" as a posh way of saying "normal" - it sounds fractionally grander.

I like "transcoding" anyway - it makes sense. When I get chance, I'm going to run some transcoding tests on lossyWAV and see which values are really worth it - so far, we're just guessing!

Given my usual speed of getting around to things, if anyone else wants to try, please jump in!

Cheers,
David.

insane

insane being 100% lossless would be a statement. :-)

Cheers,
SG

I'd like to see another option name changed: --lowpass.
With --lowpass everybody not intimate to the lossyWAV details has the impression that the signal is lowpassed whereas it's lossyWAV's major advantage that there's nothing in the signal path changed at all - for the sake of significantly reducing the bitrate just a little bit of quantization noise is added and controlled so that it's inaudible in the sense the quality settings tell.

The --lowpass parameter addresses the frequency limits of the FFT analyses, so a name like --FFTlimit is more appropriate.

Quote from: 2Bdecided on 2008-05-23 12:36:52

insane

insane being 100% lossless would be a statement. :-)

LOL! I don't think anyone who suggested that could take the flames...

Cheers,
David.

... just suggesting "standard" as a posh way of saying "normal" - it sounds fractionally grander. ...

I feel like this as well, and I really like your naming schemes 'medium'/'standard'/'extreme'.

I'd like to see another option name changed: --lowpass.
With --lowpass everybody not intimate to the lossyWAV detail has the impression that the signal is lowpassed whereas it's lossyWAV's major advantage that there's nothing in the signal path changed at all - for the sake of significantly reducing the bitrate just a little bit of quantization noise is added and controlled so that it's inaudible in the sense the quality settings tell.

The --lowpass parameter addresses the frequency limits of the FFT analyses, so a name like --FFTlimit is more appropriate.

I would prefer --hflimit or --upperlimit or something like that - --FFTlimit is a bit ambiguous....

I'd like to see another option name changed: --lowpass.
With --lowpass everybody not intimate to the lossyWAV detail has the impression that the signal is lowpassed whereas it's lossyWAV's major advantage that there's nothing in the signal path changed at all - for the sake of significantly reducing the bitrate just a little bit of quantization noise is added and controlled so that it's inaudible in the sense the quality settings tell.

The -lowpass parameter addresses the frequency limits of the FFT analyses, so a name like -FFTlimit is more appropriate.

I agree. I'd suggest "analysislimit" but it's longer to type!

I would prefer --hflimit or --upperlimit or something like that - --FFTlimit is a bit ambiguous....

Did you leave "lower limit" internally within the code, or is it gone? Not that I need it - just curious.

EDIT:

Maybe just -limit with the explanation "frequencies higher than this are not analysed"

... and maybe extended help "lossyWAV adds white noise based on analysis of the signals _below_ this frequency; if signals above this frequency are at an even lower level, they they can be swamped by the added noise. This is usually inaudible, but the behaviour can be changed by specifying a higher -limit. For many audio signals, there is little content at very high frequencies, and forcing lossyWAV to keep the noise lower than the content at these frequencies can increase the bitrate dramatically"

or something!

Cheers,
David.

No Nick - calm down!

<deep breath>

I wasn't demanding all of these - just suggesting "standard" as a posh way of saying "normal" - it sounds fractionally grander.

Agreed.

I like "transcoding" anyway - it makes sense. When I get chance, I'm going to run some transcoding tests on lossyWAV and see which values are really worth it - so far, we're just guessing!

Does --transcoding not imply some sort of internal transcoding process rather than the-output-is-considered-to-be-good-enough-for-transcoding?

So, in the spirit of keeping to tradition  :

--insane = -q 10;
--extreme = -q 7.5;
--standard = -q 5.0;
--portable = -q 2.5.

I agree. I'd suggest "analysislimit" but it's longer to type!

-A, --analysislimit or -U, --upperlimit?

Did you leave "lower limit" internally within the code, or is it gone? Not that I need it - just curious.

It's gone - I could relatively easily re-instate it (20<=n<=1378.125) --lowerlimit.

I would prefer --hflimit or --upperlimit or something like that - --FFTlimit is a bit ambiguous....

I would avoid a word that can be associated as a synonym for 'lowpass'.
My first thought was 'analysislimit' like 2Bdecided suggested, but for the sake of shortness I changed it to FFTlimit. 'analysislimit' is more precise of course, so why not use it.

see my edited post above (I can't keep up!)

...
--insane = -q 10;
--extreme = -q 7.5;
--standard = -q 5.0;
--portable = -q 2.5.

I like it - though I'd prefer 'medium' as all the other names refer to quality and not application. But it's not really important.

see my edited post above (I can't keep up!)

--limit is ok IMO though --analysislimit is clearer.

It would be necessary to add an lowpass limit for portable preset.

see my edited post above (I can't keep up!)

-l, --limit <n> it is then. And I'll add something akin to your added text to the --longhelp.

It would be necessary to add an lowpass limit for portable preset.

So you want the --portable quality to be equivalent to what is now -q 2.5 --lowpass 17000?

lossyWAV beta 1.0.1j attached to post #1 in this thread.

Quote from: Mardel on 2008-05-23 13:05:53

It would be necessary to add an lowpass limit for portable preset.

So you want the --portable quality to be equivalent to what is now -q 2.5 --lowpass 17000?

Maybe. We have to vote for the number of a limit. I think the preset not simply sets the quality, but makes fine-tuning too.

... I think the preset not simply sets the quality, but makes fine-tuning too.

I think fine-tuning should be applied to the still available -q quality levels.
So if a --limit aka former --lowpass value of say 17000 is considered useful for -portable, we should use it not primarily with -portable but with -q 2.5 (and probably for every -q value or perhaps for every -q setting below a certain -q value).
Because of the mapping of -portable to -q 2.5 fine tuning is automatically applied to -portable this way.

As you suggested to use an explicit  --limit aka former --lowpass value with -portable, it would be nice to hear your suggestion.

I checked it now, it seems that about what I would have liked to talk was repaired already, so void. Sorry.

You could tie the default -limit value into the -q scale, but people might want to change it far more dramatically because they can't hear above a certain frequency (so no point keeping it noise free) or because they can hear above a certain frequency.

In the latter case, it's really unlikely they can hear lossyWAV noise up there. The threshold of hearing slopes upwards quite steeply up there (though it's age dependent) and masking spreads upwards too - both act to make low level high frequency noise inaudible, even if there's nothing up there to hide it.

So whatever the q scale, 14-18k ish is about right - it would be mad to make it change further automatically.

Cheers,
David.

Here it is a test for those that like to call -q 0 "ugly" "horrible" and the likes.

Here you have a .flac (1' 32'' , a snippet of a song of mine, so no problems with copyrighs), which encodes to 292kbps (flac -5 -b 512) from 948 (flac -8) and which i haven't been able to ABX at the -q 0 setting.

And yes, i've ABX'd easily the problem sample posted here earlier at this same setting.

http://psycle.free.fr/josepma/gameplay-snip.flac (http://psycle.free.fr/josepma/gameplay-snip.flac) (10.4MB)


EDIT:

What i want to point out is that while problem samples are problem samples, the route that should *not* be followed is "up the bitrate/constraints until problem samples are ok". This problem sample could actually be pointing to a flaw, rather than a lower limit, and could be as been suggested, that the fluctuation in bits change could be the root of the cause.
Also, looking at the sample itself, it also produces clipped-like samples (reducing that much the precision, that a sinusoidal signal is converted to a straight line), so there is where the eyes should be looking, rather than to up the internal settings.

Nick, there's a problem with --insane: it's identical to --extreme.

What do you think about using --shaping for --extreme and --insane, or, more precisely, for a -q setting above a treshold? If you like to listen to the correction file you'll hear the added noise is so much lower and often inaudible when only listening to the noise.
Moreover other than with the low quality settings the bitrate penalty of noise shaping isn't large, especially when keeping away a bit from --shaping 1.0.
My suggestion: For -q >= 6: use --shaping 0.4+(q-value - 6)/10.

@ [JAZ]: You're right: -q 0 usually isn't so bad as it looks like after the discussion of Mardel's problem sample.
But I also think we should start a bit higher for the lowest preset and leave the lower bitrate usage to the older -q scheme.
There may be something specific with Mardel's sample that can be fixed, but among my problem samples there are several ones (with definitely no clipping) that are also easily abxable with -q 0. Though I agree they are problem samples, and most of the time the situation is more favorable with -q 0. So it's ok IMO to have -q 0, but for a robust quality which I guess is asked for by many users of a codec like lossyWAV, -q 0 and also -q 1 are a bit low.

Here it is a test for those that like to call -q 0 "ugly" "horrible" and the likes.

Here you have a .flac (1' 32'' , a snippet of a song of mine, so no problems with copyrighs), which encodes to 292kbps (flac -5 -b 512) from 948 (flac -8) and which i haven't been able to ABX at the -q 0 setting.

And yes, i've ABX'd easily the problem sample posted here earlier at this same setting.

http://psycle.free.fr/josepma/gameplay-snip.flac (http://psycle.free.fr/josepma/gameplay-snip.flac) (10.4MB)

EDIT:

What i want to point out is that while problem samples are problem samples, the route that should *not* be followed is "up the bitrate/constraints until problem samples are ok". This problem sample could actually be pointing to a flaw, rather than a lower limit, and could be as been suggested, that the fluctuation in bits change could be the root of the cause.
Also, looking at the sample itself, it also produces clipped-like samples (reducing that much the precision, that a sinusoidal signal is converted to a straight line), so there is where the eyes should be looking, rather than to up the internal settings.

You're right, of course - I'm falling into the same trap as before when I kept trying to make -3 transparent.... I'll have another look at the mechanics and see if anything springs to mind regarding this issue.

Nick, there's a problem with --insane: it's identical to --extreme.

What do you think about using --shaping for --extreme and --insane, or, more precisely, for a -q setting above a treshold? If you like to listen to the correction file you'll hear the added noise is so much lower and often inaudible when only listening to the noise.
Moreover other than with the low quality settings the bitrate penalty of noise shaping isn't large, especially when keeping away a bit from --shaping 1.0.
My suggestion: For -q >= 6: use --shaping 0.4+(q-value - 6)/10.

Mea culpa - I copied the parameter code from --extreme to --insane and forgot to change the 7.5 to 10.0.

I like the idea of some quality related shaping. Say:

quality_shaping_factor : array[0..quality_presets] = (0,0,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0);

as an example?

I'll carry out some comparisons and post the bitrates....

[edit] I immediately thought back to when I was using:

maximum_bits_to_remove:=(bitspersample - minimum_bits_to_keep -1);

and realised that this static method was removed when the dynamic maximum_bits_to_remove was introduced, i.e.:

maximum_bits_to_remove_channel[channel]:=(log2(max(1,RMS of all samples in channel))-quality_minimum_bits-to_keep);

I've reinstated the static method in series with the dynamic method and it stops the 11 bits removed for Mardel's sample.... [/edit]

...
quality_shaping_factor : array[0..quality_presets] = (0,0,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0);
...

When playing around with shaping my personal conclusion was: do it only with high quality settings as otherwise the bitrate increase is too severe. I'm also a bit afraid that with moderate quality settings and a very moderate noise shift unmasked hiss may concentrate in the 6...11 kHz zone where our ears are still pretty sensitive to hiss.
I'd welcome to do it only from a certain -q setting above like from -q 6.

I'm also a bit afraid that with moderate quality settings and a very moderate noise shift unmasked hiss may concentrate in the 6...11 kHz zone

That's odd because the filter (at least at 1.0) doesn't amplify anything below 13 kHz. 11 kHz is still rejected by about 5 dB.

In other news, I've finished implementing & testing new filter code for adaptive noise shaping. This could be one building block for improving lossyWAV or WavPack's lossy-only mode. (Note: It doesn't include a psychoacoustic model. This would be another building block.)

Cheers,
SG

Quote from: Nick.C on 2008-05-23 20:26:05

...
quality_shaping_factor : array[0..quality_presets] = (0,0,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0);
...

When playing around with shaping my personal conclusion was: do it only with high quality settings as otherwise the bitrate increase is too severe. I'm also a bit afraid that with moderate quality settings and a very moderate noise shift unmasked hiss may concentrate in the 6...11 kHz zone where our ears are still pretty sensitive to hiss.
I'd welcome to do it only from a certain -q setting above like from -q 6.

From my discussion with SG, it would seem that the effect moves the noise to the same frequency range no matter how big the factor is (0 to 1, obviously none for 0.0 ). Think of the factor a bit like the interpolation factor between in this case pure white noise and shaped noise. 0.5 would then be 50% of each....

Quote from: halb27 on 2008-05-23 20:47:49

I'm also a bit afraid that with moderate quality settings and a very moderate noise shift unmasked hiss may concentrate in the 6...11 kHz zone

That's odd because the filter (at least at 1.0) doesn't amplify anything below 13 kHz. 11 kHz is still rejected by about 5 dB.

In other news, I've finished implementing & testing new filter code for adaptive noise shaping. This could be one building block for improving lossyWAV or WavPack's lossy-only mode. (Note: It doesn't include a psychoacoustic model. This would be another building block.)

Cheers,
SG

Excellent news SG!

[edit] I processed my (now) 55 problem sample set (Mardel's sample and udial added) and the results are as follows:

|-------------|-------------|-------------|-------------|-------------|-------------|
|  -s factor  |    -q 0     | --portable  | --standard  | --extreme   |  --insane   |
|-------------|-------------|-------------|-------------|-------------|-------------|
|     0.00    |312.11 kbit/s|406.03 kbit/s|485.17 kbit/s|565.05 kbit/s|640.03 kbit/s|
|-------------|-------------|-------------|-------------|-------------|-------------|
|     0.25    |322.66 kbit/s|410.41 kbit/s|487.22 kbit/s|565.80 kbit/s|640.24 kbit/s|
|-------------|-------------|-------------|-------------|-------------|-------------|
|     0.50    |338.95 kbit/s|418.19 kbit/s|491.29 kbit/s|567.58 kbit/s|640.81 kbit/s|
|-------------|-------------|-------------|-------------|-------------|-------------|
|     0.75    |354.77 kbit/s|429.28 kbit/s|498.20 kbit/s|571.04 kbit/s|642.13 kbit/s|
|-------------|-------------|-------------|-------------|-------------|-------------|
|     1.00    |374.46 kbit/s|444.93 kbit/s|508.96 kbit/s|577.15 kbit/s|644.75 kbit/s|
|-------------|-------------|-------------|-------------|-------------|-------------|

Based on these results I would like to propose that shaping_factor:= quality_level/10 for 1.0.1k for testing (will be able to be overridden with -s 0 or --shaping 0).

Quote from: halb27 on 2008-05-23 20:47:49

I'm also a bit afraid that with moderate quality settings and a very moderate noise shift unmasked hiss may concentrate in the 6...11 kHz zone

That's odd because the filter (at least at 1.0) doesn't amplify anything below 13 kHz. 11 kHz is still rejected by about 5 dB.

Thanks for your correction. So there's nothing to fear. clarification.
Oh, I've overseen your '(at least at 1.0)'. That's what I'm a bit afraid of: the lower --shaping factors.

... Think of the factor a bit like the interpolation factor between in this case pure white noise and shaped noise. 0.5 would then be 50% of each....

If this is really so there's nothing to fear.

How about some preset for -q 0, such as "MP3 replacement"?

lossyWAV beta 1.0.1k attached to post #1 in this thread.

How about some preset for -q 0, such as "MP3 replacement"?

mp3 replacement may be the motivation for the one or other user but it's a very personal decision to prefer lossyWAV -q 0 over, say, Lame 3.98b8 --abr 280. Not everybody would agree with such a preference.

Oh, I've overseen your '(at least at 1.0)'. That's what I'm a bit afraid of: the lower --shaping factors.

Here's what happens at --shaping 1.0
(http://img528.imageshack.us/img528/687/ns10xk0.th.png) (http://img528.imageshack.us/my.php?image=ns10xk0.png)

This is the filter at --shaping 0.6
(http://img528.imageshack.us/img528/6995/ns06oc0.th.png) (http://img528.imageshack.us/my.php?image=ns06oc0.png)

At 0.0 it'll be a flat line (0 dB). So, it's like what Nick said + some smoothing.

HTH,
SG

Not everybody would agree with such a preference.

Ok. Newer mind  I meant that once there is a preset for -q 2.5/5/7.5/10, it is logical that also should be a preset for -q 0.

p.S. sbmewb.

... Here's what happens at --shaping 1.0 ...
... This is the filter at --shaping 0.6 ....
So, it's like what Nick said ...

OK, thank you. So we should only expect a quality increase from noise shaping.
Nick's kind of doing it brings up bitrate only to a small degree, so this is expected to be another progress.

... I meant that once there is a preset for -q 2.5/5/7.5/10, it is logical that also should be a preset for -q 0. ...

I only adressed the name 'mp3 replacement' which IMO isn't a lucky choice. As we have 2 settings above standard there is some logic in having two settings below it.

What is the general opinion towards this?

Quote from: no404error on 2008-05-23 22:17:51

... I meant that once there is a preset for -q 2.5/5/7.5/10, it is logical that also should be a preset for -q 0. ...

I only adressed the name 'mp3 replacement' which IMO isn't a lucky choice. As we have 2 settings above standard there is some logic in having two settings below it.

What is the general opinion towards this?

I've lost touch a little. Am I right in thinking:
10 is pretty much the old -1
7.5 is pretty much the old -2
5 is pretty much the old -3

and 2.5 is a new setting that is lower than any setting since LossyWAV moved away from the olden days when life was simple?

If that's approximately the case, I wouldn't go any lower than 2.5, especially since transparency is pretty much guaranteed with MP3 at lower bitrates, why would you choose a "mp3 replacement" (or whatever term you like) when LossyWAV, at sub 2.5 bitrates, is not as safe as MP3? Surely, no one is going to use much less than level 2 as a transcode setting either; as I would have thought you want to guarantee transparency for your transcode source.

But then I don't understand the LossyWAV as DAP audio file. MP3 et al are too competitive aren't they?

EDIT: Though I guess for the sake of symmetry a setting between 2.5 and 5 would work.

C.

Your are kind off right. I still think lossywav can take on and beat Lame 320k at similar bitrates simply due to the mp3 native defect.

Joint frequency encoding

Joint frequency encoding is an encoding technique used in audio data compression to reduce the data rate.

The idea is to merge a given frequency range of multiple sound channels together so that the resulting encoding will preserve the sound information of that range not as a bundle of separate channels but as one homogenous data stream. This will naturally destroy the original channel separation for good, as the information cannot be accurately reconstructed, but this process will greatly lessen the amount of required storage space. It should be noted that only some forms of joint stereo use the joint frequency encoding technique, such as intensity stereo coding.

Quote

Joint frequency encodingJoint frequency encoding is an encoding technique used in audio data compression to reduce the data rate.

The idea is to merge a given frequency range of multiple sound channels together so that the resulting encoding will preserve the sound information of that range not as a bundle of separate channels but as one homogenous data stream. This will naturally destroy the original channel separation for good, as the information cannot be accurately reconstructed, but this process will greatly lessen the amount of required storage space. It should be noted that only some forms of joint stereo use the joint frequency encoding technique, such as intensity stereo coding.

Does not happen in the lossless codecs that lossyWAV is aimed at therefore it makes no sense to use it in lossyWAV. All lossyWAV does is round lower significant bits to zero (now with added noise shaping).

Your are kind off right. I still think lossywav can take on and beat Lame 320k at similar bitrates simply due to the mp3 native defect.

In bitrate/quality perhaps, but not in filesize. Average filesize of my testset Q1 lossyflacs is almost twice the size of my mp3 V1, let alone V4. DAP users won't like that.

Quote from: shadowking on 2008-05-24 05:47:13

Your are kind off right. I still think lossywav can take on and beat Lame 320k at similar bitrates simply due to the mp3 native defect.

In bitrate/quality perhaps, but not in filesize. Average filesize of my testset Q1 lossyflacs is almost twice the size of my mp3 V1, let alone V4. DAP users won't like that.

No arguments there. I think the typical losswav user is:

- Experienced user wanting very high quality and predictable transcoding at half or less bitrate of lossless. Maybe this applies to a clueless user too.
- A more naive user want good enough Flac sound in his / her DAP saving themselves transcoding. They can use lower bitrates and still be very happy. They really should make correction files because normal codec @ 160..190k is similar quality to 260 k hybrid but they are much smaller.

In bitrate/quality perhaps, but not in filesize.

Really?    For bitrate above 256, lossywav better then anybody lossy-codecs.

lame --preset insane - 9.502.369
lame --abr 280 - 8.653.434
lame -b 256 - 7.636.113

oggenc -b 270 - 7.961.898
oggenc -b 280 - 8.350.866

lossywav -q 0 / flac -5 -b 512 --keep-foreign-metadata - 8.087.723

It is an example, but I will make a test for 88 albums if you need.

In bitrate/quality perhaps, but not in filesize. Average filesize of my testset Q1 lossyflacs is almost twice the size of my mp3 V1, let alone V4. DAP users won't like that.

filesize = duration*bitrate.

You've got a point with your reply, but the point itself is a bit out of place:

-V1 or -V4 are not CBR 320kbps ( just like the number 3 is not the number 5 ).
When someone uses such a high bitrate with mp3, the main reason is to get as high quality as possible, with a compatible-enough format. (i.e. not just "transparency")

The point of lossyWav origin was to couple it with the most supported lossless codec (or so was the pretention). Nowadays, lossywav not only works with FLAC, but with other lossless codecs, so possibly increasing the range of situations where it can be used.

Said that, what an user may want from a CBR 320kbps file, may more probably be achieved with lossywav -q 4 or -q 5, which is around 400~450kbps.
But 400~450kbps is the half of what the lossless file requires.

Personally, i see -q 0 and -q 1 as a way to give a lossless-like file at a reduced bitrate, with the added advantage that if the produced file sounds wrong, the quality setting can go up until it dissapears. (not like cbr 320)

I've lost touch a little. Am I right in thinking:
10 is pretty much the old -1
7.5 is pretty much the old -2
5 is pretty much the old -3

It's rather:
--extreme (-q 7.5) is old -1
--standard (-q 5.0) is old -2
--portable (-q 2.5) is old -3.

... I wouldn't go any lower than 2.5, especially since transparency is pretty much guaranteed with MP3 at lower bitrates, why would you choose a "mp3 replacement" (or whatever term you like) when LossyWAV, at sub 2.5 bitrates, is not as safe as MP3? ...

I feel like you for the lowest quality settings though due to the mp3 restrictions shadowking mentioned on occasion lossyWAV may yield the better quality even below 300 kbps.
The real benefits of lossyWAV start pretty much exactly where mp3 has its highest bitrate. So lossyWAV --portable drops in fine where mp3 quality stops.
Moreover with nowaday's DAPs capacities a bitrate of more than 300 kbps can be used without pain by more and more people.

But as is yours this is just my opinion towards a very low bitrate setting.
No matter that I wouldn't like to use a very low bitrate setting I wouldn't care however to have such a quality preset. Maybe it's just asthetics, but with 2 quality presets above --standard it would be nice IMO to have 2 below it. Maybe the lowest preset quality shouldn't be exactly -q 0 (though this would perfectly correspond to the -q setting / quality preset mapping we have so far). v1.0.1k --portable yields a bitrate of 362 kbps with my full length regular music test set. Maybe -q 1 (307 kbps) or -q 1.5 (327 kbps) may be an attractive candidate for the lowest preset quality. Both are quite a bit more attractive than --portable bitratewise, and quality even for problem samples is expected not to be annoying (at least for -q 1.5).

So I'd like to see a lowest quality preset, and my proposal is -q 1.5 for the incarnation. My proposal for the name is just '--low' (pretty much of an understatement at least for -q 1.5, but IMO this matches the quality scale of the other quality presets).

ADDED later:
It just comes to my mind that in a way lossyWAV has a BIG advantage over other lossy codecs: it is LOSSLESSLY FUTURE-SAFE.
Assume we use FLAC as the final codec. Assume in 10 years from now you can't play FLAC on your DAP. You then simply transcode to the then preferred lossless codec which is a lossless process.
Taking this into account it's ok IMO to call the lowest qualty preset (in case it's a little bit higher than -q 0) an 'mp3 alternative' - an alternative of comparable quality as very high bitrate mp3 but with this extra advantage of future-safety.

I'm sorry I didn't follow this thread as closely as I would have liked, but I've observed that LossyWAV 1.01k produces significantly higher bitrates at q0 than 1.01f, which had the lowest I've seen so far and which I was quite satisfied with.  Is there any possibly of remapping the quality scale such that that the low bitrate of 1.01f is reestablished at q0?  It seems a lot like I'm going up a whole quality scale (0 to 1 or the like) from f to k with the same settings...

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

' date='May 24 2008, 05:47' post='566920']
You've got a point with your reply, but the point itself is a bit out of place:
-V1 or -V4 are not CBR 320kbps ( just like the number 3 is not the number 5 ).
When someone uses such a high bitrate with mp3, the main reason is to get as high quality as possible, with a compatible-enough format. (i.e. not just "transparency")

V4 is not that high with 160 kbps. And I've seen numerous discussions about smaller filesizes for limited diskspace on mp3-DAPs. But I only interfered here because the lower q settin was proposed to be 'mp3' somewhere. Whether transcoding from a flac image to lower flac or to mp3 doesn't matter for me..

The names are only important to the non-technical users. Tweakers will test the various q-settings.

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

Sounds like there's hope for improving the very low bitrate settings.

BTW, Nick, thanks for changing the scale factor and for adding the --writelog.

All of a sudden redirecting stopped working for "lossywav --longhelp>>longhelp.txt" a few versions ago. 

... I am really perplex about people telling lossywav -q0 or -q1 is a viable alternative to  any modern lossy codec at VBR 256 or 320Kbps for any use ... first using 320Kbps instead of 256Kbps even for a paranoid user is always sub-optimal, no matter the modern lossy codec 256Kbps is always a better choice than 320Kbps, 320Kbps has always been the "my-bitrate-is-higher-than-yours-so-my-files-are-better-than-yours" setting of newbies, if there would be a 321Kbps setting these knowledge-less newbies would use it anyway.
So I wouldn't compare lossywav -q 0/-q 1 to lossy 320Kbps but to lossy 256Kbps.

Now comparing any modern lossy codec at 256Kbps with lossywav -q0, you compare, heavyly tested techniques (DCT), on files that are smaller & 99.9% transparent (& even when it's not transparent, it's listenable & only ABXable by people with golden ears on specific problems samples) ... with a technique that is still beta, have been proven faulty at -q0 & -q1 & produces bigger files.

I like lossywav, but I would moderate your enthousiasm, there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise or by quality vs. space. As far as I tested any modern lossy codec is more likely to be transparent at 256Kbps VBR than lossywav -q 0.

At -q2.5 it seems it achieves the same quality as pure lossy 256Kbps (transparency), but what you gain in transcodability you lose it in hard disk space, there is no magic. As far as I tested lossywav is not the magic codec that have all the advantage of lossless (quality) & lossy (space) ...

I see it as spreading audio bullshit ... so please people stop telling lossywav -q 0 is "almost" the same as lossless, it is definitly not. That being said I understand the enthousiasm as lossywav is the first really interesting hybrid codec IMHO. It seems a lot of people are using -q 0 thinking its quality is like -q 2.5 (transparent) ... it's not & that's exactly why it was left out of the presets. If you want an alternative to lossy 256Kbps, use -q2.5.

Quote from: lvqcl on 2008-05-24 15:30:27

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

Sounds like there's hope for improving the very low bitrate settings.

Well, if noise added by lossyWAV is big enough so one can hear it -- he also can hear how it changes its volume (by 6db) several times per second. And these jumps are more annoying for me than the noise itself. 

BTW, Nick, thanks for changing the scale factor and for adding the --writelog.

All of a sudden redirecting stopped working for "lossywav --longhelp>>longhelp.txt" a few versions ago. 

Sorry - all text output now goes to "con" and not STDOUT - so that the --stdout parameter works correctly.

I'll make sure that I include the --longhelp text with each beta release at post #1.

Quote from: halb27 on 2008-05-24 15:39:31

Quote from: lvqcl on 2008-05-24 15:30:27

I tried to encode "ginnungagap..." sample using lossyWAV 1.0.1k and various options.
For me, "-q 0 -m 5.333" sounds better than "-q 1" for that particular sample. Both results in 335 kbps bitrate.

Sounds like there's hope for improving the very low bitrate settings.

Well, if noise added by lossyWAV is big enough so one can hear it -- he also can hear how it changes its volume (by 6db) several times per second. And these jumps are more annoying for me than the noise itself. 

Unfortunately, bits being what they are, if you round off an extra bit the noise will jump by 6dB.

My suggestion would be to use noise shaping and possibly increase --minbits from 2.333. You may also want to push --limit up to circa 17kHz.

... I am really perplex about people telling lossywav -q0 or -q1 is a viable alternative to  any modern lossy codec at VBR 256 or 320Kbps for any use ... first using 320Kbps instead of 256Kbps even for a paranoid user is always sub-optimal, no matter the modern lossy codec 256Kbps is always a better choice than 320Kbps, 320Kbps has always been the "my-bitrate-is-higher-than-yours-so-my-files-are-better-than-yours" setting of newbies, if there would be a 321Kbps setting these knowledge-less newbies would use it anyway.
So I wouldn't compare lossywav -q 0/-q 1 to lossy 320Kbps but to lossy 256Kbps.

Now comparing any modern lossy codec at 256Kbps with lossywav -q0, you compare, heavyly tested techniques (DCT), on files that are smaller & 99.9% transparent (& even when it's not transparent, it's listenable & only ABXable by people with golden ears on specific problems samples) ... with a technique that is still beta, have been proven faulty at -q0 & -q1 & produces bigger files.

I like lossywav, but I would moderate your enthousiasm, there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise or by quality vs. space. As far as I tested any modern lossy codec is more likely to be transparent than lossywav -q 0.

At -q2.5 it seems it achieves the same quality as pure lossy 256Kbps (transparency), but what you gain in transcodability you lose it in hard disk space, there is no magic. As far as I tested lossywav is not the magic codec that have all the advantage of lossless (quality) & lossy (space) ...

I see it as spreading audio bullshit ... so please people stop telling lossywav -q 0 is "almost" the same as lossless, it is definitly not. That being said I understand the enthousiasm as lossywav is the first really interesting hybrid codec IMHO. It seems a lot of people are using -q 0 thinking its quality is like -q 2.5 (transparent) ... it's not & that's exactly why it was left out of the presets. If you want an alternative to lossy 256Kbps, use -q2.5.

Go and read the development thread. The lowest quality presets (also, coincidentally numerically the lowest....) are not pretending to be anything other than a lower bitrate version of the higher quality presets - but with a greater probability of artifacts. The lowest bitrate settings have always been pushed by myself and a few others who wish a lower bitrate version and are aware enough of the method not to fool themselves into thinking that the resulting files are comparable in any way to lossless.

If you don't like what lossyWAV is - it's a free world (thus far) - so don't use it.

As to "there is no way lossywav -q0 or -q1 actually beats any modern lossy codec at 256kbps transparency-wise" - have you posted your ABX logs on this one. While you're at it go searching for "killer samples" and see how other methods deal with them. Transparent at 320kbps - not always....

I'm sorry I didn't follow this thread as closely as I would have liked, but I've observed that LossyWAV 1.01k produces significantly higher bitrates at q0 than 1.01f, which had the lowest I've seen so far and which I was quite satisfied with.  Is there any possibly of remapping the quality scale such that that the low bitrate of 1.01f is reestablished at q0?  It seems a lot like I'm going up a whole quality scale (0 to 1 or the like) from f to k with the same settings...

You could try: -q 0 -m 1.5 --spf 22222-22223-22224-12234-12345-12356 as this should revert to the previous spreading function that you were happy with and also I remember that in 1.0.1f the dynamic minimum-bits-to-keep parameter was not being applied. Have fun!

Quote from: Nick.C on 2008-05-21 18:40:12

Quote from: Mardel on 2008-05-21 18:27:49

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

Thanks for the sample, at what position in the audio are you hearing the distortion? ...

I just abxed sec. 6.4-8.6 8/10 (and I'm sure those who know this track will do better). I wouldn't call this spot heavily distorted though, but that's a matter of taste.
We know -q 0 isn't a real quality mode (though usually it's quite okay). My personal favorite for low bitrate is -q 1.5. It's not significantly higher in bitrate (312 kbps on average for my regular music test set) but to me quality does a little jump there. The spot I mentioned with your track, Mardel, is ok to me using -q 1.5.

Mardel and sauvage78, do you mind trying -q 1.5?

Hmmm.. Its really obvious and i didn't even try to abx - blocky flactuating noise @ 300 k . Its what i hear with wavpack @ 196 k . The trouble is that the track isn't very dynamic either so there could be much bigger problems for classical music. Its too dangerous (Q0) IMO for lossywav. Q0 could be a quality collapse zone.

Quote from: halb27 on 2008-05-21 19:29:02

Quote from: Nick.C on 2008-05-21 18:40:12

Quote from: Mardel on 2008-05-21 18:27:49

There is a somthing up.
I'm hearing heavily distorted sound at -q0 (http://href.hu/x/5rff).

Thanks for the sample, at what position in the audio are you hearing the distortion? ...

I just abxed sec. 6.4-8.6 8/10 (and I'm sure those who know this track will do better). I wouldn't call this spot heavily distorted though, but that's a matter of taste.
We know -q 0 isn't a real quality mode (though usually it's quite okay). My personal favorite for low bitrate is -q 1.5. It's not significantly higher in bitrate (312 kbps on average for my regular music test set) but to me quality does a little jump there. The spot I mentioned with your track, Mardel, is ok to me using -q 1.5.

Mardel and sauvage78, do you mind trying -q 1.5?

Hmmm.. Its really obvious and i didn't even try to abx - blocky flactuating noise @ 300 k . Its what i hear with wavpack @ 196 k . The trouble is that the track isn't very dynamic either so there could be much bigger problems for classical music. Its too dangerous (Q0) IMO for lossywav. Q0 could be a quality collapse zone.

Maybe this would be a problem that could be dealt with by using an FFT length longer than 1024 samples, or a codec-block length greater than 512 samples.

We kept the codec-block length short to specifically allow the bits-to-remove to change rapidly rather than calculating over longer blocks. This may, at -q 0, be counter productive. As David said earlier - if no artifacts are noticable at -q 0 there will be questions as to why the bitrate is not pushed lower by more aggressive settings.

As an aside, I've just finished transcoding the portion of my collection in FLAC on my server (3838 tracks, 11 days 12 hours) : 102GB, 886kbps FLAC > lossyWAV --portable / FLAC -5 -b 512: 42.1GB, 363kbps.

I've encoded that sample with wavpack 4.5b @ 235 kbit -x ABR and the quality is way better. Tried wavpack @ 196 k and I still find the quality better. Perhaps low bitrate + noiseshaping + ABR has a strength here.

Abx:

-lossywav sample - obvious - no need
- Wv 196 - obvious - no need
- Wv 235 - hiss on drum crunch 8/8 easy
- Wv 270 - can't abx

Maybe its not a fair comparison as lossywav is working way outside its quality trigger areas here while wavpack can often operate nicely even in lowish bitrates vs other hybrids (shorten, rkau, maybe losswav) as Bryant has stated once.

I've encoded that sample with wavpack 4.5b @ 235 kbit -x ABR and the quality is way better. Tried wavpack @ 196 k and I still find the quality better. Perhaps low bitrate + noiseshaping + ABR has a strength here.

Maybe its not a fair comparison as lossywav is working way outside its quality trigger areas here while wavpack can often operate nicely even in lowish bitrates vs other hybrids (shorten, rkau, maybe losswav) as Bryant has stated once.

I think that this sample is one of those where the particular properties which cause lossyWAV to produce suboptimal output need to be determined, understood and mitigated. It reminds me of when we were having problems with Atem_Lied and David Bryant pointed out that the lowest bins were in the circa 200Hz area - we fixed that with the skewing parameter....

Time to get the thinking cap out again and work out this particular problem.

I've been abx fatigued for a while + a switch to linux. I've got foobar abx working with WINE and my strength is coming back so hopefully I can provide more abx input and samples. I still need to setup lossywav encoder settings etc.

@shadowking:
Yes, I think wavPack lossy does a better job below 300 kbps, as is expected from most of the usual codecs at a comparable bitrate.
I guess it's favorable in this bitrate area that wavPack lossy has a) the lossy variant integrated into the usual machinery with especially no restrictions to the blocksize and b) a dynamic noise shaping procedure which seems to work pretty well.
Moreover lossyWAV at such a low bitrate runs very much below its safe quality control mechanisms so that it's rather a miracle that it does work so relatively good.
The low bitrate settings have the advantage of being easy to test and to improve the machinery. Maybe enlarging the minimum_bits_to_keep is enough to increase the quality.

Anyway and apart from all improvement we will get hopefully I'd like to repeat my proposal for another '--low' preset which maps to 1.0.1k's -q 1.5.
It may bring down the discussion a bit on -q 0.

Sure I understand. People naturaly explore the lowest settings in any codec. Wavpack newbies probably went straight for 196 k esp non HA people, then were shocked.

@shadowking:
Yes, I think wavPack lossy does a better job below 300 kbps, as is expected from most of the usual codecs at a comparable bitrate.
I guess it's favorable in this bitrate area that wavPack lossy has a) the lossy variant integrated into the usual machinery with especially no restrictions to the blocksize and b) a dynamic noise shaping procedure which seems to work pretty well.
Moreover lossyWAV at such a low bitrate runs very much below its safe quality control mechanisms so that it's rather a miracle that it does work so relatively good.
The low bitrate settings have the advantage of being easy to test and to improve the machinery. Maybe enlarging the minimum_bits_to_keep is enough to increase the quality.

Anyway and apart from all improvement we will get hopefully I'd like to repeat my proposal for another '--low' preset which maps to 1.0.1k's -q 1.5.
It may bring down the discussion a bit on -q 0.

Yes - a --lowest parameter might be a good idea. I'm going to look more closely at the detailled output for Mardel's sample (at present the --detail parameter only outputs one channel....  ). I will also have a look at implementing an optionto limit the increase in number of bits to remove between blocks (something we tried before but discounted.....).

@shadowking - your ear-time will be greatly appreciated.

Motivated by lvqcl's good experience with a higher -m value I played around a bit with my full length regular music track set:

a) bitrate increase for -m 5:

--insane:     611 kbps > 611 kbps
--extreme:  534 kbps > 534 kbps
--standard: 452 kbps > 460 kbps
--portable:  362 kbps > 385 kbps
-q 1.5:        327 kbps > 358 kbps
-q 0:           266 kbps > 332 kbps

b) bitrate increase for -m 4:

--standard: 452 kbps > 453 kbps
--portable:  362 kbps > 367 kbps
-q 1.5:        327 kbps > 334 kbps
-q 0:           266 kbps > 290 kbps

Looking at this it looks reasonable to use -m 5 for the -q settings which correspond to --insane, --extreme, and maybe --standard.
For the lower settings the bitrate penalty is a bit high though we may well reconsider the internal mappings of -q x to -nts y and -snr z in case a relatively high -m value proves to be essential.
-m 4 however IMO is appropriate also for the low bitrate settings (if we allow -q 0 to go up significantly in bitrate), and we should never use a lower -m value.

Thinking about the meaning of minimum-bits-to-keep I wonder now about the use of a very low value of ~2.5. I guess we should want to have a little bit more bits to be kept even for the lowest setting.

So I think lvqcl's listening test with a relatively high -m value (compared to what we had so far) is very valuable, and going up with -m was already taken into consideration by yourself, Nick.

What about mapping unsafe settings like Q0 or similar to Q 1.5 ?? Advanced users can manually use switches to get the real Q0. The lame devs did similar mappings.

What about mapping unsafe settings like Q0 or similar to Q 1.5 ?? Advanced users can manually use switches to get the real Q0. The lame devs did similar mappings.

Do you mean make the "new" -q 0 aka --lowest = current -q 1.5? This would require all of the internal parameters that go to make up the preset to be available from the command line (no real problem).

or a "hidden" preset selection parameter for old -q 0? 

I guess -m 4 for -q <5, -m 5 for -q >= 5 (or a gliding -m like -m 4+0.15*q), and a --low preset which maps to -q 1.5 does it all.

I downloaded source code for 1.0.0b version.
Nick.C, can I ask you to add a new option? BTRlimit or something like this.
Then, in procedure process_this_codec_block insert one line

if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit;

just before calling remove_bits:

...
          if min_fft_result.sminbin>min_fft_result.savebin then
            fft_average:=fft_average+1
          else
            fft_minimum:=fft_minimum+1;
        end;
      end;

>>  if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit; << that line

    remove_bits;

    if (detailled_output=1) and (silent=0) then
      remove_bits_output
    else
      inc(current_codec_block);
...

I downloaded source code for 1.0.0b version.
Nick.C, can I ask you to add a new option? BTRlimit or something like this.
Then, in procedure process_this_codec_block insert one line

Code: [Select]

if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit;

just before calling remove_bits:

Code: [Select]

...
          if min_fft_result.sminbin>min_fft_result.savebin then
            fft_average:=fft_average+1
          else
            fft_minimum:=fft_minimum+1;
        end;
      end;

>>  if min_fft_result.btr>BTRlimit then min_fft_result.btr:=BTRlimit; << that line

    remove_bits;

    if (detailled_output=1) and (silent=0) then
      remove_bits_output
    else
      inc(current_codec_block);
...

I'd already re-introduced that as the static_maximum_bits_to_remove method I reintroduced at 1.0.1k.

However.... Thanks to Mardel's sample and my desire to see what was happening with it, I "fixed" the detailled output to show per-channel bits-to-remove and I think I have found a problem whereby some channels are having more bits-to-remove than I think they should. This will not necessarily be a quick fix. I would hope to post beta 1.0.1m tomorrow night.

However.... Thanks to Mardel's sample and my desire to see what was happening with it, I "fixed" the detailled output to show per-channel bits-to-remove and I think I have found a problem whereby some channels are having more bits-to-remove than I think they should. This will not necessarily be a quick fix. I would hope to post beta 1.0.1m tomorrow night.

I didn't find anything, although I had suspicions. However, I re-implemented the old max-inter-block-change in bits to remove (bits-to-remove can only increase by 1 bit per codec block). This wastes a few bits but should limit the change in noise level to 6dB every 11.6ms (for 44.1kHz audio). I would appreciate feedback to see if this has improved Mardel's sample at all at -q 0.

lossyWAV beta 1.0.1m attached to post #1 in this thread.

later today I will read the last pages of this thread, I am willing to test Mardel's sample at -q 0 V1.0.1l vs. -q 0  V1.0.1m, it shouldn't be too long/hard, unfortunatly I don't have V1.0.1l, I only have V1.0.1j & m, so if someone could repost V1.0.1l, I could do the job. Also if you have any sample with heavy problem at -q 0 to -q 2 like Mardel's sample, I am interested as long as it's not beyond my earing possibilities.

I also plan to do a quick transcodability test for my own use, I'll pass the Castanets sample thru lossywav -q 2.5 + nero aac VBR 96Kbps vs direct nero aac VBR 96Kbps & see if I can ear a difference, I test at 96Kbps because at 128Kbps it becomes hard to ABX (but still possible) & at 64Kbps with SBR I think it's too agressive if there would be a difference it would be destroyed anyway.

later today I will read the last pages of this thread, I am willing to test Mardel's sample at -q 0 V1.0.1l vs. -q 0  V1.0.1m, it shouldn't be too long/hard, unfortunatly I don't have V1.0.1l, I only have V1.0.1j & m, so if someone could repost V1.0.1l, I could do the job. Also if you have any sample with heavy problem at -q 0 to -q 2 like Mardel's sample, I am interested as long as it's not beyond my earing possibilities.

I also plan to do a quick transcodability test for my own use, I'll pass the Castanets sample thru lossywav -q 2.5 + nero aac VBR 96Kbps vs direct nero aac VBR 96Kbps & see if I can ear a difference, I test at 96Kbps because at 128Kbps it becomes hard to ABX (but still possible) & at 64Kbps with SBR I think it's too agressive if there would be a difference it would be destroyed anyway.

Thanks for volunteering your listening time, it is much appreciated. The beta 1.0.1l that you are looking for never existed as the character 'l' is too similar to '1' and '|' in some fonts. Things have moved on a bit and I have made enough changes to warrant the release of a new beta version.

From memory (correct me if I'm wrong, halb27) eig, triangle_2_1644ds, furious and badvilbel would be good (i.e. problematic) samples to try.

lossyWAV 1.0.1n attached to post #1 in this thread.

sorry I was meaning V1.01k,

I started by my little transcoding test ... so far it's an ABX fiasco  , I couldn't tell the difference between a Direct nero AAC Q0.35 (96Kbps) Vs. a Lossywav -q2.5+Nero Q0.35 (96Kbps) transcoding ... I tried at Nero 64Kbps & I failed too, everything sounded terrible but I couldn't spot any regression due to lossywav q2.5 ... so I am currently testing a nero Q0.65 (256Kbps)+nero Q0.35 (96Kbps) transcoding Vs. a lossywav q2.5+nero Q0.35 (96Kbps) transcoding, just to see how bad is nero aac 256Kbps as a source compared lossywav q2.5 which I can't ABX. All this on the castanets sample.

sorry I was meaning V1.01k,

I started by my little transcoding test ... so far it's an ABX fiasco  , I couldn't tell the difference between a Direct nero AAC Q0.35 (96Kbps) Vs. a Lossywav -q2.5+Nero Q0.35 (96Kbps) transcoding ... I tried at Nero 64Kbps & I failed too, everything sounded terrible but I couldn't spot any regression due to lossywav q2.5 ... so I am currently testing a nero Q0.65 (256Kbps)+nero Q0.35 (96Kbps) transcoding Vs. a lossywav q2.5+nero Q0.35 (96Kbps) transcoding, just to see how bad is nero aac 256Kbps as a source compared lossywav q2.5 which I can't ABX. All this on the castanets sample.

In terms of ABX'ing lossyWAV output, the most beneficial (to me) method would be to ABX the lossless original sample against the lossyWAV output rather than ABX processed samples from two versions of lossyWAV. I am very interested to find out if the quality of Mardel's sample when processed with 1.0.1n is as easy to ABX as it was with the lossyWAV version available when he introduced the sample.

Ok, I will test it soon.

If anyone is interested by the result of my personnal little transcoding test, lossywav -q 2.5 vs. Nero q0.65 as a source, it happened that the result was a success: lossywav -q 2.5 is a better source than Nero q0.65 for transcoding, the usual DCT artefact for the castanets sample (flat metallic attack) was slightly more pronounced with nero q0.65 as a source. In the end it is not surprising, but even if I can ABX it 9/12, I wouldn't tell the difference was incredible, you have to know exactly what to listen to, close your eyes & concentrates. So for me the interest of lossywav relies much more in its "splittability+no psychoacoustic" (CDImage.lossy.tak as archive) than in its "transcodability" alone. For me lossywav stands as a lossy codec by itself ... if I ever start using it when it will become stable, I think I will simply stop using Nero AAC & DCT codecs.

Ok, as you wanted me to ABX each lossywav version separatly vs. original, I first tested Lossywav V1.01n -q 1.0 Vs. Original:

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 15:36:25

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q1.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.original.wav

15:36:25 : Test started.
15:36:44 : 01/01  50.0%
15:37:07 : 02/02  25.0%
15:37:28 : 03/03  12.5%
15:37:49 : 04/04  6.3%
15:38:14 : 05/05  3.1%
15:38:46 : 06/06  1.6%
15:40:01 : 07/07  0.8%
15:40:18 : 08/08  0.4%
15:41:24 : 09/09  0.2%
15:42:28 : 10/10  0.1%
15:42:57 : 11/11  0.0%
15:43:23 : 12/12  0.0%
15:43:28 : Test finished.

 ----------
Total: 12/12 (0.0%)

The artefact is still here, still very pronounced. Was easyly ABXable from the first try.
I didn't separatly re-ABX the original sample with a problem by Mardel vs. original as I think it is useless. (See the old log, it would be the same)

Instead I started ABXing the original sample with a problem by Mardel vs. Lossywav V1.01n -q 1.0, as I first wanted to do, in order to see if there was any progression/regression. I could ABX it too so easyly that I stopped after 4 try as I was 100% sure of myself.

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 15:46:19

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q1.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossywithnoise.wav

15:46:19 : Test started.
15:46:41 : 01/01  50.0%
15:47:08 : 02/02  25.0%
15:47:40 : 03/03  12.5%
15:48:21 : 04/04  6.3%
15:48:26 : Test finished.

 ----------
Total: 4/4 (6.3%)

(The following conclusion is wrong see edit.)
The good news is that there is a progression, the bad news is that the artefact is still here & still severe.
I would describe the progression as less deep tearing, softer noise, shorter distortion duration.
The sound return to stability faster after the distortion which has less amplitude.
But this is still comparing rotten apples with rotten oranges, if you compare to the original, the artefact is still severe. I would say it's like a 0.5 quality increase in the old scale of V1.01f not much more than that.

Major Edit:
Bad news, I just realized that I tested -q 0 vs -q 1 in the 2nd ABX test (1st test is valid even if I tested a better setting than I should have) so the improvement I heard must be 100% normal. I must re-test  ... but even if the test is bad, it now almost means for certain that there is near to zero improvement because it can only be worst for V1.01n at -q 0 than at -q 1

Edit2: I re-did the test with the correct parameters, it was un-ABXable  , I cannot tell any progression/regression at all. I fear what you changed between V1.01f & V1.01n has no effect on this specific sample. Sorry ...

lvqcl got good results with Mardel's problem sample using a significantly higher bits-to-keep value of -m 5.333.
-m 5.333 is  bit of a hard bitrate penalty for the very low bitrate settings, but maybe a value of -m 4 brings a significant progress (we don''t need full transparency with -q 0 or -q 1, non-annoyance should be enough).

sauvage78, do you mind trying lossyWAV with the additional option of -m 4?

ok I will try it to see if it helps.

Edit:

V1.01n -q0 -m 4 Vs. original

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 17:35:50

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0-m4.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.original.wav

17:35:50 : Test started.
17:36:13 : 01/01  50.0%
17:36:30 : 02/02  25.0%
17:36:42 : 03/03  12.5%
17:37:03 : 04/04  6.3%
17:37:20 : 05/05  3.1%
17:37:34 : 06/06  1.6%
17:38:05 : 07/07  0.8%
17:38:20 : 08/08  0.4%
17:38:25 : Test finished.

----------
Total: 8/8 (0.4%)

Conclusion 1:
Easy to ABX from original, always the same flaws more or less pronounced.

V1.01n -q0 -m 4 Vs. -q0

foo_abx 1.3.3 report
foobar2000 v0.9.5.3
2008/05/26 17:39:29

File A: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0.wav
File B: C:\Documents and Settings\Bureau\Test\Ginnungagap.lossyV1.01n-q0-m4.wav

17:39:29 : Test started.
17:41:02 : 01/01  50.0%
17:42:16 : 02/02  25.0%
17:43:38 : 03/03  12.5%
17:44:32 : 04/04  6.3%
17:44:43 : Test finished.

----------
Total: 4/4 (6.3%)

Conclusion 2:
There is a minor improvement but if I recall well the -q1 setting which I tested by mistake in my previous post sounded almost same or better (but not worst 100% sure), so if there is a big bitrate increase I am not sure it worth it.

for efficiency I dunno, so far I only tested lossywav as pure wav so I have no clue what ends in a bigger file, -q 0 -m 4 or -q 1 ... it be must tested further with & without -m 4 at comparable bitrate ... for non-annoyance as far as I tested use -q 2.5 on this killer sample.

I still think the main problem for that sample is that added noise changes between -36 and -42dBFS several times in a second. (at 1.1...1.7 s.)

Maybe program should look ahead and stores results of analysis of several frames after and before current? Then, some sort of filter (median?) can be introduced. So the problem of 'unstable' noise can be fixed.

I re-implemented the old max-inter-block-change in bits to remove (bits-to-remove can only increase by 1 bit per codec block). This wastes a few bits but should limit the change in noise level to 6dB every 11.6ms (for 44.1kHz audio). I would appreciate feedback to see if this has improved Mardel's sample at all at -q 0.

Of course it is OK to test this. I just remember last time this came up that 2Bdecided said something to the effect that gradualy changing the bits to removed wouldn't help, it even might have a chance of giving some kind of modulation effect, and inaudible noise should be inaudible anyway (and if it's not when it should, tune somewhere). O.t.h. that was only valid for the transparent settings, so who knows 

...
V1.01n -q0 -m 4 Vs. original
...
Easy to ABX from original, always the same flaws more or less pronounced.

Thanks for testing.
I didn't expect it to bring transparency (the bitrate penalty for -m 4 isn't so large), but I hoped it would make things less annoying. No need for a detailed comparison -q 0 -m 4 vs. -q 1 IMO. There should have been some progress IMO with -m 4 to justify further investigations.

In case no new ideas come up it looks like we can't seriously improve in the bitrate range of current -q 0 or -q 1, and --portable is a good solution for the lowest preset.

... From memory (correct me if I'm wrong, halb27) eig, triangle_2_1644ds, furious and badvilbel would be good (i.e. problematic) samples to try. ....

These are good samples to try, as is Bruhns, Living In The Future, and other tracks from my potential problem collection which can be downloaded from here (http://home.arcor.de/horstalb/lossyWAVpotentialProblems) for a while.

In case no new ideas come up it looks like we can't seriously improve in the bitrate range of current -q 0 or -q 1, and --portable is a good solution for the lowest preset.

I introduced the -H, --highskew <n> (0<=n<=36) parameter at beta 1.0.1n. This is analogous to the low-frequency skew which is built in to lossyWAV but acts above 3.45kHz rather than below. This in conjunction with -l, --limit may provide a solution for the Ginnungagap sample.

Of course it is OK to test this. I just remember last time this came up that 2Bdecided said something to the effect that gradualy changing the bits to removed wouldn't help, it even might have a chance of giving some kind of modulation effect, and inaudible noise should be inaudible anyway (and if it's not when it should, tune somewhere). O.t.h. that was only valid for the transparent settings, so who knows 

Yes.

The basic problem here is that lossyWAV's tuning is aiming for transparency. For argument's sake, let's imagine that in tuning, there will probably have been moments when things were well above transparency, and bits were being wasted, but no one really cared. The worst moments will be "just above" transparency.

If you force things into non-transparent territory, two things happen. Firstly, some moments will stay transparent because they have a high margin, while others will plunge into dramatically non transparent because they were on the edge already. Secondly, the various tweaks that are in there to gain transparency don't perform "linearly" in the range of non-transparent encoding; they are set up to keep the noise inaudible - they have no mechanism to make audible noise "least bad".


Please be aware that these are two separate mechanisms. If you can't hear something, you can't hear it. Simple as that. If you can hear something, figuring out how annoying it is is a whole different ball game.

So I guess (and it's only a guess) that the wildly fluctuating noise is exactly what lossyWAV should do to keep the noise inaudible. However, for settings where the noise does become audible, it'll be very annoying.

I emphasise this is all me guessing - I'm not trying to say "this is the way it is" because I don't know.

Cheers,
David.

Yes.

The basic problem here is that lossyWAV's tuning is aiming for transparency. For argument's sake, let's imagine that in tuning, there will probably have been moments when things were well above transparency, and bits were being wasted, but no one really cared. The worst moments will be "just above" transparency.

If you force things into non-transparent territory, two things happen. Firstly, some moments will stay transparent because they have a high margin, while others will plunge into dramatically non transparent because they were on the edge already. Secondly, the various tweaks that are in there to gain transparency don't perform "linearly" in the range of non-transparent encoding; they are set up to keep the noise inaudible - they have no mechanism to make audible noise "least bad".


Please be aware that these are two separate mechanisms. If you can't hear something, you can't hear it. Simple as that. If you can hear something, figuring out how annoying it is is a whole different ball game.

So I guess (and it's only a guess) that the wildly fluctuating noise is exactly what lossyWAV should do to keep the noise inaudible. However, for settings where the noise does become audible, it'll be very annoying.

I emphasise this is all me guessing - I'm not trying to say "this is the way it is" because I don't know.

Cheers,
David.

Thanks for the insight David. I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" () when I increase the bitrate at -q 0 with some revised setting or other.

I suffered a bit of a setback as the USB stick I was using to transport lossyWAV about failed and I had to revert to the source from beta 1.0.1e. The changes were mostly cosmetic or parametric and so no real trouble. In doing so, I have not yet re-enabled the "bits-to-remove increase of one per codec-block" code.

In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.

The spreading function string was:

  Frequency_Limits                  : Array [0..spread_zones+1] of Double = (20,1378.125,3445.3125,8268.75,12403.125,16000);
  spreading_function_string         : string[precalc_analyses*(spread_zones+2)-1]='22222-22222-22223-12223-12234-12345';

I have increased the number of spread-zones to 8 from 5 by making the inter-boundary frequency change constant (didn't have to move any boundaries, just interleaved 3 more).

  Frequency_Limits                  : Array [0..spread_zones+1] of Double = (20,1378.125,3445.3125,5512.5,8268.75,10335.9375,12403.125,14470.3125,16000);
  spreading_function_string         : string[precalc_analyses*(spread_zones+2)-1]='22222222-22222222-22222223-12222223-12222224-11222234';

This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).

Bitrates for my 55 problem sample set are as follows:

|-------------|------------|------------|------------|------------|------------|
| Version     | --insane   | --extreme  | --standard | --portable |    -q 0    |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1n      | 645 kbit/s | 571 kbit/s | 491 kbit/s | 410 kbit/s | 312 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1o      | 652 kbit/s | 580 kbit/s | 504 kbit/s | 421 kbit/s | 321 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

lossyWAV beta 1.0.1o attached to post #1 in this thread.

...

Code: [Select]

|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

What's the --LC option, Nick?

Quote from: Nick.C on 2008-05-29 08:34:46

...

Code: [Select]

|-------------|------------|------------|------------|------------|------------|
| 1.0.1o --LC | 670 kbit/s | 601 kbit/s | 527 kbit/s | 441 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

What's the --LC option, Nick?

--linkchannels (but it's a bit wide for the table).

I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" when I increase the bitrate at -q 0 with some revised setting or other.

Yeah, we like to complain .  But seriously, I only would not like it if the bit rate at -q 5 would rise because something at -q 0 was fixed..

In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.
[..]
This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).

Although I don't understand what you are trying to accomplish, in general it is normal to take wider bands for higher freqs. To the ear 1 octave is a doubling of the frequency ( 2kHz - 4 kHz is the same (tonal) difference as 10 kHz - 20 kHz).
But maybe this has no relevance to lossyWav whatsoever

I used 1.0.1o with my regular music test set.
For --portable bitrate is the same as with 1.0.1k (362 kbps), for --standard it's 449 kbps against 1.0.1k's 452 kbps, for --extreme bitrate comes down to 529 kbps (1.0.1.k: 534 kbps), and for --insane bitrate is 607 kbps (1.0.1k: 611 kbps).
So from --standard upwards, bitrate is a little bit lower than when using 1.0.1k.

Bitrate of -q 0 is 270 kbps (I didn't keep the 1.0.1k's bitrate).

I did a short listening test using -q 0 on Mardel's sample and to me it's fine. I'm not sensittive towards this problem however.

I used 1.0.1o with my regular music test set.
For --portable bitrate is the same as with 1.0.1k (362 kbps), for --standard it's 449 kbps against 1.0.1k's 452 kbps, for --extreme bitrate comes down to 529 kbps (1.0.1.k: 534 kbps), and for --insane bitrate is 607 kbps (1.0.1k: 611 kbps).
So from --standard upwards, bitrate is a little bit lower than when using 1.0.1k.

Bitrate of -q 0 is 270 kbps (I didn't keep the 1.0.1k's bitrate).

I did a short listening test using -q 0 on Mardel's sample and to me it's fine. I'm not sensittive towards this problem however.

Thanks for the listening time - it's always appreciated.

I've been working on simultaneous piped input and output within foobar2000. I have managed to convert a single file using this method!  However if I try to create a single album image from a number of tracks it crashes.... I'll work on this further as there must be a simple reason why it's working for a single file and not working for multiple files.

Quote from: Nick.C on 2008-05-29 08:34:46

I realise that -q 0 is pushing the envelope when bits-to-remove is concerned - but at the same time there are several user out there who "complain" () when I increase the bitrate at -q 0 with some revised setting or other.

Yeah, we like to complain .  But seriously, I only would not like it if the bit rate at -q 5 would rise because something at -q 0 was fixed..

Quote

In re-modifying the code to bring it up to what 1.0.1n was (from memory and the change log) I had another thought about the spreading-function.
[..]
This allows finer control by halving the width of the 3 highest spread-zones to approximately 2kHz each (except the highest one).

Although I don't understand what you are trying to accomplish, in general it is normal to take wider bands for higher freqs. To the ear 1 octave is a doubling of the frequency ( 2kHz - 4 kHz is the same (tonal) difference as 10 kHz - 20 kHz).
But maybe this has no relevance to lossyWav whatsoever

The increase in bitrate at --standard is due (at least in part) to the increase from 3.25 minimum-bits-to-keep to 3.50, also slightly from the tightening of the spreading-function.

The increase in the number of spread-zones still allows consecutive zones to have the same number of bins averaged, but it also allows for the possibility of averaging different numbers of bins in each "half" of the old zone (for those that were split).

Nick, halb27
Small issue, but I thought you might be interested in this (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=63225&view=findpost&p=567350) (re. foobar setup batch file in LossyWAV wiki).
I posted in the other thread, but this seems to be the place now for all things lossyWAV.

C.

Nick, halb27
Small issue, but I thought you might be interested in this (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=63225&view=findpost&p=567350) (re. foobar setup batch file in LossyWAV wiki).
I posted in the other thread, but this seems to be the place now for all things lossyWAV.

C.

Thanks for that - I saw it at the time, but my loss of code kind of focussed my attention elsewhere.... I'll amend the wiki.

I wondered why bitrate came down a bit with 1.0.1o as compared to 1.0.1k and played around with --shaping. The default shaping (--shaping q-value/10) has gone. With the corresponding --shaping bitrate goes up.

I wondered why bitrate came down a bit with 1.0.1o as compared to 1.0.1k and played around with --shaping. The default shaping (--shaping q-value/10) has gone. With the corresponding --shaping bitrate goes up.

  (sorry - I missed that one).

[edit] ps. Thanks for spotting it.

Revised bitrates for my 55 problem sample set (FLAC -8 - 784 kbit/s) :

|-------------|------------|------------|------------|------------|------------|
| Version     | --insane   | --extreme  | --standard | --portable |    -q 0    |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1n      | 645 kbit/s | 571 kbit/s | 491 kbit/s | 410 kbit/s | 312 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1p      | 656 kbit/s | 585 kbit/s | 509 kbit/s | 425 kbit/s | 321 kbit/s |
|-------------|------------|------------|------------|------------|------------|
| 1.0.1p --LC | 672 kbit/s | 605 kbit/s | 531 kbit/s | 445 kbit/s | 336 kbit/s |
|-------------|------------|------------|------------|------------|------------|

lossyWAV beta 1.0.1p attached to post #1 in this thread. [/edit]

Thanks for that - I saw it at the time, but my loss of code kind of focussed my attention elsewhere.... I'll amend the wiki.

Oh, I see how loss of code would capture the attention  . Sorry about that; wasn't up-to-date with recent happenings.

On a different matter, I've been doing some tests with my classical CDs to see how lossyWAV does versus pure lossless:

In rare cases (e.g. solo piano) FLAC -5 was smaller than lossyWAV v1.0.1.14 at -q7.5.
So I thought perhaps it would be a good idea, where lossyWAV can't beat the lossless, it might skip the process and encode direct to FLAC or come up with some kind of notice. If I was encoding my whole collection I wouldn't stop to check versus FLAC each time, and later I'd be a little annoyed to find that the lossy was bigger than the lossless.

Glenn Gould 1980 Goldberg Variations FLAC -5 = 177 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 = 183 MB

Perhaps this could be done with a quick replay gain check, the above CD's recommended album gain is  +1.13 dB and I imagine this has a lot to do with lossyWAV's performance.

Thoughts?

C.

Oh, I see how loss of code would capture the attention  . Sorry about that; wasn't up-to-date with recent happenings.

On a different matter, I've been doing some tests with my classical CDs to see how lossyWAV does versus pure lossless:

In rare cases (e.g. solo piano) FLAC -5 was smaller than lossyWAV v1.0.1.14 at -q7.5.
So I thought perhaps it would be a good idea, where lossyWAV can't beat the lossless, it might skip the process and encode direct to FLAC or come up with some kind of notice. If I was encoding my whole collection I wouldn't stop to check versus FLAC each time, and later I'd be a little annoyed to find that the lossy was bigger than the lossless.

Glenn Gould 1980 Goldberg Variations FLAC -5 = 177 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 = 183 MB

Perhaps this could be done with a quick replay gain check, the above CD's recommended album gain is  +1.13 dB and I imagine this has a lot to do with lossyWAV's performance.

Thoughts?

C.

I re-introduced the possibility to change the codec-block-size at beta 1.0.1o.

In this instance, increasing the length of the codec-block would probably improve the situation.

Use the -b <n> parameter with n=512,1024,2048 or 4096. The corresponding change to -b must be made in the flac encoding command line to take the benefit. I would probably use wavegain on the WAV file initially and set the codec-block-size depending on the result : higher positive = bigger codec-block-size. How to do it in batch I haven't yet had time to work out.

... In rare cases (e.g. solo piano) FLAC -5 was smaller than lossyWAV v1.0.1.14 at -q7.5. ...

I've run upon this too.
Quiet music originating from a solo instrument (or music with very few instruments) is suspected in first place to yield these results.

When I had a lossless archive I produced the lossyFLAC results in the same folder as were the lossless tracks. So it's easy to compare.
The procedure is only necessary in folders with tracks that can be characterized the way I did it above.

BTW I found that in these cases FLAC often compresses in a poor way. TAK or wvPack often have a significantly better compression ratio. Sure I don't know whether this is specific to my music. But you can give it a try.

1.0.1p's average bitrate for my regular music test set:

--portable:  371 kbps
--standard: 463 kbps
--extreme:  544 kbps
--insane:    619 kbps

In this instance, increasing the length of the codec-block would probably improve the situation.

Use the -b <n> parameter with n=512,1024,2048 or 4096. The corresponding change to -b must be made in the flac encoding command line to take the benefit.

Thanks Nick & Halb27, interesting.

FYI:
Glenn Gould 1980 Goldberg Variations FLAC -5 = 177 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 512) = 183 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 1024) = 178 MB (- 5MB)
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 2048) = 175 MB (- 3MB)
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 4096) = 174 MB (- 1MB)

So it looks as though as the block size increases you get diminishing returns.

C.

With -b 4096 option, there's strange clicks in right channel...

Also Mardel's sample sounds better at -q 0 with -b 2048 option, though 100% ABXable. Bitrate is 314kbps (-q 0 -b 2048) vs. 310kbps (-q 0 -b 512).

With -b 4096 option, there's strange clicks in right channel...

That "sounds" like a bug - I'll get onto it.
[edit] Listening to it, it sounds like an overflow in one (or more) codec-blocks of the four that are stored in memory. It should only happen at -b 4096. I'll get a fix out later. [/edit]

When it's about changing block size (for the sake of a potentially increased efficiency of FLAC or whatever lossless codec is used) the essential thing is changing the block size of FLAC.
lossyWAV's blocksize can stay at 512.
The only thing to take care of is that FLAC block size is a multiple of 512 (or, if lossyWAV's block size is changed: a multiple of the lossyWAV blocksize).

Quote from: Nick.C on 2008-05-29 21:53:15

In this instance, increasing the length of the codec-block would probably improve the situation.

Use the -b <n> parameter with n=512,1024,2048 or 4096. The corresponding change to -b must be made in the flac encoding command line to take the benefit.

Thanks Nick & Halb27, interesting.

FYI:
Glenn Gould 1980 Goldberg Variations FLAC -5 = 177 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 512) = 183 MB
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 1024) = 178 MB (- 5MB)
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 2048) = 175 MB (- 3MB)
Glenn Gould 1980 Goldberg Variations Lossy.FLAC -q7.5 (-b 4096) = 174 MB (- 1MB)

So it looks as though as the block size increases you get diminishing returns.

C.

Oh, I forgot to say: in these cases where lossless yields a smaller file size than lossyWAV + FLAC I just stick to lossless. I don't think it's worth while optimizing the lossyWAV + FLAC procedure: just use the lossless encoding in these cases. With emphasis when using wvPack or TAK instead of FLAC as usually file size goes down further, in many cases at a rate of ~10% which is significantly more than the usual ~2% difference between various lossless codecs.

lossyWAV beta 1.0.1q attached to post #1 in this thread.

Echoing what halb27 has said...

The lack of efficiency in these cases in mainly down to FLAC blocksize, rather than lossyWAV itself.

As long as you don't apply positive ReplayGain values, increased bitrates of lossy vs lossless are nothing to worry about, even if you deleted your lossless original. Just take the lossy.flac file and re-encode it with normal FLAC - there will be no further loss (FLAC is lossless!) but it defaults to a different block length, which probably reduces the bitrate for such tracks.

You can go back to the lossless original, and re-encode it with a different block size via lossyWAV and then FLAC, potentially giving a slightly lower bitrate still - but if this isn't possible, don't worry - the above approach will still bring the bitrate down (usually).


Do you think it would be worth adding a utility to check for this? e.g.

If original = FLAC, then
  If filesize lossyFLAC > filesize lossless FLAC, then
    re-encode lossyFLAC with same parameters (blocksize, compression) as lossless FLAC to generate new lossy FLAC.
    If lossless FLAC still smaller, replace lossy with lossless; endif
  endif
endif

If the original isn't FLAC, it's not worth encoding everything to lossless FLAC just to check - but if it's already there, the speed hit should be tiny.

You could, of course, do the whole lot (lossyWAV and FLAC) with multiple block sizes and pick the smallest, but this would be overkill (though you could add an --overkill option Nick!) - whereas the above almost comes for free, and would solve the problem neatly if transcoding from FLAC.

Cheers,
David.

.. the above almost comes for free, and would solve the problem neatly if transcoding from FLAC.

The same would apply if it was from another lossless codec (TAK, WavPack).

I would not call "re-encode lossyFLAC" "almost for free".  I also think this is not a job for lossyWav, especially with piped input/output it couldn't tell. Maybe the script (that most use anyway at the moment) could be adapted for these cases.

I realise it can't go comfortably into lossyWAV - even when not "piped" it doesn't necessarily know what or where the source FLAC is, because the input is WAV. If it can be handled in the script though, that would be great.

By "almost for free" I meant the checking.


I guess it works for other codecs to, but not necessarily when converting from one lossless codec to another (via lossyWAV). In that case, an increase in filesize might be just because codec Y was less efficient than codec X, so changing the block size might make it even worse.

Cheers,
David.

So the default noise shaping is --shaping (q_value/10). But... longhelp still says

-s, --shaping <n>  enable fixed noise shaping; (0.00<=n<=1.00); default=0;
                    0.00 = 0%, 1.00=100% effectiveness, 0.5=50%, etc.

And yes, default=0; but this means that user should use --shaping 0.001 if he wants to disable noise shaping. 

So the default noise shaping is --shaping (q_value/10). But... longhelp still says

Quote

-s, --shaping <n>  enable fixed noise shaping; (0.00<=n<=1.00); default=0;
                    0.00 = 0%, 1.00=100% effectiveness, 0.5=50%, etc.

And yes, default=0; but this means that user should use --shaping 0.001 if he wants to disable noise shaping. 

Longhelp is obviously wrong and will be corrected at the next revision;
Use --shaping 0 if you really want to disable noise shaping.

Longhelp is obviously wrong and will be corrected at the next revision;

  Great.

Use --shaping 0 if you really want to disable noise shaping.

I  tried but failed    It looks like --shaping 0 has no effect.

Quote from: Nick.C on 2008-05-30 23:27:39

Longhelp is obviously wrong and will be corrected at the next revision;

  Great.

Quote from: Nick.C on 2008-05-30 23:27:39

Use --shaping 0 if you really want to disable noise shaping.

I  tried but failed    It looks like --shaping 0 has no effect.

Sorry - you're right - I'll re-write that bit of code so that if -s 0 or --shaping 0 is used then shaping will (definitely!!!) be disabled....

I've been working on a method of speeding up the code by using a combination of the fxtract assembly instruction and two lookup tables (2048 values for exponent, 4097 values for mantissa) to reduce the time spent calculating square roots. The final method reduces the processing time by about 10% at -q 0 (13.6 seconds down to 12.2) and is considered to be suitably accurate for its intended purpose.

I have also "fixed" the --scale parameter to accept values in the range 0<n<=8 as it was before I had my flash drive failure....

lossyWAV beta 1.0.1r attached to post #1 in this thread.

I have also "fixed" the --scale parameter to accept values in the range 0<n<=8 as it was before I had my flash drive failure....

Thanks for that. I can't use --dither in versions 101q and 101r; in version 1.0.1k it worked well.
"Unknown error in unknown module" and the program bails out.

I've been working on a method of speeding up the code by using a combination of the fxtract assembly instruction and two lookup tables (2048 values for exponent, 4097 values for mantissa) to reduce the time spent calculating square roots. The final method reduces the processing time by about 10% at -q 0 (13.6 seconds down to 12.2) and is considered to be suitably accurate for its intended purpose.

Maybe worth having a look here, maybe there's more speed to be had from the square root optimisations..
http://www.codemaestro.com/reviews/9 (http://www.codemaestro.com/reviews/9)

Also, if you're maybe going to get your hands dirty with a smidge of assembler, maybe there'll be gains to be had if you can work with the reciprocals instead..

Edit: Actually scratch that.. I've just run some tests and it wasn't pleasant

Maybe worth having a look here, maybe there's more speed to be had from the square root optimisations..
http://www.codemaestro.com/reviews/9 (http://www.codemaestro.com/reviews/9)

Also, if you're maybe going to get your hands dirty with a smidge of assembler, maybe there'll be gains to be had if you can work with the reciprocals instead..

Edit: Actually scratch that.. I've just run some tests and it wasn't pleasant

I had a look there - I don't know to convert that for a 64-bit float. I do use reciprocals and multiplication wherever possible - it is a lot quicker. Thanks for taking the time to make the suggestion!

[edit] One of the links points to a PDF with a 64-bit float version included - I'll see how it goes.... [/edit]

I ran into something little with version 1.0.1q.
At last I have a script for foobar2000 that takes either a sourcefile (%s) or stdin (-).
What I try to do is use stdin with lossyWav, then lossywav.lossy.wav as input file for FLAC. (Yes, trying to get the best of both worlds, one temp file less, keep the metadata and have normal seektables in FLAC.)

As I was trying to find out why it didn't work , I noticed FLAC --keep-foreign-metadata didn't seem to work when lossywav is fed from stdin, even though FLAC can read from a normal file.
Flac barfs: lossywav.lossy.wav: ERROR reading foreign metadata: invalid WAVE file: unexpected EOF (010)

And when I try FLAC on the same file without --keep-foreign-metadata

lossywav.lossy.wav: WARNING: skipping unknown sub-chunk 'fact' (use --keep-foreign-metadata to keep)
lossywav.lossy.wav: WARNING: unexpected EOF; expected 1073741812 samples, got 10825216 samples

now the file should have 10825668 samples according to foobar (I guess also the billion sample value comes from the foobar converter) ... and these last two are just warnings. The resulting file has the same length as the input.
So there is at least something strange with the FACT chunk when lossyWav uses stdin (or is that the result of the wrong sample value too?)

Maybe this can help to solve the stdin strangeness.


BTW am I being silly by still sticking to the FACT chunk idea? 

I ran into something little with version 1.0.1q.
At last I have a script for foobar2000 that takes either a sourcefile (%s) or stdin (-).
What I try to do is use stdin with lossyWav, then lossywav.lossy.wav as input file for FLAC. (Yes, trying to get the best of both worlds, one temp file less, keep the metadata and have normal seektables in FLAC.)

As I was trying to find out why it didn't work , I noticed FLAC --keep-foreign-metadata didn't seem to work when lossywav is fed from stdin, even though FLAC can read from a normal file.
Flac barfs: lossywav.lossy.wav: ERROR reading foreign metadata: invalid WAVE file: unexpected EOF (010)

And when I try FLAC on the same file without --keep-foreign-metadata

lossywav.lossy.wav: WARNING: skipping unknown sub-chunk 'fact' (use --keep-foreign-metadata to keep)
lossywav.lossy.wav: WARNING: unexpected EOF; expected 1073741812 samples, got 10825216 samples

now the file should have 10825668 samples according to foobar (I guess also the billion sample value comes from the foobar converter) ... and these last two are just warnings. The resulting file has the same length as the input.
So there is at least something strange with the FACT chunk when lossyWav uses stdin (or is that the result of the wrong sample value too?)

Maybe this can help to solve the stdin strangeness.


BTW am I being silly by still sticking to the FACT chunk idea? 

Meh - it looks like I broke the stdin file output.... Sorry! I'll try to get beta 1.0.1s out tonight.

[edit] Actually, it is that foobar2000 tells an "encoder" using STDIN that the "WAV file" is Max size, i.e. 4,294,967,294 - 12 bytes long (12 bytes for 'RIFF'/0FFFFFFF2h/'WAVE').

I will put in a check to see if STDIN is being used without STDOUT and if so seek to the beginning of the file and put sensible values in for the overall size and the 'data' chunk size. [/edit]

Quote from: GeSomeone on 2008-06-03 19:43:41

[..](I guess also the billion sample value comes from the foobar converter) ...

[edit] Actually, it is that foobar2000 tells an "encoder" using STDIN that the "WAV file" is Max size, i.e. 4,294,967,294 - 12 bytes long (12 bytes for 'RIFF'/0FFFFFFF2h/'WAVE').

I will put in a check to see if STDIN is being used without STDOUT and if so seek to the beginning of the file and put sensible values in for the overall size and the 'data' chunk size. [/edit]

Yes it is the foobar2000 converter. We (Halb27 and myself) tried to make you aware of this. But don't worry, this is what betas are for . I found an old posting (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=41287&view=findpost&p=362981) where Peter replies about this.
If you could fix it (put the correct number of samples in the header) that would be nice.

Check the first post.

Change log 1.0.1q: 30/05/08

Maybe you should attach list of programs wich supporting lossywav?

p.S. GXTranscoder

Check the first post.

Quote

Change log 1.0.1q: 30/05/08

Maybe you should attach list of programs wich supporting lossywav?

p.S. GXTranscoder

The list of known supported codecs is in the wiki article linked from the first page in the thread.

lossyWAV beta 1.0.1s attached to post #1 in this thread.

[edit] The speed of transcoding in foobar2000 has been dramatically increased by the use of STDIN and STDOUT. I transcoded my Mike Oldfield collection (366 tracks, 33h29m) in 22m18s or approximately 90.09x overall {Intel E6600 C2D @ 3.0GHz, 4GB DDR2-800, Windows MCE2005 SP2, source on server, destination 160GB Maxtor SATA HDD}. This was definitely limited by the processor cores as the utilisation plot for the duration of the transcode was a virtual flat-line at 100%.

What this does mean though is that the 'fact' chunk will not be preserved - so be careful not to mix-up your archive FLAC files with your lossyFLAC files. [/edit]

I will put in a check to see if STDIN is being used without STDOUT and if so seek to the beginning of the file and put sensible values in for the overall size and the 'data' chunk size.

Thanks Nick,

Something has changed with STDIN but it seems still not quite right.

When I feed the intermediate file lossywav.lossy.wav to STDIN of FLAC (without --keep.. ) says
-: WARNING: skipping unknown sub-chunk 'fact' (use --keep-foreign-metadata to keep)
-: WARNING: unexpected EOF; expected 5869416 samples, got 5868544 samples
BTW according to foobar2000 the number of samples of the source is also 5869416 (and the lossywav and resulting lossy.flac are 5869056)  I'm not sure foobar displays correct values.

When I let FLAC read the lossy file (with --keep.. )
lossywav.lossy.wav: ERROR reading foreign metadata: invalid WAVE file: unexpected EOF (010)

Is that FACT chunk becoming a nuisance now, or is it something else?

Quote from: Nick.C on 2008-06-03 19:57:03

I will put in a check to see if STDIN is being used without STDOUT and if so seek to the beginning of the file and put sensible values in for the overall size and the 'data' chunk size.

Thanks Nick,

Something has changed with STDIN but it seems still not quite right.

When I feed the intermediate file lossywav.lossy.wav to STDIN of FLAC (without --keep.. ) says
-: WARNING: skipping unknown sub-chunk 'fact' (use --keep-foreign-metadata to keep)
-: WARNING: unexpected EOF; expected 5869416 samples, got 5868544 samples
BTW according to foobar2000 the number of samples of the source is also 5869416 (and the lossywav and resulting lossy.flac are 5869056)  I'm not sure foobar displays correct values.

When I let FLAC read the lossy file (with --keep.. )
lossywav.lossy.wav: ERROR reading foreign metadata: invalid WAVE file: unexpected EOF (010)

Is that FACT chunk becoming a nuisance now, or is it something else?

Could you please post your converter settings so that I can duplicate the problem?

Many thanks.

[edit] Problem found here too - average of 249 samples dropped off the end of 85 album lossyFLAC files. It may be something to do with the last read (incomplete codec-block [probably]) - I'll get onto it and revert soon. Until such times, STDIN input is not recommended to be used (--stdout seems fine....) [/edit]

[edit] Problem found here too - average of 249 samples dropped off the end of 85 album lossyFLAC files. It may be something to do with the last read (incomplete codec-block [probably]) - I'll get onto it and revert soon. Until such times, STDIN input is not recommended to be used (--stdout seems fine....) [/edit]

Many thanks for the heads up GeSomeone - it was as I surmised and it now seems to work properly in beta 1.0.1t.

lossyWAV beta 1.0.1t attached to post #1 in this thread.

lossyWAV beta 1.0.1t attached to post #1 in this thread.

Thank you, Nick, for the new version.
I encoded my regular music test set with piping and with my bat file.
Speed is great when using piping (though it is good with the bat file).
I tested the musical content just to make sure, and with all the tracks content is identical no matter which encoding way.

Just one thing is strange: the lossy.flac files generated when using piping are 0.5...1.0 per cent bigger than those generated by the bat file. (I don't use meta information with the piped version, but I do so with the bat file solution).

Quote from: Nick.C on 2008-06-11 08:08:43

lossyWAV beta 1.0.1t attached to post #1 in this thread.

Thank you, Nick, for the new version.
I encoded my regular music test set with piping and with my bat file.
Speed is great when using piping (though it is good with the bat file).
I tested the musical content just to make sure, and with all the tracks content is identical no matter which encoding way.

Just one thing is strange: the lossy.flac files generated when using piping are 0.5...1.0 per cent bigger than those generated by the bat file. (I don't use meta information with the piped version, but I do so with the bat file solution).

I would guess that FLAC is inserting a larger number of seek points due to the fact that (in foobar2000) the assumed size of the piped file is maximum, i.e. 4,294,967,294 bytes.

Processing on my laptop, using two intermediate WAV files, used to be a real HDD thrashing exercise - 40 to 50 minutes for my 10 album test set (3.36GB, 9h22m57s total duration). Using piped input and output, the transcode now takes approximately 10 minutes - a really nice reduction in total processing time.

I've just finished a "stress test" of sorts: all of my FLAC'ed CD's on my server to lossyFLAC (-q 1.25). 12d4h24m45s (4064 tracks, 316 CD's, 108GB, 887kbit/s average) transcoded in 3h33m23s (40.5GB, 331 kbit/s average) - 82.22x overall and exactly the same number of samples out as went in . Bearing in mind that previously my 3686 track 100GB(ish) transcode took approximately 7h30m, that's an appreciable saving.

If there are no reported problems and if no-one has any further suggestions then now may be the time to move towards a release candidate for 1.0.2.

Quote from: Nick.C on 2008-06-11 08:08:43

lossyWAV beta 1.0.1t attached to post #1 in this thread.

Just one thing is strange: the lossy.flac files generated when using piping are 0.5...1.0 per cent bigger than those generated by the bat file. (I don't use meta information with the piped version, but I do so with the bat file solution).

Yes thanks a lot, this time my little script works as intended 

Except from the metadata it could be the larger seektable that FLAC allocates for a file of unknown length, as foobar2000's converter generates (http://www.hydrogenaudio.org/forums/index.php?showtopic=41287&hl=seektable).

I'll show my script (batfile) that (with this fix)

• handles either - or %s as input.
• one intermediate file less than the "old" script
• the FLAC files are a few kB smaller because no large seektables and 2kB padding (more than enough for me) instead of 8kB.
• Foreign metadata is added to the FLAC file, though I doubt lossyWav will ever see it again when converting from STDIN (with foobar).

Cons

• can run only 1 converter thread at once
• takes less advantage of multi core/hyperthreading than piping directly from lossyWav into FLAC

foobar2000:
Encoder:    C:\WINDOWS\system32\cmd.exe
Extension:  lossy.flac
Parameters: /D /C C:\PROGRA~1\FLAC\lossyFlac.bat - %d -q 4.5  (or any lossyWav settings)

and my lossywav.bat

@echo off
set lossyWAV_path="C:\Progra~1\FLAC\lossyWAV.exe"
set FLAC_path="C:\Progra~1\FLAC\flac"
set Mylog="C:\Progra~1\FLAC\lossyFlac.log"

echo # lossyWAV.exe "%~N2.flac" %3 %4 %5 %6 %7 %8 --quiet >> %Mylog%
%lossyWAV_path% %1 %3 %4 %5 %6 %7 %8 --quiet --nowarnings --low --writetolog
type lossyWav.log >> %MyLog%
del lossyWav.log

set MyLossyFile="%~N1.lossy.wav"
if NOT %1 == - goto :FLAC
set MyLossyFile="lossywav.lossy.wav"

:FLAC
%FLAC_path% -5 -e -s -b 512 --force --padding=2048 --keep-foreign-metadata %MyLossyFile% -o%2 >>%Mylog%

del %MyLossyFile%

set MyLossyFile=
set Mylog=
set lossyWAV_path=
set flac_path= 

Of course that logging stuff can be removed and on Vista it should be in another location.

[edit] some tidying up and removing a debug line from the script [/edit]

I would guess that FLAC is inserting a larger number of seek points ...

Yes, it's the seektable. When using --no-seektable with FLAC, I arrive at an expected file size.

Is the seektable necessary? I had no problems skipping part of the track (using foobar) with the --no-seektable version.

What kind of problems can I encounter when using --no-seektable?

What kind of problems can I encounter when using --no-seektable?

Did Josh not explain this once - I think that it only takes a bit longer to seek when moving about the file.

I don't encounter any disadvantage using --no-seektable when playing on my DAP.

So I'll use --no-seektable together with GeSomeone's --padding=2048 proposal and enjoy piping.

...If there are no reported problems and if no-one has any further suggestions then now may be the time to move towards a release candidate for 1.0.2.

I second this.

I don't know if anyone uses REACT to do their EAC tasks, but I have a rough (but working) model of hacking the track-config file to use lossyWAV/lossFlac.

IF NOT @lossyFlac@==1 GOTO end_lossyFlac_tracks
    IF NOT EXIST %TrackDir_lossyFlac% MKDIR %TrackDir_lossyFlac%
    PUSHD %TrackDir_lossyFlac%
        IF @various@==1 SET VA_tag=-T "album artist=@VA@"
        IF %embed_cover%==1 SET Cover_tag=--picture="|image/jpeg|||@cover@"
        ECHO ON
    @tools@\lossywav.exe "@source@" @Opt_lossyWAV@
    RENAME *lossy.wav "temp.lossy.wav"
    DEL "@source@"
        @tools@\flac.exe @Opt_lossyFlac@ %Cover_tag% %VA_tag% %Disc_Flac% -T artist="@artist@" -T album="@album@" -T tracknumber="@track@" -T title="@title@" -T date="@year@" -T genre="@genre@" -T comment="@comment@" -T encoding="Flac @Ver_Flac@ @Opt_lossyFlac@" "temp.lossy.wav" -o "%TrackName%.lossy.flac"
    DEL "temp.lossy.wav"
        @ECHO OFF
        IF %have_cover%==1 IF NOT EXIST folder.jpg COPY "@cover@" folder.jpg
    POPD
:end_lossyFlac_tracks

(and these were in the REACT.ini:)

Opt_lossyFlac=-5 -b 512 --keep-foreign-metadata
Opt_lossyWAV=-q 5

Not groundbreaking, I guess, but a start.  I don't know how to use the temp variables in batch files, so I use a static temp name and then delete it.  But, it works fine.  I get resulting lossyFlac files with proper tagging, album art embedded, and Replay Gain tagging.

  Scott

Quote from: halb27 on 2008-06-12 18:50:12

What kind of problems can I encounter when using --no-seektable?

Did Josh not explain this once - I think that it only takes a bit longer to seek when moving about the file.

At first, when ripping to images, I added seekpoints for every 90 seconds. Now I use -S- and noticed that the  cue points from the embedded cue sheet are sufficient. And because I can use the cuesheet to play the disc image I can jump to any track or movement I like I like.

hi, today I played with lossywav again in order to have it working for good with F2K & tak, so I copy/pasted your command line from post N°1, & it worked like a charm with flac

/d /c c:\data_nic\bin\lossywav - --standard --silent --stdout|c:\data_nic\bin\flac - -b 512 -5 -f -o%d

then I changed the directory path & flac setting according to my own taste c:/z directory + Tak, & I ended with:

/d /c c:\z\lossywav - --portable --silent --stdout|c:\z\Takc -e -p3 -fsl512 -ihs - %d

it worked like a charm too even if I didn't tested expected size against actual size.

But my knowledge being very limited I don't undertand the use of /d /c ...
I understand that I call cmd.exe with c:\windows\system32\cmd.exe then lossywav with c:\z\lossywav - --portable --silent --stdout| then tak with c:\z\Takc -e -p3 -fsl512 -ihs - %d, but I don't get the use of /d /c, but if I delete it it doesn't work anymore so could someone explain the use of /d /c ? What is it needed for ? Also is there any mistake in my own lossyTak command line ? Thks

hi, today I played with lossywav again in order to have it working for good with F2K & tak, so I copy/pasted your command line from post N°1, & it worked like a charm with flac

/d /c c:\data_nic\bin\lossywav - --standard --silent --stdout|c:\data_nic\bin\flac - -b 512 -5 -f -o%d

then I changed the directory path & flac setting according to my own taste c:/z directory + Tak, & I ended with:

/d /c c:\z\lossywav - --portable --silent --stdout|c:\z\Takc -e -p3 -fsl512 -ihs - %d

it worked like a charm too even if I didn't tested expected size against actual size.

But my knowledge being very limited I don't undertand the use of /d /c ...
I understand that I call cmd.exe with c:\windows\system32\cmd.exe then lossywav with c:\z\lossywav - --portable --silent --stdout| then tak with c:\z\Takc -e -p3 -fsl512 -ihs - %d, but I don't get the use of /d /c, but if I delete it it doesn't work anymore so could someone explain the use of /d /c ? What is it needed for ? Also is there any mistake in my own lossyTak command line ? Thks

C:\Documents and Settings\User>cmd /?
Starts a new instance of the Windows XP command interpreter....

/C      Carries out the command specified by string and then terminates
/D      Disable execution of AutoRun commands from registry (see below)

thks for cmd /? ... didn't knew about it 

thks for cmd /? ... didn't knew about it 

No problem - I'm glad that the command line is working properly for you. Thanks for the Tak command line - I'll include it in the first post.

From what collector and halb27 have said, maybe --no-seektable -P=4096 would be a sensible way of reducing the size of the seektable when using STDIN / STDOUT in foobar2000. I processed my 10 Album test set and I think this saved about 1MB (in 1.37GB processed). Still, 1MB is 1MB....

then I changed the directory path & flac setting according to my own taste c:/z directory + Tak, & I ended with:

/d /c c:\z\lossywav - --portable --silent --stdout|c:\z\Takc -e -p3 -fsl512 -ihs - %d

it worked like a charm too even if I didn't tested expected size against actual size.

Also is there any mistake in my own lossyTak command line ? Thks

Looks good 

In my evaluations of LossyWav processed TAK test file sets there was no advantage of choosing a preset higher than -p2m. Some file sets even compressed about 0.05 percent worse with -p3 to -p5...

  Thomas

edit: Content of duplicate post deleted...

Nick.C:
... well I haven't yet read all the discussion of collector and halb27 on seektable ... I only come to read this thread once a week/month to see how things go on ... I know I will not use lossywav for production as long as it's not stable ... so I am not in a hurry to read it all

TBeck:
Thks for the hint ... I was planning to test the efficiency of flac -q6 vs tak -p3 vs wv -hm on lossywav processed files as I think I read somewhere not all lossless codecs where equal when encoding from lossywav ... I think I read wavpack wasn't optimal compared to tak & flac ... will add tak at different settings when I will test ... post lossywav V1.2 ... with some luck maybe by then I will have a frozen tak too

Quote from: sauvage78 on 2008-06-14 14:15:33

then I changed the directory path & flac setting according to my own taste c:/z directory + Tak, & I ended with:

/d /c c:\z\lossywav - --portable --silent --stdout|c:\z\Takc -e -p3 -fsl512 -ihs - %d

it worked like a charm too even if I didn't tested expected size against actual size.

Also is there any mistake in my own lossyTak command line ? Thks

Looks good 

In my evaluations of LossyWav processed TAK test file sets there was no advantage of choosing a preset higher than -p2m. Some file sets even compressed about 0.05 percent worse with -p3 to -p5...

  Thomas

Thanks for the clarification, Thomas.

Just tested with wavpack 4.5 & it worked too, here is my 3 settings for Tak/Flac/Wv:

Encoder:    c:\windows\system32\cmd.exe
Extension:  lossy.tak
Parameters: /d /c c:\z\lossywav - --portable --silent --stdout|c:\z\Takc -e -p3 -fsl512 -ihs - %d
Format is:  lossless or hybrid
Highest BPS mode supported: 24

Encoder:    c:\windows\system32\cmd.exe
Extension:  lossy.flac
Parameters: /d /c c:\z\lossywav - --portable --silent --stdout|c:\z\flac - -6 -b 512 -f -o%d
Format is:  lossless or hybrid
Highest BPS mode supported: 24

Encoder:    c:\windows\system32\cmd.exe
Extension:  lossy.wv
Parameters: /d /c c:\z\lossywav - --portable --silent --stdout|c:\z\wavpack -hm --blocksize=512 --merge-blocks -i - %d
Format is:  lossless or hybrid
Highest BPS mode supported: 24

wavpack works but even with --merge-blocks its efficiency seems reduced compared to real lossless source, for exemple on my single test album flac -6 compressed more than wavpack -h on this lossywav file while according to my experience wavpack -h usually beats flac -6 on real lossless music.

also when I tried to add --keep-foreign-metadata to the flac command line, it crashed.

also when I tried to add --keep-foreign-metadata to the flac command line, it crashed.

That option can only be used when reading from a file (not from STDIN).

lossyWAV 1.0.1u RC1 attached to post #1 in this thread.

lossyWAV 1.0.1u RC1 attached to post #1 in this thread.

Thanks for the new RC.
I'm looking forward to the next release.

I've greatly enjoyed following the progress of this project, with almost continuous updates and all sorts of interesting feedback from all quarters. But I am somewhat ashamed to admit that only this evening have I actually begun playing with lossyWAV in foobar2000 (I had previously only done a few tests here and there, from the command line).

Based on what I've tried this evening, I have a couple of questions. They have probably been answered elsewhere, but I have not yet spotted the answers this evening... and if I've read them in the multitude of lossyWAV pages I've read over the past few months, I've forgotten. 

Briefly:

• It was very easy to get lossyWAV working using the suggested structure of C:\bin\lossywav, but it is odd to have to place my codecs in a root-level directory. Every other audio codec resides in a "codecs" sub-folder, within my foobar2000 directory. Is the number of accessible directory levels a limitation of calling the process via cmd, as the encoder? Or will this eventually be altered, to allow a complex structure along the lines of "C:\Program Files\assorted audio stuff\foobar2000\codecs\"? (The program seems to work fine if run from a complex directory structure, but only when called directly from the command line.)

• Since lossyWAV-encoded Redbook audio is still two channels, with 16 bit wordlength, 1411 kbps, and 44100 Hz, is there any reason lossyWAV could not be used to archive material that will later be burned back to a CD-R? (Effectively, isn't the VBR nature of lossyWAV closer to a "padded" CBR?)

Thank you!

    - M.

I'm running it from foobar using the following batch file, and it works fine (doesn't need to be at the root):

@echo off
D:\library\software\audio\active_encoders_all\lossywav\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
D:\library\software\audio\active_encoders_all\lossywav\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac" 
del "%~N1.lossy.wav"

Note: No spaces in my address, whereas:
C:\Program Files\assorted audio stuff\foobar2000\codecs\
does have spaces, and that may be the issue. See here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=63225&view=findpost&p=567350).

C.

• It was very easy to get lossyWAV working using the suggested structure of C:\bin\lossywav, but it is odd to have to place my codecs in a root-level directory. Every other audio codec resides in a "codecs" sub-folder, within my foobar2000 directory. Is the number of accessible directory levels a limitation of calling the process via cmd, as the encoder? Or will this eventually be altered, to allow a complex structure along the lines of "C:\Program Files\assorted audio stuff\foobar2000\codecs\"? (The program seems to work fine if run from a complex directory structure, but only when called directly from the command line.)

I am becoming more convinced that the "spaces-in-the-path-to-the-executable" problem is in some way due to the way that cmd.exe is being called - I'll spend a bit of time investigating*....

• Since lossyWAV-encoded Redbook audio is still two channels, with 16 bit wordlength, 1411 kbps, and 44100 Hz, is there any reason lossyWAV could not be used to archive material that will later be burned back to a CD-R? (Effectively, isn't the VBR nature of lossyWAV closer to a "padded" CBR?)

PCM in a WAV file is always most significant bit (msb) justified. So, a 9 bit sample occupies the top 9 bits in a 16-bit word. All lossyWAV does is vary the number of msb's in use per codec-block while maintaining the original sample size (in bytes). The audio is still fully WAV format compliant and there is no reason not to burn back to CD-R (I assume as CDDA rather than data?).

I'm glad that you're enjoying lossyWAV - I've enjoyed it this far as well - although, without David's publication of his method just over a year ago, it would never have happened.

[edit] * I may have cracked it - try:

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

[/edit]

That seems to have gotten it... thank you! (Also, you are correct in the assumption of CDDA versus data.)

    - M.

* I may have cracked it - try:

Code: [Select]

Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d

That's another variation, mine was the "short name", just one page ago

set FLAC_path="C:\Progra~1\bin\flac"

I just tested lossyWAV 1.0.1u RC1 using --standard with my usual problem set as well as some regular tracks.
Everything's fine (as was expected - but there's always the possibility with changes in code that something goes wrong).

Quote from: Nick.C on 2008-06-25 08:06:06

* I may have cracked it - try:

Code: [Select]

Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d

That's another variation, mine was the "short name", just one page ago

Code: [Select]

set FLAC_path="C:\Progra~1\bin\flac"

I was thinking about that too; since cmd is a dos-(related) command short filenames could work too.
Or one adds the dir with lossywav, flac, lame and other audio utils to the path:

In Windows XP this can be done in: (iirc)

  Control Panel > System > Advanced > Environment Variables > Path > edit..

That's another variation, mine was the "short name", just one page ago

Code: [Select]

set FLAC_path="C:\Progra~1\bin\flac"

Hmm, if it's because of how cmd is launching, and using 8.3 path in... well... path, fixes it...

Then try setting the encoder to be "C:\WINDOWS\system32\cmd.exe" with those quotes around it.  That *should* force the method that Foobar is using to launch cmd to use long filenames.  There's still a possibility that Foobar is explicitly forcing 8.3 names, and why that would be is beyond me.

Hi all,

Firstly lossyWAV is excellent, really impressed!

Just ran a test on a sample of classical FLACs encoded at -5.
The sample was 110 files that were all between 540 and 560 kbps.
The reason for this selection is that originally I thought that below 550 the savings rarely warrant using lossyWAV (however, this may not be so - but it's tricky to identify which files are suitable and which aren't).

Results:

Files: AVG kbps / Total (MB) (Saving (MB)) / % of orig size
Original FLAC -5:  551 / 2,267 / - / 100%
lossyWAV -7.0 - 535 / 2,203  (64 MB) / 97%
lossyWAV -6.5 - 526 / 2,167 (100 MB) / 96%
lossyWAV -6.0 - 516 / 2,124 (143 MB) / 94%

The saving was less than I expected.
I had a look at a few individual cases and found that a good number of lossyWAV tracks were larger than the originals (from -6 to -7).
So where lossyWAV worked it worked really well, where it couldn't make reductions it increased the file size.

I don't mean this as a criticism of lossyWAV in any way whatsoever, in fact I've been enormously impressed !!!

What is clear is that one cannot tell by the original bitrate whether or not lossyWAV can make a difference. For example I've had good reductions on files < 500 kbps.

So what I'd really like to see is a program that runs the lossyWAV analysis without doing the encoding (like WAVGain).
- User sets a minimum reduction requirement in %.
- Programs identifies all files where that reduction cannot be achieved.
- Then creates an m3u playlist of all the files that can make the user specified reduction.
- The user then loads playlist into encoder UI (i.e. foobar2k) and off you go.

Just a thought / wish.
In the mean time I'm going to do this manually.

By the way, great idea to allow non-integer values for lossyWAV encoding 
Very nice.

C.

Better to use lossywav -5 as reference . It is 'standard'.

...Just ran a test on a sample of classical FLACs encoded at -5.
The sample was 110 files that were all between 540 and 560 kbps.
The reason for this selection is that originally I thought that below 550 the savings rarely warrant using lossyWAV ...

Yes, it's a basic problem.
Guess your tracks consists of rather 'simple' music (from a lossless codec's view), that is music which is rather quiet at least at considerable amount of spots and/or is made up from one or few instruments only.

First thought: for this kind of music it's really disputable whether it's worth while not to use a lossless codec (and I guess you'll often find that when replacing FLAC by wavPack -hx3 or -hhx3 or even better TAK -p4 or even better Monkey -c4000 the situation is even more shining for the lossless codec).

Second thought:
The first thing to consider is the lossyWAV quality demand. In case you don't use lossyWAV for archiving there's not much use in going beyond the --standard quality, and in this case you do get some savings with most of your samples as compared to lossless. And probably --standard is already some overkill which you can find out for your needs by listening tests.

If you use lossyWAV as a replacement for lossless archiving like me the situation is worse of course. But if the percentage of tracks where lossyWAV brings a remarkable relief in file size is large (my situation) I don't care about the few tracks which I encode losslessly. And I don't care too much about some tracks where I may have chosen a suboptimal decision.
If on the other hand your collection most of the time consists of such tracks which are easy to encode for a lossless codec I wouldn't care about lossyWAV (though I might save some file size on some tracks) and would use lossless straight ahead.

Better to use lossywav -5 as reference . It is 'standard'.

Good point, -5.0 added below:

Files: AVG kbps / Total (MB) (Saving (MB)) / % of orig size
Original FLAC -5: 551 / 2,267 / - / 100%
lossyWAV -7.0 - 535 / 2,203 (64 MB) / 97%
lossyWAV -6.5 - 526 / 2,167 (100 MB) / 96%
lossyWAV -6.0 - 516 / 2,124 (143 MB) / 94%
lossyWAV -5.0 - 491 / 2,022 (245 MB) / 89%

This mini snapshot illustrates the issue very well. In red are the lossyWAV at -5 versions (bitrate at the end):

053. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
054. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
055. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 554
056. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 359
057. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 555
058. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 559

On 55 lossyWAV achieves a huge reduction (i.e. 56)
On 53 lossyWAV makes it larger (see 54)
Yet the initial bitrate is very similar.

If there was a tool to identify this prior to encoding, then clearly I'd process 55 but leave 53 alone.

@halb27

Yes, I am using lossyWAV as a replacement for lossless archiving. The reason for the test was to identify the cut-off where it was not useful to employ lossyWAV. There are plenty of files in my classical collection > 550 kbps and lossyWAV has a huge effect and is very worthwhile anywhere between -6 to -7.5 (I've not bothered testing > -7.5).

But, as I said, what was interesting were the many samples that were < 500 kbps where lossyWAV also achieved a significant reduction. So the problem, as I mentioned was not simply a case of saying "ah, below x kbps lossyWAV has no value, thus I shall only use it on files over x kbps". The issue is one of prior identification.

Where a reduction of less than, say, 5% is possible, I'm going to leave them as pure lossless FLACs - that's fine. So far - 6.5 looks good/very safe to me. LossyWAV is still young and I'm sure problem samples will arrive as its take-up increases -- though I may be wrong.

C.

lossywav 1.0.1u

flac -x -b 512 --keep-foreign-metadata %1
(-q 10) - 89,96%
(-q 7.5) - 77,15%
(-q 5) - 65,38%
(-q 2.5) - 51,35%
(-q 0) - 34,96%

for example

flac -x %1
(-q 10) - 96,52%
(-q 7.5) - 83,55%
(-q 5) - 70,93%
(-q 2.5) - 56,45%
(-q 0) - 39,69%

In case this is in some way related to my last post:
I'm using lossywav 1.0.1u with the wiki settings for foobar (i.e. -b 512)

C.

Ran into a strange quirk while using piped input and output:

I first noticed a filesize discrepancy when I was testing stuff from the command line and then from foobar2000.

I used these command lines when testing from the console:

wvunpack -q testfile.wv - | lossywav - -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o test1.lossy.flac

lossywav testfile-fromfb2k.wav -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o test2.lossy.flac

and this for encoding through foobar2000 and saved as test3.lossy.flac:

cmd.exe /d /c lossywav - -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o %d

I noticed that the output file from foobar was 7.11MB while everything else was 7.02MB, so I stripped tags from all the files using Mp3Tag and saw that nothing changed.

test1.lossy.flac: 7,363,354 bytes (MD5: bd2ceb8a15bb390cd2d2e4e74efca13b)
test2.lossy.flac: 7,363,354 bytes (MD5: bd2ceb8a15bb390cd2d2e4e74efca13b)
test3.lossy.flac: 7,464,204 bytes (MD5: a472581649d91ec91f35ef17ef4806e5)

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

Ran into a strange quirk while using piped input and output:
I first noticed a filesize discrepancy when I was testing stuff from the command line and then from foobar2000.

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

And differences in number of seekpoints and/or padding ?

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

This was discussed here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=63254&view=findpost&p=570541) and it is to do with piped input from foobar2000, basically the seektable is larger than it ought to be.

This mini snapshot illustrates the issue very well. In red are the lossyWAV at -5 versions (bitrate at the end):

053. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
054. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
055. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 554
056. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 359
057. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 555
058. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 559

On 55 lossyWAV achieves a huge reduction (i.e. 56)
On 53 lossyWAV makes it larger (see 54)
Yet the initial bitrate is very similar.

If those numbers in brackets are dates, then I'd guess the 1955 recording is quite hissy, while the 1980 recording is not. Lossless encoders have to preserve this hiss perfectly intact - lossyWAV doesn't.


It could be the default lossyFLAC blocksize isn't appropriate for these recordings. Take the lossyFLAC versions and re-encode them to FLAC, but with default FLAC settings. That might help.

Otherwise, it can simply be that lossyWAV can't find anything to remove.

But, as I said, what was interesting were the many samples that were < 500 kbps where lossyWAV also achieved a significant reduction. So the problem, as I mentioned was not simply a case of saying "ah, below x kbps lossyWAV has no value, thus I shall only use it on files over x kbps". The issue is one of prior identification.

I think we've had this discussion before - it would be useful to have a tool to simply keep the original if it was smaller (or similar).

Cheers,
David.

lossyWAV 1.0.1v RC2 attached to post #1 in this thread.

If those numbers in brackets are dates, then I'd guess the 1955 recording is quite hissy

They are, and it is.

It could be the default lossyFLAC blocksize isn't appropriate for these recordings. Take the lossyFLAC versions and re-encode them to FLAC, but with default FLAC settings. That might help.

Tried that - the difference is minimal.

I think we've had this discussion before - it would be useful to have a tool to simply keep the original if it was smaller (or similar).

We have (see below) and it would.

With lossy processes there is normally one frontier, but with lossyWAV there are two:
1) TRANSPARENCY (the traditional problem): what's the lowest we can go and still achieve transparency
2) FILESIZE (a problem unique, I think, to lossyWAV): what kind of original file can defeat lossyWAV, not in terms of transparency but in terms of filesize.

Although the latter is not as important as the former, it is still an issue, as no one expects a lossy process to increase bit rate.

C.

Although the latter is not as important as the former, it is still an issue, as no one expects a lossy process to increase bit rate.

It's not lossyWAV per se which is increasing the bitrate, rather the -b 512 used in the FLAC encoding command line. For a track which gets bigger, try, as David suggested, re-encoding with FLAC without the -b 512 part of the command line. This will default to the original block length (probably 4096 samples) and the processed file should be the same size or maybe slightly smaller.

Yes, apologies Nick.
Increasing bitrate was inaccurate of me.
Results with your amendment to FLAC's default blocksize:

Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 554  (default FLAC blocksize)

The point is with such a reduction (559 to 554 with lossyWAV -5) I wouldn't have bothered.
A complimentary tool to pre-scan and identify such tracks, would certainly be welcome. That's really all my point is - and I shall leave it be from now on.

When it comes to practical use of lossyWAV I'm not going to be editing my batchfile each time I get an increase in bitrate  ; instead I'm going leave that file alone.

C.

EDIT: It's just hit me that I may have a wrong assumption about lossyWAV:
If, hypothetically, lossyWAV can't do anything to reduce the original, does that mean that its process is lossless?
e.g.
ORIGINAL = 500
lossyWAV -2.5 = 500
lossyWAV -7.5 = 500

1) Will the 2 lossyWAV versions be identical?
2) Will the 2 lossyWAV versions be identical to the original?

EDIT: It's just hit me that I may have a wrong assumption about lossyWAV:
If, hypothetically, lossyWAV can't do anything to reduce the original, does that mean that its process is lossless?
e.g.
ORIGINAL = 500
lossyWAV -2.5 = 500
lossyWAV -7.5 = 500

1) Will the 2 lossyWAV versions be identical?
2) Will the 2 lossyWAV versions be identical to the original?

Possibly.... - or at least very close to identical.

Ah! So lossyWAV is even safer than I thought. This makes sense of halb27's response a while back as to not being too bothered about the lossyWAV processed files that had almost zero reduction. He knew they were practically lossless.

Thanks Nick. That's cleared up a strange misconception on my part.

C.

They can be absolutely 100% lossless if lossyWAV judges that this is required.

Try a pure digitally generated sine wave, for example.

Cheers,
David.

Sample 3 - Sine Wave - A440  (generated in Cool Edit)
RG = -11.43 dB

Original FLAC -5 (318 kbps)

-q 10 -b 512 (363)
-q 7.5 -b 512 (363)
-q 5.0 -b 512 (363)
-q 2.5 -b 512 (363)
-q 0 -b 512 (352)

-q 7.5 -b 4096 (296)
-q 5.0 -b 4096 (296)
-q 2.5 -b 4096 (296)

Should I be getting identical bitrates to the original, or is the point that regardless of -q setting lossyWAV is not compromising quality (except at -q0, but hey)?

C.

Sample 3 - Sine Wave - A440  (generated in Cool Edit)
RG = -11.43 dB

Original FLAC -5 (318 kbps)

-q 10 -b 512 (363)
-q 7.5 -b 512 (363)
-q 5.0 -b 512 (363)
-q 2.5 -b 512 (363)
-q 0 -b 512 (352)

-q 7.5 -b 4096 (296)
-q 5.0 -b 4096 (296)
-q 2.5 -b 4096 (296)

Should I be getting identical bitrates to the original, or is the point that regardless of -q setting lossyWAV is not compromising quality (except at -q0, but hey)?

C.

It would be interesting to see the bitrate of original FLAC -b 4096 and the identical compression option you used with lossyWAV (I only guess you use -5 too). As the lossyWAV bitrate is the same with -q 2.5 to -q 7.5 I expect original FLAC will get close to this bitrate when using -b 4096 as with lossyWAV.
Other than that this is a good sample to see that a short blocksize can make the lossless encoder increase bitrate pretty much with 'simple' music.

If you have Cool Edit, you can simply load the original .wav and the .lossy.wav and subtract one from the other (mix paste, 100%, invert) to see if there is any difference at all. Or use the foobar2k wave comparator. Or ask lossyWAV how many bits it's removing.

Cheers,
David.

If you have Cool Edit, you can simply load the original .wav and the .lossy.wav and subtract one from the other (mix paste, 100%, invert) to see if there is any difference at all. Or use the foobar2k wave comparator. Or ask lossyWAV how many bits it's removing.

Cheers,
David.

Or use the --correction parameter and look at the .lwcdf.wav file.

It would be interesting to see the bitrate of original FLAC -b 4096 and the identical compression option you used with lossyWAV (I only guess you use -5 too). As the lossyWAV bitrate is the same with -q 2.5 to -q 7.5 I expect original FLAC will get close to this bitrate when using -b 4096 as with lossyWAV.

Original FLAC -5 -b 512 = 363
Very close!
Cool.

Didn't think of using mix paste, I'll try all the suggestions -- correction file looks appealing.
Thanks Nick, David, Halb27, much appreciated.

C.

Thanks for your sample.

Makes me reconsider using FLAC for the cases of 'simple' music (solo instruments or very quiet music) where I use wavPack right now.
wavPack is great - especially in these cases - but I'd prefer not to use various codecs (thinking of another DAP in the far future). Your sample encourages me to try FLAC with an explicit large blocksize.

lossyWAV 1.0.1w RC3 attached to post #1 in this thread.

(hopefully I've nailed the WINE crashing issue this time....)

Should I be getting identical bitrates to the original, or is the point that regardless of -q setting lossyWAV is not compromising quality (except at -q0, but hey)?

A sine wave concert A (440 Hz) is essentially a pure tone at 440 Hz (the windowed FFT will spread it somewhat) with nothing more than spectrally flat dither noise at other frequencies, which should come to about -120 dBFS per bin with a 1024-point FFT except for the bins around 440 Hz which are presumably full-scale (close to 0dBFS save for spreading caused by windowing)

LossyWAV will look for the noise floor over the frequencies below 16 kHz (the lowest frequency bin). As this is close to -120 dBFS and probably a little lower thanks to the random nature of dither noise, lossyWAV is very likely indeed to retain all 16-bits. The exception is where there's a large negative safety margin to reduce bitrate at the expense of added hiss, as is the case at -q 0.

That explains why you get essentially consistent bitrate with 512 blocksize. I'm not sure what the difference is between flac -5 (lossless, which normally give -b 4096 for 44.1 kHz audio) and lossyFLAC -q 5.0 and -b 4096, which gave 318 kbps and 296 kbps respectively. I guess you should try using just flac -5 or just flac -5 -b 4096 for both lossless and lossyFLAC to compare like-with-like and then see if the lwcdf.wav file has non-zero content (Cool Edit analysis can tell you).

Did a mix-paste in Cool Edit, and just ran a bit compare in foobar:
No differences in decoded data found.

Yet in foobar, bitrates are different:
FLAC Original -b 4096 = 318
LossyFLAC -b 4096 -q 5 = 296

Anyway, the point is made and to be honest I'm just happy that lossyWAV is very careful indeed with the data, and thus it's fine to use for archiving and using as transcode source.

Thanks Dynamic for the explanation.

C.

Did a mix-paste in Cool Edit, and just ran a bit compare in foobar:
No differences in decoded data found.

Yet in foobar, bitrates are different:
FLAC Original -b 4096 = 318
LossyFLAC -b 4096 -q 5 = 296

Anyway, the point is made and to be honest I'm just happy that lossyWAV is very careful indeed with the data, and thus it's fine to use for archiving and using as transcode source.

Thanks Dynamic for the explanation.

C.

Remember, when piping in foobar2000, the [edit] FLAC [/edit] seektable will be large and the padding will be 65536 bytes per file. This may be affecting your bitrate (assuming you are carrying out the transcoding using pipes in foobar2000, of course).

Both FLACs were created using foobar2000 (but perhaps only one is using piping? - not even really sure what piping is).  One FLAC is created from the other -- piping?

Original FLAC created from WAV.
LossyFLAC created from Original FLAC.

The difference in filesize = 79,214 bytes.
The files are exactly 30 secs in length.

C.

I'll be releasing 1.1.0 final tomorrow night - but I wonder which of the more advanced options included for developmental purposes post 1.0.0b should be removed?

My guess is that --limit should stay but noone seems to be using --highskew. --blocksize is also not necessarily a good idea to keep as it may confuse users as it may be better to keep it at 512 samples throughout.

I've been sorting out the logfile mechanism - it will no longer crash with two or more foobar2000 transcoding threads. Also, I've expanded on --bitdist - now renamed as --blockdist and supplemented by --sampledist which both give information about the block / sample distribution of lsb's and msb's. This will also be written to a logfile.

For the logfile when used with STDIN, I have implemented a --stdinname parameter which is followed by the name which the user wishes to be displayed on the console and also written to the log.

Any suggestions, likes, dislikes, etc will be very much appreciated.

I'll be releasing 1.1.0 final tomorrow night - but I wonder which of the more advanced options included for developmental purposes post 1.0.0b should be removed?

My guess is that --limit should stay but noone seems to be using --highskew. --blocksize is also not necessarily a good idea to keep as it may confuse users as it may be better to keep it at 512 samples throughout.

I've been sorting out the logfile mechanism - it will no longer crash with two or more foobar2000 transcoding threads. Also, I've expanded on --bitdist - now renamed as --blockdist and supplemented by --sampledist which both give information about the block / sample distribution of lsb's and msb's. This will also be written to a logfile.

For the logfile when used with STDIN, I have implemented a --stdinname parameter which is followed by the name which the user wishes to be displayed on the console and also written to the log.

Any suggestions, likes, dislikes, etc will be very much appreciated.

I don't use any of the advanced options as I don't really understand what some of them do and under what circumstances it's appropriate to use them. However, I guess that some people do use them so they're probably all worth keeping just in case someone finds them useful. The thing I like about LossyWav is that you can use very basic settings, like the quality level, or even just let it default everything and get good results while on the other hand you have very fine control over how the program behaves if you want to use the advanced settings. Having said that it would be best to remove anything that was put in for monitoring/debugging especially if it's likely to confuse somebody. On that subject is using the same letter but different case for different parameters likely to confuse? EG -s/-S, -d/-D.

Just a small confirmation that iTunes 7.7 still can't make use of lossywav to save bits. Too bad, as my iPod is /the/ place I'd use this ... Ah well, thanks for the good work, it's a very interesting project

I'll be releasing 1.1.0 final tomorrow night ...

Wonderful. Much appreciated.

... but I wonder which of the more advanced options included for developmental purposes post 1.0.0b should be removed? ...

IMO most of the advanced options should be removed in a final release.
--highskew, --blocksize as you think, but if it is up to me I would always make the last beta version available, and keep the advanced options there, but remove the advanced options from the final verison with very few exceptions.
Valuable exceptions IMO are -q, --scale, -s.
The problems with many of the advanced options is also that an average user can't understand them, and a really useful precise description will probably offend many users.
The other problem is the mere number of advanced options which at the end of the day has more of a negative effect than a positive one.

So IMO there should be a high threshold, and an advanced option should have a very promising effect at least to some people in order to justify a specific advanced option within the final version.

The situation is different with beta versions where there are users which want to be a bit experimental.

So for the final version I'd even prefer not to have the --noclips option any more.

IMO most of the advanced options should be removed in a final release.
--highskew, --blocksize as you think, but if it is up to me I would always make the last beta version available, and keep the advanced options there, but remove the advanced options from the final verison with very few exceptions.
Valuable exceptions IMO are -q, --scale, -s.
The problems with many of the advanced options is also that an average user can't understand them, and a really useful precise description will probably offend many users.
The other problem is the mere number of advanced options which at the end of the day has more of a negative effect than a positive one.

So IMO there should be a high threshold, and an advanced option should have a very promising effect at least to some people in order to justify a specific advanced option within the final version.

The situation is different with beta versions where there are users which want to be a bit experimental.

So for the final version I'd even prefer not to have the --noclips option any more.

I'll strip out most of the advanced options from 1.1.0 and also leave 1.0.1x RC4 available with the advanced options left in.

To confirm: --fft32, --analyses, --highskew, --blocksize, --minbits, --noclips & --wine will all be removed for the release of 1.1.0.

...To confirm: --fft32, --analyses, --highskew, --blocksize, --minbits, --noclips & --wine will all be removed for the release of 1.1.0.

That's fine.

lossyWAV 1.1.0 attached (with source) to post #1 in this thread.

Thanks a lot, Nick.