Skip to main content

Topic: lossyWAV Development (Read 385654 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #675
I'll have a look at the -1 btr issue, although, thinking about it, it should have no impact at all as when btr falls to zero then the block is merely stored and is not processed at all.
The bug is merely presentational - when btr is indicated as -1 then it is actually 0 and the codec_block is stored. This will be amended for beta v0.5.9

As an aside, could this method be applied to 32-bit float values, i.e. round the mantissa only? And would any of the lossless codecs make use of zero LSB's in the mantissa?
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #676
b) The replaygain procedure doesn't process the sound. It just computes a volume correction value and stores it in the file so that the playback machinery is able to adjust volume according to this value (in case the playback machinery has a replaygain feature).
[snip]
The only form of 'sound processing' because of replaygain can occur on playback when there is something like a 'soft clipping prevention' feature with replaygain which usually can be switched off if not wanted.


Well that's not entirely true.  In fact, if the replaygain code is handled in 16bits, it can easily cause an audible effect to be heard.
No, not "easily". "Potentially".

Of course, the effects of ReplayGain are almost always audible - it changes the volume!

However, 16bits are more than enough to accomplish this transparently for most audio signals. Not all, but most.

The potential for detecting the re-quantisation (or re-dithering) exists with extremely dynamic signals, where ReplayGain reduces the volume because substantial sections are relatively loud, but the listener cranks the volume up to concentrate on some relatively quiet sections. In these sections, the dither or quantisation may be audible.


A far greater and more common problem is owning a poor DAP with a hissy amplifier, low power etc, where ReplayGain will make everything too quiet and drop it into the noise floor.

Cheers,
David.


Sorry for not posting these sooner. I've been out all day.
I've attached 10 second samples. The issue happens 5 seconds in when encoding the sample.flac file.
Sorry, the M5_issue.flac isn't great quality but you can clearly hear the issue I'm experiencing.
Just to check...

M5_issue is mono. I assume your DAP is actually playing back in stereo?

Cheers,
David.

  • stel
  • [*]
lossyWAV Development
Reply #677
OK, I've spent some time this evening playing around with different encoder settings on my problem sample and I'm kicking myself with the results. The good news is that it's not lossyWAV that's at fault...

My problem was caused by the 'Bit Depth Control' settings in foobar2000. The 'Format is:' was set to lossy and 'Highest BPS mode supported:' set to 24. Setting the format to lossless(or hybrid) fixed the issue. I guess I've encoded everything at 24bit and my DAP isn't happy about it.

Apologies to everyone who spent time looking into it.

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #678
OK, I've spent some time this evening playing around with different encoder settings on my problem sample and I'm kicking myself with the results. The good news is that it's not lossyWAV that's at fault...

My problem was caused by the 'Bit Depth Control' settings in foobar2000. The 'Format is:' was set to lossy and 'Highest BPS mode supported:' set to 24. Setting the format to lossless(or hybrid) fixed the issue. I guess I've encoded everything at 24bit and my DAP isn't happy about it.

Apologies to everyone who spent time looking into it.
Many thanks for the subsequent investigations! My iPAQ won't even play 24bit.....

[edit] Interestingly, the difference between my 53 sample test set at 16 bit and 24 bit (Foobar WAV output) is about 40kB in 38.4MB [/edit]
  • Last Edit: 17 December, 2007, 04:52:43 PM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • jesseg
  • [*][*][*]
  • Banned
lossyWAV Development
Reply #679
The 'Format is:' was set to lossy and 'Highest BPS mode supported:' set to 24. Setting the format to lossless(or hybrid) fixed the issue.


You're the second person in this thread alone to run into problems from that setting.  I would think they would have it set to lossless by default, but I guess not (or is it?)

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #680
I've been toying with looking at implementing (& optimising in IA-32) mixed-radix FFT's (i.e. non-power-of-two-length).

At present, they're *very* slow.

Is there a feeling that changing the timeframe of the fft analysis from 1.45msec (64 samples @ 44.1kHz) to, say, 1.497msec (66 samples @ 44.1kHz) or 23.22msec (1024 samples @ 44.1kHz) to 20msec (882 samples @ 44.1kHz) would be beneficial?

This would also allow better handling of WAV's which are other than 44.1kHz as the optimal FFT length could be calculated directly rather than using the existing 64 / 128 / 256 / 512 / 1024 sample lengths, e.g. 72 samples = 1.5msec @ 48kHz; 960 samples = 20msec @ 48kHz.

If it is considered to be of merit then I will progress the implementation / optimisation over the festive period, initially alongside the existing power-of-two FFT routine unless (until?) the mixed-radix method becomes acceptably fast.
  • Last Edit: 18 December, 2007, 07:14:41 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • halb27
  • [*][*][*][*][*]
lossyWAV Development
Reply #681
The best of possibilities that arise from such a generalization IMO comes with the long FFTs where a finer differentiation becomes possible. By now we only have the choice between a 512 and a 1024 sample FFT.

Anyway I can't imagine ist's a real benefit, even for a 48 kHz source which makes up for a ~10% difference in timing.

I once was worried about the overlapping of the 1024 sample FFT into the neighboring blocks (and I would still prefer the 5/8 partitioning I once suggested) which can also be reduced by using a say 882 sample FFT, but because of the very good quality we have I'm content with the way you do the FFT right now.
lame3995n -Q0.5

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #682
I think it's a waste of time. The exact numbers aren't important, and even in lossy codecs (where the exact numbers are important) they still stick to fixed values irrespective of the sample rate, with reasonable results.

Cheers,
David.

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #683
The best of possibilities that arise from such a generalization IMO comes with the long FFTs where a finer differentiation becomes possible. By now we only have the choice between a 512 and a 1024 sample FFT.

Anyway I can't imagine ist's a real benefit, even for a 48 kHz source which makes up for a ~10% difference in timing.

I once was worried about the overlapping of the 1024 sample FFT into the neighboring blocks (and I would still prefer the 5/8 partitioning I once suggested) which can also be reduced by using a say 882 sample FFT, but because of the very good quality we have I'm content with the way you do the FFT right now.
I think it's a waste of time. The exact numbers aren't important, and even in lossy codecs (where the exact numbers are important) they still stick to fixed values irrespective of the sample rate, with reasonable results.

Cheers,
David.
Thanks guys - I'll park it. Only the presentational bug of -1 btr to be corrected for beta v0.5.9.

-shaping may get the bullet as it is really beyond my grasp of audio processing.

As things seem to be ticking along nicely, I think I'll concentrate on producing the correction file next.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #684
I probably said this before, but I strongly advise borrowing the wavpack file format for that, as much as possible/relevant, because I'm sure David Bryant has worked through some of the issues that might be faced here. It might not be entirely applicable, but it's always good to re-use someone else's (good) work when they're happy for you to do so!

Cheers,
David.

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #685
I probably said this before, but I strongly advise borrowing the wavpack file format for that, as much as possible/relevant, because I'm sure David Bryant has worked through some of the issues that might be faced here. It might not be entirely applicable, but it's always good to re-use someone else's (good) work when they're happy for you to do so!

Cheers,
David.
My approach was going to be even more simplistic - write two WAV files, one lossy, one correction, then encode both in FLAC. I'm working on adding a FACT chunk to the WAV file - containing a null terminated string which will include lossyWAV version information, parameters used and date/time of processing (and ultimately the CRC32 of the lossless and lossy data) which would be written to both files.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #686
I'm working on adding a FACT chunk to the WAV file - containing a null terminated string which will include lossyWAV version information, parameters used and date/time of processing (and ultimately the CRC32 of the lossless and lossy data) which would be written to both files.
I've successfully implemented a mechanism to insert a 160 byte FACT chunk immediately after the FMT chunk in the WAV file. This takes the form:
Code: [Select]
fact/152/lossyWAV v0.5.9 : 20/12/2007 08:46:57
-2 -cbs 512 -nts 0.00 -snr 21.00 -skew 36.00
-spf 22224-22235-22336-12347-12358 -fft 10101
If a file has already been processed, the FACT chunk will be found and lossyWAV will exit. When encoding in FLAC the --keep-foreign-metadata switch must be used to preserve the lossyWAV FACT chunk.

Thinking about it, I should make a bit more effort and make the FACT chunk variable length (up to a sensible maximum). In this way, the total length of the FACT chunk will be (8+string_length+(string_length and 1).
  • Last Edit: 21 December, 2007, 06:23:34 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #687
lossyWAV beta v0.5.9 attached at post 1 of the thread.

Fixed btr -1 bug;
Implementation of FACT chunk inclusion in output when processed. lossyWAV will exit if it finds a lossyWAV FACT Chunk in a WAV file. FLAC required --keep-foreign-metadata switch to retain FACT chunk.
  • Last Edit: 01 January, 2008, 09:25:21 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • jesseg
  • [*][*][*]
  • Banned
lossyWAV Development
Reply #688
Could you have the exe return... something besides error level 1, when it actually has an error, such as the WAV file already having the lossyWAV flag?  That will allow me, and others, to add handling for it into batch files and software.

Very good idea though.  I'll release a new batch for iFLCDrop when I can get back a specific code for the "lossyWAV flag exists" error.  With the error code returned I can make the script copy the source-file into the temp directory, and have it get encoded into FLAC anyways.

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #689
Could you have the exe return... something besides error level 1, when it actually has an error, such as the WAV file already having the lossyWAV flag?  That will allow me, and others, to add handling for it into batch files and software.

Very good idea though.  I'll release a new batch for iFLCDrop when I can get back a specific code for the "lossyWAV flag exists" error.  With the error code returned I can make the script copy the source-file into the temp directory, and have it get encoded into FLAC anyways.
Very good idea, I'll start thinking about the codes needed and will post as beta v0.6.0 in a couple of days.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #690
Could you have the exe return... something besides error level 1, when it actually has an error, such as the WAV file already having the lossyWAV flag?  That will allow me, and others, to add handling for it into batch files and software.

Very good idea though.  I'll release a new batch for iFLCDrop when I can get back a specific code for the "lossyWAV flag exists" error.  With the error code returned I can make the script copy the source-file into the temp directory, and have it get encoded into FLAC anyways.
Very good idea, I'll start thinking about the codes needed and will post as beta v0.6.0 in a couple of days.
Another thought - would it be useful to, say, "lossyWAV <wavfile.wav> -check" to allow the user to check for a processed file without trying to process it again?
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • jesseg
  • [*][*][*]
  • Banned
lossyWAV Development
Reply #691
Yes, especially if it returned an error code if it was already processed.  It would allow an application to check or batch check for lossyWAV or non-lossyWAV files... for whatever reason someone might want to do that.  But even without an error code, it would be ok for non-automated or non-batch use I guess.

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #692
Yes, especially if it returned an error code if it was already processed.  It would allow an application to check or batch check for lossyWAV or non-lossyWAV files... for whatever reason someone might want to do that.  But even without an error code, it would be ok for non-automated or non-batch use I guess.
I would of course include an error code  following your request. You can even see the rifffact chunk stored at the beginning of FLAC files thanks to Josh's latest --keep-foreign-metadata switch, and Tiny Hexer file binary viewer / editor.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • singaiya
  • [*][*][*][*]
lossyWAV Development
Reply #693
Is lossyWAV's method similar to Wavpack lossy's method? I thought it was a different approach, but then seeing mention of correction files maybe they're more similar than I thought?

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #694
The concept of the correction file will be similar in that the lossless original will be able to be recomposed from the lossy.wav and the lwcdf.wav files by simple sample addition.

lossyWAV rounds LSB's to zero where the added noise of the rounding is calculated to be below a threshold value.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #695
I would of course include an error code  following your request. You can even see the rifffact chunk stored at the beginning of FLAC files thanks to Josh's latest --keep-foreign-metadata switch, and Tiny Hexer file binary viewer / editor.
lossyWAV beta v0.6.0 attached at post 1 of the thread.

Error code = 16 on exit if WAV file has already been processed.

-check parameter included to allow checking without trying to process the file, error code = 16 if processed. [edit]It's not clear in the parameters that -check should only be used in the context: "lossyWAV <wavfile.wav> -check" as it will always exit after determining whether a lossyWAV FACT chunk exists.
  • Last Edit: 01 January, 2008, 09:24:44 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • jaybeee
  • [*][*][*][*]
lossyWAV Development
Reply #696
Thanks for the latest update Nick.C.
I personally think it's best to add download links to the first post in the thread; that way it's always easy to find 

I use foobar, the set up as per the lossywav wiki entry, and so do you think it's possible to preserve Replaygain tags? (I'm unsure if this is a foobar or lossywav issue).
  • Last Edit: 29 December, 2007, 12:27:01 PM by jaybeee

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #697
Thanks for the latest update Nick.C.
I personally think it's best to add download links to the first post in the thread; that way it's always easy to find 

I use foobar, the set up as per the lossywav wiki entry, and so do you think it's possible to preserve Replaygain tags? (I'm unsure if this is a foobar or lossywav issue).
David will correct me if I am wrong, but I've seen slightly different Replaygain values for the same track pre as opposed to post processing, so I don't know if retaining them is a good idea. It wasn't much though. lossyWAV does nothing at all with tags - that's all Foobar (thankfully).

I will edit the first post in this thread to reflect both its content and as the download point.
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps

  • halb27
  • [*][*][*][*][*]
lossyWAV Development
Reply #698
How do you make foobar pass the replaygain information to the resulting (say) FLAC file?
I'd love to do that as my input ape files do have replaygain information.

I personally wouldn't care about slightly incorrect replaygain values (and I can't but imagine they are small).
The bigger problem to me is with replaygain that sometimes the replaygain values have to be corrected manually to achieve an equal loundness impression. I'd be happy if I could do these manual corrections just once in my ape files, and take profit of it whenever I encode them.
  • Last Edit: 29 December, 2007, 03:38:04 PM by halb27
lame3995n -Q0.5

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV Development
Reply #699
lossyWAV beta v0.6.1 now appended to post #1 of this thread.
  • Last Edit: 03 January, 2008, 11:29:32 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 1| FLAC -5 -e -p -b 512 -P=4096 -S- ~= 320kbps