Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lossyWAV Development (Read 561394 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lossyWAV Development

Reply #75
... Maybe "-f" for FLAC and "-w" for WavPack? I am a fan of simplistic command lines with single character switches (if possible - and this is not going to be *too* complex......).
...

Fine, however - for definiteness and greater clarity - what about -<format-extension> like -flac or -wv as the optimization option?

I'll go and find fft and conv implementations.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #76
Fine, however - for definiteness and greater clarity - what about -<format-extension> like -flac or -wv as the optimization option?
Yes, I see your point and agree - "-flac" and "-wv" it is! Thanks for volunteering to go hunting for code...... I'll start on the wavread / wavwrite implementations tonight.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #77
... I'll start on the wavread / wavwrite implementations tonight. ...

Oh.. I forgot that. I can transcode my Euphoria reading and writing of wav files to Delphi.
Can do it this weekend if you can wait for that.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #78
Oh.. I forgot that. I can transcode my Euphoria reading and writing of wav files to Delphi.
Can do it this weekend if you can wait for that.
Absolutely! I'll try to get the rest of the algorithm side of it as far advanced as possible while waiting for fft, conv, wavread and wavwrite.

Many thanks!

[edit] May have found viable FFT / CONVOL routines - TPMAT036 - certainly look promising, and free! Available at http://www.unilim.fr/pages_perso/jean.debo...math/tpmath.htm [/edit]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

 

lossyWAV Development

Reply #79
[edit] May have found viable FFT / CONVOL routines - TPMAT036 - certainly look promising, and free! Available at http://www.unilim.fr/pages_perso/jean.debo...math/tpmath.htm [/edit]

Oh, you're real fast !!!
I looked up the documentation and it looks very promising as you said.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #80
Well, I've done some Delphi Coding and created
  • a unit wavIO which does the IO of wav files for our purpose
  • a unit cliParameter which does the CLI parameter handling
  • a test and demonstration program LossyWavTest that shows how to use these units and which together with test unit MakeLossy makes up for a stupid preprocessor which simply sets the 5 LSBs in each sample to zero.
As for bits per sample I think 8 bits per sample need not be supported. I reject such input files.
At the moment I also reject 24 bit per sample files. The structure of the unit is thus that 24 bit are supported but with actual reading and writing this is not the case. I will add this within the next days (now I'm gonna prepare dinner for friends).

As for the sampling frequency I reject any sample frequency below 32 kHz and above 48 kHz. I am afraid the logical details of the preprocessing procedure will depend on the sampling frequency as the number of samples taken into account correspond to a certain time period. If everything is optimized for 44.1 kHz things may work fine for 48 kHz cause this is just ~10% off. With 32 kHz it's worse (~30% off).
Anyway I think we should be conscious about it and take care of everything we support.
In order to make things precise (what we support) I've restricted sample frequency to the range mentioned which probably is the most common range anyway.

[attachment=3625:attachment]
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #81
Thanks for the code - I'm back from a weekend away, so should be able to devote some time to the project this week.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #82
24 bit input files now supported in unit wavIO:

[attachment=3628:attachment]
Samples are read and written blockwise where a block corresponds to a FLAC/wavPack/TAK etc. block.
wavIO deals with sample blocks for channel 0 and 1 (in the case of stereo) of the kind:
sampleBlockCh0, sampleBlockCh1: array[0..blocksize] of LongInt;

Thus sample values are always 32 bit integers. With 16 (24) bit files the 16 (24) bit make up for the 16 (24) MSBs and the remaining bit are set to 0.

(In my previous version the 16 bit samples were just 16 bit ints (judging from value range) in an 32 bit integer container which was not a good idea as 16 bit and 24 bit input files would not have a matching representation).

Edited: Link is to new zip file. Sorry I forgot to remove testing statements in the previous version.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #83
I'm just playing round with the bibilolo sample from recent 64 kbps listening test.
As it's a bandwidth testing sample I wanted to find out whether or not my 32 kHz resampling does have an audible effect for me with this sample. However what I found was much more of concern: it's a very problematic sample for wavPack lossy, for instance at sec. 17.2-19.2.

So it may be worth while testing with this preprocessor. Nick.C., do you mind processing it?
AlexB showed me it's sample 3 from Gabriel's samples for an 48 kbps AAC test:

http://www.mp3-tech.org/tests/aac_48/samples/.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #84
I'm just playing round with the bibilolo sample from recent 64 kbps listening test.
As it's a bandwidth testing sample I wanted to find out whether or not my 32 kHz resampling does have an audible effect for me with this sample. However what I found was much more of concern: it's a very problematic sample for wavPack lossy, for instance at sec. 17.2-19.2.

So it may be worth while testing with this preprocessor. Nick.C., do you mind processing it?
AlexB showed me it's sample 3 from Gabriel's samples for an 48 kbps AAC test:

http://www.mp3-tech.org/tests/aac_48/samples/.
Not a problem at all - attachment processed using -2 presets as agreed in the previous posts.
I'm having "fun" with Delphi - my head hurts after a few hours with it and it's late now. I will try to have a (very limited) version of lossyWAV.exe available later this week.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #85
Thank you.
Result is very good, no audible problem to me (though I will do it again more carefuly tomorrow).
The preprocessor really shines. It knows when not to throw away a lot (negligible saving in bitrate with this sample).
wavPack lossy does it the other way around and uses a bitrate lower than the nominal one (rare with wavPack). This is a sample where kind of a quality control with wavPack lossy is missed badly.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #86
  Okay then - v0.0.1 of lossyWAV.exe.
It *will* crash occasionally. badvilbel always makes it crash for instance;
Settings are not yet fully implemented.
Quality checks are not yet implemented.
Only one fft length (1024) is used at present.
Posting just for those who want to play with it at this early stage.
syntax: lossyWAV <inputfilename> <outputfilename>
Have fun!


[edit 20070825] Too little, too early - sorry. File removed.[/edit]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #87
Foobar2000 compatible batch file to use as an external encoder:

Code: [Select]
@echo off
set lossyWAV_path="c:\data_nic\_wav\lossyWAV.exe"
set flac_path="c:\program files\flac\flac.exe"
%lossyWAV_path% %1 "%~D1%~P1%~N1.ss.wav"
%flac_path% -8 -f -b 1024 -o"%~D1%~P1%~N2%~X2" "%~D1%~P1%~N1.ss.wav"
del "%~D1%~P1%~N1.ss.wav"
set lossyWAV_path=
set flac_path=
Remember (on multi-processor / multi-core processor PC's) to set affinity of Foobar2000 to only one processor - or it will crash when trying to process the second file on the convert list.

See attached image for settings in Foobar2000. Superseded.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #88
Code: [Select]
.WAV   59317484  Same Thing -org.wav      1411   org

.FLA   31141612  Same Thing -1.flac        741   org
.FLA   29462093  Same Thing -8.flac        701   org
.FLA   13009242  Same Thing -lf.flac       309   flossy
.FLA   20282435  Same Thing shi.flac       482   32 kHz samplerate

Just a quick test. For dos/win98/ lovers: BatchEnc 1.51 from Speek works too 
I think it's a promising project. Thanks. Not abx'ed yet.
Noticed that not only the high frequencies are cut off at 32 k, but I'm also missing the deep bass in my test sample. Which is Same thing from Bonnie Raitt

lossyWAV Development

Reply #89
For resampling I suggest to use ssrc_hp with the --twopass option (to avoid clipping). Haven't found any problem with it so far.

32 kHz is a standard sample rate and as such has its own merits, but maybe something like 35 kHz together with flossy is a very attractive choice.
35 kHz can be played back for instance by foobar, winAmp (and also my rockbox armed iRiver DAP). It may depend on your sound card however.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #90
Thanks for this tool (and the screenshot!!!). I was curious to see how much space it would spare with some quiet classical music (lossless bitrate <400 kbps).
I give a try: as expected it didn't spare that much bitrate (7 kbps). But the bad thing is that it's easily ABXable, even at low playback volume (no replaygain and volume knob on a quiet position):

http://www.megaupload.com/?d=WAEP6D5F





EDIT:

I found even worse:
now lossy encoding is ~50% bigger than lossless but awfully noisy ?!

http://www.megaupload.com/?d=WL4G98P7

lossyWAV Development

Reply #91
But guru, this isn't a complete port of lossyFLAC - it's a first stab. It's missing half the analysis and won't be anywhere near transparent!

And you're right - it's less useful for "your kind of music" which is usually (intelligently) encoded with little loss, which is exactly what's required.

If you have any test samples which you'd like encoded properly with the MATLAB version, just post them here.

Cheers,
David.

lossyWAV Development

Reply #92
Disappointed (but not *really* surprised) to hear that - sorry if I raised false hopes / expectations.

I will try to implement the multi-length FFT analyses and also re-introduce the noise_threshold_shift tonight at the same time as reinstating the settings derived from Wombat and Halb27's ABXing earlier in the thread.

Additional comment as the build quality increases / becomes measurably closer to the Matlab script will be very gratefully received - observations are always useful.

I intend to carry out some side-by-side testing to allow codec-block-by-codec-block checking of the bits_to_remove for each analysis fft_length - to see if the Delphi version matches the Matlab version.

I also freely admit that v0.0.1 is a "quick win", i.e. the first build that actually uses the fft analysis and threshold_index values to determine number of bits to remove, and does not (hopefully) represent the quality of output of later versions.

Possibly I shouldn't have posted it publicly at such an early stage - I may need to resort to a more private alpha test scenario.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #93
I'm sorry... I thought this tool include the full analysis.
If you have any test samples which you'd like encoded properly with the MATLAB version, just post them here.

Cheers,
David.

The problem is: I don't know what kind of sample may be interesting with this kind of processing. That's why I was waiting to experiment on my side with a wide library of sample.
Anyway, I recall that my gallery of 150 samples is still online and if something must go wrong with this kind of PCM processor this collection may help to find it.

lossyWAV Development

Reply #94
For resampling I suggest to use ssrc_hp with the --twopass option (to avoid clipping).

I already do, thanx to you. The aac 'problem' isn't a problem to me, it was merely a test. I only use flac, and mp3 for the wife's portable.

lossyWAV Development

Reply #95
I intend to carry out some side-by-side testing to allow codec-block-by-codec-block checking of the bits_to_remove for each analysis fft_length - to see if the Delphi version matches the Matlab version.


As long as you have dither switched off, you can compare the resulting .wav files. They'll be identical _if_ you use the same reference noise thresholds for both. In reality, since the reference thresholds are set by measuring a sample of noise, they probably won't be - don't let that surprise you or make look for bugs that aren't there!

Cheers,
David.

lossyWAV Development

Reply #96
...I intend to carry out some side-by-side testing to allow codec-block-by-codec-block checking of the bits_to_remove for each analysis fft_length - to see if the Delphi version matches the Matlab version. ...

I understand very well that you are proud of being so extraordinary quick in creating this first Delphi version of lossyWave but I also thought it produces what you've arrived at with the MATLAB script.
An intermediate state isn't so useful I think.
It's a very good idea to test parts of the Delphi version so that it arrives at exactly the same result as the MATLAB version (with the exception of possibly errors found in the MATLAB version). After all you've arrived already at a quality with the MATLAB version which seems to be very good and which should be preserved.
So I think it's worth waiting some more days (or weeks) and a having a real productive version for public testing.

With what parts can I help getting on? After I have started contributing a little bit I also want to go ahead right now. Cleaning up the photos from my summer holidays with which I'll be busy for some weeks can wait.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #97
@halb27: I would very much appreciate it if you could further develop the cliParameters unit to allow settings (extra settings), of the type "-'char'" followed by numeric or text parameter depending on the char in the parameter, e.g. -b 16 to force 16 bit output regardless of input bitdepth; -c 1024 to set codec_block_size to 1024 bytes; etc, etc.

I would also expect to automatically derive the output filename from the input, i.e. outfile = name.lossy.wav; or possibly specify an output directory (-d pathname\ ?)

I know that the latter concept was to limit the user specifiable options to one, i.e. -1, -2 or -3, but at this stage it might be useful to allow the user to over-ride certain settings in the pursuit of settings less easy to ABX.........

p.s. please PM me an e-mail address and I will forward the latest project code.
p.s.2. code is getting neater and more understandable - there are some no-parameter functions and procedures - horrible coding, but fast and it makes the code clearer - as you already mentioned. I will be looking at multiple analyses tonight and coding a CONV routine - very simplistic with the [0.25,0.25,0.25,0.25] spreading_function.

Best regards,

Nick.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #98
@halb27 - ygem.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #99
Anyway, I recall that my gallery of 150 samples is still online and if something must go wrong with this kind of PCM processor this collection may help to find it.

Really looking forward to this input!
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!