Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lossyWAV Development (Read 561406 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

lossyWAV Development

Reply #450
...I will continue my quest to further optimise and speed-up the code. FP assembly language is not as painful as I first thought. I did download the Intel IA-32 Software Developers Manual and it's got lots of nice instructions in it.... However I would be worried about using instructions only available on later processors as I don't wish to alienate any users (and am not in the position [yet] to maintain separate builds).

Nice you do optimizing. IMO you're absolutely right in not going too far spezializing. Speed is welcome but even more is using your exe without getting into trouble (including your personal trouble as extreme optimizing can be troublesome).
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #451
I will continue my quest to further optimise and speed-up the code. FP assembly language is not as painful as I first thought. I did download the Intel IA-32 Software Developers Manual and it's got lots of nice instructions in it.... However I would be worried about using instructions only available on later processors as I don't wish to alienate any users (and am not in the position [yet] to maintain separate builds).

Do you know the bible of IA-32 optimization? If not: Optimizing assembly code (Agner Fog)

  Thomas

lossyWAV Development

Reply #452
I will continue my quest to further optimise and speed-up the code. FP assembly language is not as painful as I first thought. I did download the Intel IA-32 Software Developers Manual and it's got lots of nice instructions in it.... However I would be worried about using instructions only available on later processors as I don't wish to alienate any users (and am not in the position [yet] to maintain separate builds).
Do you know the bible of IA-32 optimization? If not: Optimizing assembly code (Agner Fog)

  Thomas
Ooooh! Thanks for that Thomas, I will certainly have a read before I get too heavily down the "Delphi wrapper around an assembly language program" route.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #453
Well, big thanks to Thomas for the pointer to Agner Fog's excellent guide to optimising assembly language. The FFT routine is now completely in IA-32 assembler using only 32bit registers and the FPU. Even so, it is considerably faster than alpha v0.4.0.

lossyWAV alpha v0.4.1 attached: Superseded.

Code optimisation of FFT routine;

Slight change to the -overlap calculations regarding number of fft analyses to carry out for a given block and size of fft_overlap. This means that the "central" fft analysis may not be exactly in the centre of the codec block, but the end_overlap value is exactly half of the fft_length.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #454
Hallo Nick,

Thank you very much for your new version.
Looks like quality has improved: With 'Under The Boardwalk' using plain -3 I'm far away now from being able to abx it. No chance at all.

I'm about to encode part of my collection using -3.
Doing so I wanted to try the 128 sample and 512 sample FFT using a full -spf string but with no effect. Are these FFT lengths reserved to -1?
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #455
Hallo Nick,

Thank you very much for your new version.
Looks like quality has improved: With 'Under The Boardwalk' using plain -3 I'm far away now from being able to abx it. No chance at all.

I'm about to encode part of my collection using -3.
Doing so I wanted to try the 128 sample and 512 sample FFT using a full -spf string but with no effect. Are these FFT lengths reserved to -1?
YGPM!
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #456
Doing so I wanted to try the 128 sample and 512 sample FFT using a full -spf string but with no effect. Are these FFT lengths reserved to -1?
Not any more. I have implemented a "-fft" parameter which takes a 5 character binary numeric input, each character of which corresponds to a specific fft_length, i.e. character 1 > 64 samples, character 2 > 128 samples, etc, character 5 > 1024 samples. So, the default for -2 would be -fft 10101, for -1 would be 10111.

I have also converted the spread and remove_bits procedures to IA-32 / FP assembly, so there's been a bit of a speed up as well. Incidentally, I found out that the size of the data segment for each unit will adversely affect the program speed if not carefully aligned to 8 or 16 byte boundaries (not exactly sure which).

lossyWAV alpha v0.4.2 attached: Superseded.[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]
Code: [Select]
lossyWAV alpha v0.4.2 : WAV file bit depth reduction method by 2Bdecided.
Delphi implementation by Nick.C from a Matlab script, www.hydrogenaudio.org

Usage   : lossyWAV <input wav file> <options>

Example : lossyWAV musicfile.wav

Quality Options:

-1            extreme quality [4xFFT] (-cbs 1024 -nts -3.0 -skew 30 -snr 24
              -spf 11124-ZZZZZ-11225-11225-11236 -fft 10111)
-2            default quality [3xFFT] (-cbs 1024 -nts -1.5 -skew 24 -snr 18
              -spf 11235-ZZZZZ-11336-ZZZZZ-1234D -fft 10101)
-3            compact quality [3xFFT] (-cbs  512 -nts -0.5 -skew 18 -snr 12
              -spf 11235-ZZZZZ-11336-ZZZZZ-1234D -fft 10101)

-o <folder>   destination folder for the output file
-force        forcibly over-write output file if it exists; default=off

Advanced / System Options:

-nts <n>      set noise_threshold_shift to n dB (-18dB<=n<=0dB)
              (reduces overall bits to remove by 1 bit for every 6.0206dB)
-snr <n>      set minimum average signal to added noise ratio to n dB;
              (0dB<=n<=48dB)
-skew <n>     skew fft analysis results by n dB (0db<=n<=48db) in the
              frequency range 20Hz to 3.45kHz
-cbs <n>      set codec block size to n samples (512<=n<=4608, n mod 32=0)
-fft <5xchr>  select fft lengths to use in analysis (1=on, 0=off)
              from 64, 128, 256, 512 & 1024 samples, e.g. 01001 = 128,1024
-overlap      enable conservative fft overlap method; default=off

-spf <5x5chr> manually input the 5 spreading functions as 5 x 5 characters;
              These correspond to FFTs of 64, 128, 256, 512 & 1024 samples;
              e.g. 44444-44444-44444-44444-44444 (Characters must be one of
              1 to 9 and A to Z (zero excluded).
-clipping     disable clipping prevention by iteration; default=off
-dither       dither output using triangular dither; default=off

-quiet        significantly reduce screen output
-nowarn       suppress lossyWAV warnings
-detail       enable detailled output mode

-below        set process priority to below normal.
-low          set process priority to low.

Special thanks:

Dr. Jean Debord for the use of TPMAT036 uFFT & uTypes units for FFT analysis.
Halb27 @ www.hydrogenaudio.org for donation and maintenance of the wavIO unit.
[/size]As a quick test, I ran my 52 sample set through at "-3 -fft 00100 -skew 36 -cbs 1024", which gives about a 3x speed increase (over -2):

WAV: 121.53MB; FLAC: 68.08MB, 790.6kbps; lossyWAV -2: 44.16MB, 512.8kbps, 46 secs.; lossyWAV -3 -fft 00100 -skew 36 -cbs 1024: 41.30MB, 479.6kbps, 16secs.

Surprisingly(?), this produces output which is satisfactory for my preferred DAP, in a third of the time and 94% of the diskspace.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #457
Good news, thank you.

As I'm about to go productive I welcome very much the possibility to have further FFTs, especially at the short edge as the 64 sample FFT has some shortcomings in the low/mid frequency range.
Going productive I try to play it safe while staying within most of the current -2 framework (but -cbs 512 and -nts -1.0).
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #458
Hallo Nick,

There seems to be a problem. I tried v0.4.2 this way:

lossyWAV.exe utb.wav -2 -cbs 512 -nts -0.5 -skew 24 -snr 18 -spf 11235-11236-11336-12348-1234D -fft 11111

and lossyWav starts to output a .lossy.wav file but immediately after that stops working. No crash, it just hangs, doesn't come back to the command line, and produces no output. Sorry.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #459
Ah.... It may be down to my inexperience with FP assembly - probably too few "FWAIT" instructions. I will amend and re-attach.

Nick.

[edit]lossyWAV alpha v0.4.3 attached: added a few more "FWAIT" instructions and reduced the permissible range of "-spf" input values back to hexadecimal characters (could cause problems at shorter FFT lengths).

Having fun with "-fft" and I was astounded to get casual listening compatible results on my 52 sample set  (as above) with "-3 -fft 00100 -skew fffff-fffff-44579-fffff-fffff -skew 36" : 33.25MB, 386.1kbps

Regarding the crashing - could anyone else with this problem please speak up and also, if you could indicate CPU type that would be very welcome. The only machines I have to test on are Intel C2D.....

On a more serious note, and with regard to IA32 / FPU assembly language: when *should* I insert an FWAIT instruction into the code?[/edit]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #460
Sorry same issue with new version and

lossyWAV.exe utb.wav -2 -cbs 512 -nts -0.5 -spf 11235-11236-11336-12348-1234D -fft 11111

lossyWav.exe echoes the options, starts producing the lossy.wav file, then hangs and produces no output. It does not crash. I can finish lossyWav by pressing Ctrl-C.

My cpu is a (32 bit) AMD mobile Athlon (= low power Barton), and I'm running Windows XP.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #461
Sorry same issue with new version and

lossyWAV.exe utb.wav -2 -cbs 512 -nts -0.5 -spf 11235-11236-11336-12348-1234D -fft 11111

lossyWav.exe echoes the options, starts producing the lossy.wav file, then hangs and produces no output. It does not crash. I can finish lossyWav by pressing Ctrl-C.

My cpu is a (32 bit) AMD mobile Athlon (= low power Barton), and I'm running Windows XP.
Thanks - now for a bit more debugging 
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #462
I should add I had no problem with v0.4.1 where you had already a lot of assembler code in it.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #463
It crashes immediately here when I run lossyWAV.exe file.wav. My processor is an AMD Athlon 64. I also had no problem with 0.4.1.

Edit: on second thought it doesn't terminate, I just get the crash report by windows, but it hangs like halb27 says.

lossyWAV Development

Reply #464
I may have found a possible culprit.....

lossyWAV alpha v0.4.3b attached.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

 

lossyWAV Development

Reply #465
Sorry, same effect.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #466
Sorry, same effect.
Is it doing this with no input parameters, or only when input parameters (other than name of file to process) are used?
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #467
Same effect with plain lossyWav.exe utb.wav.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #468
Seems to be a problem with AMD processors at the moment......

lossyWAV alpha v0.4.3c attached: Maybe?

lossyWAV alpha v0.4.3d attached: FWAIT instructions removed. Just to see if that is it.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV Development

Reply #469
Nothing changes with v0.4.3c and with v0.4.3d.
lame3995o -Q1.7 --lowpass 17

lossyWAV Development

Reply #470
It seems to work on my Athlon64X2
Code: [Select]
E:\dev-privat\lossy-wav>lossyWAV "Q:\CD\Anastacia\2000-Not That Kind\01 Not That Kind.wav" -o .\
lossyWAV alpha v0.4.3 : WAV file bit depth reduction method by 2Bdecided.
Delphi implementation by Nick.C from a Matlab script, www.hydrogenaudio.org
Processing : 01 Not That Kind.wav
Format     : 44.10kHz; 2 ch.; 16 bit; 8858220 samples; 200.87 sec.
Average    : 6.4714 bits; [55984/8651]; 11.72x; CBS=1024]
%lossyWAV Warning% : 47 bits not removed due to clipping.


lossyWAV Development

Reply #472
Does it happen often, that there are no bits removed at all?
Code: [Select]
E:\dev-privat\lossy-wav>lossyWAV "Q:\CD\Various\1990-Classic Hits der 20er Jahre
- CD 1\01 Am Sonntag will mein Süsser mit mir segeln gehn - Edith d'Amara.wav"
lossyWAV alpha v0.4.3 : WAV file bit depth reduction method by 2Bdecided.
Delphi implementation by Nick.C from a Matlab script, www.hydrogenaudio.org
Processing : 01 Am Sonntag will mein S³sser mit mir segeln gehn - Edith d'Amara.
wav
Format     : 44.10kHz; 2 ch.; 16 bit; 7926828 samples; 179.75 sec.
Average    : 0.0000 bits; [0/7742]; 11.95x; CBS=1024]

lossyWAV Development

Reply #473
It doesn't work on my Notebook, CPU is a Pentium-M.

lossyWAV Development

Reply #474
lossyWAV 0.4.2 has no issues with my laptop's AMD Mobile Sempron 3000+ (32-bit) CPU, except it crashes when the specified output folder doesn't exist.
lossyFLAC (lossyWAV -q 0; FLAC -b 512 -e)