lossyWAV 1.0.0b released:
lossyWAV 1.0.0b, Copyright © 2007,2008 Nick Currie. Copyleft.
This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.
This program is distributed in the hope that it will be useful,but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program. If not, see <http://www.gnu.org/licenses/>.
Usage : lossyWAV <input wav file> <options>
Example : lossyWAV musicfile.wav
Quality Options:
-q, --quality <n> quality preset (10=highest quality, 0=lowest bitrate;
-q 5 is generally accepted to be transparent)
default=-q 5.
Standard Options:
-c, --check check if WAV file has already been processed; default=off.
errorlevel=16 if already processed, 0 if not.
-C, --correction write correction file for processed WAV file; default=off.
-f, --force forcibly over-write output file if it exists; default=off.
-h, --help display help.
-L, --longhelp display extended help.
-M, --merge merge existing lossy.wav and lwcdf.wav files.
-N, --noclips set allowable number of clips / channel / codec block to 0;
default=3,3,3,3,2,1,0,0,0,0,0 (-q 0 to -q 10)
-o, --outdir <dir> destination directory for the output file(s).
-v, --version display the lossyWAV version number.
Special thanks:
David Robinson for the method itself and motivation to implement it.
Don Cross for the Complex-FFT algorithm used.
Horst Albrecht for valuable tuning input and feedback.
[/size]
Explanation, history and lossless codec settings in the wiki article (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV)
[edit] 1.0.0b necessitated by unknown WAV chunk handling error and 24-bit --correction / --merge parameter error.
Link to GNU GPLv3+ source code for lossyWAV 1.0.0b (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=4479) [/edit]
N.B.: lossyWAV 1.1.0b released here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=64666&view=findpost&p=577042).
lossyWAV 1.0.0 released:
Congratulations !
22 minutes late (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=56129&view=findpost&p=563788). Bitterly disappointed.
Kidding of course. Congrats on the big 1.0(.0). Have been watching the development of this with interest, even if I didn't understand any of the technical discussion.
lossyWAV 1.0.0 released:
Congratulations !
Many thanks!
22 minutes late (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=56129&view=findpost&p=563788). Bitterly disappointed.
Kidding of course. Congrats on the big 1.0(.0). Have been watching the development of this with interest, even if I didn't understand any of the technical discussion.
I got hung up with the GNU GPL text....
Just Thanks
Edit: plz validated news & frontpage mods
Just Thanks
Edit: plz validated news & frontpage mods
I'm glad it got this far too. Unfortunately, I cannot add a topic to the Validated News forum - it returns a lack of privilege error.
Unfortunately, I cannot add a topic to the Validated News forum - it returns a lack of privilege error.
What about "News Submissions"?
Great work!
Thomas
Nice work. Taking this experimental concept and following through with a usable, high-quality implementation is amazing.
Unfortunately, I cannot add a topic to the Validated News forum - it returns a lack of privilege error.
What about "News Submissions"?
Great work!
Thomas
Ah.... That would explain a lot.... .... and thanks!
All in all, this release came pretty fast, and it's Free Software. Great! Now I just hope someone will port it to unix
Aside from that, does lossyWAV compile as a 64-bit program out of the box, or would that need some adaptations too? Also, would a 64-bit build benefit from your ASM optimisations as well? Would upgrading from x87 to SSE improve performance?
All in all, this release came pretty fast, and it's Free Software. Great! Now I just hope someone will port it to unix
Aside from that, does lossyWAV compile as a 64-bit program out of the box, or would that need some adaptations too? Also, would a 64-bit build benefit from your ASM optimisations as well? Would upgrading from x87 to SSE improve performance?
lossyWAV is vanilla Delphi with 32-bit x86 and x87. No SSE instructions have (intentionally) been used - there was a problematic bug in an early build where I inadvertently used an SSE3 instruction (FISTTP) to truncate and store the result of the spreading function which caused compatibility problems for some users. I will be releasing the code shortly (certainly within a few days) and after I have a go at re-writing the chunk-mapping element of the wavIO unit to properly handle un-recognised chunks in the WAV file before the 'data' chunk. (the latest NIN album has a 'PAD ' chunk before the 'data' chunk in the FLAC and 24bit/96kHz versions, so lossyWAV exits with an error code....).
[edit] Using SSE may improve performance, using FISTTP almost certainly would shave a few cycles per codec_block. [/edit]
Well done Nick, and thank you.
Thank you too to all the people who helped with testing.
Where's the launch party?
Cheers,
David.
Well done Nick, and thank you.
Thank you too to all the people who helped with testing.
Where's the launch party?
Cheers,
David.
Thanks David, that means a lot. It has been a really enjoyable (in the sense that bashing one's head off a wall occasionally is "enjoyable" ) project to be involved in. Without the help of all the contributors / commenters / constructive critics it would not have been possible.
Thanks again,
Nick.
lossyWAV 1.0.0 released:
Usage : lossyWAV <input wav file> <options>
Example : lossyWAV musicfile.wav
Quality Options:
-q, --quality <n> quality preset (10=highest quality, 0=lowest bitrate;
-q 5 is generally accepted to be transparent)
default=-q 5.
Hello,
I guess this is a beginners question , but when I have a wave-file and use lossyWav.exe, I get an exact copy of the original Wave file. Is this correct?
I get no difference if I try "lossyWAV <input wav file> -q 0" or "lossyWAV <input wav file> -q 10" or just "lossyWAV <input wav file>". I always ends up with a new file with lossy.wav filetype, but the exact same file-size as the original.
Balnes
Very good work. Much appreciated.
Hello,
I guess this is a beginners question , but when I have a wave-file and use lossyWav.exe, I get an exact copy of the original Wave file. Is this correct?
I get no difference if I try "lossyWAV <input wav file> -q 0" or "lossyWAV <input wav file> -q 10" or just "lossyWAV <input wav file>". I always ends up with a new file with lossy.wav filetype, but the exact same file-size as the original.
Balnes
It should not be exactly the same size, there is a 'fact' chunk inserted immediately after the 'fmt ' chunk, probably about 50 to 100 bytes long. The WAV file then requires to be encoded with a compatible lossless codec, e.g. FLAC, TAK, WavPack, taking care to set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC). When you compare the filesizes of the FLAC and the lossy.FLAC you should notice a difference.
Hallo Nick,
I've just come home from the extended weekend and seen the good news.
Congratulations, and thank you very much!
Hallo Nick,
I've just come home from the extended weekend and seen the good news.
Congratulations, and thank you very much!
Glad to be of service!
[edit] lossyWAV 1.0.0b released, fixes unknown WAV chunk problem and 24-bit correction file / merge problem. [/edit]
The WAV file then requires to be encoded with a compatible lossless codec, e.g. FLAC, TAK, WavPack, taking care to set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC).
Please explain why. A flac will be generated anyway. What are the risks when I don't keep that metadata ?
Please explain why. A flac will be generated anyway. What are the risks when I don't keep that metadata ?
If you don't keep the metadata, you lose the 'fact' chunk, which in turn will not prevent lossyWAV processing the file again.
What are the risks when I don't keep that metadata ?
If you don't keep the metadata, you lose the 'fact' chunk, which in turn will not prevent lossyWAV processing the file again.
Thanks. So, sadly, that's another way to mislead people. Well, it's not your fault.
What are the risks when I don't keep that metadata ?
If you don't keep the metadata, you lose the 'fact' chunk, which in turn will not prevent lossyWAV processing the file again.
Thanks. So, sadly, that's another way to mislead people. Well, it's not your fault.
However, it would not be that difficult to check for the lowest set bit per codec-block and "estimate" whether the file had been previously processed. Unfortunately, if someone were to dither the lossyWAV processed file you would lose even that possibility.
Or if someone dithered to 16bit and saved as 24bit anyways (i've seen it a few times from my mastering clients) it would trigger a false positive. But that's splitting hairs. Hopefully FLAC (and others) will just default to including that information. I already was using it before lossyWAV anyways, because I include lots of meta-data in (some of) my WAV files since i've stopped using DDP.
Most importantly, I add my congrats.
The technical discussion was way over my comprehension abilities , but I did pick up that very clever tricks are used.
I'm hoping that there will be A-B testing by the silver-eared and examination of what happens after *.lossy.flac is turned into *,lossy.mp3, *ogg, *m4a or whatever.
Not only do I mull replacing a portion of my full-size 14,000 FLAC files, I'm thinking about a DJ I know who has scruples about using lossy files, but still wants to carry his dance itinerary on a laptop. I.e., can we assure him that a Q 5 *.lossy.flac is as good as "the real thing"?
A little late, but congratulations Nick.C, halb27 & 2Bdecided!
Nick, I hope you got to run your program (and I assume if you did, it didn't crash, even if you did):
Procedure Celebrate;
Begin
Repeat
Success:=Drink_Beer and not Spill_Beer;
Until Success=False;
Goto Bed;
End;
... also thanks to jesseg for FLACdrop.
LossyWAV's future is going to be very interesting; I see it as an excellent format for artists to release their music.
C.
I have not followed the development of lossyWAV closely. In my understanding it is a kind of pre-processor that takes out some information from the audio file that is unlikely to be heard anyway. After that you can use your favorite lossless encoder to encode the pre-processed WAV and reach better compression.
How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).
I have not followed the development of lossyWAV closely. In my understanding it is a kind of pre-processor that takes out some information from the audio file that is unlikely to be heard anyway. After that you can use your favorite lossless encoder to encode the pre-processed WAV and reach better compression.
How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).
lossyWAV rounds lower significant bits to zero. This adds white noise to the audio. The level of the added noise has been pre-calculated for each bit removed. The number of bits to remove is determined by processing the results of overlapping FFT analyses of at least 2 different lengths (64 samples and 1024 samples by default). The quality settings take the processed FFT results and make the processor remove more or less bits dependent on internal settings. --quality 5 is generally accepted to be transparent. It should be stated clearly now that lossyWAV is pure variable bitrate: different audio will result in a different lossless encoded bitrate for the same --quality setting (in a similar way that the lossless audio files encoded losslessly may have different bitrates).
I cannot comment on comparison with Wavpack and Optimfrog as to my knowledge no direct comparison has been carried out.
...
--quality 5 is generally accepted to be transparent. It should be stated clearly now that lossyWAV is pure variable bitrate: different audio will result in a different lossless encoded bitrate for the same --quality setting (in a similar way that the lossless audio files encoded losslessly may have different bitrates).
I cannot comment on comparison with Wavpack and Optimfrog as to my knowledge no direct comparison has been carried out.
You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?
I mean no offense to you or anyone, I am just curious.
You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?
I mean no offense to you or anyone, I am just curious.
I can claim an MP3 is transparent without comparing it to Wavpack and/or Optimfrog simply by ABXing it against the original WAV file. What's different here?
C.
You cannot comment on Wavpack and Optimfrog, yet you claim that --quality 5 of your pre-processor is 'generally accepted to be transparent'. May I ask if any comparison was made to support that claim?
I mean no offense to you or anyone, I am just curious.
I can claim an MP3 is transparent without comparing it to Wavpack and/or Optimfrog simply by ABXing it against the original WAV file. What's different here?
C.
Carpman is correct. ABX testing has been carried out by individuals with better ears than mine and -q 5 is as I "claimed" generally accepted to be transparent.
Take a look at the "lossyWAV Development" thread in the uploads forum for more details.
... How does this compare to Wavpack lossy and Optimfrog Dualstream qualitywise? Is it on pair with those or is it superior/inferior? It is my understanding that lossyWAV uses the same concept as Wavpack or Dualstream (correct me if I am wrong).
WavPack lossy controls the prediction error, lossyWAV controls the total error. In a sense lossyWAV has the better quality control in at least targeting the 'real' thing. Which does not necessarily mean that lossyWAV has the better quality - at least not at a bitrate which is low for these approaches (~ 300 kbps and below).
Apart from quality considerations where there will always be a lack in finally objective judgement (due to ever limited listening experience), there are practical advantages with lossyWAV as you have a choice of final codecs relevant for playback. FLAC can be used which is considered to be the lossless codec which is available best in the universe of players.
Can you please link me to the source code?
Its released under the GPL license, but I am unable to find it for download.
Can you please link me to the source code?
Its released under the GPL license, but I am unable to find it for download.
Download lossyWAV 1.0.0b Source (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=4479) Have fun!
I have not followed the development and just discovered this now. I read the older threads and the wiki article and did some testing myself, and I have to say, this software is really great . Even the high quality preset sometimes give a tremendous decrease in bitrate. Also I checked out the thread for the next version, and everything (and more) I could think of is already on the 'wishlist'. Keep up the good work
Download lossyWAV 1.0.0b Source (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=4479) Have fun!
Thanks a lot!
Downloaded, and will see if I can get this thing going on Mac OS X.
Downloaded, and will see if I can get this thing going on Mac OS X.
Go, krmathis, Go!
Downloaded, and will see if I can get this thing going on Mac OS X.
Go, krmathis, Go!
If you require any of the assembly routines converted back to Delphi, let me know - it should be a relatively simple task - although it will slow down processing fairly dramatically, especially if the FFT unit needs to be converted back.
Also, I could make sure that the remove_bits procedure is modified as per beta 1.0.1, i.e. independent channel bit removal rather than the same bits-to-remove for all channels.
This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?
This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?
We had this argument before. The answer is (for FLAC use as an example):
a) the lossyWAV preprocessed FLAC files get the extension .lossy.flac
b) the FLAC metadata carry the information about lossyWAV preprocessing
c) when looking at a lossless codec file you never know what the input was. It might originate from a 192 kbps DRM WMA file from a company offering DRM protected downloads but which you want to play on a non WMA DAP while not giving away quality. Three years after encoding theere's a chance you don't memorize where it came from.
This seem to be really interesting but one question comes to my mind ?
With this lossy treatment applied to future lossless files aren't we bluring everything ? I mean, we may not be able now to distinguish a real lossless file from an "almost lossless" file, which is finally a lossy one, don't you think?
If you can't distinguish a lossless file from an almost lossless file, is there any problem with having an almost lossless file?
As halb27 said, if you do not create the lossless file yourself, you will never really know whether the audio contained in the lossless container really is lossless.
Yes, lossyWAV blurs the boundary - however if the almost lossless version cannot be ABX'ed from the original then....
For the record, again, I will be using lossyWAV & FLAC to allow me to fit as much of my music as possible onto as big a Compact Flash as I can afford for use on my iPAQ running TCPMP beta v0.81. My FLAC archive will remain complete and unmolested - lossyWAV & FLAC is purely my DAP transcode method of choice.
I see.
Point c) from halb27 is indeed a good remark.
If you can't distinguish a lossless file from an almost lossless file, is there any problem with having an almost lossless file?
I do not use myself flac for my music because I only care about transparency. But I think about people who uses flac for perfect losslessness whatever the transparent result you can get with lossy treatments.
We had this argument before
Oops sorry I didn't know.
BTW, thank you for your answers.
It should be stated clearly now that lossyWAV is pure variable bitrate
perhaps a better way to word it would be that lossyWAV is variable bit depth. just a thought. again amazing work, and the change in v1.0.1 with the processing of individual channels... brilliant. i can't wait to see how it changes the bitrates (post flac of course, lest i be caught in my own trap), and of course it should be higher quality in some cases.
... set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC)...
Is (-b 512 AND --keep-foreign-metadata) mandatory for this to work with FLAC? You mention that it keeps LossWAV from processing the file a second time, but if I am only running the process once can I leave this out and still get the desired results?
Thank you for your time and effort.
--
E
... set the block size to 512 samples (-b 512 in FLAC) and also to preserve WAV chunks (--keep-foreign-metadata in FLAC)...
Is (-b 512 AND --keep-foreign-metadata) mandatory for this to work with FLAC? You mention that it keeps LossWAV from processing the file a second time, but if I am only running the process once can I leave this out and still get the desired results?
Thank you for your time and effort.
--
E
The --keep-foreign-metadata is optional in this case, the -b 512 is not (well, it is, but you would be best not to omit it).
The -b 512 parameter tells FLAC to use a 512 sample block length for encoding. This matches the 512 sample codec-block length in lossyWAV and will result in optimal FLAC encoding in most cases (excepting cases where bits-to-remove does not vary between codec-blocks).
is there a more in-depth "how to" for foobar planned (compared to what is in the wiki article, that is)?
it's probably my inability to grasp something fully, but i can replicate the settings shown in the image (with differences in path) and copy+paste+save the batch file and eventually go so far as to rip a test wav and watch the encoder interface run for less than a minute, only to hang for a few seconds afterward and then throw up an error indicating that the encoder halted because it could not find the .wav being used in the process.
on the contrary, getting it to run through XP's command line saw next to no error, aside from clipping issues with the file since i left that parameter out. i don't find myself crawling to the command line often, so i was hoping to rely on foobar to handle the brunt of the work once i could round up files to run the encoder on.
is there a more in-depth "how to" for foobar planned (compared to what is in the wiki article, that is)?
it's probably my inability to grasp something fully, but i can replicate the settings shown in the image (with differences in path) and copy+paste+save the batch file and eventually go so far as to rip a test wav and watch the encoder interface run for less than a minute, only to hang for a few seconds afterward and then throw up an error indicating that the encoder halted because it could not find the .wav being used in the process.
on the contrary, getting it to run through XP's command line saw next to no error, aside from clipping issues with the file since i left that parameter out. i don't find myself crawling to the command line often, so i was hoping to rely on foobar to handle the brunt of the work once i could round up files to run the encoder on.
Have you used a path with spaces in it? I ask as this was an early problem I had with foobar2000 using lossyWAV via a batch file. Also, make sure the extension you type in is "lossy.flac" not ".lossy.flac" as the latter will also crash.
Don't be too concerned about clipping (in this case) - even at the lowest quality setting lossyWAV only allows up to a maximum of 3 samples to be "clipped" per channel per codec-block (512 samples in length).
Nick
I think there's a problem with the recommended batch file code for the foobar setup in the LossyWAV wiki (http://wiki.hydrogenaudio.org/index.php?title=LossyWAV):
@echo off
z:\bin\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
z:\bin\flac.exe -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"
I, like Jalan was having all sorts of difficulty getting lossyWAV to encode via foobar.
Then I noticed that it's calling flac.exe whereas it's calling lossyWAV (.nothing).
I changed it to flac (.nothing) and it worked fine. So maybe this is the problem. Here's the batch file that worked for me.
@echo off
z:\bin\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
z:\bin\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"
Perhaps if Jalan could try this and see if it works too, then if it does it perhaps it's worth amending the Wiki?
I'm not sure, but is the "no spaces" tip mentioned in the wiki (didn't see it). Something like this might be helpful:
@echo off
d:\address_with_no_spaces\lossywav\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
d:\address_with_no_spaces\lossywav\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac"
del "%~N1.lossy.wav"
C.
Have you used a path with spaces in it? I ask as this was an early problem I had with foobar2000 using lossyWAV via a batch file.
indeed, there were spaces in the path that foobar was attempting to call the batch from.
Perhaps if Jalan could try this and see if it works too, then if it does it perhaps it's worth amending the Wiki?
tried it with the amended path via Nick C's suggestion (no spaces, direct to the main drive) and your removal of the .exe extension to the flac encoder in the batch text and it produced a full file without issue. thanks for making note of it.
tried it with the amended path via Nick C's suggestion (no spaces, direct to the main drive) and your removal of the .exe extension to the flac encoder in the batch text and it produced a full file without issue. thanks for making note of it.
Glad it worked for you.
I'm pretty sure it's the .exe issue, as I tried it with no spaces first and it still didn't work, then got rid of the .exe and it did. And if that's not scientific evidence then my full name isn't Carpman Sherlock Einstein
C.
I'd like to integrate lossyWAV into foobar2000. But I always receive an error at the end (Obejct not found). Could you please add an output option that we can set output file names in the codec? I can't let foobar ignore that %d is missing in my encoding paramters
I'd like to integrate lossyWAV into foobar2000. But I always receive an error at the end (Obejct not found). Could you please add an output option that we can set output file names in the codec? I can't let foobar ignore that %d is missing in my encoding paramters
Doesn't it work the way shown in the wiki?
Hello there everyone, I want to convert some lossyWAV files to lossyFLAC (to be able to edit metadata, etc.)
The Wiki article says i should use the following command line options with FLAC encoder:
-5 -b 512 --keep-foreign-metadata
My question is: why is [-5] compression quality deliberately used in this example? Should I obey this compression quality? I'm always encoding my FLAC files with [-8] compression setting, is it uncompatible with lossyWAV ?
All clarification is very welcome. Thanks.
You can use whichever compression setting/level you want.
The -b 512 is important to match the blocksize, so minimising bitrate.
The --keep-foreign-metadata keeps an extra field in the wavefile which marks it as lossyWAV processed, which can be useful but isn't needed.
Cheers,
David.
The -5 quality level is recommended, but not mandatory.
Due to the usage of the -b 512 switch (which in turn, is needed for what 2bdecided said), the quality levels higher than 5 are not able to reduce the size as much as if they used the default setting, but they still use more CPU.
All in all, it's up to you to try and decide if it is worth using -8 or not.