The third (and probably last) beta release of WavPack 4.0 includes the following:
- Adobe Audition (Cool Edit) filter with 32-bit float & extra info support
- more robust error decoding (with error muting)
- new library interface to encoding functions
- improved hybrid mode noise shaping
- md5 signatures
download win32 executables and plugins (http://wavpack.com/beta3.zip)
download sources (http://wavpack.com/beta3src.zip)
keep up the good work
With the addition of the Audition file filter, this package now incorporates all the important functionality of the 3.97 version. Therefore, assuming nothing untoward is discovered, I will be releasing 4.0 in a week or two.
I have incorporated all of robUx4's changes for Linux and OS-X compiles in the sources, so except for adding md5.c to the source list and fixing anything else I messed up, that should go painlessly.
Minor improvements / changes:
- extra mode (-x) now has default level based on mode
- new option to store floats (and larger ints) with resolution limited to 25 bits (saves space with virtually no audio degradation)
- new option to minimize console messages (-q)
- new option to ignore .wvc file during decode (-i)
- new option to decode to raw audio only (-r)
- full error reporting in foobar plugin
keep up the good work
Thanks!
Sounds nice indeed, now to do the same with WavPack as done with MAC - insert CUE-file and EAC LOG-file into an APEv2 tag and voila, singlefile joy ;o)
I'll be checking this out at once.
New foobar2000 wavpack input by Case available here (http://www.saunalahti.fi/cse/foobar2000/foo_wavpack.zip).
Supports handling of internal cuesheets the same way monkeys input does allready.
Simply add a "cuesheet" tag to the file and paste your correct cuesheet into it.
Many thanks to Case for updating this.
@bryant:
Nice work. Just finally played around with this a bit and it seems the normal compression setting is a very nice tradeoff between filesize and decoding speed. Might even concider switching over entirely to wavpack now that case has added internal cuesheet support.
Here is a small comparison I made between the different wavpack modes and Monkeys Audio normal. Only did this on one album though, so I can't promise it is entirely valid for all types of music. Also compared to monkeys audio 3.97 and not 3.99. 3.99 should be at least about 10% slower then 3.97 on decoding.
Done on Athlon XP 3000 with 1024 mb of ram.
Results:
Wavpack 4.0 beta3 decoding speed (Asian Dub Foundation - Enemy of the Enemy) :
normal: ~70 times 351 mb
high: ~36 times 341 mb
fast: ~93 times 358 mb
Monkeys Audio 3.97:
normal: ~46 times 340 mb
picmixer & Case:
Thanks for the cuesheet support; I'll give that a try myself. Is this something I could easily integrate into my foobar source?
BTW, picmixer, if decoding speed vs. filesize is your primary concern, you might want to give the -X switch a try. It slows encoding way down, but decoding is not affected and it can sometimes get you almost to the next higher mode with respect to compression ratio. (of course, maybe that's what you used...)
BTW, picmixer, if decoding speed vs. filesize is your primary concern, you might want to give the -X switch a try. It slows encoding way down, but decoding is not affected and it can sometimes get you almost to the next higher mode with respect to compression ratio. (of course, maybe that's what you used...)
Hmh, no havent tried that yet. Thanks for the hint.
High decoding speed with the least tradeoff in filesize is indeed what I am after. Curious to see the results.
Good stuff David, as always.
I'll have a play with your "new and improved" hybrid mode noise shaping and let you know my thoughts.
Thanks for this.
Den
New foobar2000 wavpack input by Case available here (http://www.saunalahti.fi/cse/foobar2000/foo_wavpack.zip).
Supports handling of internal cuesheets the same way monkeys input does allready.
Simply add a "cuesheet" tag to the file and paste your correct cuesheet into it.
Is there any way to automate this from a batch file?
I have incorporated all of robUx4's changes for Linux and OS-X compiles in the sources, so except for adding md5.c to the source list and fixing anything else I messed up, that should go painlessly.
Do you have a linux makefile for b3? I'm not having much luck getting it to compile.
If the makefile is not included, you can try the one I did from the sources there (http://mukoli.free.fr/wavpack/).
Thanks for the cuesheet support; I'll give that a try myself. :) Is this something I could easily integrate into my foobar source?
Hi Dave,
Case made the source also available: foo_wavpack_src.zip (http://www.saunalahti.fi/~cse/foobar2000/foo_wavpack_src.zip)
@ robUx4,
I've tried the beta2 makefile, but it didn't work. I did add the md5.c file to Makefile.wavpack and Makefile.wvunpack. Could you have a look?
Sure, as soon as I have a Linux system back working
Good luck with it!
The user documentation says that for every 14kbps over 320kbps the quantisation noise is lowered by about 1 dB. How does the -h and the new -x[n] option affect the hybrid quality? Also; does encoding to a 320kbps wv hybrid file result in a higher quality file than i.e. mp3 or ogg vorbis at the same bitrate?
I guess my question put simple is this: Is it worth waiting 30 extra minutes using the -x switch when encoding an album in hybrid mode?
I guess my question put simple is this: Is it worth waiting 30 extra minutes using the -x switch when encoding an album in hybrid mode?
The -x switch does improve the quality of the hybrid mode slightly (you can see this using the -n switch), but I don't think that alone would be worth the extra time. However, there are times when the -x switch makes a big difference (at least in the normal mode), like on this WavPack torture sample:
http://wavpack.com/Furious.wv (http://wavpack.com/Furious.wv)
So, I would say that if have the time then it's worth using -x as an insurance against problem samples. If you don't have the time then it's not worth it... ;-)
As for comparing WavPack's lossy mode with other lossy codecs at high bitrates, this is very difficult because they are almost all transparent (to most people and with most samples). However, there is some evidence that WavPack (and the other hybrid encoder OptimFROG) perform better with transcoding.
Also, WavPack lossy is not subject to the accumulation of artifacts that you get with repeated encode / decode cycles. For example, you can do dozens of encode / decode cycles with WavPack lossy at 512 kbps and the result will still sound fine (maybe equivalent to a single cycle at 320 kbps). Do this with other codecs and the non-linearities in frequency response and time-domain smearing will make the sample unlistenable. I'm not sure this has any relevance to normal usage, but it does demonstrate a difference in the techniques.
Hope this helps...
Yes that helped a bunch. Thanks
The code that compiles on OS X and Linux (http://mukoli.free.fr/wavpack/wavpack.b3a.tar.bz2). The MD5 is OK on Linux but not on OS X. I dunno if it's just due to a endianess problem or something else...
@ David :
Where would our guys have to search if we want a simple lib to parse your framing ? Mosu thought about adding wavpack support to mkvtoolnix lately, but stopped it right after he learned from robux4 that there is both an encoder and decoder library now, but there is no explicit part of the code allowing to parse a Wavpack stream and mux the packets in matroska.
Can you help us on this ? Is there a parsing code already, and we just dont find it ?
Thanks robUx4. Two observations:
1. Wildcards don't work in Linux ('wavpack *.wav' doesn't work)
2. 'wvunpack -vm file.wv' gives this in Windows:
WVUNPACK Hybrid Lossless Wavefile Decompressor Win32 Version 4.0b3 28-06-04
Copyright (c) 1998 - 2004 Conifer Software. All Rights Reserved.
original md5 signature: 783bc77ac749566e2870d2a002d96236
unpacked md5 signature: 783bc77ac749566e2870d2a002d96236
verified 05 - A Forest.wv in 19.71 secs (lossless, 50.75%)
and this in Linux:
WVUNPACK Hybrid Lossless Wavefile Decompressor Linux Version 4.0b3 28-06-04
Copyright (c) 1998 - 2004 Conifer Software. All Rights Reserved.
unpacked md5 signature: b23456ef10fe4bacc64771ee8779fcdb
verified track_01.wv in 35.71 secs (lossless, 48.18%)
The original md5 signature is missing in Linux.
Speek:
First of all, thanks to you and Case for making that foo_wavpack source available. I really appreciate it.
As for the Linux MD5 thing, I see that the files are not the same one. Have you tried both versions decoding the exact same file? I can't tell if the problem is that the Linux encoder is not putting the MD5 in the .wv file in the first place, or whether the decoder just can't see it. I assume you realize that you must use the -m when encoding to enable writing the MD5 into the file, that option is not just for displaying it.
BTW, thanks for your testing!
ChristianHJW:
Not only can I help, but I want to help. If there's any functionality that you need that's not in there, I'm sure I can put it together very quickly.
As robUx4 says, there are library routines for both decoding existing WavPack files and for creating new WavPack files. The functions for creating WavPack files have no I/O calls in them; you simply pass raw audio data to the functions and they call you back with complete (no parsing required!) WavPack blocks (which I assume you guys just put into your container as is). You can put exactly as many samples in each block as you like because I provide a "flush" routine.
On the decode side, there is parsing code that is mixed with the file reading code and the code for seeking. However, I assumed that you would not use this because you would already know where the blocks started and ended because that's how I passed them to you in the first place. A couple calls to the decoder will unpack a single block into its raw samples again.
The only complication here is perhaps how multichannel audio is handled. On the encode side my functions expect the data to be interleaved through all channels (like a .wav file) and it automatically separates it into mono and stereo streams and packs them into their own blocks (all transparent to you). However, on decode your code would have to do the re-interleaving. I could provide some helping functionality here if needed.
But I think this is all very easy to accomplish; I just need to be made to understand exactly what you guys want. The WavPack code is done and ready to go.
Speek:
First of all, thanks to you and Case for making that foo_wavpack source available. I really appreciate it.
As for the Linux MD5 thing, I see that the files are not the same one. Have you tried both versions decoding the exact same file? I can't tell if the problem is that the Linux encoder is not putting the MD5 in the .wv file in the first place, or whether the decoder just can't see it. I assume you realize that you must use the -m when encoding to enable writing the MD5 into the file, that option is not just for displaying it.
The foo_wavpack modification is entirely Case's work. I just supplied the link.
The MD5sum works fine in Linux too. I just forgot to add the -m switch when encoding.
One small thingy: in line 78 of wavpack.c you removed the '\n' at the end of the help text. This results in the prompt being on the same line as the last line of the help text in Linux.
1. Wildcards don't work in Linux ('wavpack *.wav' doesn't work)
They do. But in Linux
wavpack *.wav
is different than
wavpack "*.wav"
Hmm, yes wavpack "*.wav" does work. But what's the difference? (or where can I read about that?)
I made a simple bash script to play wavpack files in Linux. It needs aplay which is part of alsa-utils and should be available on most modern Linux distributions.
Usage:
1. Copy this text in a text file named something like 'wavpack123' (you can make up your own filename) and save it.
2. Put it somewhere in the PATH (i.e. /usr/local/bin).
3. Make it executable (chmod +x wavpack123)
4. cd into directory with wavpack files and do 'wavpack123 *' or 'wavpack123 filename' or 'wavpack123 file1 file2 file3' or 'wavpack123 file[1-3]'.
5. Improve the script, because I'm just a beginner and I'm sure there are may Linux users here that can do better
#!/bin/bash
if [ "$1" = "" ] || [ "$1" = "--help" ] || [ "$1" = "-h" ]
then
echo "Usage: `basename $0` <files>. Wildcards can be used."
else
until [ -z "$1" ]
do
if [ "$TERM" != "linux" ]
then
# We're on a X terminal (xterm, rxvt, etc.). Put filename in title bar.
title=`basename "$1" .wv`
echo -n -e "\e]2;$title\a"
fi
wvunpack "$1" - | aplay -q
shift
done
if [ "$TERM" != "linux" ]
then
echo -n -e "\e]2;Finished playing\a"
fi
fi
exit 0
Hmm, yes wavpack "*.wav" does work. But what's the difference? (or where can I read about that?)
The difference is that in the first case the shell resolve the wildcard and then passes all the matching files to the program. In the second case it passes the wildcard to the program. The wavpack command-line apps only support the later.
I can't test your script from here. But at least you should replace /bin/bash by /bin/sh. On most systems it's the same, but it's cleaner this way.
One small thingy: in line 78 of wavpack.c you removed the '\n' at the end of the help text. This results in the prompt being on the same line as the last line of the help text in Linux.
Yeah, I removed the newline so that the whole display would fit on a DOS screen, even though it gets a little ugly. I think we can just send another newline in Linux.
I don't know what I'm going to do when I add more switches...
I don't know what I'm going to do when I add more switches...
Use pause inbetween screens, just like ARJ did
Hmm, yes wavpack "*.wav" does work. But what's the difference? (or where can I read about that?)
The difference is that in the first case the shell resolve the wildcard and then passes all the matching files to the program. In the second case it passes the wildcard to the program. The wavpack command-line apps only support the later.
Well behaved unix applications should never try to expand their own wildcard arguments. They should just process whatever they're passed as arguments by their parent shell. Luckily doing the right thing here is actually a lot easier than trying to write your own shell expansion code.
I agree, but in the other hand the apps need a bit of intelligence to know that for a given file the .wvc file should be used (for example).
The latest source code (http://mukoli.free.fr/wavpack/wavpack.b3b.tar.bz2) with a fix for MD5 on OS X and some minor fixes.
The latest source code (http://mukoli.free.fr/wavpack/wavpack.b3b.tar.bz2) with a fix for MD5 on OS X and some minor fixes.
This compiles cleanly on my debian linux box. Thanks!
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
It would also be nice if wavpack showed what x-level is used when using the -x option without giving an explicit number, and a higher resolution "progress meter"; maybe add one digit and a comma? (looks nicer when doing slow encodings imho)
Oh and another thing; when i rip an album to a single file + cuesheet, encode it to, say, 400kbps wavpack lossy, add the cuesheet as a tag and replaygain the tracks when opened in foobar2000.... where and how is the replaygain info stored?
Oh and another thing; when i rip an album to a single file + cuesheet, encode it to, say, 400kbps wavpack lossy, add the cuesheet as a tag and replaygain the tracks when opened in foobar2000.... where and how is the replaygain info stored?
ReplayGain values are stored inside the cuesheet tag as REM comments.
For those who might be interested I have created a new wvselfx.exe stub for WavPack 4.0 so that someone can now create (by hand) self-extracting WavPack files. Just use the DOS copy command to prepend this file to any WavPack 4.0 .wv file and it will become a self-extracting archive for any win32 machine:
copy/b wvselfx.exe+music.wav music.exe
I'll put the -e option back into wavpack.exe before the release. An nice improvement over the previous versions of this is that you can rename the resulting .exe back to .wv and it will be a perfectly fine WavPack file again (there can be up to 1 meg of garbage before the first WavPack block in a file).
http://wavpack.com/wvselfx.zip (http://wavpack.com/wvselfx.zip)
BTW, I won't be releasing the source for this stub because it's a big mess.
Seems I can't decode files using absolute path, starting with slash, eg
wvunpack /mnt/c/wv/pj\ harvey/uh\ huh\ her/uh\ huh\ her.wv - | aplay
will result in
illegal option: n !
illegal option: t !
illegal option: / !
illegal option: c !
illegal option: / !
illegal option: w !
illegal option: v !
<snip>
illegal option: w !
can't delete in verify mode!
aplay: playback:1878: read error
Symlink (wvunpack uh\ huh\ her.wv) or relative path (wvunpack ../../../../mnt/ ...) will work otoh.
Seems I can't decode files using absolute path, starting with slash, eg
wvunpack /mnt/c/wv/pj\ harvey/uh\ huh\ her/uh\ huh\ her.wv - | aplay
will result in
illegal option: n !
illegal option: t !
illegal option: / !
illegal option: c !
illegal option: / !
illegal option: w !
illegal option: v !
<snip>
illegal option: w !
can't delete in verify mode!
aplay: playback:1878: read error
Symlink (wvunpack uh\ huh\ her.wv) or relative path (wvunpack ../../../../mnt/ ...) will work otoh.
Yes, this will be fixed right away. The problem is that I had "/" in there as a very old option introducer, but this will go away for Linux. Thanks...
Hmm. I should have just fixed it, then posted.
Here are the source fixes for that and the other problem Speek found:
http://wavpack.com/wp_options.zip (http://wavpack.com/wp_options.zip)
David, where is our library !
Toff and Mosu have TTA support and filters more or less finished, they would look at wavpack now ....
A few files have been updated. First, I found a bug in the CoolEdit / Audition filter that would cause an error when storing 20-bit or 24-bit files. The resulting files will still load fine in Audition and can be played fine, but if they are converted to wavs the RIFF header will indicate the wrong amount of data and there could be trouble. I have uploaded a new version in beta3.zip (see link at top of thread). To fix eny existing file (after updating the filter) simply load and re-save them in Audition. Sorry I didn't catch this in testing.
Also, I found a problem introduced into the Linux source (does not affect the original Win32 source or beta executables). This could cause the command-line programs to crash on termination, but shouldn't cause any corrupted files to be written. Get the updated files here:
http://wavpack.com/wp_options.zip (http://wavpack.com/wp_options.zip)
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)
It would also be nice if wavpack showed what x-level is used when using the -x option without giving an explicit number, and a higher resolution "progress meter"; maybe add one digit and a comma? (looks nicer when doing slow encodings imho)
The problem is that the X mode can take several seconds to work on a single block and I can't update the progress during that without a lot of trouble. So, even if I put in tenths of a percent it would still update slowly.
BTW, the X mode default is always X6 for fast mode, X4 for the normal mode and X3 for the high mode.
edit: added detail to clarify problem
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)
Automatically splitting from cuesheets would be excellent! One vote from me!
[That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)
Automatically splitting from cuesheets would be excellent! One vote from me!
[a href="index.php?act=findpost&pid=224482"][{POST_SNAPBACK}][/a]
... if we had this in matroska, the feature could be used for any supported audio format .... soon also for wavpack ....
The problem is that the X mode can take several seconds to work on a single block and I can't update the progress during that without a lot of trouble. So, even if I put in tenths of a percent it would still update slowly. sad.gif
BTW, the X mode default is always X6 for fast mode, X4 for the normal mode and X3 for the high mode.
I see. Well, thanks for the clarification anyways. Thanks for a great product. Any idea when v4.0 will go final, and what's left to do before it does?
I have created a tiny version of the WavPack 4.0 decoder for possible use in "resource limited" CPU devices (Palm, Pocket PC, ...?) or as the basis for hardware implementation. It uses less than 32k of code space, less than 4k of data space and does not require floating-point math. I included a demo that accepts WavPack files on stdin and outputs wav files to stdout.
http://wavpack.com/tiny_decoder.zip (http://wavpack.com/tiny_decoder.zip)
Thanks for a great product. Any idea when v4.0 will go final, and what's left to do before it does?
[a href="index.php?act=findpost&pid=224515"][{POST_SNAPBACK}][/a]
Thanks. This was the first thing I wanted to complete. All that is left is to add reader callbacks to the regular library (I have decided to do that sooner rather than later) and update all the user documentation. There are no functional changes planned for the release, which
should (assuming no surprises) be in the 1-2 week time frame.
I have created a tiny version of the WavPack 4.0 decoder for possible use in "resource limited" CPU devices (Palm, Pocket PC, ...?) or as the basis for hardware implementation. It uses less than 32k of code space, less than 4k of data space and does not require floating-point math. I included a demo that accepts WavPack files on stdin and outputs wav files to stdout.
http://wavpack.com/tiny_decoder.zip (http://wavpack.com/tiny_decoder.zip)
Wow !!! Very nice !
Does it support hybrid mode ? (with and without the complement file)
If so you should send it to all portable player makers (starting from iRiver by which I plan to buy the H-340) ! Then I could use WavPack everywhere
[BUG] WavPack 4.0b3 Cannot Handle Double-byte Characters Properly
Hi,
I might be wrong, but I believe this is a bug in WavPack.exe:
[ Phenomenon ]
- It cannot handle 0x5c at the 2nd byte of a double-byte character properly.
WavPack will mistake it as the path separator (Backslash),
and will try to look for the file to encode in a wrong path,
confused by that pseudo path separator, and will complain "Can't open file FOO!"
[ Example ]
For instance, in my locale, Japanese, there is a character which would look like -\ in the ASCII code page, but it is actually one letter, SHIFT_JIS [0x96][0x5c], or U+66B4.
Now, if you try
wavpack foo[0x96][0x5c]bar.wav
then, wavpack will say,
Can't open file foo[0x96][0x5c]foo[0x96][0x5c]bar.wav!
Wavpack is right, it can't open file foo[0x96][0x5c]foo[0x96][0x5c]bar.wav!
because such a file does not exist, but I asked Wavpack to compress foo[0x96][0x5c]bar.wav which DOES exist.
Note that [0x96][0x5c] here stands for one double-byte char like i said before.
[ Demo ]
http://www.faireal.net/tmp/wv-problem.zip (http://www.faireal.net/tmp/wv-problem.zip)
[ Note ]
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...
Except this small problem, Wavpack seems really great.
Keep up a good job
If I may ask, what happened to the plans for fixing quality hybrid mode for 4.0?
Wow !!! Very nice !
Does it support hybrid mode ? (with and without the complement file)
If so you should send it to all portable player makers (starting from iRiver by which I plan to buy the H-340) ! Then I could use WavPack everywhere
It supports hybrid mode, but not the correction files. The correction files are probably the most complicated part of WavPack decoding, and also the most difficult part to integrate into other applications. I doubt that many portable players would easily handle streaming two files at the same time as they would have to have lots of RAM to buffer the data or be seeking back and forth like crazy. This wouldn't apply to flash players, but I can't see wanting to put lossless audio on those because of the limited space. Of course I can always add stuff back later, but I wanted to start with something as simple as possible to not scare anyone away.
I
will be sending this to the player makers (and iRiver is definitely on the list), but it would help alot if users (and potential users) did the same as well (and posted on their boards). They're not going to just take my word for it...
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...
Hmm. This is a tough one because your test files work fine here, and I was not able to find any directly applicable information on the Internet. Was the problem fixed in TTA? Maybe I could look at what they did. It probably won't help, but you might try putting the whole filename in quotes.
Can anyone else shed any light on this?
If I may ask, what happened to the plans for fixing quality hybrid mode for 4.0?
Ah yes, I remember something about a quality mode...
Basically I decided that I had to get something out or WavPack 4.0 was going to start sounding like vaporware and that was one feature that could easily become a quagmire. I could tackle float support and multichannel without outside help, but getting the quality mode right takes lots of outside help (my old ears and eaqual only catch so much).
So, I left in all the hooks and as much flexibility as I could think of and figured I would return to the quality mode when there was less pressure to get something out that people could use, and hopefully I will be able to get something nice without breaking decoders. In fact, if all goes well then breaking decoders won't even be an option.
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...
Hmm. This is a tough one because your test files work fine here, and I was not able to find any directly applicable information on the Internet. Was the problem fixed in TTA? Maybe I could look at what they did. It probably won't help, but you might try putting the whole filename in quotes.
The test file works because your Windows doesn't use multibyte character set for legacy ANSI programs and Zip packed the filename as ANSI and it uncompresses incorrectly for anyone but Japanese.
You can read all about multibyte programming on MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_support_for_multibyte_character_sets_.28.mbcs.29.asp).
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...
Hmm. This is a tough one because your test files work fine here, and I was not able to find any directly applicable information on the Internet. Was the problem fixed in TTA? Maybe I could look at what they did. It probably won't help, but you might try putting the whole filename in quotes.
Well, I think you can reproduce the problem by setting your locale (system langauge) = Japanese. In Windows XP, control panel -> Regional and Language Options -> Advanced -> Language for non-unicode program.
You dont need to have Japanese version OS to reproduce this bug. My OS is not Japanese version either. And possibly the same problem may happen in another language using multibute char. such as Chinese.
problematic 2nd half of double-byte char is one of the most common problems in i18n. There should be more than one solutions, one of which is to use Unicode internally, like many internationalized apps do.
A double-byte char uses 2 bytes to express 1 glyph. The problem is, the 2nd half of those 2 bytes might be confusing, like, it can be 0x5c and innocent parser thinks 0x5c as the path separator, or the letter for an escape sequence, and will be messed up. But it actually is not such a special (meta)character, but just the 2nd half of a common double-byte character.
The author of TTA, Alexander aka ald, said "I think that I know the reason of this problem. Going to fix it now." a few days ago.
-o switch in ttaenc can already handle double-byte char decently, and WinTTA (tau producer) is more perfect. As for WavPack, this problem is not conspicuous, because it supports pipeline, so if the user uses foobar2000 as the front-end, then the filename handling will be done by foobar2000. TTAENC cannot do that, not supporting pipeline, so it should parse %s from foobar2000 by itself anyway. So you might want to check his source when ready, or someone else's who knows this problem well...
The reason of this problem is very simple, but possibly fixing it is not that easy.
I hope you will find an elegant way to solve it.....
tyia
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...
Unfortunately, changing my language settings did not change how that test folder unpacked, but I think I figured out what was going on anyway. Please let me know if this helps:
http://wavpack.com/wp4-jis.zip (http://wavpack.com/wp4-jis.zip)
Thanks again, Case.
Unfortunately, changing my language settings did not change how that test folder unpacked, but I think I figured out what was going on anyway. Please let me know if this helps:
http://wavpack.com/wp4-jis.zip (http://wavpack.com/wp4-jis.zip)
Did you "install files for East Asian languages" (Language tab)? Maybe you have to change the "Text services and input languages" too. I can reproduce this problem on my Windows XP English version...
Anyway, thanks a lot for your work.
I tested "Win32 Version 4.0b3x 11-7-04" and the bug related 0x5C has been fixed
Yes, fixed! Thanx again!
But this fix has this terrible side effect
- now wavpack can not handle all the
other double-byte chars (that is, all the double-byte chars that do
not include 0x5c in the trailing byte)
Since its not that easy to describe what is happening by words, I upped this pic.
Could you please chack this one?
http://www.faireal.net/tmp/wv4b3x.png (http://www.faireal.net/tmp/wv4b3x.png)
wavpack [965C].wav
DOES work, while
wavpack [82A0].wav
fails.
The error message is: "file [82A0].wav.wav not found!"
The original WavPack 4.0b3 can do:
wavpack [82A0].wav
successfully, so your last tweak includes both debugging and enbugging...
I hope you wont start hating double-byte chars for that.
It may be a dumb question, but does wavpack 4.0 support adding tags while encoding? For example when using Exact Audio Copy for extractio directly to wavpack. Would be nice if the files could be tagged directly, kind of like flac, rather than with some other app. If this is possible I'd be pleased to hear the command line string to do so. That would be the only thing slowing my down from switching. So far it looks like damn good work.
It may be a dumb question, but does wavpack 4.0 support adding tags while encoding? For example when using Exact Audio Copy for extractio directly to wavpack. Would be nice if the files could be tagged directly, kind of like flac, rather than with some other app. If this is possible I'd be pleased to hear the command line string to do so. That would be the only thing slowing my down from switching. So far it looks like damn good work.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=225279")
Well, actually it is through another little application, but in the end it is transparent when using EAC, so learn on this [a href="http://www.saunalahti.fi/cse/EAC/]page[/url] how to do it.
But this fix has this terrible side effect
- now wavpack can not handle all the other double-byte chars (that is, all the double-byte chars that do not include 0x5c in the trailing byte)
Yeah, I was afraid that might happen. The problem is that shift-jis cannot be reliably parsed backward because the second byte of the two byte codes can also be above 0x80 (thanks, Bill). What I did should work in all cases except when the double byte code is the last character in the filename (I think).
I know how to fix this correctly, but it's a little more work. Thanks for helping me out on this.
Well, it wasn't as hard as I thought. I think I have got every case this time:
http://wavpack.com/wp4-jis.zip (http://wavpack.com/wp4-jis.zip)
Thanks very much for your assistance and patience on this.
The problem is that shift-jis cannot be reliably parsed backward because the second byte of the two byte codes can also be above 0x80 (thanks, Bill).
Exactly.... You cannot parse it backward.
Quick hack would be like this:
char szArg[] = "C:\\My Music\\[Some DB Charcters]\\foo.wav";
char szBuf[ MAX_PATH + 1 ];
char szPathSeparated[ MAX_PATH + 1 ][ MAX_PATH + 1 ];
int iPathSeparated = 0;
memset( szBuf, '\0', sizeof( szBuf ) );
if( GetSystemDefaultLangID() == 0x411 ) // Japanese
{
int len = lstrlen( szArg );
for( int i = 0, j = 0; i < len; )
{
if( (UINT) szArg[ i ] > 0x80 )
{
szBuf[ j++ ] = szArg[ i++ ];
szBuf[ j++ ] = szArg[ i++ ];
}
else
{
if( szArg[ i ] == '\\' )
{
lstrcpy( szPathSeparated[ iPathSeparated++ ], szBuf );
i++;
j = 0;
memset( szBuf, '\0', sizeof( szBuf ) );
}
else
{
szBuf[ j++ ] = szArg[ i++ ];
}
}
}
}
But this is ugly. You might want to process everything in Unicode internally....
ADDED: Sorry, not yet fixed.I forgot to test the "half-width" case. In Shift_JIS Code Page, 0x80-FF can be used
as a single-byte char too, like 0x80-FF used in French or German for special characters. Wavpack cannot handle this case yet.
Heres a demo result (pic):
http://www.faireal.net/tmp/0xb1.png (http://www.faireal.net/tmp/0xb1.png)
/ADDED
Well, it wasn't as hard as I thought. I think I have got every case this time:
http://wavpack.com/wp4-jis.zip (http://wavpack.com/wp4-jis.zip)
Thanks very much for your assistance and patience on this.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=225398")
Thank you very much
You posted that one while I'm posting another one,
and YES, looks like this newest build finally fixes all the problems related to Shift_JIS.
Im really happy for that, and at the same time, let me add this for the future development:
That build is not yet perfect, as many other same tools are not, when it should handle Unicode filenames.
For instance, it cannot handle the path which contains both Japanese and Latin special characters such as ä (a umlaut).
Youll get an error like this (yeah, this is Bill's fault, basically):
[a href="http://www.faireal.net/tmp/mixed_filename.png]http://www.faireal.net/tmp/mixed_filename.png[/url]
So, users would be in trouble if they had a folder in Japanese name and if they had some music files in German or French titles in it. Moreover, on win2k/xp, you can mix, say, Korean, Japanese, Arabic, and French in one filename. I guess that's why modern windows programs need to be Unicode-based.
It is very difficult to handle this gap for commandline tools because cmd.exe does not support Unicode properly while Windows per se uses Unicode. But ofcos normal Win32 GUI applications can manage unicode file names, like foobar2000 can open files in Japanese and files in Thai and files in Korean in the one same playlist.
Thanx again anyway
Okay, this is obviously not the way to fix this problem because even if I get this language working right, there are hundreds of others. One of these days I will look into this fully.
In the meantime I have generated one final version which might just fix all cases for shift-jis because I now use only 81-9f and e0-fc for the 2-byte introduction and I query the default language.
http://wavpack.com/wp4-jis.zip (http://wavpack.com/wp4-jis.zip)
Thanks again for assistance...
Fixed anyway. Thank you, bryant
Okay, this is obviously not the way to fix this problem because even if I get this language working right, there are hundreds of others. One of these days I will look into this fully.
Thats exactly what I thought, as there are even "unicode-only" languages, like Tatar, now working on Windows somehow. But on the other hand, this confusing situation is fully understandable when I remember that WavPack has a long history, begining from the time when Unicode was not common at all.
Thanks again for assistance...
[a href="index.php?act=findpost&pid=225459"][{POST_SNAPBACK}][/a]
You're welcome
I am the one who should be thanking you, for your cool freeware, for its high speed (one of the fastest as lossless), and for its imaginative hybrid mode.
Keep up a good work!
Greetings from the east end of Asia
- Liisachan
Yeah Liisachan is always a precious help for developpers. You can ask Mosu about it
We even named a Matroska release "Liisachan / CHIP release" (as it was also mentioned in the CHIP german magazine) !
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)
Automatically splitting from cuesheets would be excellent! One vote from me![a href="index.php?act=findpost&pid=224482"][{POST_SNAPBACK}][/a]
yes, this is a useful feature. some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data. I just checked in just such a --cue option to CVS; will be in the next release pretty soon.
Josh
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)
Automatically splitting from cuesheets would be excellent! One vote from me![a href="index.php?act=findpost&pid=224482"][{POST_SNAPBACK}][/a]
yes, this is a useful feature. some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data. I just checked in just such a --cue option to CVS; will be in the next release pretty soon.
Josh
[a href="index.php?act=findpost&pid=226622"][{POST_SNAPBACK}][/a]
I'll definitely be making use of this. Thanks!
I just wanted to comment on how nice wavpack is looking. From what I can tell it seems overall better than most lossless compressors. When compared with things like optimfrog and ape, the compression is bit less, but damn does it encode fast. I've testing it using -hm for encoding. I like how it stores the wav files md5 in the file. Not found any bugs yet either. Nice work. Just waiting for the final release now.
yes, this is a useful feature. some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data. I just checked in just such a --cue option to CVS; will be in the next release pretty soon.
Josh
Just tried this out. Seems to be working great. Thanks!