Folks,
I have a lot of wavpack hybrid tracks. I'm finding that dealing with the two files for each track is cumbersome. I'm looking for a way to convert these wavpack hybrid file pairs back to the single *.wv lossless files, while keeping the tags intact. Does anyone know of a good Linux solution. Command line is preferred but a gui is alright.
I've googled this and nothing seems to work well.
Thanks
making an example encode
wavpack -b2 -c -w "sometag=VaLuE" file.wav
(will generate file.wc and file.wvc with sometag=VaLuE)
ffmpeg_static -i file.wv whatever.wv
(will generate whatever.wv with sometag=VaLuE)
Using ffmpeg from http://johnvansickle.com/ffmpeg/ (http://johnvansickle.com/ffmpeg/)
I'am sure there is a way with just wavpack tools, but too dumb to understand the wavpack man.
------------------------------------------
edit: ok, with wavpack 4.75 a simple
wavpack file.wv -o whatever.wv
is working just fine.
or convert all *.wv to subdir ./out
wavpack *.wv -o ./out
I'd use caudec (http://caudec.net), because it will be automatically using n threads, just like foobar on windows. It also uses /tmp to get around HDD limitations. Tags are kept, as well.
caudec -c wv -q hx6 -o /my/output/dir *.wv
Currently, I'm using DeadBeef (http://deadbeef.sourceforge.net/)to convert hybrid Wavpack to MP3 under LXLE (a variant of Lubuntu) and so far so good: it's been working like a charm, tags and everything!
So I can't see why it wouldn't do the same flawlessly to your required hybrid-to-lossless conversion.
Edit: as I'd previously posted elsewhere on HA, I made sure the correction files were actually being recognized by DeadBeef during conversion* - or else I could've ended up with MP3 files resulting from lame lossy-to-lossy conversion.
*which, unfortunately, (and thanks to some arcane reason) wasn't the case with my copy of foobar2000 under Wine.
The only thing I can add is that it's possible do do this in place (if you have the stomach, although it should be safe because tmp files are always used and the output is verified before the source is deleted):
wavpack -vdy *.wv
or even something like this (although too many files will create problems for the shell):
wavpack -vdy *.wv */*.wv
Note that the -d is not normally required when overwriting the original file, but is required here to delete the .wvc files (although you could easily delete them manually afterwards too).
And, of course, you might want to use some other encoding mode options (-h, etc.). Note that this will
not use multiple threads.
Note that the second example is Linux only (not Windows), and that you can use this same format to convert back to hybrid if you change your mind again (and then the -d would be redundant).
Good luck!
edit: fixed typo in second example and added final note
@bryant, nice about the overwrite, but how deep should that */wv */*.wv glob traverse?
ls */mp3 */*.mp3 | wc -l
ls: dostop do */mp3 ni mogoč: No such file or directory
407
find . -type f \( -iname "*.mp3" ! -path "*.Apple*" \) | wc -l
1911
ls */*.mp3 **/*.mp3 | wc -l
814
Pretty sure only find gets them all (GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)).
@bryant, nice about the overwrite, but how deep should that *.wv */*.wv glob traverse?
I had a typo in my original example which I also fixed in your quote (hope that's not against the rules ), but yeah, I have not figured out a way to traverse recursively just using the shell like that. But in reality I would just go one level deeper every command until I got no files (and would do an "ls" first at each level to make sure something funny wasn't going to happen). I quite often use
grep like that when I'm looking for things is large source trees.
Ideally I should have a --recursive option, but I'd be a little hesitant to implement that...
Ideally I should have a --recursive option, but I'd be a little hesitant to implement that...
I think a few find examples in man should be enough, + advanced would be something like parallel, I have this example noted:
parallel --gnu mpcenc {} {.}.mpc ::: *.flacUnless I guess stuff like this is out of the man scope for specific thingy.
parallel --gnu mpcenc {} {.}.mpc ::: *.flac
Wow, thanks, I had never heard of the
parallel command!
I will experiment with that and see how well it works doing batch WavPack operations with multiple threads.
-David
Apparently my post didn't take last night. Thanks all for your suggestions. Bryant's works great. I had no idea that Wavpack preserved tags! That's makes me love it even more.
Hi, longtime lurker here.
First of all, I want to thank bryant, and I love wavpack, etc, etc.
A couple of months ago, I ran into a similiar problem the OP did. I then discovered GNU Parallel (as previously mentioned,) and wrote a shell script for batch encoding. I'll leave it here in case anyone would like to use it.
#! /bin/sh -
#
# WVBATCH - wavpack batch encode script
#
## CONSTANTS ##
readonly TRUE=0
export TRUE
readonly FALSE=1
export FALSE
## VARIABLES ##
export WAVPACK_OPTS="-hhx6"
export DELETE_INPUT=$FALSE
MAX_PROCS="100%"
### FUNCTIONS ##################################################################
print_help()
{
cat << EOF
WVBATCH - wavpack batch encode script
Usage:
wvbatch [options] <input.files>
Options:
-d)
delete all input files
(same as '-o "[other-options] -d"')
--help)
print this message
-j)
maximum number of processors to use ||
percentage of the available processors to use
(default = '100%')
e.g.)
-j 6
-j 40%
-o)
wavpack options
(default = '-hhx6'; '-qzymvr' is hardcoded)
e.g.)
-o -hx3b320c
-o "-hhx4 --raw-pcm=48000,16,6 --channel-order=FL,FR,FC,LFE,SL,SR"
Read the wavpack man page for all available options.
Dependencies:
1 WavPack
2 GNU Parallel
EOF
}
### TRAPS ######################################################################
trap '
printf "\naborting...\n" >&2
exit 255
' HUP INT QUIT ABRT KILL PIPE TERM
### GET OPTS ###################################################################
while [ $# -gt 0 ] ; do
case $1 in
-d)
DELETE_INPUT=$TRUE
shift
;;
--help)
print_help >&2
exit 1
;;
-j)
MAX_PROCS="$2"
shift 2
;;
-o)
WAVPACK_OPTS="$2"
shift 2
;;
--)
shift
break
;;
*)
break
;;
esac
done
if [ $# -eq 0 ] ; then
print_help >&2
exit 1
fi
if [ $DELETE_INPUT -eq $TRUE ] ; then
WAVPACK_OPTS="$WAVPACK_OPTS -d"
fi
### MAIN #######################################################################
wavpack 2>&1 | sed 4q >&2
parallel -j "$MAX_PROCS" '
TIME_START=$(date +%s.%N)
wavpack -qzymvr $WAVPACK_OPTS {}
EXIT_STATUS=$?
TIME_STOP=$(date +%s.%N)
case $EXIT_STATUS in
0)
TIME=$(echo "$TIME_STOP - $TIME_START" | bc)
OUTFILE={.}.wv
WVINFO="$(wvunpack -qzs "$OUTFILE")"
MD5_SIG="$(
echo "$WVINFO" |
grep -F "original md5:" |
sed "s/original md5:[[:space:]]*//"
)"
MODALITY=$(echo "$WVINFO" | grep -F "modalities:")
if echo "$MODALITY" | grep -Fq "lossless" ; then
if echo "$MODALITY" | grep -Fq "hybrid" ; then
HYBRID="(+.wvc) "
fi
COMP_TYPE="lossless"
COMP_STAT="$(
echo "$WVINFO" |
grep -F "compression:" |
sed "s/compression:[[:space:]]*//"
)"
else
COMP_TYPE="lossy"
COMP_STAT="$(
echo "$WVINFO" |
grep -F "ave bitrate:" |
sed "s/ave bitrate:[[:space:]]*//"
)"
fi
printf "%s:\n" {}
if [ $DELETE_INPUT -eq $TRUE ] ; then
printf "deleted source file %s\n" {}
fi
printf "original md5 signature: %s\n" "$MD5_SIG"
printf "created (and verified) %s %sin %.2f secs (%s, %s)\n" \
"$OUTFILE" "$HYBRID" $TIME "$COMP_TYPE" "$COMP_STAT"
printf "\n"
exit 0
;;
*)
# wavpack will print the error here
printf " \033[41m>>>\033[0m %s\n\n" {} >&2
exit $EXIT_STATUS
;;
esac
' ::: "$@"
EXIT_STATUS=$?
case $EXIT_STATUS in
0)
if [ $# -ne 1 ] ; then
printf " **** %d files successfully processed ****\n" $# >&2
fi
exit 0
;;
254)
printf " **** warning: errors occured in more than 253 of %d files! ****\n" $# >&2
exit 254
;;
255) # parallel error
exit 255
;;
*)
printf " **** warning: errors occured in %d of %d files! ****\n" $EXIT_STATUS $# >&2
exit $EXIT_STATUS
;;
esac
@rdtsh: That looks great. I was playing around with it and i altered line 15 to read
export WAVPACK_OPTS="-vyhhx3m"
When i do that and run
wvbatch -d 1977-going_for_the_one/*.wv
i get
WAVPACK Hybrid Lossless Audio Compressor Linux Version 4.70.0
Copyright © 1998 - 2013 Conifer Software. All Rights Reserved.
illegal option: !
illegal option: - !
what am i doing wrong here?
What shell are you using?
I just tried your changes, and it worked with dash.
@rdtsh; No bashism detected (checkbashisms (in jessie devscripts)), so I guess it should work on most shells? (Thanks for sharing, quite an elaborate script).
p.s. There are some 'comments' if you paste the script to http://www.shellcheck.net (http://www.shellcheck.net/)
What shell are you using?
I just tried your changes, and it worked with dash.
I'm just using zsh (bash compatable). I made a postbin of the code here https://gist.github.com/dc4e4a151a81f2e17db4 (https://gist.github.com/dc4e4a151a81f2e17db4) but doing a diff on it and your code, i only changed that line.
p.s. There are some 'comments' if you paste the script to http://www.shellcheck.net (http://www.shellcheck.net/)
Thanks for showing me shellcheck. I wrote the script after reading the O'Reilly book "Classic Shell Scripting", so everything should be POSIX. About the two comments in shellcheck:
1)
There's some bug in parallel that won't let it take a shell function as an argument. I could have put the meat of the script into a seperate file, but that would have been silly.
2)
I put the backslash in there so that the line wouldn't go over 80 characters. If it weren't in quotes, it wouldn't matter, and that is how parallel sees it. Parallel's fine with it, so I don't see a problem with it.
I did notice a somewhat redundant 'printf' on line 164 though.
@BearcatSandor
I figured it was some shell issue. It's probably the '-' in the crunchbang. That tells the shell not to take any additional arguments (security feature); it must mess zsh up.
A couple of months ago, I ran into a similiar problem the OP did. I then discovered GNU Parallel (as previously mentioned,) and wrote a shell script for batch encoding. I'll leave it here in case anyone would like to use it.
Very cool...thanks!
WAVPACK Hybrid Lossless Audio Compressor Linux Version 4.70.0
Copyright © 1998 - 2013 Conifer Software. All Rights Reserved.
illegal option: !
illegal option: - !
There's a special debug feature built into the WavPack command-line programs that might help debug issues like this. You create a copy of the binary with "_debug" appended to the name (like "wavpack_debug") and when you use that it will dump its command-line arguments to stderr when it's run, one arg per line. It also dumps some other info, but this look like an args problem.
Also, I noticed you're still using the 4.70 version...there's a new faster one available, you know!
Also, I noticed you're still using the 4.70 version...there's a new faster one available, you know!
You should email some package maintainers. Most FOSS people only care about Xiph.org stuff.
Also, I noticed you're still using the 4.70 version...there's a new faster one available, you know!
You should email some package maintainers. Most FOSS people only care about Xiph.org stuff.
@Bryant Well, I'm on Gentoo which can be a bit conservative in some ways and some packages like wavpack don't get a lot of love.
@rdtsh I'll try the script in plain old bash and see what it does.
Thanks both of you
@rdtsh I'll try the script in plain old bash and see what it does.
Bash isn't plain. Try dash, it only does POSIX with a few BSD extensions (It's what /bin/sh links to in Debian by default.) I think that my earlier hypothesis was wrong, as I didn't notice the wavpack copyright stuff.
@rdtsh: That looks great. I was playing around with it and i altered line 15 to read
export WAVPACK_OPTS="-vyhhx3m"
When i do that and run
wvbatch -d 1977-going_for_the_one/*.wv
i get
WAVPACK Hybrid Lossless Audio Compressor Linux Version 4.70.0
Copyright © 1998 - 2013 Conifer Software. All Rights Reserved.
illegal option: !
illegal option: - !
what am i doing wrong here?
1- I highly doubt that your /bin/sh is actually zsh.
2- zsh is not bash-compatible, it's neither a superset, nor a subset. Example:
a=b; b=c; echo "${(P)a}" # zsh syntax
a=b; b=c; echo "${!a}" # bash syntax3- Do you happen to have a file that starts with a dash?
All of my files start with numbers, as i prepend the disc number to the file name.
I just switched to bash via "bin/bash" at my cli and ran the command. i get the same results with /bin/bash and /bin/dash
Thanks for the help so far folks. I really do appreciate it.
All of my files start with numbers, as i prepend the disc number to the file name.
I just switched to bash via "bin/bash" at my cli and ran the command. i get the same results with /bin/bash and /bin/dash
Thanks for the help so far folks. I really do appreciate it.
Your interactive shell is irrelevant. The script has a shebang(the 1st line) that calls /bin/sh.
Anyway, I don't think what /bin/sh links to is relevant here.
Sharing more of the output you get should help us figure out what's really going on.
Also, I noticed you're still using the 4.70 version...there's a new faster one available, you know!
You should email some package maintainers. Most FOSS people only care about Xiph.org stuff.
@Bryant Well, I'm on Gentoo which can be a bit conservative in some ways and some packages like wavpack don't get a lot of love.
Yes, I know the adoption is pretty slow in some distros. Raspian (Debian/ARM) is still on 4.60 which has bug there (because of default unsigned chars) that causes
high and
very high modes to produce corrupt files.
But it's pretty safe to build a static version locally (--disable-shared) and install it in the default location (/usr/local/bin). You can undo it by deleting those three files. That will also make it easy to try the "wavpack_debug" trick I described above and see the actual args being passed.
Yes, I know the adoption is pretty slow in some distros.
I just noticed that my distro upgraded
This thread has caused me to revisit the script, so I upgraded it so that you can set an output directory and pipe to stdin like
find ./ -type f \( -iname "*.wav" \) | wvbatch -
#! /bin/sh -
#
# WVBATCH - wavpack batch encode script
#
## CONSTANTS ##
readonly TRUE=0
export TRUE
readonly FALSE=1
export FALSE
## VARIABLES ##
export WAVPACK_OPTS="-hhx6"
export DELETE_INPUT=$FALSE
MAX_PROCS="100%"
TARGET_DIR=""
#
READ_FROM_STDIN=$FALSE
INPUT=""
TMP_INPUT=""
### FUNCTIONS ##################################################################
print_help()
{
cat << EOF
WVBATCH - wavpack batch encode script
Usage:
wvbatch [options] <input.files>
find ./ -type f \( -iname "*.wav" \) | wvbatch [options] -
Options:
-d)
delete all input files
(same as '-o "[other-options] -d"')
--help)
print this message
-j)
maximum number of processors to use ||
percentage of the available processors to use
(default = '100%')
e.g.)
-j 6
-j 40%
-o)
set an output directory
-O)
wavpack options
(default = '-hhx6'; '-qzymvr' is hardcoded)
e.g.)
-O -hx3b320c
-O "-hhx4 --raw-pcm=48000,16,6 --channel-order=FL,FR,FC,LFE,SL,SR"
Read the wavpack man page for all available options.
Dependencies:
1 WavPack
2 GNU Parallel
EOF
}
### TRAPS ######################################################################
trap '
printf "\naborting...\n" >&2
exit 255
' HUP INT QUIT ABRT KILL PIPE TERM
### GET OPTS ###################################################################
if [ $# -eq 0 ] ; then
print_help >&2
exit 1
fi
while [ $# -gt 0 ] ; do
case $1 in
-d)
DELETE_INPUT=$TRUE
shift
;;
--help)
print_help >&2
exit 1
;;
-j)
MAX_PROCS="$2"
shift 2
;;
-o)
TARGET_DIR="$2"
shift 2
;;
-O)
WAVPACK_OPTS="$2"
shift 2
;;
-)
shift
READ_FROM_STDIN=$TRUE
break
;;
--)
shift
break
;;
*)
break
;;
esac
done
if [ $DELETE_INPUT -eq $TRUE ] ; then
WAVPACK_OPTS="$WAVPACK_OPTS -d"
fi
if [ "$TARGET_DIR" ] ; then
if [ ! -e "$TARGET_DIR" ] ; then
mkdir -p "$TARGET_DIR"
fi
if [ "$(echo -n "$TARGET_DIR" | tail -c 1)" != "/" ] ; then
TARGET_DIR="$TARGET_DIR""/"
fi
fi
export TARGET_DIR
if [ $READ_FROM_STDIN -eq $FALSE ] ; then
INPUT=$1
shift
while [ $# -gt 0 ] ; do
INPUT=$(printf "%b\n" "$INPUT" ; printf "%b\n" "$1")
shift
done
else
read TMP_INPUT
INPUT="$TMP_INPUT"
while read TMP_INPUT ; do
INPUT=$(printf "%b\n" "$INPUT" ; printf "%b\n" "$TMP_INPUT")
done
fi
### MAIN #######################################################################
wavpack 2>&1 | sed 4q >&2
parallel -j "$MAX_PROCS" '
OUTDIR="$TARGET_DIR$(echo {//} | sed "s/.\///1")"
if [ ! -e "$OUTDIR" ] ; then
mkdir -p "$OUTDIR"
fi
OUTFILE="$OUTDIR"/{/.}.wv
TIME_START=$(date +%s.%N)
wavpack -qzymvr $WAVPACK_OPTS {} -o "$OUTFILE"
EXIT_STATUS=$?
TIME_STOP=$(date +%s.%N)
case $EXIT_STATUS in
0)
TIME=$(echo "$TIME_STOP - $TIME_START" | bc)
WVINFO="$(wvunpack -qzs "$OUTFILE")"
MD5_SIG="$(
echo "$WVINFO" |
grep -F "original md5:" |
sed "s/original md5:[[:space:]]*//"
)"
MODALITY=$(echo "$WVINFO" | grep -F "modalities:")
if echo "$MODALITY" | grep -Fq "lossless" ; then
if echo "$MODALITY" | grep -Fq "hybrid" ; then
HYBRID="(+.wvc) "
fi
COMP_TYPE="lossless"
COMP_STAT="$(
echo "$WVINFO" |
grep -F "compression:" |
sed "s/compression:[[:space:]]*//"
)"
else
COMP_TYPE="lossy"
COMP_STAT="$(
echo "$WVINFO" |
grep -F "ave bitrate:" |
sed "s/ave bitrate:[[:space:]]*//"
)"
fi
printf "%s:\n" {}
if [ $DELETE_INPUT -eq $TRUE ] ; then
printf "deleted source file %s\n" {}
fi
printf "original md5 signature: %s\n" "$MD5_SIG"
printf "created (and verified) %s %sin %.2f secs (%s, %s)\n\n" \
"$OUTFILE" "$HYBRID" $TIME "$COMP_TYPE" "$COMP_STAT"
exit 0
;;
*)
# wavpack will print the error here
printf " \033[41m>>>\033[0m %s\n\n" {} >&2
exit $EXIT_STATUS
;;
esac
' ::: "$INPUT"
EXIT_STATUS=$?
case $EXIT_STATUS in
0)
if [ $# -ne 1 ] ; then
printf " **** %d files successfully processed ****\n" $# >&2
fi
exit 0
;;
254)
printf " **** warning: errors occured in more than 253 of %d files! ****\n" $# >&2
exit 254
;;
255) # parallel error
exit 255
;;
*)
printf " **** warning: errors occured in %d of %d files! ****\n" $EXIT_STATUS $# >&2
exit $EXIT_STATUS
;;
esac
I think that it should be portable.
There seems to be a bug in parallel that causes it to mess up on files and directories with backspaces in their names. It seems to be related to a similar bug I reported a few months back. I'll send a bug report later (no time right now.)
Yes, I know the adoption is pretty slow in some distros.
I just noticed that my distro upgraded
I just saw that Arch Linux updated, but they had to add --disable-asm, which means it's actually slower than 4.70 now. I've e-mailed the maintainer asking if I could help.
I also got a report that the asm build on MaxOS is broken too...
backspaces in their names.
I mean't the escape character "\", sorry
I guess the bug has been fixed. The one in my distro is just ancient.
Yes, I know the adoption is pretty slow in some distros.
I just noticed that my distro upgraded
I just saw that Arch Linux updated, but they had to add --disable-asm, which means it's actually slower than 4.70 now. I've e-mailed the maintainer asking if I could help.
I also got a report that the asm build on MaxOS is broken too...
If building from X64 and targeting X86. The right asm arch is not detected correctly if --host is not passed explicitly!
That's because you're using $host_cpu for detection in configure.
Wavpack 4.75.2 are released. Arch Linux package is up. Arch still uses --disable-asm. Why is this?
https://projects.archlinux.org/svntogit/pac...ackages/wavpack (https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/wavpack)
^ You might want to ask the question on the arch mailing list or the maintainer directly, instead.
Wavpack 4.75.2 are released. Arch Linux package is up. Arch still uses --disable-asm. Why is this?
https://projects.archlinux.org/svntogit/pac...ackages/wavpack (https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/wavpack)
I noticed that for 4.75.0 and e-mailed the maintainer (I had corresponded with him in the past), but did not hear back.
I'll try again.