Synchronize FLAC library with the lossy version

2011-05-05 20:36:14

I don't think that I was too explicit in the title.

My library is 98.8% FLAC (I haven't made that number up), and it is way too large for my small laptop external HDD. I try to keep a smaller, lossy, version of it, but as I add new stuff in it I forget to make the lossy version. What happens is that when I'm away and I try to listen to my latest music, it's never there.

Is there a easy way to keep my 2 libraries in synch?

Synchronize FLAC library with the lossy version

Reply #1 – 2011-05-10 06:07:12

I finished a script to do just that a few days ago. Are you on Linux? Here it is:

Code: [Select]

#! /usr/bin/env tclsh8.5

# ---------------------------------------------------------------------
# mclone.tcl <in_dir> <out_dir>
#
# music file cloner by David Gravereaux
# Sat Apr 30 21:56:53 PDT 2011
#
# Generate MP3 clones of the whole music collection
# for my &^$% Samsung BD-C5500 player because it doesn't
# understand FLAC nor does it work with MediaTomb's
# transcoding feature.  Its DLNA certification is a joke.
#
# ---------------------------------------------------------------------

package require Tcl 8.5	;# for {expand}
# requires LAME 3.99, too

if {$argc != 2} {
	puts stderr "barfed!  need FROM and TO directories as args"
	exit 1	
}
set from [file normalize [lindex $argv 0]]
set to [file normalize [lindex $argv 1]]

proc tryexec {args} {
	global done

	#puts "% calling: $args"
	set pipe [open "|$args |& [info name] cat.tcl" r]
	#puts "pid(s) [pid $pipe] started... please wait..."
	fconfigure $pipe -buffering line -blocking 0
	fileevent $pipe readable [list gotpipereadable $pipe]
	set done 0
	vwait done
	#puts "% done!"
}

proc gotpipereadable {pipe} {
	global done

	set got [gets $pipe]
	if {[string length $got]} {
		#puts "*** $got"
	}

	### check for incompleted lines
	if {[fblocked $pipe]} {
		#puts -nonewline "*** [read -nonewline $pipe]"
		#flush stdout
	}

	if {[eof $pipe]} {

		set done 1
		fconfigure $pipe -blocking 1
		set status [catch {close $pipe} result]

		if {$status == 0} {

			# The command succeeded, and wrote nothing to stderr.

		} elseif {[string equal $::errorCode NONE]} {

			# The command exited with a normal status, but wrote something
			# to stderr, which is included in $result.
			puts "% stderr was: $result"

		} else {

			switch -exact -- [lindex $::errorCode 0] {

				CHILDKILLED {

					foreach { - pid sigName msg } $::errorCode break
						# A child process, whose process ID was $pid,
						# died on a signal named $sigName.  A human-
						# readable message appears in $msg.
						puts "pid $pid died on signal $sigName: $msg"
						exit 1
				}

				CHILDSTATUS {

					foreach { - pid code } $::errorCode break

						# A child process, whose process ID was $pid,
						# exited with a non-zero exit status, $code.
						puts "pid $pid exited with code: $code"
						puts "stderr: $result"
						exit 1
				}

				CHILDSUSP {

					foreach { - pid sigName msg } $::errorCode break
						# A child process, whose process ID was $pid,
						# has been suspended because of a signal named
						# $sigName.  A human-readable description of the
						# signal appears in $msg
						puts "pid $pid has been suspended for $sigName: $msg"
						exit 1
				}

				POSIX {

					foreach { - errName msg } $::errorCode break
						# One of the kernel calls to launch the command
						# failed.  The error code is in $errName, and a
						# human-readable message is in $msg.
						puts "Can't start executable because of $errName: $msg"
						exit 1
				}
			}
		}
	}
}

proc forceInt { x } {
	set count [scan $x %d%s n rest]
	if { $count <= 0 || ( $count == 2 && ![string is space $rest] ) } {
		return -code error "not an integer: \"$x\""
	}
	return $n
}

# reference links:
# [url=http://www.id3.org/id3v2.3.0]http://www.id3.org/id3v2.3.0[/url]
# [url=http://xiph.org/vorbis/doc/v-comment.html]http://xiph.org/vorbis/doc/v-comment.html[/url]
# [url=http://musicbrainz.org/doc/Picard_Tag_Mapping]http://musicbrainz.org/doc/Picard_Tag_Mapping[/url]
#
# return id3v2.3 data
#
#  This routine only needs to be good enough to let MBP
#  do a rescan on the output later and fill the data properly
#
proc gen_metacmds {metafile} {
	set cmd [list]
	if {[catch {set f [open $metafile r]}]} {
		return [list]
	}
	set metaList [split [read $f] \n]
	foreach meta $metaList {
	if {$meta ne ""} {
		foreach {name value} [split $meta =] break
		switch -- [string toupper $name] {
		"ALBUM" {lappend cmd --tl $value}
		"TITLE" {lappend cmd --tt $value}
		"ARTIST" {lappend cmd --ta $value}
		"DATE" {
			if {[string length $value] == 4} {
			# year only
			lappend cmd --ty $value
			} else {
			if {![catch {set seconds [clock scan $value]}]} {
				lappend cmd --ty [clock format $seconds -format "%Y"]
				lappend cmd --tv TDAT=[clock format $seconds -format "%d%m"]
			}
			}
		}
		"ORIGINALDATE" {
			lappend cmd --tv TORY=[clock format [clock scan $value] -format "%Y"]
		}
		"TRACKNUMBER" {lappend cmd --tn [set track [forceInt $value]]}
		"TRACKTOTAL" {set total [forceInt $value]}
		"ALBUMARTIST" {lappend cmd --tv "TPE2=$value"}
		"PERFORMER" {}
		"COPYRIGHT" {}
		"LICENSE" {}
		"ORGANIZATION" {lappend cmd --tv "TOWN=$value"}
		"DESCRIPTION" {lappend cmd --tc $value}
		"GENRE" {}
		"LOCATION" {}
		"CONTACT" {}
		"ISRC" {
			# see [url=http://www.ifpi.org/isrc/]http://www.ifpi.org/isrc/[/url]
			lappend cmd --tv "TSRC=$value"
		}
		default {
			# this *REQUIRES* lame 3.99 or better
			lappend cmd --tv "TXXX=$meta"
		}
		}
	}
	}
	return $cmd
}

proc convert {input output} {
	if {[file exists $output]} {
		puts "skipping [file tail $input]"
		return
	}
	
	puts "% About to convert: $input"
	tryexec flac --force --decode $input --output-name=temp.wav
	tryexec metaflac --export-tags-to=temp.meta $input

	#tryexec metaflac --block-number=3 --export-image-to=Cover_front.jpg $input

	# we don't know what the image file format really is, but let convert
	# change it to really be a jpg
	#tryexec convert -resize 500x500 -quality 75 Cover_front.jpg Cover_front.jpg
	
	set dir [file dirname $output]
	if {![file exists $dir]} {
	file mkdir $dir
	}

	tryexec lame -m s -V 0 -q 0 --preset insane --replaygain-accurate \
	--tg 79 --add-id3v2 {*}[gen_metacmds temp.meta] \
	temp.wav $output

	file delete temp.wav temp.meta
}

proc scanAndConvert {fromD toD} {
   if {[file exists [file join $fromD skip]]} {
	   puts "% skipping $fromD"
	   return
   }
   set dirs [lsort [glob -nocomplain -tails -directory $fromD -type d *]]
   foreach dir $dirs {
	   # yes, this is recursion
	   scanAndConvert [file join $fromD $dir] [file join $toD $dir]
   }
   set files [lsort [glob -nocomplain -tails -directory $fromD *.flac]]
   foreach f $files {
	   convert [file join $fromD $f] [file join $toD [file rootname $f]].mp3
   }
}

scanAndConvert $from $to

You'll need this too, called cat.tcl in the same directory as mclone.tcl

Code: [Select]

# cat.tcl
fconfigure stdin -buffering none -translation binary -blocking 0
fconfigure stdout -buffering none -translation binary

proc DoIt {} {
	set got [read stdin]
	if {[string length $got]} {
		puts -nonewline stdout $got
		flush stdout
	}
	if {[eof stdin]} {set ::done 1}
}

fileevent stdin readable DoIt
set done 0
vwait done

Synchronize FLAC library with the lossy version

Reply #2 – 2011-05-10 21:15:29

I had a similar problem, but did not automate it like that. I used foobar to sort my library by the codec used and grouped lossy formats under one file tree and the lossless codecs under another tree.

That way I could compare the two trees and if the lossy version was missing I could see it easily enough. I then would use foobar to create the lossy version with a simple right click and convert. The file trees and naming schemes were already saved, so it was pretty simple.
JXL

Synchronize FLAC library with the lossy version

Reply #3 – 2011-05-11 03:15:43

J. River Media Center supports stacking different versions of the same file. One change affects the whole stack.

Notice