Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC re-encoding (Read 165848 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

FLAC re-encoding

Reply #75
If you use my latest release of Tag, which has been compiled with libFLAC 1.1.3, on a pre-1.1.3 file, it will change the vendor string to 1.1.3.

Which is incorrect also, since Tag only changes tags, right ? When checking my flacs I'd get the impression all or many seperately tagged ones are v.113 

Quote
I don't really understand the vendor string, but it seems odd to me that simply tagging a file can change the perceived version of FLAC used to encode the file.

For me it's the reason to avoid Tag and Tag&Rename, because they get the credits while only adding/changing the tags. Those credits, imho, should go to the encoding flacversion  Maybe the taggers could be mentioned in a comment string. But that's for Josh to decide.

FLAC re-encoding

Reply #76
Which is incorrect also, since Tag only changes tags, right ? When checking my flacs I'd get the impression all or many seperately tagged ones are v.113
Yes.  Well, not 'incorrect' (see below), but confusing and/or illogical.  Sorry, I didn't mean to sound like it was a recommendation, but that it was an equal problem to that mentioned by agentk7.

For me it's the reason to avoid Tag and Tag&Rename, because they get the credits while only adding/changing the tags. Those credits, imho, should go to the encoding flacversion smile.gif Maybe the taggers could be mentioned in a comment string. But that's for Josh to decide.
I haven't touched Case's tagging code; however, I very much doubt that either of these apps are purposefully changing the vendor string.  I suspect it is libFLAC, used to tag the files, which does.  Therefore these apps are not acting 'incorrectly'.

However, as stated above, I do agree that the vendor string changing simply because you add an artist field seems very strange.  The idea of a string to distinguish the version of codec used to encode the file is great, but having it change with a simple tag update is bizarre.  I can only assume that I misunderstand the purpose of the vendor string.

Edit: Hmm.. that said, using the latest metaflac on a file encoded with 1.1.0 does not change the vendor string, so maybe these apps are doing something wrong to not stop the vendor string being amended... hopefully Josh can shed some light.  Edit2: I've just looked at the Tag code (tagwrite.cpp, WriteFLACTag()) and, although I have no idea what is going on, I can't see anywhere where the vendor string is being specified.  There may be some method to stop the default behaviour of updating the vendor string that has not been implemented.  It would be great is a competent C++ programmer could provide a patch...
I'm on a horse.

FLAC re-encoding

Reply #77
Well, not 'incorrect' (see below), but confusing and/or illogical.

OK. Incorrect in a way that, without any tags at all, my flacs are still the old streams.

Quote
Edit2: I've just looked at the Tag code (tagwrite.cpp, WriteFLACTag()) and, although I have no idea what is going on, I can't see anywhere where the vendor string is being specified.  There may be some method to stop the default behaviour of updating the vendor string that has not been implemented.

There are programs that have the 'honour' to be mentioned in the vendor string. Tag and Tag&Rename are some of them. As part of the flac-packet it seems logical to me that metaflac is doing its job allright.

FLAC re-encoding

Reply #78
There are programs that have the 'honour' to be mentioned in the vendor string. Tag and Tag&Rename are some of them. As part of the flac-packet it seems logical to me that metaflac is doing its job allright.
Yes, but i was testing whether amending the vendor string was actually the norm (metaflac behaviour).  As I say, Tag uses FLAC's libFLAC, compiled from the source distributed by Josh.  So, is it libFLAC's normal behaviour to change the vendor string?  It appears so, given that the Tag code doesn't specifically change the string; inferring that, by using libFLAC, any tagging (or any tagging app) will amend the vendor string.

I don't have enough experience with other taggers to know which use libFLAC and which amend the vendor string, and I don't have enough knowledge to pursue this further.

I suppose we're getting off-topic, but I had hoped that we may resolve something with this... well, not 'we', but someone with more knowledge taking an interest...
I'm on a horse.

FLAC re-encoding

Reply #79
what happens with the vendor string depends on how the metadata was edited.  the vendor string should always be preserved unless during the process of editing, the whole original VORBIS_COMMENT metadata block is deleted by the app (or if there was none in the first place, more unlikely) and a new one created.

but there may be a bug about that; if you can give me specific info about how it is calling libFLAC I can take a look.

Josh

FLAC re-encoding

Reply #80
but there may be a bug about that; if you can give me specific info about how it is calling libFLAC I can take a look.
I'm afraid I can't be specific as such, as I don't really understand the code. 

I have uploaded the cpp file in question, here, and the specific function is below:

Code: [Select]
int WriteFLACTag ( const char* filename, FileInfo* Info )
{
    const char* Items_OggApe[] = {
        "TITLE=",          APE_TAG_FIELD_TITLE,
        "VERSION=",        APE_TAG_FIELD_SUBTITLE,
        "ARTIST=",          APE_TAG_FIELD_ARTIST,
        "ALBUM=",          APE_TAG_FIELD_ALBUM,
        "TRACKNUMBER=",    APE_TAG_FIELD_TRACK,
        //"ORGANIZATION=",    APE_TAG_FIELD_PUBLISHER,
        //"DESCRIPTION=",    APE_TAG_FIELD_COMMENT,
        "GENRE=",          APE_TAG_FIELD_GENRE,
        "DATE=",            APE_TAG_FIELD_YEAR,
        //"LOCATION=",        APE_TAG_FIELD_RECORDLOCATION,
        //"COPYRIGHT=",      APE_TAG_FIELD_COPYRIGHT,
        //"ISRC=",            APE_TAG_FIELD_ISRC
    };

    char*              item;
    char*              value;
    char*              p;
    size_t              item_len;
    size_t              i, j;
    int                tag_written = 0;
    int                needs_new_block = 1;

    struct FLAC__Metadata_SimpleIterator*      si;

    si = FLAC__metadata_simple_iterator_new ();
    if ( !FLAC__metadata_simple_iterator_init ( si, filename, 0, 0 ) ) {
        FLAC__metadata_simple_iterator_delete ( si );
        return 1;
    }
    if ( !FLAC__metadata_simple_iterator_is_writable ( si ) ) {
        FLAC__metadata_simple_iterator_delete ( si );
        return 1;
    }

    do {
        FLAC__MetadataType  type = FLAC__metadata_simple_iterator_get_block_type ( si );

        if ( type == FLAC__METADATA_TYPE_VORBIS_COMMENT || type == FLAC__METADATA_TYPE_PADDING ) {
            needs_new_block = 0;
            break;
        }
    } while ( FLAC__metadata_simple_iterator_next ( si ) );

    if ( !needs_new_block ) {
        FLAC__metadata_simple_iterator_delete ( si );

        si = FLAC__metadata_simple_iterator_new ();
        if ( !FLAC__metadata_simple_iterator_init ( si, filename, 0, 0 ) ) {
            FLAC__metadata_simple_iterator_delete ( si );
            return 1;
        }
        if ( !FLAC__metadata_simple_iterator_is_writable ( si ) ) {
            FLAC__metadata_simple_iterator_delete ( si );
            return 1;
        }
    }

    do {
        FLAC__MetadataType  type = FLAC__metadata_simple_iterator_get_block_type ( si );

        // Vorbis comment block or padding block
        if ( needs_new_block || (type == FLAC__METADATA_TYPE_VORBIS_COMMENT || type == FLAC__METADATA_TYPE_PADDING) ) {
            if ( !tag_written ) {
                FLAC__StreamMetadata*                      data;
                FLAC__StreamMetadata_VorbisComment_Entry    entry;
                size_t  comments = 0;

                if ( (data = FLAC__metadata_object_new ( FLAC__METADATA_TYPE_VORBIS_COMMENT )) == NULL ) {
                    fprintf ( Options.output, "Failed to create FLAC metadata block.\n" );
                    FLAC__metadata_simple_iterator_delete ( si );
                    return 1;
                }

                for ( i = 0; i < Info->TagItemCount; i++ ) {
                    if ( Info->TagItems[i].Item[0] != '\0' && Info->TagItems[i].ValueU[0] != '\0' )
                        comments++;
                }

                if ( !FLAC__metadata_object_vorbiscomment_resize_comments ( data, comments ) ) {
                    fprintf ( Options.output, "Failed to create FLAC metadata block.\n" );
                    FLAC__metadata_object_delete ( data );
                    FLAC__metadata_simple_iterator_delete ( si );
                    return 1;
                }

                comments = 0;

                for ( i = 0; i < Info->TagItemCount; i++ ) {
                    int replaced_item = 0;
                    if ( Info->TagItems[i].Item[0] == '\0' || Info->TagItems[i].ValueU[0] == '\0' )
                        continue;

                    for ( j = 0; j < (sizeof (Items_OggApe) / sizeof (*Items_OggApe)) / 2; j++ ) {
                        if ( stricmp ( (char *)Info->TagItems[i].Item, (char *)Items_OggApe[j*2+1] ) == 0 ) {
                            replaced_item = 1;
                            item_len = strlen ((char *)Items_OggApe[j*2]) - 1;
                            if ( (item  = (char *)malloc ( item_len + 1 )) == NULL ) {
                                fprintf ( Options.output, "WriteFLACTag: Memory allocation failed.\n" );
                                exit (1);
                            }
                            strncpy ( item, Items_OggApe[j*2], item_len );
                            item[item_len] = '\0';
                            break;
                        }
                    }

                    if ( !replaced_item ) {
                        item_len = Info->TagItems[i].ItemSize;
                        if ( (item  = (char *)malloc ( item_len + 1 )) == NULL ) {
                            fprintf ( Options.output, "WriteFLACTag: Memory allocation failed.\n" );
                            exit (1);
                        }
                        strcpy ( (char *)item, (char *)Info->TagItems[i].Item );
                    }

                    p = item;
                    while ( *p != '\0' ) {
                        int ch = *p;
                        if ( ch == '=' || ch < 0x20 || ch > 0x7D ) ch = '_';
                        *p++ = ch;
                    }

                    entry.length = strlen (item) + 1 + Info->TagItems[i].ValueUSize;
                    if ( (value = (char *)malloc ( entry.length + 1 )) == NULL ) {
                        fprintf ( Options.output, "WriteFLACTag: Memory allocation failed.\n" );
                        exit (1);
                    }
                    sprintf ( value, "%s=%s", item, Info->TagItems[i].ValueU );
                    entry.entry = (unsigned char *)value;

                    if ( !FLAC__metadata_object_vorbiscomment_set_comment ( data, comments++, entry, 1 ) ) {
                        fprintf ( Options.output, "Failed to create FLAC metadata block.\n" );
                        FLAC__metadata_object_delete ( data );
                        FLAC__metadata_simple_iterator_delete ( si );
                        return 1;
                    }

                    free ( item );
                    free ( value );
                }

                //FLAC__metadata_object_vorbiscomment_set_vendor_string ( data, entry, 1 );
                /*
                FLAC__metadata_object_vorbiscomment_resize_comments ( FLAC__StreamMetadata *object, unsigned new_num_comments);
                FLAC__metadata_object_vorbiscomment_set_comment ( FLAC__StreamMetadata *object, unsigned comment_num, FLAC__StreamMetadata_VorbisComment_Entry entry, FLAC__bool copy);
                FLAC__metadata_object_vorbiscomment_insert_comment (FLAC__StreamMetadata *object, unsigned comment_num, FLAC__StreamMetadata_VorbisComment_Entry entry, FLAC__bool copy);
                FLAC__metadata_object_vorbiscomment_delete_comment (FLAC__StreamMetadata *object, unsigned comment_num);
                */

                if ( !needs_new_block ) {
                    if ( !FLAC__metadata_simple_iterator_set_block ( si, data, 1 ) ) {  // 1 = use padding
                        fprintf ( Options.output, "Failed to write FLAC metadata block.\n" );
                        FLAC__metadata_object_delete ( data );
                        FLAC__metadata_simple_iterator_delete ( si );
                        return 1;
                    }
                } else {
                    if ( !FLAC__metadata_simple_iterator_insert_block_after ( si, data, 1 ) ) {  // 1 = use padding
                        fprintf ( Options.output, "Failed to write FLAC metadata block.\n" );
                        FLAC__metadata_object_delete ( data );
                        FLAC__metadata_simple_iterator_delete ( si );
                        return 1;
                    }
                }

                FLAC__metadata_object_delete ( data );

                tag_written = 1;
            } else if ( type == FLAC__METADATA_TYPE_VORBIS_COMMENT ) {
                if ( !FLAC__metadata_simple_iterator_delete_block ( si, 0 ) ) {    // 0 = no padding
                    fprintf ( Options.output, "Failed to remove FLAC metadata block.\n" );
                    FLAC__metadata_simple_iterator_delete ( si );
                    return 1;
                }
            }
        }
    } while ( FLAC__metadata_simple_iterator_next ( si ) );

    FLAC__metadata_simple_iterator_delete ( si );

    fprintf ( Options.output, "FLAC tag written.\n" );

    return 0;
}
Unfortunately, if you, or another developer familiar with the FLAC library, are not prepared to look at the code then we're stuck.  Perhaps, if you could help resolve this for version 2.0.50, it could be distributed with the FLAC Windows installer, and Windows FLAC users can sleep easy...

That said, I don't expect you to resolve this.  I was mainly querying whether this was standard for the library or not.  As you say it is not then I must put it down to the specific code and forget about it, until someone with the proper skills takes on the mantle.
I'm on a horse.

FLAC re-encoding

Reply #81
ok, I can see in that loop is prepending it's own created VORBIS_COMMENT, then deleting any other it finds afterward.  so it will get the vendor string of the current library in the created VORBIS_COMMENT.

2 ways to fix: update existing VORBIS_COMMENT instead of delete and replace, or copy vendor string into new comment.

Josh

FLAC re-encoding

Reply #82
Thanks for the confirmation Josh.

If only I knew my arse from my elbow...

I'm on a horse.

FLAC re-encoding

Reply #83
@Synthetic Soul

does you'r batch file support future flac versions? I mean if you use FlacGetV.exe.
For example if the flac.exe is for example v1.1.4. Does it just re-encode files lower 1.1.4?

FLAC re-encoding

Reply #84
It didn't, but it does now.  Version 2.2.2.

It's a very simple change but to make it easier I have turned it into a variable so the user can set it at the top with all the others.

Code: [Select]
REM ######################################################
SET pathToFLAC="FLAC.EXE"
SET flacOptions=-8
SET retainListOfFailedFiles=0
SET retainListOfProcessedFiles=0
SET flacVersion=113
REM ######################################################

Change 113 to 114 to skip files that are 1.1.4.

Of course, the batch file name may be a little confusing...
I'm on a horse.

FLAC re-encoding

Reply #85
thanks :-)

FLAC re-encoding

Reply #86
I'm running into problems re-encoding a set of Flac files that were encoded using Flac 1.1.1.  I can run Flac 1.1.3 using -d and it decodes the file to wav without a problem.  Is this a known issue?

Here's the output I get on one of the files when using Flac 1.1.3 to either re-encode or to verify the file.  It's dbPowerAMP's Windows Explorer extension that tells me it was encoded by Flac 1.1.1.

Code: [Select]
C:\temp\test>flac -f test.flac

flac 1.1.3, Copyright (C) 2000,2001,2002,2003,2004,2005,2006  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

WARNING: test.flac is not a WAVE file; treating as a raw file
ERROR: for encoding a raw file you must specify a value for --endian, --sign, --channels, --bps, and --sample-rate
Type "flac" for a usage summary or "flac --help" for all options


Edit: My bad.  I meant to run a test in the second example.  The file tests fine, but it won't re-encode.

Code: [Select]
C:\temp\test>flac -t test.flac

flac 1.1.3, Copyright (C) 2000,2001,2002,2003,2004,2005,2006  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

test.flac: ok

FLAC re-encoding

Reply #87
I'm running into problems re-encoding a set of Flac files that were encoded using Flac 1.1.1.  I can run Flac 1.1.3 using -d and it decodes the file to wav without a problem.  Is this a known issue?

Ok. I went back and read through this entire thread.  Looking at the file in a hex editor I can see that it does indeed have an ID3v2.3.0 tag in the beginning.

My only question is - Why is Flac able to decode this file without a hitch, but it not _re_encode it?  That doesn't really make sene to me.  It means that using this method of re-encoding is of limited use.  You need to test the file somehow, then the old method of first decoding to WAV if it fails.  May as well just use the older method to be safe until this can be fixed.

FLAC re-encoding

Reply #88
Not really.  If you have 1000 FLAC files and 2 with ID3v2 tags at the start it's well worth getting 998 converted with tags and then only having to deal with the 2 you know need special attention.

NB: This is not a FLAC issue.  FLAC never has supported ID3v2 tags, so should not be expected to deal with files that have them.  It is a user/EAC issue.
I'm on a horse.

FLAC re-encoding

Reply #89
My only question is - Why is Flac able to decode this file without a hitch, but it not _re_encode it?  That doesn't really make sene to me.

yeah, the reason for this is that the actual decoder can skip id3v2 if it already knows it's a FLAC file.

but the flac program itself does not yet know for sure the file type.  it does not go by just the extension (some platforms do not have it and some users can mess this up believe it or not), so it reads the beginning of the file to try and figure out what it is.

but, in order to support pipes properly it has to buffer all that lookahead because it can't seek the input back to the beginning after figuring out the file.  but with id3v2 that could be a lot and you can't really know how long except with multiple reads, so I punt in this case.

there are three possible fixes, but all seem like more than acceptable than to just reject invalid files:

1) add another option like --force-flac-format
2) split the filetype detection code to work for seekable input, then the bug will only be with pipes
3) make the detection code more complicated to buffer the whole id3 tag

in the meantime I've at least fixed the code to notice the id3 tag and give a better error.  I guess I could also just trust every input file whose names start with .flac.  but this will not fix it on platforms like (I think) BeOS where there is no extension.

Josh

FLAC re-encoding

Reply #90
but, in order to support pipes properly it has to buffer all that lookahead because it can't seek the input back to the beginning after figuring out the file.  but with id3v2 that could be a lot and you can't really know how long except with multiple reads, so I punt in this case.

Thanks Josh.  It makes a lot more sense now.  Clarifying the error message is a good start and may be sufficient.

I've been able to work around this in my script by first trying the native re-encoding and if Flac throws a non-zero exit code, then I fall back to the old method of decoding first to wav and encoding again, passing all the tags along.

FLAC re-encoding

Reply #91
I use the following batch file to convert my old flac files to 1.1.3

Code: [Select]
@echo off
pause
for /r %%i in (*.flac) do start /low /wait flac -f -8 -A "tukey(0,5)" "%%i"


But unfortunately, the batch never stops. If for example I run it in a directory with 2 flac files named 1.flac and 2.flac, it will keep alternating between the two. It seems the FOR somehow recognizes that a file has changed and therefor adds it to the list of files to be processed...?

Any way to stop this? If I encode to another file with -o "%%i.new" or something it works perfect, but I see no easy way to delete the old files afterwards and rename all the .flac.new to .flac

FLAC re-encoding

Reply #92
Yes, that happened to my early version.  As you say, I think XP (it doesn't happen on 2K) dynamically alters the collection of files, seeing the amended file as new.

I resolved this in mine by creating a list of files using that syntax, and then using that list as the source files.  You can see my code here.

Edit:  Here's a cut-down version:

Code: [Select]
FOR /R %%G IN (*.flac) DO ECHO "%%G">>list.txt
FOR /F "tokens=* delims=" %%H IN (list.txt) DO START /LOW /WAIT FLAC.EXE -f -8 -A "tukey(0,5)" %%H
DEL list.txt
I'm on a horse.

FLAC re-encoding

Reply #93
I use the following batch file to convert my old flac files to 1.1.3

Code: [Select]
@echo off
pause
for /r %%i in (*.flac) do start /low /wait flac -f -8 -A "tukey(0,5)" "%%i"


But unfortunately, the batch never stops. If for example I run it in a directory with 2 flac files named 1.flac and 2.flac, it will keep alternating between the two. It seems the FOR somehow recognizes that a file has changed and therefor adds it to the list of files to be processed...?

Any way to stop this? If I encode to another file with -o "%%i.new" or something it works perfect, but I see no easy way to delete the old files afterwards and rename all the .flac.new to .flac



Wy don't you use this batch file?
It's really helpfull!


Oh someone was faster :-)

FLAC re-encoding

Reply #94
Thanks, I'll just use your batch file instead of worrying too much to find my own solution.

I took the liberty of modifying line #12
Code: [Select]
SET flacOptions=-8 -A "tukey(0,5)"


Again, thanks for providing the batch file. Works like a charm.

FLAC re-encoding

Reply #95
You could also just use the flac.exe version from http://www.rarewares.org/lossless.html.  (You need to copy the libmmd.dll in the "Windows" folder, You find it on the same page)
This is new compile of the official release + the comma bugfix.

Then you don't need -A "tukey(0,5)"

FLAC re-encoding

Reply #96
You could also just use the flac.exe version from http://www.rarewares.org/lossless.html.
This is new compile of the official release + the comma bugfix.
"MSVC6 compile"... wow... are people still using that old compiler piece of crap?

It does not work however: "Die Anwendung konnte nicht gestartet werden, weil libmmd.dll nicht gefunden wurde. Neuinstallation der Anwendung könnte das Problem beheben."

I'll stay with the official version and the tukey(0,5)

EDIT: Hooray, managed to shrink an 4825 MB directory down to 4770 MB

FLAC re-encoding

Reply #97
The file has ID3v2 tags at the beginning, as suspected.  It looks like EAC added them
FLAC will accommodate ID3v1 at the end of the file, but ID3v2 are totally non-standard.

Sorry, for jumping back in time. I don't know if it's an id3version two tag, but tonight I noticed that eac adds that horrible id3tag when I tick [v] add id3 tag at the compression options tab/page.

FLAC re-encoding

Reply #98
@Synthetic Soul

I have another suggestion for the batch file.
You write all the added files in the beginning to the text file for the processed files.
Maybe you could just add them when they are really successful processed. So you could read all processed files. In the case you abort the transcoding in the middle.

And maybe you could include a option to pursue an old log file.


What do you think?  :-)

FLAC re-encoding

Reply #99
The processed list has to contain all files, as this is the list that is used to detirmine which files to process.  However, as well as writing all failed files to a log I could also easily write all successful files to a log (and add another config flag to specify whether it is kept or not).

I'm a little confused by your other request, but two interesting actions spring to mind.

Firstly, and most easily, I could allow the user to drag a log file onto the batch file to start, rather than a file or folder.  The script would then use all files in the log as the process list, rather than create it's own.  This may be useful to re-process the "flac-113-failed.txt" log, once the user has made some changes to the files (like removing ID3v2 tags).

          Secondly, but more difficultly, the script could somehow restart processing a folder and its subfolders, by using the newly suggested "flac-113-successful.txt" and "flac-113-processed.txt" to restart the list, skipping any files that exist in "flac-113-successful.txt".  Presumably this would be initiated by dragging "flac-113-processed.txt" onto the batch file, and having "flac-113-successful.txt" exist in the same folder as flac-113.bat.  Or something.

Anyway, thanks for the ideas.  I'll see what I can do.  I'm a little busy at the moment.
I'm on a horse.