HydrogenAudio

Hosted Forums => foobar2000 => General - (fb2k) => Topic started by: Peter on 2017-04-11 11:24:32

Title: Tag Sanitizer component
Post by: Peter on 2017-04-11 11:24:32
https://www.foobar2000.org/components/view/foo_sanitizer

Removes excessive or otherwise unwanted tags from your music files.

Use with caution, always make a backup of your files first.

Usage: Select one or more files to operate on, right click, choose Tagging / Sanitize tags...

For best results, make sure you're running foobar2000 1.3.15 or newer before using this component, as version 1.3.15 introduces MP4 tag manipulation improvements meant specifically for Tag Sanitizer use.

[edit] it's "Tagging" not "Utilities", thanks to jazzthieve for pointing out
[edit] added note about 1.3.15
Title: Re: Tag Sanitizer component
Post by: jazzthieve on 2017-04-11 12:03:36
I found it under tagging - sanitize tags instead of utilities - sanitize tags.
Title: Re: Tag Sanitizer component
Post by: mjm716 on 2017-04-11 16:14:23
Foobar has really needed something like this. I've always had to use MP3Tag or another utility to get rid of extraneous tags, which has been a big pet peeve.

However this component still does not seem to do the job.

I just tried a test case on some sloppy files and it did not remove
COMMENT ITUNNORM
COMMENT ITUNPGAP
COMMENT ITUNSMPB
COMMENT ITUNES_CDDB_IDS

Why exactly can't Foobar simply display **all** existing Tags via Properties dialog? It would be far easier to delete them from there.

Here is a sample based on tags I haven't cleaned [MP3Tag (93) vs FB (80)]:

(http://i.imgur.com/lgWD0BR.png)

(http://i.imgur.com/DCk2lXH.png)
Title: Re: Tag Sanitizer component
Post by: Peter on 2017-04-11 16:30:12
Note that the component doesn't touch files by itself but tells relevant file format handler to perform a tag nuke cycle.
If some fields don't go away, it's the decoders that need updating not the component.

As for iTunSMPB, we cannot remove it as it holds gapless info. iTunNORM holds SoundCheck (ReplayGain equivalent). I'll check about the other fields.
Title: Re: Tag Sanitizer component
Post by: mjm716 on 2017-04-11 16:41:22
As for iTunSMPB, we cannot remove it as it holds gapless info. iTunNORM holds SoundCheck (ReplayGain equivalent). I'll check about the other fields.

I understand that FB will map certain tags, however I would expect the "nuke tags" option to kill those custom tags along with other gapless/RG/... if they were not calculated by FB.

e.g. I've used FB for all of my replaygain info, so I want to ensure consistency in all my tags by using the same "converters/calculators".

also RE: Why exactly can't Foobar simply display **all** existing Tags via Properties dialog?
Sorry, my lightbulb finally went on and I realized that the tags need to be coded in FB parsing before they can be displayed - hopefully the images above help identify some discrepancies.
Title: Re: Tag Sanitizer component
Post by: q-stankovic on 2017-04-11 21:11:15
I am not sure if i understand the component: What fields are removed?
Title: Re: Tag Sanitizer component
Post by: marc2003 on 2017-04-11 21:57:09
You see this dialog before it runs...

(https://i.imgur.com/InlyJBZ.png)
Title: Re: Tag Sanitizer component
Post by: Peter on 2017-04-12 06:02:25
Regarding MP4 fields-
Everything works exactly as intended.
For optimal performance you want to upgrade foobar2000 to version 1.3.15 which contains latest & greatest MP4 tag manipulation fixes - hence the release of this component right after 1.3.15 stable came out.

I'll update the component description to mention this.

Also, no, we're not going to remove iTunSMPB, it would make the file impossible to decode correctly it cannot be trivially recreated.
Title: Re: Tag Sanitizer component
Post by: davideleo on 2017-06-04 17:19:39
Is this different than manually deleting fields in the properties dialog?
Title: Re: Tag Sanitizer component
Post by: Coreda on 2017-06-05 03:35:30
This would be great for the transcoding (conversion) options. Often wanted to only retain certain tags in the output during the operation. Problem is the output in my case isn't monitored by fb2k, since it's not intended for the main library, so it's a longer process to strip the tags.
Title: Re: Tag Sanitizer component
Post by: Case on 2017-06-05 07:14:33
Is this different than manually deleting fields in the properties dialog?
Yes, this removes fields that the properties dialog doesn't show. If you for example examine iTunes purchase run through this you should see how much extra data was wiped.

This would be great for the transcoding (conversion) options. Often wanted to only retain certain tags in the output during the operation. Problem is the output in my case isn't monitored by fb2k, since it's not intended for the main library, so it's a longer process to strip the tags.
You can hit Ctrl-A in the converter output dialog to select all files and run the command that way.
Title: Re: Tag Sanitizer component
Post by: Porcus on 2017-06-05 10:51:53
Is it true that fb2k can read "everything" APE, ID3 and Vorbis, right? So for those tag types, this is no different than using the Properties, is that true? (Or does it delete info tags as well for those, cf. the last item below? Forgot to check, editing ... )

* A feature request: Import exceptions tag list from library. Meaning, click to auto-populate the list of exceptions with the tags that fb2k see in the library files.

* Another: if front cover present, export it to <filename>.jpg (or more accurately, <filename>.<fileextension> in the same directory) prior to deleting.

* A quirk: It seems that if I have the Explorer window with the directory open, then Explorer starts updating itself and reading the tags, and I do often get the error message that the file is already in use. I cannot tell for sure that this is the cause.

* So it seems that for some .m4a files, fb2k can read the tool tag (AtomicParsley: "©too") and display it under %__tool%; others it cannot.  No way to tell the difference with AtomicParsley, which when I tested, reports for both files
Atom "©too" contains: iTunes v5.0.0.35, QuickTime 7.0.2
That arguably defeats the purpose of what I was going to post as a wishlist item:
Also keep info tags. (Listing __tool or tool does not help.)
Title: Re: Tag Sanitizer component
Post by: Porcus on 2017-06-05 17:22:13
By the way, is "pgap" atom that COMMENT ITUNPGAP? If so, then fb2k cannot always find it. And maybe it is needed if "true"? (Having given Apple the middle finger long time ago, I only have a very few iTunes encodes and cannot find anything with a pgap other than "false" in my library.)
Title: Re: Tag Sanitizer component
Post by: Porcus on 2017-08-05 18:57:01
Played around with it a bit, and found this, which could possibly be an issue for gapless playback, I do not know (I never found it in anything where I have more than a single track).  This one had Tool: iTunes 9.1.1.11, QuickTime 7.6.6, and sanitizing tags seemingly made it half a second longer:

Code: [Select]
Length mismatch : 5:33.553197 vs 5:33.601088, 14709696 vs 14711808 samples.
Compared 14709696 samples, discarded last 2112 samples from the longer file.
Differences found within the compared range: 29368448 values, starting at 0:00.346848, peak: 2.0140601 at 3:05.184921, 2ch
Detected offset as 2112 samples.

Comparing again with corrected offset...
Compared 14707584 samples, with offset of 2112 discarding last/first samples from total of 14709696.
No differences in decoded data found within the compared range.
Title: Re: Tag Sanitizer component
Post by: marc2003 on 2017-08-05 19:42:05
Check the ITUNSMPB values with mp3tag before and after?? I have no problems with any of my AAC files.

edit: assuming they actually are AAC files based on tool you mention???
Title: Re: Tag Sanitizer component
Post by: Porcus on 2017-08-06 11:40:46
No ITUNSMPB before nor after.
I think I have found a suspect, an atom called "chpl" that is unknown to AtomicParsley, but google brought me to https://www.adobe.com/content/dam/Adobe/en/devnet/flv/pdfs/video_file_format_spec_v10.pdf (and to https://lists.ffmpeg.org/pipermail/ffmpeg-cvslog/2014-August/080079.html ).

Comparing output using the -T + command, there are two atoms removed: @too and the chpl. From the original:
                Atom ©too @ 61297 of size: 56, ends @ 61353
                    Atom data @ 61305 of size: 48, ends @ 61353

This is inside the ilst atom inside the meta atom inside the udta atom inside mov atom.
Then a couple of lines below in the output:
        Atom chpl @ 495169 of size: 58, ends @ 495227 ~
This is inside the udta atom inside the mov atom - not in the ilist though.
Finally, the line
~ denotes an unknown atom
is gone when sanitizing. Not so strange.

(Also, some atoms are reduced in size and the atoms after the removed ones are relocated).

AtomicParsley with -t + reports just one of these in the original: the line
Atom "©too" contains: iTunes 9.1.1.11, QuickTime 7.6.6
is gone from the sanitized.


In case it is of any interest: if I use ffmpeg with -acodec copy -vn (the -vn removes the album art) then the chpl atom stays in, but AtomicParsley reports a tiny length difference: 333.601 becomes 333.602. But fb2k does not see any difference. And reports yet another length, 5:33.553.
Title: Re: Tag Sanitizer component
Post by: kode54 on 2017-08-07 04:49:40
Chapters are the Nero specification for encoding gapless information in MP4 files.
Title: Re: Tag Sanitizer component
Post by: Porcus on 2017-08-07 07:12:45
... and thus, should not be sanitized away, I guess?

(I actually prefer gapless information to stay in a tag that fb2k's "Properties" dialogue does not delete, rather than as COMMENT ITUNES BLAHBLAHBLAH - but I have no idea what applications support what.)
Title: Re: Tag Sanitizer component
Post by: cryptic99 on 2017-11-26 13:09:48
Thank you, Thank you Thank you.  I've been looking for something to remove the purchase tags for a while.  Worked great for me on the first try!
Title: Re: Tag Sanitizer component
Post by: Mrakobes on 2018-01-16 19:55:00
why the plugin created to clean up a littered file to FLAGS DCP??? me I'm very angry
Title: Re: Tag Sanitizer component
Post by: foxyshadis on 2018-03-05 10:56:10
Is it true that fb2k can read "everything" APE, ID3 and Vorbis, right? So for those tag types, this is no different than using the Properties, is that true? (Or does it delete info tags as well for those, cf. the last item below? Forgot to check, editing ... )]
There are a number of tags that are specifically coded to not be shown in the Properties page, with absolutely no way to make them visible. I have no idea why there's no way to see them or why it's such a state secret about exactly which tags, though album art and replay gain are definitely included. The only way to manipulate those hidden tags is a special component or a 3rd party program.
Title: Re: Tag Sanitizer component
Post by: Case on 2018-03-05 14:46:18
why the plugin created to clean up a littered file to FLAGS DCP??? me I'm very angry
What are you asking? I certainly don't understand what the question is.

Is it true that fb2k can read "everything" APE, ID3 and Vorbis, right? So for those tag types, this is no different than using the Properties, is that true? (Or does it delete info tags as well for those, cf. the last item below? Forgot to check, editing ... )]
There are a number of tags that are specifically coded to not be shown in the Properties page, with absolutely no way to make them visible. I have no idea why there's no way to see them or why it's such a state secret about exactly which tags, though album art and replay gain are definitely included. The only way to manipulate those hidden tags is a special component or a 3rd party program.
Special tags such as file sample lengths or gapless info are hidden to prevent people from breaking their files. Unsupported tags like ID3v2 rating that require special handling are hidden. ReplayGain is not shown together with your typical text-based tags because it isn't similar to those. Regular tags can contain anything user wishes and have no syntax to follow. ReplayGain tags have very specific rules. And the RG info isn't stored in tags with all formats.
Title: Re: Tag Sanitizer component
Post by: Porcus on 2018-12-10 13:58:27
Played a bit with it on my recently updated system. Every now and then it gives the error that file is in use. And then every now and then it wipes tags.

I have foo_external_tags installed, but took care not to test on files with external tags.

* A feature request: Import exceptions tag list from library. Meaning, click to auto-populate the list of exceptions with the tags that fb2k see in the library files.
Or a checkbox. Or even: a checkbox with "keep fb2k-displayable tags". Sanitize away what I cannot use.
Title: Re: Tag Sanitizer component
Post by: fuffi on 2019-06-23 05:01:53
Since this component was updated the last month, I post here.

I'd like to say, that it looks very 1990 ;-)

I like retro, but
it would be useful and more convenient to have a resizable (at least in length) window, for ppl who want to keep more than 10 tags and dont want so scroll throgh a single comma seperated row with tiny text.

Even more convenient, if there were checkboxes for each usergiven tag, where we can en-/disable several tags and do not need to delete/restore them for each run. (some ppl may want to delete different tags for flacs, mp3s or apes, etc, so typing all the tags again and again (aka restore them from a saved text-file) is kind of time-wasting. especially with this single smallish line to text.
(we're all getting older and sight may vary 8-)

The import feature (also mentioned in above posts) will be also very handy.

Question:
Will it delete all other embeded imgages than the front cover, when "preserve att. front cover" is checked?
Title: Re: Tag Sanitizer component
Post by: Warrex on 2019-12-28 23:25:30
I am trying to keep iTunNORM by adding it to the exeption list (tried iTunNORM, itunnorm and ITUNNORM) but Tag Sanitizer still removes it. What am I doing wrong?
Title: Re: Tag Sanitizer component
Post by: MusicHound on 2020-01-02 07:51:19
iTunNORM should work just make sure there's no spaces before or after the apostrophe.
Title: Re: Tag Sanitizer component
Post by: Peter on 2020-02-18 12:09:27
Regarding the 'retro' appearance,
I've given it some thought, but I don't want to drastically alter it from where it is.
The general idea of this component is to clean up newly downloaded files, retaining only tags that you keep in your library, remembering the last used config to apply again next time you download some album.

If the component imports a list of known tags from the library, it will seem as if it lists tags present in the selected files. Wrong!
If the component imports a list of known tags from the selected files... then it just clones the functionality of the Properties dialog, so what's the point?

Re embedded images-
Everything not specifically checked as preserved is discarded - including stuff that foobar2000 doesn't even show (and that's the point of this component), basically tags are discarded and then selected data is written back to fresh tags.
All embedded images are removed, and if "preserve front cover" is checked, just front cover is written to the fresh tags.

Re iTunNORM-
It seems there's currently no way to retain it, I'm looking into it.
Title: Re: Tag Sanitizer component
Post by: Johnny Blade on 2020-08-22 05:59:30
What is the difference between this component and the Masstagger "Remove all except for ..." function?
Title: Re: Tag Sanitizer component
Post by: Johnny Blade on 2020-09-21 14:39:56
No one??…

Well... After all this time doing tests, I came to the conclusion that the function integrated in Masstagger is reasonably better than Tag Sanitizer.

The Tag Sanitizer appears to be a more specific component with a few functions dedicated to the label it proposes, but in several directories containing entire discographies, it ended up removing more tags than I had stipulated, which caused me considerable headache to re-tag the affected files. Situation that I did not see happen with the function integrated to Masstagger.
Title: Re: Tag Sanitizer component
Post by: Shot2 on 2021-02-17 23:43:35
Hi,
thanks for the new foo_sanitizer 1.3 component.

When trying to extract the embedded front album art from audio files (AAC or FLAC), what kind of format is expected in the "Move front cover to external file" input box?
Whatever I try (e.g. %filename%.jpg, or folder.jpg, or full path to a filename) the operation fails with the message "Please specify a JPEG file name to save to". See screenshot here: https://ibb.co/fxyYfGV

Foobar2000 v1.6.4 (portable install), foo_sanitizer v1.3, Win10 x64.
Title: Re: Tag Sanitizer component
Post by: tordenflesk on 2021-02-23 17:08:26
Hi,
thanks for the new foo_sanitizer 1.3 component.

When trying to extract the embedded front album art from audio files (AAC or FLAC), what kind of format is expected in the "Move front cover to external file" input box?
Whatever I try (e.g. %filename%.jpg, or folder.jpg, or full path to a filename) the operation fails with the message "Please specify a JPEG file name to save to". See screenshot here: https://ibb.co/fxyYfGV

Foobar2000 v1.6.4 (portable install), foo_sanitizer v1.3, Win10 x64.
Same here
Title: Re: Tag Sanitizer component
Post by: Mikko_H on 2021-03-03 11:00:57
How about adding a results window or log file to track what changes were made?

Also would be nice to have an option which cleans space characters at the start and end of tags.
Title: Re: Tag Sanitizer component
Post by: Porcus on 2021-03-03 14:30:37
Also would be nice to have an option which cleans space characters at the start and end of tags.

You don't need the component for that. In a file's "Properties", right-click a field and select "clean up".

(I wish for fb2k to also allow a config to strip leading or trailing newlines, and to convert Unix newlines to Windows newlines - but that is outside this component.)
Title: Re: Tag Sanitizer component
Post by: Mikko_H on 2021-03-03 14:53:28
Wow I was not aware of that, thanks for the tip. Although it also converts underscores to spaces which is not really desirable for me.
Title: Re: Tag Sanitizer component
Post by: fuffi on 2021-03-29 01:37:00
Re embedded images-
Everything not specifically checked as preserved is discarded - including stuff that foobar2000 doesn't even show (and that's the point of this component), basically tags are discarded and then selected data is written back to fresh tags.
All embedded images are removed, and if "preserve front cover" is checked, just front cover is written to the fresh tags.
How can I tell this component to keep ARTIST, BACK and DISC images?


Regarding the 'retro' appearance,
I've given it some thought, but I don't want to drastically alter it from where it is.
Just an expanded window would be enough, preferably resizeable.
Its a pain to alter the tags I want to keep. (I use a lot of self-created tags to store remarks in my files, like: FUFFYTAG_THIS_IS_ONE_OF_MY_FAVORITES or FUFFYTAG_GREAT_ALBUM or FUFFYTAG_GENRE_KARNEVAL)

If the component imports a list of known tags from the selected files... then it just clones the functionality of the Properties dialog, so what's the point?
hmmm...IMHO The point was, that the Properties dialog won't remove all those hidden tags not shown in the properties dialog. So that dialogue can't be used to remove all those tags.
I not only santizite my own collection. Also friends and foes/customers/clients ones. Therefore, other settings/tags are needed to keep.

Title: Re: Tag Sanitizer component
Post by: NoToneS on 2021-07-27 18:13:19
My current settings
title,artist,album artist,album,tracknumber,discnumber,date,lyrics,unsyncedlyrics

Has this component broke, I've used it for a long time and am pretty sure it used to work, now even with above line stripped to just "album artist" it still removes the album artist tag %album artist%
Title: Re: Tag Sanitizer component
Post by: sveakul on 2021-07-28 04:20:37
Try "Masstagger" instead (https://www.foobar2000.org/components/view/foo_masstag (https://www.foobar2000.org/components/view/foo_masstag)) :

(https://i.imgur.com/ei57mRO.png)
Title: Re: Tag Sanitizer component
Post by: oberon0470 on 2022-09-24 15:50:52
Is there anyway to preserve the last modified date when sanitizing tags. I'm using v2 beta 10 and have 'Preserve file creation/access/modification...' checked in preferences. It changes the last modified date on all files sanitized even on ones that should not have needed any changes. I have the same behavior in v1.6.12.
Title: Re: Tag Sanitizer component
Post by: Sue Dunham on 2022-11-20 22:06:59
Played a bit with it on my recently updated system. Every now and then it gives the error that file is in use. And then every now and then it wipes tags.

I'm having this issue too; it'll give an error message like "Error processing file: [filepath] File is already in use", and the tags will all be wiped. This happens if I'm processing a very large number of files, like, well, all of them.

Now I'm using foobar2000 v2 beta 13, but I had this issue before too with the standalone component in v 1.6.12.