Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: The evilness of APEv2 :) (Read 10314 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

The evilness of APEv2 :)

Open them in fb2k - you'll see that they are correct APE tags.
Due to the APE tag uses ID3v1 "TAG" signature in their header "APETAGEX", there are some cases, in that the program that supports ID3v1 WILL think that file have ID3v1, when actually no. <open in any prog except fb2k>

This prog WILL corrupt APE tag on editing.

The second case occures when the tag frames contains "TAG" combination.

So, APE cannot be used without ID3v1, if not backward compatible to pre-existing tagging scheme.

I think what this fault is because APE was NOT designed specially for MP3.

Now i investigating if it can cause false syncs...
[attachment=1384:attachment]
[attachment=1385:attachment]

The evilness of APEv2 :)

Reply #1
Just to clarify - an ape2 tag will not always give these results, only if the apeTAG part is 128 bytes from the end, it could be possible to write built in padding into an ape2 tag to stop this from ever happening, but then again said older programs will not read any tags. Adding an ID3v1 tag after the ape2 tag will give backwards compatibility.

I am all for APEv2 tags in mp3 files, and agree with others 100% who have implemented it, this problem only comes about because of older software that has not caught up with this new standard, unfortunately mp3 is getting long in the tooth and companies / developers are lazy to implement it.

The evilness of APEv2 :)

Reply #2
Quote
it could be possible to write built in padding into an ape2 tag to stop this from ever happening,
[a href="index.php?act=findpost&pid=280547"][{POST_SNAPBACK}][/a]


It is, but one would have to add a key called 'PADDING' with zeroes for data or something like that. That would look bad in a tagger. APEv2 format doesn't have any special handling for padding, at least not to my knowledge.

The whole thing of the 'TAG' part of 'APETAGEX' being at exactly 128 bytes from the end of the file is very improbable, but it's still possible. I never even thought of that, but thanks to this thread, I've updated the tag scanning code of my tagging library to take it into account. I'm glad that this thing came up


The evilness of APEv2 :)

Reply #4
Quote
It is, but one would have to add a key called 'PADDING' with zeroes for data or something like that. That would look bad in a tagger. APEv2 format doesn't have any special handling for padding, at least not to my knowledge.


Not necessarily, a tag value in APE2 is:

4 byte size,
4 byte flag
ELEMENT\0TAGVAL

for older ape1 there was a \0 on the end of TAGVAL (off the top of my head), on the special case that the 'tag' as 128 bytes back you could add a null (or extra null for ape1), set the size to accomodate that extra null, and it shouldn't cause any problems.

The evilness of APEv2 :)

Reply #5
Quote
I am all for APEv2 tags in mp3 files, and agree with others 100% who have implemented it, this problem only comes about because of older software that has not caught up with this new standard, unfortunately mp3 is getting long in the tooth and companies / developers are lazy to implement it.


If somebody develops a somethins new, he must throughtly analyse the interaction of new technology with the existing ones, and be sute that it AT LEAST not to corrupt any previous. I.e. adding new tagging scheme would take this essential steps:

0. Think if it can be done using pre-existing technologies only (fully backward compatible). so the old tag will be readable by new programs, and the older programs will read new tag.
1. Think if it can be done in forward compatible way. (addind tracknum in ID3v1.1 tags), so the old tag will be readable by new programs, and the older programs will read new tag except for new feature added.
2. ...If two previous are impossible or inappropiate, then new tag should at least not to interfere with the old tagging scheme in any way, so the any program with proper old technology support can read or write old tag perfectly, without any knowledge of new tagging scheme.

Current inplementation of APEv2 breaks the ID3v1 TAG detection mechanism. ID3v1 is the most widely supported and used tagging scheme for MP3. So in no way new tags MUST NOT affect the programs supporting only ID3v1 (Rule#2)
I.e. old programs should not need for recompile to work with (file with both versions of tags) flawlessly.

Examples:
ID3v1 is forward compatible to MP3
ID3v1.1 is forward compatible to ID3v1.0
ID3v2 does not interfere with ID3v1.x
[one from beginning, other from end]
ID3v2 is forward compatible to MP3
[because it uses syncsafe integers (and) unsynchronisation scheme to prevent false syncs which could interfere with old MP3 decoders. MP3 standard allows arbitrary data _without_sync_patterns_ before and after the audio frames AFAIK less than ~8kb]
APEv2 _interferes_ with ID3v1
[because informal standard for ID3v1 says that if three bytes from 127byte from the end of mp3 are "TAG", file have an ID3v1 tag. APE does nothing to prevent this]

I dont know if APE is forward compatible to MP3 by my definitions
[some string length could produce sync patterns, which is not allowed by MP3 standard] ))

IMO it is what about developers should think, and not about the complexity... One could not just take tag format X from audio format Y, and apply it to audio format Z...

The evilness of APEv2 :)

Reply #6
When you need syncsafe encoding and you rely on sync patterns for things to work properly, you know you have a badly designed standard.

For all tags to be compatible with each another, there must be some "skip these N bytes" specification, which allows older players to reliably skip unsupported tagging schemes. Anything else is a hack.

Look at what's in MP4 or Matroska files, solid containers, storing data into atoms, now *that* is proper design: older players can skip what they don't support.

A well-designed tagging scheme like APEv2 is useless, if the rest of the file was never designed for tagging to begin with.

The audio players/tag editors are forced to rely on probabilities and guesswork to distinguish between audio and tags..  so it's inevitable that some byte patterns will fool them.

The evilness of APEv2 :)

Reply #7
Quote
When you need syncsafe encoding and you rely on sync patterns for things to work properly, you know you have a badly designed standard.
[a href="index.php?act=findpost&pid=280722"][{POST_SNAPBACK}][/a]


Exactly - the majority of players only work with ID3v1 and ID3v2 because they know of their existance - I can name a few early flash based portable mp3 players that will not play an id3v2 tagged file, so how is that for forward compatibility - atleast with an ape2 on the end, worse case is tiny static with an older player (although very unlikely).

I gave an example of how to make ape2 100% non detectable to id3v1 programs - am I going to write it? no because if you are using ape2 in a mp3 file you are best to have APE2 then an ID3v1 tag.

 

The evilness of APEv2 :)

Reply #8
Quote
Look at what's in MP4 or Matroska files, solid containers, storing data into atoms, now *that* is proper design: older players can skip what they don't support.
[a href="index.php?act=findpost&pid=280722"][{POST_SNAPBACK}][/a]

Quote
When you need syncsafe encoding and you rely on sync patterns for things to work properly, you know you have a badly designed standard.
[a href="index.php?act=findpost&pid=280722"][{POST_SNAPBACK}][/a]

So, the MP3 is badly designed. If it have proper container as you said nobody needs such a things. But we have MP3, so we should use various workarouns to keep compatibility with _proper_ mp3 decoders what follows the standard. No _MP3_ tagging scheme will be elegant, easy and simple. If we use comletely new audio format - i'll probably vote for APE

Quote
For all tags to be compatible with each another, there must be some "skip these N bytes" specification, which allows older players to reliably skip unsupported tagging schemes. Anything else is a hack.
[a href="index.php?act=findpost&pid=280722"][{POST_SNAPBACK}][/a]

Every tagging scheme for MP3 unfortunately is a hack. But they works with mp3 decoders what follows the standard. Do not think that developers rush to patch their programs to even skip some bytes from something new. (in many cases it is hard or impossible, remember hardware mp3 players with DSP). This new things should be compatible to standard (not/and to them).

The evilness of APEv2 :)

Reply #9
Quote
I gave an example of how to make ape2 100% non detectable to id3v1 programs - am I going to write it? no because if you are using ape2 in a mp3 file you are best to have APE2 then an ID3v1 tag.

Too bad 
I dislike id3v1 because its encoding is not invariant relative to system codepage. Also i dislike ugly truncated strings. And if some program will use such crappy strings for something, say it update id3v2 or file names? Then i removed all my id3v1 (where were no warnings about that in my tagging program)

Now i fear if this situation appear in the retagging process, and i lose some of MP3Gain_undo which is saved in APE... Furthermore IMO it is almost inevitably, because of "APETAGEX", on medium sized music collection.

But dont you think that adding even a warning that APE must come with ID3v1 will be nice?

The evilness of APEv2 :)

Reply #10
Quote
Quote
I gave an example of how to make ape2 100% non detectable to id3v1 programs - am I going to write it? no because if you are using ape2 in a mp3 file you are best to have APE2 then an ID3v1 tag.

Too bad 
I dislike id3v1 because its encoding is not invariant relative to system codepage. Also i dislike ugly truncated strings. And if some program will use such crappy strings for something, say it update id3v2 or file names? Then i removed all my id3v1 (where were no warnings about that in my tagging program)

Now i fear if this situation appear in the retagging process, and i lose some of MP3Gain_undo which is saved in APE... Furthermore IMO it is almost inevitably, because of "APETAGEX", on medium sized music collection.

But dont you think that adding even a warning that APE must come with ID3v1 will be nice?
[a href="index.php?act=findpost&pid=280858"][{POST_SNAPBACK}][/a]


If you're using APEv2 tags, then obiously whatever software you're writing tags in supports APEv2 correctly, otherwise you would not have been able to write the APEv2 tags. 

The only way I could see this ever being a problem is if you're tried to update your tags with some program that did not support APEv2, in which case you would not be able to update your APEv2 tags anyway.  Therefore I do not think this is a problem.

The evilness of APEv2 :)

Reply #11
I do not use APE for tagging, just mp3gain uses them to store some info.
I'll use id3v2 for RG as soon it will be possible.