Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Frame length magic (Read 6572 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Frame length magic

MP3 frame length is usually calculated with a formula like this:

    C * bit_rate / sample_rate + padding ,

with C being some constant value. So the variables entering the equation are bit rate, sample rate and padding (which IMO serves the purpose of distributing bit rate over an integral number of frames). While it seems perfectly reasonable to calculate the size of audio data blocks this way, I'm perplexed that it also accomodates the frame header (constant 4 bytes) and even the _optional_ CRC (constant 2 bytes). How on earth is that possible?

Out of curiosity,
Andreas

Frame length magic

Reply #2 –
bitrate includes mp3 data + overhead due to header and CRC

Frame length magic

Reply #3 –
Quote from: benski on
bitrate includes mp3 data + overhead due to header and CRC


Thanks, of course you're right. Viewing the whole format as raw data stream, the bit rate should apply to the whole stream, not just the actual audio data. But still I don't fully understand. In case there's no CRC, does the encoder insert two fill bytes?

Or are there just more frames with CRC, assuming the same input data and compression settings?

Frame length magic

Reply #4 –
Quote from: spambouncer on
Quote from: benski on

bitrate includes mp3 data + overhead due to header and CRC


Thanks, of course you're right. Viewing the whole format as raw data stream, the bit rate should apply to the whole stream, not just the actual audio data. But still I don't fully understand. In case there's no CRC, does the encoder insert two fill bytes?

Or are there just more frames with CRC, assuming the same input data and compression settings?

The latter. Use of CRC is discouraged because 1) no player (so far as I know) uses it and 2) it theoretically decreases audio quality (cbr) or increases size with no quality improvement (vbr).

Frame length magic

Reply #5 –
If CRC is used, there will be less space for the audio data, but the stream rate will be the same, as simple as that.

Frame length magic

Reply #6 –
Aw, you guys are confusing me. What is it now - more compression or more frames? I skimmed the ISO, but couldn't retrieve much info from it, short of diving right into the audio data syntax. So I encoded a short wave (29444 bytes, 44kHz) to 128kbps/44kHz with the LAME ACM.

Resulting file sizes:
With CRC 2016 bytes.
W/o CRC 2018 bytes.

Two bytes shorter with CRC! Now what do you make of that? 

Excited,
Andreas

Frame length magic

Reply #7 –
*your test sample is way too small to be able to conclude anything
*if you are using an ACM filter, then the data has to be further encapsulated into a wav container, so it's hard to conclude anything.

Frame length magic

Reply #8 –
Hi Sebastian

I recognise your name -  from xtremevbtalk, of course!

My guess is this: for each frame, the encoder determines the minimum size in bytes, MINS, required to encode the next set (1152) of samples (given the requested quality options, etc).

Then it adds 4 or 6 to allow for the header and CRC if requested.

It then selects the minimum bitrate that corresponds to a frame of at least this size, and encodes it at that rate.

The encoding algorithms never seem to waste space, so the final encoding will always fill the frame - I guess further that when there is room,  extra sample data that is "locally redundant",  that is, not essential for the required setting, is added in.  In other words, the "quality" of the frame is improved when there is room.

This lovely little theory doesn't quite sit with reality, however.  I encoded a wav file to mp3 VBR with LAME 3.97, with and without CRC, with highest quality options (-V0 -q0), and then stripped off any LAME-tagged frames.

Both files have 8435 frames, of which 2471 have different size in the two files. I'd expect (assuming the theory) these 2471 frames to all be 1 "size" bigger in the CRC version, corresponding to 2471 frames where the local redundancy measure didn't allow the 2-byte CRC to fit.

Not so!  Of those 2471 frames, only 1315 are bigger in the CRC version, while the other 1156 are bigger in the non-CRC version.

Here's an extract (front and back) from a listing of comparative frame sizes when they differ. The "net diff" is the difference between the total frame sizes to that point, a negative value means the CRC version is bigger.

Note that there are several points where the CRC version would be smaller than the non-CRC version, the last such occurrence being frame 282 (the CRC file ends up about 15k bytes longer). 

So it is definitely true that an extract from a non-CRC file can be  longer than in the CRC version.

The obvious question now is, given that frames are meant to be more or less independent (ie can be copied and pasted together arbitrarily), why do so many nonCRC frames get shortened in the CRC version?   

Code: [Select]
Frame    Frame Size    Net
 no.    noCRC   CRC   diff
--------------------------
   22    -522  +626   -104
   23    +626  -522      0
   24    -522  +626   -104
   25    +626  -522      0
   26    -522  +626   -104

   28    +626  -522      0

   31    -522  +626   -104
   32    +626  -522      0
   33    -522  +626   -104
   34    +626  -522      0

   36    -522  +626   -104
   37    +626  -522      0
   38    -522  +626   -104

   42    +626  -522      0
   43    -522  +626   -104

   47    +626  -522      0
   48    -522  +626   -104
   49    +626  -522      0
   50    -522  +626   -104

   70    -626  +731   -209
   71    +731  -626   -104
   72    -626  +731   -209
   73    +731  -626   -104

   76    +731  -626     +1
   77    -626  +731   -104
   78    +731  -626     +1

   82    -626  +731   -104
   83    +731  -626     +1
   84    -626  +731   -104
   85    +731  -626     +1

   87    -626  +731   -104
   88    +731  -626     +1
   89    -731  +835   -103
   90    +731  -626     +2

   93    -835 +1044   -207
   94    +731  -626   -102
   95    +731  -626     +3

  104    +835  -731   +107

  106    -626  +731     +2
  107    +731  -626   +107
  108    -626  +731     +2

  112    -731  +835   -102
  113    +731  -626     +3
  114    -731  +835   -101

  116    +731  -626     +4
  117    -626  +731   -101
  118    +731  -626     +4
  119    -626  +731   -101

  136    -626  +731   -206
  137    +731  -626   -101
  138    -626  +731   -206
  139    +731  -626   -101

  141    -626  +731   -206
  142    +731  -626   -101

  155    -835 +1044   -310
  156    +731  -626   -205
  157    +731  -626   -100

  171    -731  +835   -204
  172   +1044  -835     +5
  173    -626  +731   -100

  175    -626  +731   -205
  176    +731  -626   -100

  181    -626  +731   -205
  182    +731  -626   -100

  205    +731  -626     +5
  206    -626  +731   -100
  207    +731  -626     +5
  208    -626  +731   -100

  212    +835  -731     +4

  221    -626  +731   -101
  222    +731  -626     +4

  225    +731  -626   +109
  226    -626  +731     +4

  241    -626  +731   -101
  242    +835  -731     +3

  246    -835 +1044   -206
  247   +1044  -835     +3

  249    -626  +731   -102

  251    +731  -626     +3
  252    -626  +731   -102

  281    -626  +731   -207
  282   +1044  -835     +2   (Last time noCRC > CRC)
  283    -835 +1044   -207
  284    +731  -626   -102
 ...
 ...
 8386    +626  -522 -15338
 8387    -417  +522 -15443

 8394    +522  -417 -15338
 8395    -417  +522 -15443

 8399    -417  +522 -15548
 8400    +522  -417 -15443

 8404    -417  +522 -15548
 8405    +522  -417 -15443

 8408    -417  +522 -15548
 8409    +522  -417 -15443
 8410    -417  +522 -15548
 8411    +522  -365 -15391
 8412    -365  +417 -15443
 8413    -365  +417 -15495
 8414    +522  -417 -15390

 8416    -365  +522 -15547
 8417    +522  -365 -15390
 8418    -365  +417 -15442
 8419    -365  +417 -15494

 8431    -313  +365 -15546
 8432    +365  -313 -15494
 8433    -313  +365 -15546


BTW, the "+" and "-" signs for the frame sizes are just there to indicate which one is bigger.

Frame length magic

Reply #9 –
Out of interest, how did you calculate the frame size? Did you use the formula above and extract the bitrate from each frame's header? Did you also take the padding into consideration? What did you use for C?

Frame length magic

Reply #10 –
Yes, yes, and 144 = 1152/8 

Frame length magic

Reply #11 –
I think that you're beeing fooled by the frame padding that has to sometimes add 1byte to some of the frames to keep. You'd better try with a different sampling rate like 48kHz.
And of course, it you don't encode in CBR you have no way to conclude anything about the impact of CRC.

Frame length magic

Reply #12 –
I can see how padding bytes give the "net" difference rates of +1, +2 etc, in my table, but not why corresponding frames would be a bitrate "size" larger in the non-CRC version.

I was testing on VBR-encoded files, and should of course also look at CBR.  But are you saying CRC is not applicable to VBR encoding?  LAME help lists it as a general option, irrespective of CBR/VBR, so I assumed it was.  It's certainly provided some interesting results!

Cheers

Frame length magic

Reply #13 –
Of course CRC works with VBR.
However, with VBR you are specifying a quality level and not a size, so if you enable CRC, the encoder will OVERALL increase bitrate to keep a similar quality level. Then, because of a feature called "bit reservoir", some bits might be shifted between adjacent frames, so it's quite pointless to look at the size of individual frames within an MP3 VBR file.

Frame length magic

Reply #14 –
Ha ha, my entire life (all 55 years of it) has been pointless, according to several of my ex-partners

Bit-reservoiring (can reservoir really be a verb?) does tend to muddy the waters, I agree.

After re-encoding with --nores, the 2 files are indeed much closer to predicted state - frame counts agree (more frames now,  8539), only 148 corrresponding frames have different sizes,  145 of those are one size up in the CRC version.

And, curiously, still 3 frames unexpectedly bigger in non-CRC version - perhaps there's a still a bit of unauthorised bit-reservoiring going on?
 

Frame length magic

Reply #15 –
Thanks for going through all that trouble, Mathimagics. (I'm talking about the MP3-dissecting, not your ex-partners!    )

@ Gabriel: I thought having small files might help single out differences faster. The riff headers shouldn't be a problem, they always have the same size (58 bytes).

So here are my findings, looking at the files with a hex editor: Both files have the same number of frames (5) with equal sizes (417 for the first one, 418 for the rest). Except for the last frame, which has 288 bytes in the non-crc version and 286 in the crc version. Puzzling, huh?