Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 Frames, Splitting, VBR, and the Bit Bucket (Read 5798 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 Frames, Splitting, VBR, and the Bit Bucket

Hi guys,

I've been browsing this forum and I can't find quite the information I need for a project I am working on.

I want to embed an MP3 file splitter into an audio app that I am writing. I found the source for mp3splt-1.9-win32 on SourceForge, and after days of searching for sourcecode, this has been the only app really helpful to me in learning how to do splitting. Although the compiled EXE works fine, it really has too many features, so I decided to spin off and do my own proggie sorta based on it.

I have been reading The Private Life of MP3 Frames and this has been instrumental in my understanding of the structure of a frame. Based on this and previous coding experience I have, I feel I will have no trouble writing my split engine for CBR-encoded files.

On the other hand, VBR-encoded files might give me some trouble. My goal is to split files into clips of a fixed time-length, such as 10 seconds or 15, etc. CBR will be easy; I just look at the headers, do the math to calculate the frame lengths, and hop and skip through the file. Once the number of examined frames add up to the desired time span, copy the data to a standalone MP3 file.

Theoretically, this should also work for VBR right? Maybe not - somewhere in my web surfing (dammit I can't find the page now), I read that some VBR models store data in a "bit reservoir". It's my understanding that this consists of unused space in previous frames, which get filled up with overflow data from the current frame - something like when a WinRAR archive is compressed using its "Solid Archive" setting.

My problem is that this whole Bit Bucket (I've taken to calling it that cuz "reservoir" it too hard to type) would seem to make it impossible to cleanly split a VBR file. This is because the audio data is smeared across multiple frames.

And then there is this "ABR" encoding model, which I did not even know existed until about an hour ago, and I have no idea whatsoever how the frames of these types of files are set up, and what splitting would do to them.

So how does one cleanly split VBR and ABR files?

A better question to ask might be: Can they be cleanly split?

I also read there are a couple of mutually-exclusive additional header data frames from Xing and FHG that contain information about the VBR data. Once I started getting into that I realized this may or may not be information that I needed to know, and decided to look for some help here.

Can anyone shed some light on this for me, or point me to some newbie-to-moderate explanation of correct file splitting techniques....??

Also, something else I have been forced to consider is this idea of "wrapped" files. I am understanding these to be multiple MP3 files that are just all appended together into one larger file, with the ID3 data sandwiched inbetween the smaller files, so that upon examination of the wrapped file, it appears there is ID3 data appearing at various points in the middle of the file, correspoding to where the "seams" between the smaller files are. How in heck do I deal with this...???

Thanks...!
Back off, or I'll flick a booger on your windshield!

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #1 –
The bit reservoir is used more extensively with CBR encodings, so clean splitting will be difficult with every format 

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #2 –
1. The bit reservoir is used in ABR, CBR and VBR files. CBR files use it most.
2. ABR means Average Bit Rate - it is actually VBR. If you encode a file in "normal" VBR mode, the bit rate can vary from 32 kbps to 320 kbps based on the sound. When using ABR encoding, you specify the average bit rate to be 192 for example, and the encoder will produce files with a bit rate between 128 and 256 for example (the bit rate allocation spectrum is not so high). ABR is used where high quality is wanted but the file size should be somehow predictable.

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #3 –
Now I've got a question... Let us assume a program can handle only MPEG 1 Layer III files. In that case, it will know that a frame has 1152 bytes. If the program starts looking for a valid frame (by seeking for 0xFFF) and founds one, it will assume that the next frame will start 1153 bytes later. Problem: this isn't always the case, right? I mean, if the file uses the bit reservoir, the frames won't always have 1152 bytes, right? How could I detect the length of a frame then? Would I need to search for valid frames bytewise?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #4 –
The frame size can be calculated by looking at the frame header (you should use both the bitrate value and the padding value).

Now, you have the problem related to bit reservoir.
When bit reservoir is used, the main_data_begin value is indicating where the data relevant for the current frame is starting in the previous frames. Those data are stored in ancillary data of previous frames.

To properly split, you have to put data relevant to the current frame but stored in previous frame back into the current frame. To do this, you should first increase the size of the current frame.

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #5 –
Quote
To properly split, you have to put data relevant to the current frame but stored in previous frame back into the current frame. To do this, you should first increase the size of the current frame.

Can it be done, if the current frame is already 320kbps ?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #6 –
If the current frame is already 320kbps, you have several options:

*It is already including data relevant to the next frame. In this case, you could also increase the next frame's size to put back the next-frame-data into the next frame.

*You could appen a silent frame before the current frame, in order to be able to put data relevant to current frame into the first (silent) frame. This is perhaps the easiest way do deal with bit reservoir.

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #7 –
Quote
The frame size can be calculated by looking at the frame header (you should use both the bitrate value and the padding value).

Yes, I know that. If I have a stereo CBR MPEG 1 Layer III file with 160 kbps and a sampling rate of 44.100 Hz, all frames will have the same size when using my calculation method: 523 bytes (when padding was used). The formula looks like this:

FrameSize = Fix(Coefficient * BitRate * 1000 / SamplingRate) + Padding
FrameSize = Fix(144 * 160 * 1000 / 44100) + 1

(Fix will convert a floating point value to an integer value, Coefficient is a number based on the MPEG version and MPEG layer, BitRate is the bit rate, SamplingRate is the sampling rate and Padding is either 1 or 0)

Now, the formula above does not include the bit reservior. If I want to skip to the next frame, would I need the bit reservior value?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #8 –
Quote
To properly split, you have to put data relevant to the current frame but stored in previous frame back into the current frame. To do this, you should first increase the size of the current frame.

Ok now... Thanks Gabriel! That gave me what I needed to know for the next step in understanding this!

I just thought of something that would clarify the problem: What I think actually needs to happen is that any data in the bit buckets of the current frame AND previous frames that contains data for upcoming frames needs to be removed or ignored. In essence, I need to "wind things down" in preparation for the cut.

Once the bit bucket data for upcoming frames is identified and removed from the current AND previous frames, and after those frames are properly resized, cut the file at the end of the current frame. Then - I think - this would qualify as a clean cut, yes?

So...

1) How do I identify bit bucket data for upcoming frames?

(also - is the bit bucket really used for anything else but upcoming frames?)

2) Once the appropriate data is identified, how do I remove it and properly resize the frames - or - how do I ignore it when copying the rest of the data to construct the ending frames in the new MP3 file...?

I am figuring that depending on how many frames the data is smeared across, I will have to process multiple frames to complete the process of "winding down".

Keep in mind I am not looking for source code (unless someone just happens to have some they want to show me), but the actual math behind this process which I describe. For instance, what bytes in the header contain the information about the bit bucket, and what format is it in? What multiplcation factors need to be used to calculate sizes of things...? Is it big-endian or little-endian and all that...

I understand the basic stuff such as FrameSize = 144 * BitRate / SampleRate, but what I will need to know is the equations to analyze and manipulate the bit bucket so I can remove or ignore that data.

Thanks guys!

Damn! I never thought I'd find a forum where people would know the answers to this kind of stuff...!!! You guys rock! B)
Back off, or I'll flick a booger on your windshield!

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #9 –
When bit reservoir is used, the data is stored into ancillary data of previous frames. As ancillary data is part of the frame, using ancillary data or no do not change the frame size (except in mpeg-2 MC it seems).

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #10 –
Quote
When bit reservoir is used, the data is stored into ancillary data of previous frames. As ancillary data is part of the frame, using ancillary data or no do not change the frame size (except in mpeg-2 MC it seems).

OK, that explains everything. Thanks Gabriel!

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #11 –
OK, there is one last problem... The maximum value for the bit reservoir is 512 bytes... Where do these 512 bytes come from?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #13 –
Quote
When bit reservoir is used, the data is stored into ancillary data of previous frames. As ancillary data is part of the frame, using ancillary data or no do not change the frame size (except in mpeg-2 MC it seems).

Ok, that statement makes me want to step back and ask a different question...

Am I making this too complicated?

Is it enough to simply split the MP3 file at the proper boundry bewteen two frames...? (and just not be concerned about unused ancillary data in the last few frames before the split?)

Or will this cause MP3 players to throw errors if there is "orphaned data" in those frames...?
Back off, or I'll flick a booger on your windshield!

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #14 –
Quote
OK, there is one last problem... The maximum value for the bit reservoir is 512 bytes

Er, just to be clear, is the max. value of the bit reservoir really 512 bytes? That's very surprising to me.

EDIT: actually I suppose it's not... just funny seeing it in writing and speaking in terms of bytes

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #15 –
Quote
Er, just to be clear, is the max. value of the bit reservoir really 512 bytes? That's very surprising to me.

In the ISO standard it is defined as 511 slots. (the value 0 is used when bit reservoir is not used)
An in case of Layer III, a slot is 1byte.



Quote
Is it enough to simply split the MP3 file at the proper boundry bewteen two frames...? (and just not be concerned about unused ancillary data in the last few frames before the split?)

The useless ancillary data in the first part of your cut will be ignored by players, so you can keep it here.
But you still have to put relevant data from the bit reservoir back into the second part, or you will loose the ability to decode a few frames at the beginning of second part.

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #16 –
Quote
The useless ancillary data in the first part of your cut will be ignored by players, so you can keep it here.
But you still have to put relevant data from the bit reservoir back into the second part, or you will loose the ability to decode a few frames at the beginning of second part.

Ahh... Ok thanks. Right now, I am only worried about the data before the split, as I am only creating "preview clips" of the MP3 source file and not actually cutting it up.

However, for my reference in case I want to actually start making cuts, can you give me a quick explanation of how to collect the anciallary data before a cut and "pre-pend" it to the frames after a cut?

I will of course seach the internet for this info, but if you can summarize it for me, or point me to any particular web pages you think would be helpful, it would be great.
Back off, or I'll flick a booger on your windshield!

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #17 –
Quote
I will of course seach the internet for this info, but if you can summarize it for me, or point me to any particular web pages you think would be helpful, it would be great.

You have a copy of the ISO standard, don't you?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #18 –
Quote
Quote
I will of course seach the internet for this info, but if you can summarize it for me, or point me to any particular web pages you think would be helpful, it would be great.

You have a copy of the ISO standard, don't you?

Do you have one, Gabriel?

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #19 –
Yes I have.
 

MP3 Frames, Splitting, VBR, and the Bit Bucket

Reply #21 –
Quote
You have a copy of the ISO standard, don't you?

No sorry, but I do not... if i google for MP3 and ISO, i get a zillion links to crack & warez sites...

can you reply with a link to the proper document for me?

thanks
Back off, or I'll flick a booger on your windshield!