Skip to main content

Topic: MP3 repacker (Read 465626 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #25
Quote
@ominion

i dont have fobar her
but i did two diskwrites of the same mp3 as before (the original and not the repackwd)
and they came out bit identical

MD5
b526274595bd1b40fa744673c8b25f08 *first.wav
b526274595bd1b40fa744673c8b25f08 *Second.wav

CRC32
first.wav 75FDC377
Second.wav 75FDC377

So it definatly seems like the repack stuff is NOT lossless to the decoded stream
[a href="index.php?act=findpost&pid=283412"][{POST_SNAPBACK}][/a]

I figured it out. It's actually probably Winamp's fault. My program outputs a VBR stream, and so adds a VBR header at the front. This header is designed such that programs which don't support it will simply see an empty frame.

Winamp suppports the header, but it still plays the frame as an empty frame. Therefore, Winamp will add an extra 1152 null samples to a file with a VBR header. If the input file to mp3packer doesn't have a header, then Winamp will add an extra 1152 samples to the output file (which always has a header), thus making the MD5 different for the decoded files.

At least, I think that's how Winamp works. Foobar thinks my two test files are exactly the same, whereas Winamp outputs files with different MD5s.

In order to make sure, I took the two Winamp output files, deleted the WAV headers of the 2 (because the files have a different number of samples, which is encoded in the header) then deleted the first 1152 samples (4608 bytes) of the file from my program. The resulting files had the same MD5. That indicates that Winamp's output is the exact same, except for 1152 samples added to the front. If that makes any sense...
  • Last Edit: 19 March, 2005, 09:39:57 PM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • moozooh
  • [*][*][*][*]
MP3 repacker
Reply #26
Okay, version 0.02 works. But still with some issues…

Code: [Select]
C:\Files\progs\Lame stuff\mp3packer>perl -v

This is perl, v5.6.1 built for MSWin32-x86-multi-thread
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2001, Larry Wall

<…>

Code: [Select]
C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl ae01.mp3 ae01repacked.mp3
WARNING: Bit reservoir overflow on frame 8
WARNING: Bit reservoir overflow on frame 9
WARNING: Bit reservoir overflow on frame 10
WARNING: Bit reservoir overflow on frame 11
WARNING: Bit reservoir overflow on frame 12
WARNING: Bit reservoir overflow on frame 13
WARNING: Bit reservoir overflow on frame 14
WARNING: Bit reservoir overflow on frame 15
WARNING: Bit reservoir overflow on frame 16
On frame 18

ERROR! Found more data than could be encoded in one frame!
Frame number 17 uses 1214 bytes, of which 1170 bytes need to be stored in
the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 0.
(The additional reservoir must be from 0 to 511)

C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl -a 511 ae01.mp3 ae01repacked.mp3

Here it does massive BR overflow on almost every frame then halts and writes this:

Code: [Select]
ERROR! Found more data than could be encoded in one frame!
Frame number 3064 uses 1257 bytes, of which 1062 bytes need to be stored in the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 511.
(The additional reservoir must be from 0 to 511)

The file was 9:23 track encoded (not by me though) in 256 kbps cbr (lame 3.91). Encspot doesn't report any problems.
What do I do now?
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #27
Quote
Okay, version 0.02 works. But still with some issues…

Code: [Select]
C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl -a 511 ae01.mp3 ae01repacked.mp3

Here it does massive BR overflow on almost every frame then halts and writes this:

Code: [Select]
ERROR! Found more data than could be encoded in one frame!
Frame number 3064 uses 1257 bytes, of which 1062 bytes need to be stored in the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 511.
(The additional reservoir must be from 0 to 511)

The file was 9:23 track encoded (not by me though) in 256 kbps cbr (lame 3.91). Encspot doesn't report any problems.
What do I do now?
[a href="index.php?act=findpost&pid=283852"][{POST_SNAPBACK}][/a]


Try increasing the -p switch. It's 100 by default. Use "mp3packer -a 256 -p 200". If THAT doesn't work, then I'll need to do something about my frame-too-big handling. I'll start working on a better solution now... The problem is that, with many LAME encodes (especially CBR 256 -- you picked the worst one) the file might require at least n bytes in the reservoir, but my program tries to minimize the amount of bytes in the bit reservoir, so many times there's fewer than n bytes, so the program panics.

Encspot wouldn't notice any problems in the input file, simply because there really aren't any problems. It's just the way my program handles (or fails to handle) some files.

I'm pretty sure I can do some sort of "pre-pass" on the file, log all the frames which are larger than 320kbps, and try to only pad the reservoir right before them. I'll get to work on it, but I have no idea when it'll be done.
  • Last Edit: 20 March, 2005, 05:36:52 PM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • moozooh
  • [*][*][*][*]
MP3 repacker
Reply #28
It works now, but only with -p 250 or more, the result is just stripping a few kbps.
Edit: “A few” = 2-8 kbps (tested on 224-256-320 kbps files).
  • Last Edit: 22 March, 2005, 02:56:38 PM by Mo0zOoH
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

  • Jojo
  • [*][*][*][*][*]
MP3 repacker
Reply #29
I haven't had the chance yet to give it a try...but is it true that CBR files will end up as VBR files?
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #30
Quote
It works now, but only with -p 250 or more, the result is just stripping a few kbps.
Edit: “A few” = 2-8 kbps (tested on 224-256-320 kbps files).
[{POST_SNAPBACK}][/a]

Hmmm... Do you know what program made the files? In my experience with both LAME and the iTunes encoder, CBR 320 files go down to around 290. I can believe that 224 won't give much improvement, though. The lower the bitrate, the more the files actually need all of it.

Quote
I haven't had the chance yet to give it a try...but is it true that CBR files will end up as VBR files?
[a href="index.php?act=findpost&pid=284651"][{POST_SNAPBACK}][/a]

Yup. That's how it works. It uses the minimal size for each frame, which almost always results in a VBR file.  (Just like I said in post [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=32379&view=findpost&p=282515]#10[/url]  ) A valid LAME VBR header is written, so programs will handle it the same way as they do every VBR file, if that's what you're wondering.

By the way, I made a new version which intelligently figures out where padding needs to be placed, so you don't need to screw around with the -a and -p switches. The downside to this is it either has to read the file twice (goes almost half as fast) or store the entire file into memory (uses lots of memory) I'll post it once I finish testing it.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #31
Just released version 0.03. Now you no longer need to fiddle with the -a and -p switches (actually they aren't even supported any more). It shouldn't complain about not having enough bits in the bit reservoir again.

I do recommend that you use the -m switch, as it tends to go about twice as fast. It crams the entire file into memory, so it's not good for large files. For standard 1-song-per-file MP3s it's fairly safe to use -m.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • kevinsham
  • [*]
MP3 repacker
Reply #32
I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.

  • Lyx
  • [*][*][*][*][*]
MP3 repacker
Reply #33
@Omion

Would it be possible to add a parameter to the m-switch where you can specify a value in megabytes. If the file is larger than the specified value, then it will not be loaded into memory, if it is smaller or equal then it will be loaded into memory. This could be useful for batch-repacking in the future when your tool becomes stable.

- Lyx
I am arrogant and I can afford it because I deliver.

  • Antonski
  • [*][*][*]
MP3 repacker
Reply #34
Quote
I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.
[a href="index.php?act=findpost&pid=285369"][{POST_SNAPBACK}][/a]


I also got the same error when I tried to re-pack a new-Xing vbr file (according to EncSpot).
However other 9 cbr files (also Xing, old and new) were repacked without errors.

BTW, I don't think that Perl is not a "real" programming language 

  • djdance
  • [*]
MP3 repacker
Reply #35
Boys, please let me simply download .EXE without any compiling, well?
Would anybody likes to upload exe and put the link here?

  • kjoonlee
  • [*][*][*][*][*]
MP3 repacker
Reply #36
Um, you don't need a repacker binary. Use perl to run it.

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #37
Quote
Quote
I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.
[a href="index.php?act=findpost&pid=285369"][{POST_SNAPBACK}][/a]


I also got the same error when I tried to re-pack a new-Xing vbr file (according to EncSpot).
However other 9 cbr files (also Xing, old and new) were repacked without errors.

BTW, I don't think that Perl is not a "real" programming language 
[a href="index.php?act=findpost&pid=285522"][{POST_SNAPBACK}][/a]

THAT's not good. Looks like I've made more bugs than I've squashed with the lastest version. I may have missed something whilst re-reading the XING tag spec. Can either of you send, say, the first 1KB of the problem file to me? I've PMed you my email address.

Unfortunately, I don't know how most people cut up a file, so I can't tell you how to send the first 1KB, but I'm sure someone here does.  I use an old hex editor called "Hedit" which doesn't even exist any more, so it's not really of any use.

By the way, does this only happen with version 0.03, or does 0.02 do it too?
[edit:] Also, does the -m switch affect the problem? [/edit]

@Lyx:
Adding a cutoff value for the -m switch would not only be possible, it would be extremely easy and useful. I don't know why I didn't think of it before. I'll add it to the next version, along with the fix to this bug people are getting.
  • Last Edit: 26 March, 2005, 02:22:51 AM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • kjoonlee
  • [*][*][*][*][*]
MP3 repacker
Reply #38
For chopping, I use dd from GNU coreutils.

dd if=input.mp3 of=chopped_file bs=1000 count=1

edit: If you're running Windows, MinGW, Cygwin, or GnuWin32 should let you run coreutils.
  • Last Edit: 26 March, 2005, 04:00:46 AM by kjoonlee

  • moozooh
  • [*][*][*][*]
MP3 repacker
Reply #39
0.3 works good for me, thanks.
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

  • Antonski
  • [*][*][*]
MP3 repacker
Reply #40
Quote
Can either of you send, say, the first 1KB of the problem file to me? I've PMed you my email address.


Sure, I'll send it in Monday (that file is on my office desktop).
Meanwhile I'll try to reproduce the problem on my home PC (if my wife allow me to go closer to the comp  )

BTW, the problem could be with that file and not with your program. I even don't remember where I have downloaded it from.

Tony

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #41
Released version 0.04.

Hopefully the previous "substr outside of string" error has been fixed. Thanks, Antonski and kevinsham, for sending me the files that broke.

Also added a "-M" option, which has one argument. If the file size is smaller than the argument, in MiB, then the "-m" option is turned on. Otherwise, it's turned off.

There was another problem which may or may not have existed, but it definitely shouldn't exist any more. The calculation for the XING TOC was off by around 100 bytes, which may have caused the XING tag to overflow and corrupt the first frame on very small files.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • Antonski
  • [*][*][*]
MP3 repacker
Reply #42
Hi Reed,

I'm afraid I've encounered a new problem with repacker, this time workng on my old home sweet Win98SE. I have Active Perl 5.6.1.
After repacking of every file the program generates a message "Illegal division by zero at mp3.pm line 785." and there is no Xing header in the output file (as far as I can see).
There is no problem with Win2000 though.
If you are interested, I would send you the the headers of the original file and both outputs - correct and incorrect.

BTW, I really like that program a lot. And Perl

~Tony

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #43
@Antonski -
Hmm.... That's where the program divides by the size of the output file. Maybe win98 doesn't update the size of open filehandles or something. Try replacing your mp3.pm with this one. If that doesn't work, then I'll have to think of something else.
  • Last Edit: 16 April, 2005, 05:07:03 PM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

  • Antonski
  • [*][*][*]
MP3 repacker
Reply #44
Quote
Try replacing your mp3.pm with this one.


Works perfectly!
Thank you 

  • Deep_Elem
  • [*][*][*]
MP3 repacker
Reply #45
I've been tinkering with v.0.04 using the latest version of Perl on Win XP. I'm getting pretty good results, even with 192 kbps CBR LAME files. But I have noticed two issues:

1. My original LAME files have proper LAME tags, but the new files have no LAME tags (so I am adding them manually via fb2k).

2. Encspot now reports the encoder used as Xing or Gogo but the original files are definitely LAME.

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #46
1. That's weird. If the input files have LAME tags, the output files should too. Otherwise, a simple XING tag is added. What version of LAME do you have, and how did you check if the files have the tag? It's possible that my program thinks the tag is incorrectly formed, and so just stores a standard XING tag. (If it doesn't write an XING tag either, that's very bad)

2. Well, the audio data is the same, so it shouldn't matter. Encspot just sees that the files are arranged differently from what LAME usually does, and guesses something else. The quality is the same as the original. (Except if a LAME tag is written, in which case EncSpot parrots the encoder string)
  • Last Edit: 17 April, 2005, 07:33:36 PM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker
Reply #47
Sorry to resurrect this thread on my first post but I really do like MP3 Packer and had a few questions...

Omion I use your program on all my old --preset insane files to remove tags and in the process shave a few kbits off the filesize. Am I right in saying that it also strips off any "bad last frames" that it encounters (I think you confirmed this in an earlier post) ?

When I recently tried to repack a 192kbps CBR mp3 Encspot showed the outputted file as having 224, 256 or 320 kpbs bit rates for some frames. Is this possible ? Is the mechanism similar to:
Quote
If LAME decides a frame needs more then 320kbps, it will take the needed bits from the bit reservoir.... As of version 0.03, very large frames are identified early enough to pack the reservoir dynamically.

I sincerely hope you are still developing this great little proggie, batch processing and perhaps an executable and a GUI would be very welcome indeed  although I am perfecty content with the app in its current form (if anyone reading could compile MP3 Repacker into a stable binary or show me how to go about it I would be eternally grateful ). As it is I currently use Florian's amazing MP3Tag program with mp3repacker inserted as a tool like so:
Code: [Select]
 Name: MP3 Packer
Path: C:\WINDOWS\system32\cmd.exe
Parameter: /c "cd /d "E:\Backup\Programs\MP3 Packer\MP3 Packer v0.04" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%vbr.mp3""
with "for all selected files" ticked. Provided the user adjusts the absolute path to the two MP3 Packer files accordingly then it allows any number of mp3s in a directory to be packed to output files in the same folder but with vbr appended to the end of the filename by simply highlighting the mp3s, right clicking and selecting the MP3 Packer tool. This pseudo-batch processing comes in quite handy so I hope those instructions help anyone else in the same situation as me. Omion I was wondering if you could tell me if it's "safe" to run so many MP3 Packers concurrently ?

Lastly I was hoping you could tell me if you are aware of any bugs that would make running Mp3 Packer on all of my old unreplaceable mp3s a bad decision...

Thanks for all your hard work and effort 

--asonicboom

  • Omion
  • [*][*][*][*]
  • Developer
MP3 repacker
Reply #48
Quote
Sorry to resurrect this thread on my first post but I really do like MP3 Packer and had a few questions...

Omion I use your program on all my old --preset insane files to remove tags and in the process shave a few kbits off the filesize. Am I right in saying that it also strips off any "bad last frames" that it encounters (I think you confirmed this in an earlier post) ?
If a bad last frame is encountered, it is simply added to the "non-MP3 junk at the end" category. That means that if you use the -s option the bad frame will be thrown out, otherwise it will be included but not optimized.
Quote
When I recently tried to repack a 192kbps CBR mp3 Encspot showed the outputted file as having 224, 256 or 320 kpbs bit rates for some frames. Is this possible ? Is the mechanism similar to:
Quote
If LAME decides a frame needs more then 320kbps, it will take the needed bits from the bit reservoir.... As of version 0.03, very large frames are identified early enough to pack the reservoir dynamically.
It is possible, expected, and happens quite a lot  . Due to the bit reservoir, the amount of data stored in an MP3 frame has very little relation to the bitrate of that frame. For example, here's a graph of a 192-CBR file. The red is the size of each frame (constant, of course) but the black dots are the actual amount of data in each frame. The blue is how many bits are in the reservoir per frame.
although I am perfecty content with the app in its current form (if anyone reading could compile MP3 Repacker into a stable binary or show me how to go about it I would be eternally grateful ).[/quote]I've been trying to figure out how to make binaries from Perl, but my Perl doesn't seem to want to.

I don't have any experience or desire to make a GUI for the program, but if you want to, feel free  .
Quote
As it is I currently use Florian's amazing MP3Tag program with mp3repacker inserted as a tool like so:
Code: [Select]
 Name: MP3 Packer
Path: C:\WINDOWS\system32\cmd.exe
Parameter: /c "cd /d "E:\Backup\Programs\MP3 Packer\MP3 Packer v0.04" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%vbr.mp3""
with "for all selected files" ticked. Provided the user adjusts the absolute path to the two MP3 Packer files accordingly then it allows any number of mp3s in a directory to be packed to output files in the same folder but with vbr appended to the end of the filename by simply highlighting the mp3s, right clicking and selecting the MP3 Packer tool. This pseudo-batch processing comes in quite handy so I hope those instructions help anyone else in the same situation as me. Omion I was wondering if you could tell me if it's "safe" to run so many MP3 Packers concurrently ?
As long as you don't have them writing to the same file, running multiple repackers in parallel isn't a problem. (If you do have them writing to the same file, they won't corrupt it; they'll just exit with a "permission denied" error)
Quote
Lastly I was hoping you could tell me if you are aware of any bugs that would make running Mp3 Packer on all of my old unreplaceable mp3s a bad decision...
I don't know of any bugs in the program; it seems fairly stable. However, if you want to extra assurance, you can run the input and output files through Foobar's "bit-compare files". If that says they're OK, then you can delete the old one. If it says they're different, then post to this thread, as it would indicate a bug.

Quote
Thanks for all your hard work and effort 

--asonicboom
[a href="index.php?act=findpost&pid=297982"][{POST_SNAPBACK}][/a]
Thank you for the feedback! It's good to know that people are using my program.

[edit from 7 years in the future: fixed the images to point to my current server]
  • Last Edit: 02 August, 2012, 01:05:23 AM by Omion
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker
Reply #49
Thanks for replying so soon and so comprehensively Omion, your answers were very helpful  Again good work on making such a great little program.
Quote
If that says they're OK, then you can delete the old one. If it says they're different, then post to this thread, as it would indicate a bug.
Will do  Till then...

--asonicboom