Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Vorbis stream with no Ogg header (Read 19577 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Vorbis stream with no Ogg header

Hi!
Last night I broadcasted signal grabbed from my soundcard's line-in. I used OddCast+IceCast. OddCast was set to save archive of stream to disk. After hour IceCast crashed resulting in OddCast reconnect. After the broadcast I found two dumped OGG files, but first of them had no OGG container header (no "vorbis" text present in header). Second file is OK. If I'm getting it right, Vorbis within Ogg container has the following format:

Code: [Select]
OggS <ogg_header> OggS <vorbis_header> OggS <vorbis_data> OggS <vorbis_data> OggS <vorbis_data> ...


but I ended only with

Code: [Select]
OggS <vorbis_data> OggS <vorbis_data> OggS <vorbis_data> ...


I would like to know if there is any chance to reconstruct header of the first file (put it again in another Ogg container) or how to play it or convert it to another format.

Thanks in advance ...

Vorbis stream with no Ogg header

Reply #1
If the stream has no header then you are probably not going to be able to salvage it. The header is the most important part of the stream due to the fact that it contains all of the setup information including codebooks, granulepos, vendor string, etc.  Try running the file through ogginfo.exe and see what it tells you about the file.
budding I.T professional

Vorbis stream with no Ogg header

Reply #2
Quote
Try running the file through ogginfo.exe and see what it tells you about the file.
Warning: Invalid header page, no packet found
Warning: Invalid header page in stream 1, contains multiple packets
New logical stream (#1, serial: 000041d0): type invalid
Warning: stream start flag not set on stream 1
Warning: sequence number gap in stream 1. Got page 166557 when expecting page 0.
Indicates missing data.
Warning: EOS not set on stream 1

Vorbis stream with no Ogg header

Reply #3
Quote
Warning: Invalid header page, no packet found
Warning: Invalid header page in stream 1, contains multiple packets
New logical stream (#1, serial: 000041d0): type invalid
Warning: stream start flag not set on stream 1
Warning: sequence number gap in stream 1. Got page 166557 when expecting page 0.
Indicates missing data.
Warning: EOS not set on stream 1


A lot of people who are streaming from an icecast or oddcast server have this problem.  Basically what I am going to tell you to do is attempt to use vcut.exe to try and cut the stream at specific granulepos. Seeking with an Ogg container is relative to wherever the granulepos is set within a page.  You might be able to salvage or whatever is left of it if you truncate it down past a specific position. This is the specific problem that of people who streaming with Vorbis complain about. Why was the Ogg bitstream designed with all of the codebooks and other specific packet information at the start of the stream? the logical and physical bitstream structure though was very clever. I can't think of anything else , although that might be worth a shot.
budding I.T professional

Vorbis stream with no Ogg header

Reply #4
Quote
Basically what I am going to tell you to do is attempt to use vcut.exe to try and cut the stream at specific granulepos.

Code: [Select]
>vcut.exe k.ogg 1.ogg 2.ogg 10

WARNING: vcut is still experimental code.
Check that the output files are correct before deleting sources.

Processing: Cutting at 10 samples
Input not ogg.
Error reading headers
Processing failed


didn't work  any other ideas?

Vorbis stream with no Ogg header

Reply #5
Quote
Quote
Basically what I am going to tell you to do is attempt to use vcut.exe to try and cut the stream at specific granulepos.

Code: [Select]
>vcut.exe k.ogg 1.ogg 2.ogg 10

WARNING: vcut is still experimental code.
Check that the output files are correct before deleting sources.

Processing: Cutting at 10 samples
Input not ogg.
Error reading headers
Processing failed


didn't work  any other ideas?
[a href="index.php?act=findpost&pid=367519"][{POST_SNAPBACK}][/a]


You can probably save it... but you'll have to do it manually.

Your post indicates you know how the headers should be at the start of the file. ie Main header, comment header, code book header.

In order to replace the headers you need to know what specifically they were... this might be a bit of trial and error, however there's a fairly good chance that they were a fairly standard configuration. There's a few things that can vary... the stuff in the first header is fairly likely to be easy to guess, ie it's probably 44100 stereo, grab the first page from some other vorbis file that is the same. The comment header is irrelevant, you can put any old comment header in there, you can always use a tool to change the comment header later.

As to the third header, this will depend on the encoder settings use to produce the file. You may be lucky and it's a fairly "default" encode, in which case you will probably be able to cut this header from a file that was encoded in a similar way, ie with the same encoder settings. There's a fixed set of codebooks, if this file was made by something other than the reference encoder, you may have to do a lot of trial and error to find one that will work.

Basically, what i would do is... get a hex editor, make a backup copy of your file. Then from the start of the file, look forward until you see the first OggS, if there's one right at the start, do nothing, if not, cut everything away upto the first one... that small bit of data is not likely to be salvagable.

Now you have a file with a clean page start. Now try inserting the headers... this will be the first 2 or sometimes 3 pages you will have to do some mental decoding to see this, though if the third page has granule pos 0, it has the continuation flag set, has only packet and it's fairly small, chances are this 3rd page is part of the headers. Cut this section from some other file, which you beleive was similarly encoded.

Save it into it's own file, then make another copy of this little file.

Now look in your broken file, look at the serial number field, now, in your little file with the headers, you need to edit the serial number field. Now you have a problem, the checksum on the header page will be wrong. There is no easy way around this unless you can do some programming to recalculate what it should be. Most player will reject the file if the checksums are incorrect.

However, i know my directshow filters have a bug and they don't check the checksum... so at this point you could maybe try and play the file, and you will at least know if you guessed the right headers, and then you can worry about fixing the checksum later.

On the downside, my filters scan the whole file at load time, so if there is something funky later on in the file... or if this file is chained (which my codecs don't handle yet), it may barf right there. A workaround, is to put it on the web, and use my filter to load url..., since it does not scan the whole file in a network stream, so it will play (assuming the headers are right) until it reaches something that is not right, and then either stop unexpectedly, or possibly crash

Once you know you have he right headers, then you fix the header page checksums... and possibly fix up any garbage that is at the end of the file.

So, all in all, it's almost definately possible to save this file... but it requires some knowledge of the bitstream, and some programming skill, there is nothing automated that i know of that will fix this, and it's unlikely there ever will be, because there's no easy way to guess which headers should be used. Though someone with a very in depth knowledge of the vorbis packets could possibly make some reasonably educated guesses.

So ask yourself... how bad do you want to save this file... and how much time are you willing to expend to do so.

If you send me the file, use www.yousendit.com and enter my email address ogg@illiminable.com after you upload it there, it will send me a link to download it from the yousendit server. I will try to have a look at it, but i can't make any promises that i will actually be able to fix it, or that i will be able to get around to quickly, so if you what i wrote above sounds like you can do it, then i'd suggest you do it, as you are obviously far more motivated to recover this file than me.

In order for this to even be possible i would need as much information about what settings you think it was encded at, also another file (probably another file from this ssame stream) that you suspect was encoded the same way, what version of icecast/vorbis encoder/other software used etc. Even in the best case this is time consuming. You should hope that it was made with the reference encoder(ie libvorbis_, if it was made with any other encoder (ie AoTuV) it may not be possible.

Vorbis stream with no Ogg header

Reply #6
I know this posting is rather old, but you can fix broken Ogg Vorbis streams with Avi-Mux GUI. These fixed files can be cut wit vcut.exe.

EDIT: My fault, it doesn't work if the header is missing.

Vorbis stream with no Ogg header

Reply #7
Havn't tried Avi-Mux but I'd be verysurprised if it could generate proper headers out of nothing


commenting on post #6:

One could also encode another sample (of the same format) with the same settings (including the serial number -- it can be forced afaik). Since the serial number is now the same you can just try to copy the first OggS pages (upto the setup header) and place'em in front of your recorded stream. No manual CRC calculation necessary.

Cheers!
SG

Vorbis stream with no Ogg header

Reply #8
Why was Ogg container designed so complex? The inability to cut Vorbis streams is a serious disadvantage compared to MPEG formats. Would it seriously hurt quality if each Ogg frame was self sufficient? To minimize overhead frames (Ogg pages?) could simply be longer.

 

Vorbis stream with no Ogg header

Reply #9
Unlike 4 Bytes MPEG frame header, Vorbis stream header is ~2 kB long, so including it in every frame would really be an overhead. I'm not such an expert to say why, it might contain some dynamic codebooks or whatever.

The claim about "inability to cut Vorbis streams" is obviously false. All what is needed is that programs which save Ogg Vorbis streams save them properly. I don't know why it was broken in this case regardless of the crash, Ogg is supposed to be written linearly.

However I wonder why nobody suggested to the OP to take the headers from the second file from the same session, which was fine. The headers would be the same because it was apparently one encoding session.

Edit: Some less BS.
Full-quoting makes you scroll past the same junk over and over.

Vorbis stream with no Ogg header

Reply #10
I have the same problem as stick. I used wget to grab a chunk of a stream, but no tool I can find will let me cut out the interesting bits. They all crash or hang. foobar2000 plays the file beautifully.

There's no way to rebuild a stream, hey?

Vorbis stream with no Ogg header

Reply #11
Just for reference, I've heard Canar later used this beautiful tool made by some great individual with great success.
Full-quoting makes you scroll past the same junk over and over.

Vorbis stream with no Ogg header

Reply #12
Just for reference, I've heard Canar later used this beautiful tool made by some great individual with great success.
And then promptly forgot to give said great individual some great props.

Seriously Yirkha, thanks a ton!