Skip to main content
Topic: OGG Vorbis header bits (Read 10158 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

OGG Vorbis header bits

Is someone able to understand the raw bit in a ogg/vorbis header.

I'm able to reverse the tag part (which is good for my needs) but not the preceding header...

The doc states that "the Vorbis text comment header is the second (of three) header packets that begin a Vorbis bitstream" but nowhere in the source can i find a clear answer on the bits composing these headers...

Every ogg files begins with "OggS" (that's for sure) and then some header bits (i guess) [ in my example file :
00000004h: 00 02 00 00 00 00 00 00 00 00 56 24 00 00 00 00
00000014h: 00 00 B9 E4 13 F2 01 1E 01
]

I may interpret it as bitstream description or header description...don't know....

Then a vorbis header begins...
0000001dh: 76 6F 72 62 69 73 00 00 00 00 02 44 AC 00 00 FF ; vorbis.....D¬..ÿ
0000002dh: FF FF FF 00 E2 04 00 FF FF FF FF B8 01                  ; ÿÿÿ.â..ÿÿÿÿ¸.

And some OGG formating
0000003ah: 4F 67 67 53 00 00 00 00 00 00 00 00 00 00 56 24 ; OggS..........V$
0000004ah: 00 00 01 00 00 00 34 3D FE CC 12 FF 03 FF FF FF  ; ......4=þÌ.ÿ.ÿÿÿ
0000005ah: FF FF FF FF FF FF FF FF FF FF FF FF 53 03                ; ÿÿÿÿÿÿÿÿÿÿÿÿS.

And the part, i successfully reverse :
00000068h: 76 6F 72 62 69 73 1D 00 00 00 58 69 70 68 2E 4F ; vorbis....Xiph.O
00000078h: 72 67 20 6C 69 62 56 6F 72 62 69 73 20 49 20 32 ; rg libVorbis I 2
00000088h: 30 30 34 30 36 32 39 0B 00 00 00 0B 00 00 00 54 ; 0040629........T
00000098h: 49 54 4C 45 3D 50 72 6F 75 74 0E 00 00              ; ITLE=Prout...

Which is "vorbis", followed with a 2 byte string length, followed with a UTF8 string description of the vendor name  (is that what the doc means by "8 bit clean"...so obscure...i am not so much a fan of this "bit vector" analogy, i must admit that i really doesn't understand it...), and a 4 byte "number of comments" (n) integer, followed with the same pattern n times : a 2 byte string length and its UTF8 string.

Am i correct.... ?

And what is the first header bit structure ?

Thanks

MaB_fr

OGG Vorbis header bits

Reply #1
The first packet (of every stream) is the stream identification packet which is defined for Vorbis in http://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#id2547578

Note that the Vorbis bitstream specification operates on a higher level -- the packet level.  Iv you haven't already, please make yourself familar with the following documents as well which describe how logical streams consisting of packets are encapsulated in Ogg pages:
http://www.xiph.org/vorbis/doc/oggstream.html
http://www.xiph.org/vorbis/doc/framing.html

HTH,
Sebi

OGG Vorbis header bits

Reply #2
Quote
The first packet (of every stream) is the stream identification packet which is defined for Vorbis in http://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#id2547578

Note that the Vorbis bitstream specification operates on a higher level -- the packet level.  Iv you haven't already, please make yourself familar with the following documents as well which describe how logical streams consisting of packets are encapsulated in Ogg pages:
http://www.xiph.org/vorbis/doc/oggstream.html
http://www.xiph.org/vorbis/doc/framing.html

HTH,
Sebi
[a href="index.php?act=findpost&pid=341404"][{POST_SNAPBACK}][/a]


I already read that....and was totally disapointed by it. (i must add, that there's a bunch of 404 in those.......)

Is there any strong headers (i mean C language headers) specifiying the API...
I wasn't able to found the #define in ogg.h for simple things like :

"header_type_flag
5  bitflags: 0x01: unset = fresh packet
                 set = continued packet
         0x02: unset = not first page of logical bitstream
                      set = first page of logical bitstream (bos)
         0x04: unset = not last page of logical bitstream
                      set = last page of logical bitstream (eos)
"
and such...

Does they even exist ????
Am i suppose to rely entirely on my own interpretation of this documentation ?


MaB_fr

OGG Vorbis header bits

Reply #3
Quote
I already read that....and was totally disapointed by it. (i must add, that there's a bunch of 404 in those.......)

But those direct links provided do answer your questions...

Quote
Is there any strong headers (i mean C language headers) specifiying the API...
I wasn't able to found the #define in ogg.h for simple things like :

"header_type_flag
5  bitflags: 0x01: unset = fresh packet
                set = continued packet
        0x02: unset = not first page of logical bitstream
                       set = first page of logical bitstream (bos)
        0x04: unset = not last page of logical bitstream
                       set = last page of logical bitstream (eos)
"
and such...

Does they even exist ????

Not in the ogg header, I guess. But when using libogg, you don't need to worry about low-level details like that.

But the above is very easy to map to C code anyway... For example, assuming you have read the byte containing the flags into the variable flag, you could write something like this:

Code: [Select]
if (flag & 0x02)
 /* beginning of stream */

OGG Vorbis header bits

Reply #4
Ok...i doesn't want to go on the philosophic level...so it's not my point to argue about the importance of having constant defined and strong defines in low levels...

My problem is, i have an ogg vorbis file, tagged and all...Correctly read by every taggers and players i tried except.....mine when i use the vorbis library....

When i ov_open my file...i get a OV_ENOTVORBIS error message.....
And i've checked every kind of mistake i could have done...

Code: [Select]
oFile = fopen( lpcFileFullPath, "r" );
if( oFile == NULL )
    return FALSE;
    
if( ov_open( oFile, &ovfFile, NULL, 0 ) != 0 )
{
    if( ov_open( oFile, &ovfFile, NULL, 0 ) == OV_ENOTVORBIS )
    {
 ov_clear( &ovfFile );
 
 oFile = fopen( lpcFileFullPath, "r" );
 return ExtractVorbisRaw( oFile, pMetaTag );
    }
     
    return FALSE;
}
ExtractWithLibVorbis( oFile, &ovfFile, pMetaTag );

I even verified it with an hex editor....

Then what i choose to do, is believing that my version of the vorbis library was not compatible with my oggvorbis tagged file (i repeat, this file works on other taggers...)

So that's where i decided to reverse the data bits of the ogg files....(as you can see when i call ExtractVorbisRaw....which doesn't use any libvorbis function...)

I'd really, REALLY, like to be able to use libvorbis and the vorbisfile API...Except that it DOES NOT WORK....(i use libvorbis-1.1.1 and libogg-1.1.2 source code, first statically linking libogg to libvorbis and then statically linking libvorbis to my dll).

Of course it can be a bug in libvorbis. But as i assume that it is a strong, released library, and i prefer to think that I AM the problem...

I repeat that i already read the ogg doc and was finally able to "understands" it (always hoping i am not misunderstanding anything as it is strongly abstracted...)
It is clear that some of the API choice are unclear to me (like how do you MODIFY a vorbis comment with just ov_comment() function in vorbisfile API)...

But people where able to do it...so should i ... (i hope :-))

And....i've think i've done it...

Rest the problem of my ogg file not being read by libvorbis....

Thanks for answering my call to help

MaB_fr

P.S. : i'm joigning what i think is missing in ogg.h for my kind of needs....may help some people somewhere in the universe

Code: [Select]
typedef struct        stOggHeaderRaw
{
    char      ID[ 4 ];      //must be "OggS"
    unsigned char    StreamVersion;    //0 seems always good (?)
    unsigned char    HeaderTypeFlag;
    unsigned char    Position[ 8 ];
    unsigned char    SerialNumber[ 4 ];
    unsigned char    Counter[ 4 ];
    unsigned char    CheckSum[ 4 ];
    unsigned char    SegmentNumber;
    unsigned char    FirstSegmentSize;
};
#define    OGGRAWHEADER      struct stOggHeader
typedef    struct stOggHeaderRaw    *    LPOGGRAWHEADER;

typedef struct        stVorbisRawCommentsHeader
{
    unsigned char    StreamVersion;    //0 seems always good (?)
    char      ID[ 6 ];      //must be "vorbis"
    unsigned char    VendorStringSize[ 4 ];
};
#define    VORBISRAWCOMMENTSHEADER    struct stVorbisRawCommentsHeader
typedef    struct stVorbisRawCommentsHeader *    LPVORBISRAWCOMMENTSHEADER;

#define    OGG_OGGHEADER_ID      "OggS"
#define    VORBIS_VORBISHEADER_ID      "vorbis"    
 
#define    OGG_FLAG_CONTINUEDPACKET  1
#define    OGG_FLAG_STREAMBEGINNING  1 << 1
#define    OGG_FLAG_STREAMENDING      1 << 2

OGG Vorbis header bits

Reply #5
Shouldn't that be calling ov_open only once and saving the return value? ( And if you did that much wrong in just these dozen lines, I'd hate to see what you've done with the whole program. )

Oh, and since you appear to be using Windows, you should be using binary mode for all files. (ie. "rb" instead of "r")

OGG Vorbis header bits

Reply #6
Quote
Shouldn't that be calling ov_open only once and saving the return value? ( And if you did that much wrong in just these dozen lines, I'd hate to see what you've done with the whole program. )
[a href="index.php?act=findpost&pid=341637"][{POST_SNAPBACK}][/a]


if i call ov_open() twice it's juste because it's doing an unexpected behaviour (saying that my file is not a ogg vorbis file when it actually IS)

and why and doesn't use a if( ( somevar = ov_open() ) == OV_ENOTVORBIS ) is juste because I REALLY WANT the vorbis lib to work and be able to ERASE the code inside the fraudulent 'if'...

It's certainly an incorrect and dirty way of doing thing, but it was a temporary hack until someone can explains where i did wrong before this "if"....

I'd really appreciate if someone can concentrate on my problem, so i reedit my code with a clean way of doing it (just for you, for example, now you will be able to read my code ) :

Code: [Select]
oFile = fopen( lpcFileFullPath, "r" );
if( oFile == NULL )
return FALSE;

ovOpResult = ov_open( oFile, &ovfFile, NULL, 0 )
if( ovOpResult != 0 )
{
  ov_clear( &ovfFile );
 
  if( ovOpResult == OV_ENOTVORBIS )    // unexpected BEHAVIOUR....
  {
     oFile = fopen( lpcFileFullPath, "r" );
     return ExtractVorbisRaw( oFile, pMetaTag );
  }
}

return ExtractWithLibVorbis( oFile, &ovfFile, pMetaTag );

OGG Vorbis header bits

Reply #7
See as I mentioned above. Open the file in binary mode on Windows platforms. fopen( path, "rb" ).

OGG Vorbis header bits

Reply #8
Quote
See as I mentioned above. Open the file in binary mode on Windows platforms. fopen( path, "rb" ).
[a href="index.php?act=findpost&pid=341641"][{POST_SNAPBACK}][/a]


Thank you so much.....it WORKS...

Can explain to me why libvorbis needs a binary mode ? (i'm always curious of the inside...)

And do you know how i'm supposed to use ov_comment() to put modified data in a vorbis file (i read the vorbisfile.c and again....i'm lost) ? Must i use ogg packet control function ?

Thanks again

MaB_fr

OGG Vorbis header bits

Reply #9
The stdio file access functions on Win32 perform line ending translation on the fly, so you need to enable binary mode if you don't want this clobbering your obviously binary (and not text) files. [span style='font-size:4pt;line-height:100%']I hereby revoke your programmer's license and delete your copy of Visual Studio.[/span]

OGG Vorbis header bits

Reply #10
Quote
The stdio file access functions on Win32 perform line ending translation on the fly, so you need to enable binary mode if you don't want this clobbering your obviously binary (and not text) files.
[a href="index.php?act=findpost&pid=341645"][{POST_SNAPBACK}][/a]


Ok, and so as a bitsrteam can contains "line ending" in the data, libogg needs to secure this...

Do you think I may suggest the ogg team to add this part into their docs, or is it "common knowledge" ?

MaB_fr

OGG Vorbis header bits

Reply #11
Quote
Ok, and so as a bitsrteam can contains "line ending" in the data, libogg needs to secure this...

Do you think I may suggest the ogg team to add this part into their docs, or is it "common knowledge" ?

MaB_fr
[a href="index.php?act=findpost&pid=341647"][{POST_SNAPBACK}][/a]

Yes, that is common knowledge.

And if you had looked at the vorbisfile_example.c file included with libvorbis, you could see that it does include code to set binary mode  for stdin/stdout...

Quote
And do you know how i'm supposed to use ov_comment() to put modified data in a vorbis file (i read the vorbisfile.c and again....i'm lost) ? Must i use ogg packet control function ?

The recommended way is to use the functions in the file vcedit.c, part of the vorbiscomment program in the vorbis-tools package.

OGG Vorbis header bits

Reply #12
Quote
Byte order: Little-endian
Offset   Length   Contents
[ 0      4 bytes  "OggS"
  4      1 byte   Stream structure version (0x00)
  5      1 byte   Packet flag:
                    bit 0:  true if page continued
                    bit 1:  true if first page
                    bit 2:  true if last page
                    bit 3..7: reserved
  6      8 bytes  The end pcm sample position (64bit integer)
14      4 bytes  Stream serial number
18      4 bytes  Page number
22      4 bytes  Check sum
26      1 byte   Number of segments(s)
27     (s)bytes  Sengment table
27+(s) (b)bytes  Body (b := header[27] + header[27+1] + ... + header[27+s-1])
]*

FROM http://www.onicos.com/staff/iz/formats/ogg.html
It works for me.

Re: OGG Vorbis header bits

Reply #13
maybe of some help to anyone wanting to read vorbis comments.

Each packet starts with 'OggS' and the comments are in the second packet.

1. open the file and search for the second 'OggS'
2. From this point search for 'vorbis'
3. The next four bytes are a little endian number  so read these four bytes as a long number

In a HEX editor you will see as an example
76 6F 72 62 69 73 34 00 00 00
which is 'vorbis' followed by a four byte number in real life 00 00 00 34 (52 in decimal remember it is little endian)

4. Read in the next 52 bytes as a UTF-8 encoded string. This is the vendor string.

5. Read the next four bytes as a number this gives you the number of comments in the file.

6. The four bytes after this are the length of the comment.

7. Read in this number of bytes as a UTF-8 encoded string

Repeat steps 6 and 7 for the number of comments retrieved in step 5

All comments are in the form 'TITLE=Tell Me Why'

The first part before the = is the comment title (not TITLE) the second part is the comment content.

Some files have picture metadata in them I have not sorted this out yet but it is still read the same way.

Writing new comments I have not looked into yet but I think it is copy file to comment start to another temp file add new comments including total and length for each then from comments end in original copy all bytes to temp file, delete original file then rename temp to original.

Hope this helps





Re: OGG Vorbis header bits

Reply #14
Understand a little more about the comments Page\Packet

4F 67 67 53 00 00 00 00 00 00 00 00 00 00 DD 5B 00 00 01 00 00 00 88 4D C7 08 10 DB FF FF FF FF FF FF FF FF FF FF FF FF FF FF

The first four bytes are always 4F 67 67 53 = "OggS"

The next part to look for is the stream serial number here it is DD 5B 00 00

After that is the sequence number 01 00 00 00 this shows that this page\packet is the first packet in the stream identified by the serial number.

Next is the page\packet CRC32 checksum 88 4D C7 08 obtained by setting these bytes to 00 and then working the CRC32 over the whole page\packet i.e. from OggS to the start of the next page which starts with OggS.
not including the second OggS

The next byte tells you the number of segments in this page\packet 10 (16 decimal)

Each segment can be anything from 0 to 255 bytes in length.

To calculate the size of lets call it a section here you read the next byte and if it is FF (255) read the next byte and add it to the previous one.

Keep adding these together until you find a byte which is less than FF (255)

So in the example above we have 10 followed by DB and we know that the comments section is the first section in this page\packet.

As DB is less than FF then length of the comments section is DB bytes in length.

The start of the first segment is the byte after the last segment read allways seems to be 03 I do not know why.

The file I have taken this example from shows the comments section as:-

03 76 6F 72 62 69 73 34 00 00 00 41 4F 3B 20 61
6F 54 75 56 20 5B 32 30 31 31 30 34 32 34 5D 20
28 62 61 73 65 64 20 6F 6E 20 58 69 70 68 2E 4F
72 67 27 73 20 6C 69 62 56 6F 72 62 69 73 29 05
00 00 00 12 00 00 00 54 49 54 4C 45 3D 42 69 63
79 63 6C 65 20 52 61 63 65 0C 00 00 00 41 52 54
49 53 54 3D 51 75 65 65 6E 09 00 00 00 44 41 54
45 3D 31 39 39 31 1E 00 00 00 4F 52 47 41 4E 49
5A 41 54 49 4F 4E 3D 48 6F 6C 6C 79 77 6F 6F 64
20 52 65 63 6F 72 64 73 3E 00 00 00 43 4F 4D 4D
45 4E 54 3D 66 72 65 3A 61 63 20 2D 20 66 72 65
65 20 61 75 64 69 6F 20 63 6F 6E 76 65 72 74 65
72 20 3C 68 74 74 70 73 3A 2F 2F 77 77 77 2E 66
72 65 61 63 2E 6F 72 67 2F 3E 01

Looks a mess but the first few bytes are   03 76 6F 72 62 69 73 which is  03 then vorbis

The next four bytes 34 00 00 00 (52) are the vendor string length

This means that the next 52 bytes are the vendor string.

This takes you to 62 69 73 29 on the fourth line down.

The next four bytes 05 00 00 00 tell you how many comments are included in this case 5.

The next part follows a sequence of length then comment repeated 5 times.

So next four bytes 12 00 00 00 (18) are the length of this comment in this case "TITLE=Bicycle Race"

and the next four bytes are length of next comment etc.

After extracting all the comments you find the last byte is 01 I have no idea why this is here but it is important that comments end with this.

The file will not play if missed out.

I am developing an ogg vorbis comment reader and writer without using libvorbis so all native code. I use PureBasic and you can find me on their forum at :-

https://www.purebasic.fr/english/

Anyone interested in helping would be welcome.

I hope this helps someone.

Re: OGG Vorbis header bits

Reply #15
Holy thread resurrection!

Ogg Vorbis uses a "framing bit" to end the comment header.  It must be non-zero.  This is required and the stream is technically unreadable if it is missing.  Most other Ogg-encapsulated codecs do not use this.

Re: OGG Vorbis header bits

Reply #16
There are numerous libraries that allows to create/edit Vorbis tags. No need to re-invent the wheel, but if you insist just analyze the source codes. Most are free. (vorbiscomment, taglib, etc). Your empirical way only make sense if no documentation or source code. Inverse engineering will not result in a trusted system unless you can generate all valid permutations and you can inverse engineer all reliably. Very hard even for the experienced people.

Just my 2 cents.

 

Re: OGG Vorbis header bits

Reply #17
The main problem when anyone approaches ogg vorbis files is understanding that ogg is not vorbis and vorbis is not ogg.

The ogg file is split into pages and each page is split into segments, these segments store in the case of a vorbis bit stream the vorbis bit stream as defined by the vorbis specification.

The first thing is to learn how to decode an ogg file. Then look at the vorbis stream away from the ogg file.

If it helps you can look at this thread on a programmers forum (all code in basic) where you can see my journey in understanding this was played out. It includes details of reading ogg files etc and some detail of the vorbis stream.

vorbis comment reading

It does include source code for a comment reader which works very well. Remember there is no standard vorbis comments each read/write program seems to do it differently.

Writing comments is a whole different ball game however and a lot more complicated.

Hope this helps anyone attempting to understand these files.

collectordave

 
SimplePortal 1.0.0 RC1 © 2008-2019