... or is each packet independent of others (assuming it is not a continuation one set in the header)? I am I safe to parse on packet boundaries, and the decoder deal with them packet at a time? Or is it more complex?
These raw packets may be used directly by transport mechanisms that provide their own framing and packet-separation mechanisms (such as UDP datagrams). For stream based storage (such as files) and transport (such as TCP streams or pipes), Vorbis uses the Ogg bitstream format to provide framing/sync, sync recapture after error, landmarks during seeking, and enough information to properly separate data back into packets at the original packet boundaries without relying on decoding to find packet boundaries.