The Ogg container format is being promoted by the Xiph Foundation for use with its Vorbis and Theora codecs. Unfortunately, a number of technical shortcomings in the format render it ill-suited to most, if not all, use cases. This article examines the most severe of these flaws.
According to the article Ogg has worse properties regarding overhead, latency and random access features than both matroska and mp4, for which also free implementations exist.
An additional criticism is that Ogg is much too complex without benefit (as higher flexibility) for real world scenarios compared to the other two.
Albeit its complexity it isn't even that flexible, for example, checksumming is mandatory, which is wasted space for numerous scenarios.
It is, for example, somewhat strange that it was specifically designed to be streamable, but you can't even setup self contained streams, that can be tuned into at any point (as webradio) without necessary quirks by a streaming server, which has to fake the beginning of a new stream as soon as you tune in.
Ogg Skeleton is supposed to describe the other streams and may very well include seek index tables.
It seems Ogg Skeleton has no way of signalling seek tables.