[vorbis-dev] Re: overhead ??

[Ralph Giles]

On Sat, Dec 06, 2003 at 03:17:01AM +0100, Francesco Gadaleta wrote:

> this encoding both avoids imposing a maximum packet size as well as
> imposing minimum overhead on small packets.
> The question of overhead is unclear.

So, you've got this bitstream produced by the codec and you want to chop it up in smaller pieces. 
You do that, and then you encapsulate it with a little data about each piece. This extra per-piece 
data makes the ogg-encapsulated bitstream larger than the original raw bitstream produced by the 
codec. This increase in size is known as 'overhead' and obviously one wants to minimize it.

One thing that needs to be part of this data is the length of the piece being encapsulated. The 
most obvious was to do that is to just put the length in there as a number as part of the 
overhead. But how many bytes should you use to store it? If the packets are all really short (less 
that 256 bytes) you can store the length in a byte. If they're just short (less than 64k bytes) 
you can store the length in two bytes. The longer the pieces you want to be able to store the 
larger the overhead is in absolute terms, and worse, the larger the fraction of the stream that is 
overhead becomes when most of the packets are small.

Obviously, then, it would be better if you could store the lengths of the pieces using a number of 
bytes appropriate the the length of each piece. There are indeed ways to do so. One way is to use 
a few bits to record the size of the length and then that many bytes to record the size of the 
piece itself. Another is the lacing technique Ogg uses.

The real strength of the lacing approach over other variable length encodings is in how it 
interacts with the rest of the Ogg encapsulation design. The above is an over-simplification:
each packet (piece) needs a length, but the lacing technique allows multiple packets to share 
other data like timestamps and start codes, limiting the overhead required to store this 
information when the packets themselves are small.

So, because the lacing technique can split a large packet over an arbitrary number of pages with 
no fixed size of length field, it imposes no limit on the size of packets. Because packets can 
also be aggregated with many to a page, each one using as little as one byte to specify its length 
and sharing all the rest of the descriptive data in the page header, the fraction of the bitstream 
devoted to overhead can be limited. This is what the paragraph is describing.

 -r
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.