[vorbis-dev] Vorbis packet sizes

[John Ripley]

> From: Beni Cherniavsky <cben at techunix.technion.ac.il>
> Sent: 05 June 2003 13:39
> 
> John Ripley wrote on 2003-06-05:
> 
> > Beni Cherniavsky wrote:
> > >
> > > There is no bound whatsoever on packets.  There is a limit on the
> > > size of a page - just below 64KB.  Packets can span multiple
> > > pages, which is the only limitation to reading entire packets.
> > > I/O should be much slower than moving around memory anyway.
> >
> > That's the point, I/O is slow and may block, which then involves a
> > context switch. If you want the ability to context switch, you
> > suddenly need a separate stack (or thread if you're running a big
> > OS). If you take all the I/O out of the decode path, it won't ever
> > need to context switch.
> >
> How do you take all I/O out of the decode path?  I'm afraid I missed
> you intent, please explain again.

You separate the tasks of reading I/O and decoding packets. Reading the
stream can be done with non-blocking I/O as one task, with the aim to read
an entire packet into memory. This entire packet is then passed to the
decoder, which will never have to worry about blocking I/O, seeing as all
the data it will ever need is already there.

That way you can run the file reads, the decoder, and whatever else is
running in your system as a pure state machine. This means you don't need
any threads. The trouble with an API which only supports blocking I/O via
callbacks is that it needs its own context (thread) to run in, or it blocks
everything else.

I'll probably end up not bothering with the method of reading entire packets
and then passing them to the decoder. It's simpler, but it's not actually
any faster, and I still have to support other codecs with similar APIs to
libvorbis. I'll probably arrange for the codec to be a state machine without
callbacks, though.

> > > Multiplying max page size by page sequence number limit, 
> you can get a
> > > theoretical limit of packet size but that would be 
> ridiculously big.
> > > Ogg intentionally supports arbitrarily big packets.
> >
> > The "unknown" and "arbitrary" upper bounds of Vorbis 
> streams is something I
> > don't particularly like. However, if you assumed stereo 
> (I'm throwing away
> > anything with more), 8192 block size, and worst case 
> entropy (all codes are
> > 32 bits), I bet it would still be reasonably small. I'll 
> calculate this when
> > I have some spare time.
> >
> It might make some sense to place bounds on packets size in specific
> use areas.  It most certainly doesn't belong in the Ogg framing
> standard, I think it's wonderful that it does not place constraints.

I agree it doesn't belong in Ogg. But for Vorbis, it makes sense to know in
advance what the structure size limits are. As far as I can tell, there's no
way you can implement a decoder which will play any arbitrary, valid 1.0
spec stream. The upper bounds on the size of things like the codebooks are
in gigabytes (or was it terabytes)? I could certainly arrange for a
perfectly valid stereo 44.1kHz stream which oggdec 1.0 couldn't play, for
example.

It would be good, in my opinion, to actually place some more constraints in
the spec. It's hard to advertise "Supports Ogg Vorbis 1.0" when you know
that's not true in all cases. What you actually meant is "Supports OggEnc
1.0 files, and YMMV".

- John Ripley.
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.