[vorbis-dev] Re: RT & codebooks [was streaming metadata requirements

[Kevin Marks]

At 8:48 am -0600 12/6/01, Jack Moffitt wrote:
>SLT - Simulated Live Transport - this kind of stream is where you
>concatenate a lot of preencoded files, and send them in realtime as if
>you were genereating them live.

[...]

>This is the normal operation that most people use.  They have a bunch of
>OGGs lying around, and then put it on random.  They all get concatenated
>and sent in realtime.  Between each OGG is a sizeable set of codebooks,
>and the comments.  For the most part this meets a good portion of title
>streaming needs.
>
>The problem with the commercials is, they are tiny.  15 seconds long
>maybe.  At low bitrates, they are similarly sized to the codebooks
>themselves.  Or at least the codebooks are a _significant_ portion of
>the OGG.  It's also not unlikely to have 4+ commercials in a row, 15 or
>30 seconds long.  So for a low bitrate person, you're probably going to
>get disconnected, because there's no way you can pull the codebooks that
>fast without pausing the audio.

[...]

>For SLTA, we really need some way to say "cache the last set of books"
>or we'll have to reencode the whole thing.  If we reencode the whole
>thing, we again need the metadata spec :)

[...]

>In either case, I'm a little hesitant to break up a live stream into
>separate streams, considering the codebook overhead for some cases, and
>for SLT streams, I think the codebook overhead is too big in some cases.
>
>Sometimes I can avoid codebook changes by using a metadata stream,
>because what I really want is metadata, not a new stream :)  Other
>times, the codebook overhead is just going to be a given, metadata
>stream or not.
>
>In the normal cases though, resending codebooks hasn't been a big
>problem.  Although if you're paying  bandwidth bills on a few hundred
>thousand listeners per month with an avg. listening time of 90 minutes
>(medium sized operation), the codebooks can add up.

OK, I've said this before and I'll say it again.

In practice, many vorbis files will be using standard codebooks. In 
previous posts, you have said that optimising codebooks for a file 
give a maybe 1% advantage.

It makes no sense to add this much overhead to a stream, and it will 
prevent anyone joining a stream in the middle if the have to wait for 
codebooks to come by.

The way JPEG coped with a very similar problem in RTP (sending the 
quantization and Huffman tables) was to set aside a byte in each 
packet to define which one is being used. They then defined 0-127 to 
be predefined ones, and 128-255 to be dynamic ones, and added a way 
to specify the dynamic ones inline at regular intervals.

This makes far more sense than just caching one set. You can define a 
few standard codebooks (by bitrate and content presumably - some 
tuned for classical music, some for metal, some for speech) and give 
them numbers 0-127 in the RTP spec. Others get numbers assigned above 
128 and are cached by the client or fetched in or out of band as you 
decide.

when you have a choice of going from 0 to 1 things cached and from 0 
to many, many is about the same work, but provide larger benefits.

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.