[vorbis-dev] Ogg as container format

[rillian]

On Tuesday, October 2, 2001, at 01:18 , Kevin Marks wrote:

> Here you seem to imply that the packets are to do with RTP networking 
> units. For DV do you use the DV RFC to decide these? Or are they 
> arbitrary? You have 2 size-limited structures overlaid on each other, 
> neither of which necessarily corresponds to a fundamental data unit of 
> the underlying media.

It sounds like you're still not understanding the intention. Packets are 
the fundamental units of data the encoder produces. In the case of 
vorbis, we use the same packets for rtp and for ogg, and I believe they 
were deliberately chosen to be of a size that made that practical. For 
other formats one is free to choose whatever's convenient.

Pages are the framing, sync, interleave and error detection layer. They 
do that on whatever opaque packets the codec gives it. Packets can span 
pages, so the 64k size limit applies only to how the framing layer 
works, not what the codec sees.

Which RFC talks about the DV format? I thought the spec was payware from 
smpte.

>
> <sarcasm>
> You call QT old-fashioned, and then include hard-limited 64k page 
> sizes? I'm having DOS flashbacks here. Looks like you have a great 
> format for video, as long as it's 320x200 8-bit  VGA Mode 13h.
> </sarcasm>
>
>> As for 471/565 lacing values, that's less than half a percent overhead
>> for potentially very fine grained packetization (hint; if you're doing
>> things right, each packet is about 200-500 bytes and the overhead is
>> *still the same*).  Doing it any other way would kill us in overhead
>> for *small* packets (like low bitrate audio) where packets are only
>> 40-50 bytes.
>
> As Steve points out, big chunk DMA is very useful for high bitrate 
> video, whereas most good networking implementations will do 
> scatter/gather to construct packets. Where did you get those packet 
> sizes? The usual bottleneck on RTP is the Ethernet frame size of 1460 
> or so data bytes.

I don't know where monty got those numbers. They fit nicely in 
everyone's mtu maybe?

Steve's points seem valid. Ogg wasn't designed with anything like that 
in mind, and in general ogg multimedia has come from a very 
software-oriented point of view. I suspect monty would call that sort of 
thing premature optimization.

> It looks like you skip to an arbitrary point and scan for 'OggS' then 
> do a 64kB CRC to make sure this isn't a fluke. Then you have some 
> packets that correspond to some part of a frame of video or audio. You 
> recover a timestamp, and thus you can pick another random point and do 
> a binary chop until you hit the timestamp before the one you wanted. 
> Then you need to read pages until the timestamp changes and you have 
> resynced that stream. Any other interleaved streams are presumably 
> being resync'd in parallel so you can then get back to the read and 
> skip framing. Try doing that from a CD-ROM.

That sounds about right. Again, I don't see any alternative but to store 
a table of pointers to each packet. You didn't confirm that's what qt 
does?

> If you happen to chain 2 files that use the same stream serial number, 
> you are hosed, as you'll get packets that belong to the wrong stream 
> when doing this. to prevent this, a server will have to rewrite each 
> page header with a unique stream number as it goes out.

Also correct.

> That's known as bait and switch. I AM arguing about general purpose 
> container formats in response to Michael's orginal comment:
>
> At 8:34 PM +1000 9/26/01, Michael Smith wrote:
>> rm goal is for ogg to be a generic media container format in
>> the same way as riff (avi/wav), qt, and so on are. Except better,
>> hopefully ;-)

Fair enough. Monty was claiming a different domain from Michael.

I'm still interested in learning more from the way qt does things. 
You've talked a lot about problems you see with the parts that have 
already been written. I'm worried about the issues with multi-track 
synchronization, metadata, and so on that are still being decided. Do 
you have a take on the large-headers-in-streaming problem? How do you 
packetize text?

  -r

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.