[xiph-rtp] header caching and chaining

[Aaron Colwell]

On Sun, Apr 10, 2005 at 01:42:44PM -0700, Ralph Giles wrote:
> On Thu, Feb 10, 2005 at 04:46:26PM -0800, Aaron Colwell wrote:
> 
> > I figure we could use the 32 bits that were proposed for the CRC field or 
> > maybe even 24 or 16 bits. This basically just limits how many chains with
> > different ident,codebook pairs you want to support in a single broadcast.
> 
> So you'd be ok with the 16 bits in my proposal?

I don't really care that much about the size. My only concern is that 
theoretically an ogg file can have 2^32-1 chains in it. I know that no one is
likely to do this, but they could. I think a 32 bit field in every packet would
be a waste of bits as would any number of bits if chaining wasn't even used.

Ideally I'd like to rearrange the flag bits a little bit so that there could be
a bit that indicates whether a chainID is present in the packet. The chainID
field could then be variable length ala UTF-8 style or perhaps a simple 
encoding like MSB being set means that there is another byte for the chainID.
The reason I like this option is that it is kind of a "pay as you go" strategy.
As you add more chains to the stream, you pay more and more for the chainID
field. Yes it is a little more complex, but it allows you to accomodate any
valid ogg file, prevents you from wasting bits when chaining isn't even used,
and provides incremental overhead when chaining is used.

> 
> Also, it occured to me that while storing actual info in the SSRC would 
> be an abuse, switching on SSRC,ident pairs might be ok. I don't suggest 
> that this is a good idea, but it does leave lots of headroom.
> 
> > I'm not sure I follow you here. The reason I am proposing the base URL scheme
> > is because you may not always know what the ident,codebooks pairs are at
> > SDP generation time. By using the base URL mechanism the client knows how to
> > take a new ID it gets and retrieve information about the codebooks involved.
> > Unless you want to send this mapping info periodically in the data stream I'm
> > not sure how else you can do this.
> 
> Ok, I understand now. I don't have enough experience with SDP to have an 
> opinion on the style issue, so if you think this is reasonable, I'm ok 
> with it. Saves typing too.
> 
> The idea would be simple concatenation, as in your example, or some kind 
> of template string? There are lots of ways to map ident to url.

I think absolute URLs should be used for the base URL and relative URLs
should be used for the ident-info. Then you can just use the relative URL
resolution rules to figure out what the URL for each ident info. That's 
basically how SETUP urls are generated for RTSP. That is where I got the idea
from.

> 
> > Here is the case where all the chains are known at SDP generation time
> > [...]
> > If you allow the urls to be relative that you could also do this
> > 
> > v=0
> > o=- 1105605563 1105605563 IN IP4 207.188.30.165
> > s=<No title>
> > i=<No author> .2000
> > c=IN IP4 0.0.0.0
> > t=0 0
> > a=control:*
> > a=range:npt=0-202.297000
> > m=audio 0 RTP/AVP 101
> > b=AS:8
> > a=control:TrackID=0
> > a=rtpmap:101 VORBIS/44100/2
> > a=fmtp:101 chain-ids=0,1,2; baseURL="http://foo.com/ident-441k/"
> > a=chain-info:0 ident=42; codebook=98
> > a=chain-info:1 ident=43; codebook=98
> > a=chain-info:2 ident=45; codebook=23
> > a=ident-info:42 url="ident-441k"; hash=987234BC8D92DFE2987234BC8D92DFE2
> > a=ident-info:43 url="ident-8k"; hash=2186461716517578792145688D92DFE2
> > a=ident-info:43 url="ident-11k"; hash=218646687642f4AEFD2145688D92DFE2
> > a=codebook-info:98 url="codebook-lowBW"; hash=309573098520975ABEFC34768D92DFE2
> > a=codebook-info:23 url="codebook-speech"; hash=4567319735186778271C34768D92DFE2
> 
> Ok, thanks for putting this together, that really helps. :)
> 
> I'm not sure being able to mix-and-match ident and codebook headers is 
> worth the indirection complexity. How about just a single id=url;hash 
> mapping? I also think it would be reasonable to (optionally) include a 
> comment header in the set, for streams where that makes sense. Yes, of 
> course we want a separate metadata stream, but there's no reason not to 
> fall back on current tech.

I think there is probably a more compact representation I could use here.
The reason I wanted to have a mix and match scheme was to reduce the amount
of duplication in the SDP. That minimizes the number of hash values in the SDP
if the same ident or codebook is used in several chains.

> 
> I'd also suggest s/hash/md5/ just so it's possible to change the hash
> later.

I agree.

> 
> > If the chain ID set is not known at SDP generation time then you could have
> > an SDP that looks like this
> > 
> > v=0
> > o=- 1105605563 1105605563 IN IP4 207.188.30.165
> > s=<No title>
> > i=<No author> .2000
> > c=IN IP4 0.0.0.0
> > t=0 0
> > a=control:*
> > a=range:npt=0-202.297000
> > m=audio 0 RTP/AVP 101
> > b=AS:8
> > a=control:TrackID=0
> > a=rtpmap:101 VORBIS/44100/2
> > a=fmtp:101 chainIDBaseURL="http://foo.com/chainIDs/"
> 
> Ok. Similar to before I have a simpler proposal. The http urls should 
> just point to an Ogg stream with the header packets. That's well 
> specified, and easy for the server to generate. Having a separate query 
> to get the hashes saves bandwidth, but also increases latency quite a 
> bit. If you're generating custom headers for every stream in a live 
> environment, I don't think cache coherency for the hashes is going to be 
> very good anyway.

I though about just pointing to an ogg stream, but that prevents the client
from just grabbing the ident or codebook. It is possible for the client to
already have one of these 2 pieces from an earlier chain or from an earlier 
file. I just wanted to have a scheme where the client is able to only get what
it needs.

> 
> Question: if you've streaming theora+vorbis using this scheme, can you 
> use the same url for both headers and just put up a multiplexed Ogg 
> file? That would make it *really* easy.

It would, but if the client already has everything but the Theora codebook then
it has to waste bits pulling down the Vorbis ident, Vorbis codebook, and Theora
ident header. Having to pull down all the pieces reduces the savings of the
client's cache because if you have a cache miss on a codebook or ident you
still have to download stuff you might already have.

> 
> > The client can differentiate the 2 cases by what fields are present in the 
> > fmtp header.
> 
> Is this why you have BaseURL vs ChainBaseURL? I think with a combined 
> header scheme this could also go away.

Yes that is why I have the 2 different URL headers. 

I'll look at this again and see if I can come up with a more compact SDP
representation.

Aaron

> 
> Still, in general this looks good, and I think it's better than trying 
> to send the headers themselves in the SDP. If we can work out these 
> details, and specify some way to point to an RTP header stream for 
> multicast, I think we're done. Broadcast applications can use the SDP 
> scheme with a fixed set of preloaded codebooks.
> 
>  -r
>