[xiph-rtp] header caching and chaining

[Aaron Colwell]

ok. I sort of lost track of what has happened here. Could someone provide a
summary of the problem and solutions being proposed?

Here is what I've gotten from this discussion.

- There are concerns about collisions in the CRC

- The use of a unique ID is being proposed so that collisions are avoided,
  but this brings up a problem of mapping the unique ID to the codebooks and
  ident header.

- There are some concerns that using a unique ID adversely effects the players
  ability to cache codebooks and avoid retrieval when it isn't necessary.

Is this correct?

If so here is a possible solution.

- Generate a unique ID for each chain. This could simply be the chain index 
  (ie the Nth chain in the file will have a unique ID of N)

- The server publishes a base URL in the SDP that allows the client to retrieve
  hashes and URLs for the codebook and ident header associated with each 
  unique ID. The url for info associated with a particular unique ID is 
  constructed by appending the unique ID to the base URL. 


- When it sees a new ID in the stream, the client can request the hash and URL
  info from the server. If the hashes match stuff it already has in it's cache
  then it just uses the info from it's cache. If it doesn't have the info then
  it can use the specified URLs to retrieve what it needs.

- If the unique IDs are known at SDP generation time, then they can be 
  advertised in the SDP. The client then can prefetch all the info associated
  with these IDs.

- Since the hash will no longer be in the payload we should probably use MD5
  so that the collision worry just goes away.

- For one-way scenarios, nothing special needs to be done because the codebooks
  and ident header will be transmitted inline. If you are dealing with a 
  completely closed system the server and client could agree on specific 
  ID -> ident, codebook mappings and not even bother transmitting the codebooks
  inline. It might be useful to reserve some of the ID space for this purpose
  in general.

Aaron
 

On Sun, Feb 06, 2005 at 05:30:56PM -0800, Ralph Giles wrote:
> On Mon, Feb 07, 2005 at 01:18:35AM +0000, Phil Kerr wrote:
> 
> > I think Derf was talking about Theora, not Vorbis, but from what I 
> > gathered he listed the maximum permutations for all the config fields.  
> > In practice there will be a large number of permutations that will never 
> > be seen.
> 
> He was. I didn't necessarily agree with his feelings about the relative 
> danger of collision, so I generalized the argument. :)
> 
> > When the idea came up for using a CRC32 field there seemed to be no real 
> > objections to it, perhaps we should find out what the frequency of hash 
> > collisions are in real-life?
> 
> The problem is it's hard to judge because this depends on the behavior 
> of future encoders. I'm more worried about custom codebooks generated by 
> more advanced multipass theora encoders. I guess I do agree the issue is 
> probably more important for theora than for vorbis.
> 
> OTOH, caching seemed like a major win on the whole chaining thing.
> 
> > >So, the only way around this is to not cache the headers. We could just 
> > >say that, but doing so breaks the broadcast use case where play-time 
> > >retrieval isn't an option.
> >
> > This is a solution, but does it harm functionality too much?
> 
> I think so. We have people interested in satellite broadcast of 
> vorbis+theora right now. That use case is important.
> 
> > >The other thing we can do is just make it an arbitrary number and say 
> > >that the sender and receiver MUST negotiate a mapping between the ident 
> > >and the decoder setup out of band, and just leave it out of the scope of 
> > >the RTP mapping (beyond the in-band transmission option anyway.) This 
> > >should work fine for the SDP uri and so on.
> > >
> > If there is no caching then this will work as there only needs to be a 
> > unique ident for each file for the session life.  How do you ensure that 
> > the arbitrary numbers are unique?  Would ov_serialnumber be unique enough?
> 
> Well, it would be up to the server to make sure it didn't reuse an ident 
> for two different decoder configs within a given RTP session. Otherwise, 
> it's arbitrary. That enforces no caching based on the ident, and isn't 
> much of a burden for the server.
> 
> IMHO,
>  -r
> _______________________________________________
> xiph-rtp mailing list
> xiph-rtp at xiph.org
> http://lists.xiph.org/mailman/listinfo/xiph-rtp
>