[theora-dev] bitstream versioning

[Dan Miller]

as per yesterday's discussion at #theora --

It has been suggested that we devise a method such that we will be able to add certain features to the bitstream (interlace, exotic color subsampling, clip length flag for download, etc), in such a way as to not obsolete streams encoded previously.  Rillian's suggestion, which is pretty much standard practice, is to add a bitstream revision number.  Another suggestion was to move to a tag structure, where parameters are tagged with text ID's; this would allow additional parameters to be added to the bitstream spec transparently to older bitstreams.  These two suggestions are not mutually exclusive, though I am suggesting we not include an explicit revision number in the normal way.

My objection to the typical revision number field is as follows.  Take for example Mau's request for a clip length field.  To add that to the spec in the traditional way would require a new rev number.  Older players would have to barf at any bitstream with the length field because the rev number would be too high.  That's unfortunate because in most cases the older player could play the stream fine without knowing about clip length.  So I guess I'm saying it makes any kind of 'forward' compatibility, where new bitstreams can play in older players in some cases, impossible.  That will inevitably create more of an impression that Theora is unstable (fairly or not).  Similar situations arise when thinking about stuff like gamma correction, side data for improved post-filtering, etc.  There are many features that can improve quality or ease of use, but don't really make the bitstream unplayable by older code.

Another example: we add an interlace option in a new revision.  Most encoder apps will naturally upgrade to the new version, and always stick the higher rev number in the bitstream.  Older players are out of luck.  With the tag method, if interlace is not used, the file will still play.  On the other hand, if interlace is specified, the older player should have a way of knowing it shouldn't try to play the stream.

Here's my suggestion: Let's add one more field right now -- a tag-style extended parameter field (ie a text string with ID's and values).  Initially, we implement exactly one extended parameter: "TH1.0-COMPAT" 0|1.  This way, future encoders can explicitly state whether the bitstream they are producing can be played in a Theora 1.0 player (any player based on our upcoming Beta 1 release).  [note: "TH1.0" refers to the bitstream, not the code release]

In the future, we can add similar compatibility tags for new revisions of the bitstream spec.  In the meantime, anyone experimenting with new features can set the compat flag to zero so old players won't crash.  Once we've got a new stable bitstream, we number it and add the new compatibility flag.

Thoughts?

 ___  Dan Miller
(++,) Founder, On2 Technologies
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.