[theora-dev] Extension to Skeleton for multi-track media

[Silvia Pfeiffer]

Hi all,

Discussions about a need for an extension to Skeleton to cater for
multi-track media files has been going on for a while. In a recent
thread here, in discussions on IRC, and at FOMS between Jan, Ralph,
Viktor and I, we discussed some fields. Viktor and I continued that
discussion to make more specific recommendations on what fields to
add.

We now have a wiki page at http://wiki.xiph.org/SkeletonHeaders that
has aggregated all the different things that were discussed.

"Language", "Role" and "Name" are fields that we want to introduce to
better expose "semantic" information about the tracks. These will help
a media player make better choices on which tracks to display and
which to put into a selection to the user. These will also be
necessary for providing the (currently proposed) HTML5 JavaScript API
(http://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI) with the
right information.

Note that right now there is no proposal for explicitly specifying
track dependencies, since the need for these are not really clear.

A further part of the wiki page is the proposal to impose an implicit
order on the tracks through the order in which their BOS pages are
given. This is nothing semantic, but only a convenience so we can
ascertain that different Web browsers will address the same track by
the same index number through JavaScript.

Finally there are two rendering related fields that we propose
introducing: Display-hint and Altitude (their names could of course
still be changed). Each of these are specified as features of the
given track, but relate to the other tracks for rendering. Altitude
specified the display ordering (as z-index in HTML/CSS), and
Display-hints right now has proposals for picture-in-picture display
relative the video's display area, for transparency to be applied to
all pixels of the track, for one color to be chosen as completely
transparent (as in green-screening), and for an image mask to be
applied. The image in the image mask would need to be either
referenced or encoded into another skeleton package - this isn't quite
solved yet.

We'd like to introduce these fields into a new version of Skeleton,
such that future formats will be able to include these fields. All of
these fields are optional, so there are no requirements really, just
additional functionality that a media player can make use of.

This email is to ask for input to the different proposals and for
suggestions of further improvements.

Cheers,
Silvia.