[theora-dev] handling multitrack Ogg

[Silvia Pfeiffer]

(cc-ed to theora-dev because that will reach a larger audience -
please reply to only one mailing list.)

Hi all,

In discussions with the video accessibility subgroup of the W3C HTML
working group, we are currently looking at how to deal with multitrack
video, e.g. such video that has a main video and audio track, plus
e.g. a sign language video track, an audio description audio track, a
caption track and several subtitle tracks in different languages.
(i.e. several theora, several vorbis and several kate tracks)

We are in the process of developing a JavaScript API for extracting
information from such files and thus be able to control them from
JavaScript, e.g. turn them on/off, find out what is there and what is
relevant etc. In this JavaScript API, it is necessary to address the
tracks.

Now, I don't think there is an explicit track order in an Ogg file -
tracks are just regarded as parallel entities and differentiated by
serial number, right?

I found out that the MPEG/QuickTime container format has an explicit
"track ID" that numbers its tracks.

The track order also determines the display order. "Lower numbered
layers are shown on top of higher-numbered layers. This layering order
is important mainly when multiple video tracks are combined though
graphics modes." (see
http://developer.apple.com/quicktime/qttutorial/movies.html) There is
also an explicit rendering mode, e.g. to alphablend, dither, blend or
copy a layer on top of another one.

Further, MPEG/QuickTime has an explicit "alternate group ID". This ID
defines tracks that belong to the same group and can thus only be
displayed alternately of each other, e.g. subtitle tracks, audio dubs,
or video tracks with differing video quality.

I now wonder whether we need to introduce an explicit "track ID" into
Ogg, which will define a fixed order independent of the parsing system
and will allow us in JavaScript to directly address tracks.

For example, the current draft looks something like this:

interface MediaTrack {
  readonly attribute DOMString title;
  readonly attribute DOMString type;
  readonly attribute DOMString role;
  readonly attribute DOMString lang;
           attribute boolean enabled;
  ...
};
interface MediaTracks {
  readonly attribute unsigned long length;
  getter MediaTrack item(in unsigned long index);
  ...
};
interface HTMLMediaElement : HTMLElement {
  ...
  readonly attribute MediaTracks tracks;
  ...
};

Which could be used for something like this:

if (video.tracks[1].role == "caption") video.tracks[1].enbled = true;
if (video.tracks[1].lang == "fr")          video.tracks[1].enbled = true;


As you can see, there is track index that runs through all the tracks.
If we want to keep them in the same order in every system, we have to
come up with an ordering scheme.

Right now, I can see two different systems: the order in which their
BOS pages are given in the Ogg header part - or the order in which the
serial numbers go, when ordered. Alternatively, we can introduce an
explicit track ID and order by that number.

I'm curious what others think.

Cheers,
Silvia.