[vorbis-dev] Ogg MIDI proposal

[Ralph Giles]

On Saturday, August 25, 2001, at 05:01 , Kenneth Arnold wrote:

> How about allowing multiple Format 0 streams (thus representing Format
> 1 and even (partially?) Format 2) in an Ogg stream, so long as the
> pages are small enough and spaced properly so everything is
> simultaneous (if the application requires it)?

Jack said the format 1 -> format 0 conversion could be done without loss of generality, so I
generally agreed the '0 only' spec made sense. I'm not entirely clear on whether this imposes a 16
'device' limitation for Ogg MIDI, or not, and how onerous that actually is.

In theory yes, you should be able to group multiple midi streams. But they might be alternate
tracks instead of tracks to be mixed in. Same would go for multiple vorbis streams, and so on,
which brings us again to the metadata can of worms. See below. :)

>> Changes to Vorbis
>> -----------------
>> libvorbisfile will need to be modified to recognize substreams and ignore
>> all but the first valid Vorbis stream.
> 
> I think it's about time to get the full Ogg stream interleave format
> together. I have not seen the MNG work (Ralph, could you commit it to
> CVS?), but I suspect this has already addressed some of the issues

Not really. Most of this is still at the design stage.

A lot of what you mention is important work from the tools point of view; mostly we've been
thinking at the spec level, which has something to do with why we missed your concerns. At the ogg
and vorbis levels, we already have everything we need.

Figuring out how to properly interleave all these bitstream types is also *hard* in a way that had
me assuming much of it belonged at the application level. And there will be so many different
types. I do expect oggmerge will eventually have either plugins or a nice interface for compiling
in support for various formats, but I haven't seen how we can have a unified library-level
architecture that handles everything. Can you explain your idea?

> [...]
> * table of contents header
>   - to provide information about what codecs are used in the stream,
>     for player selection or desktop environments -- more of a
>    future-proofing than actually needed now

Right, I maintain we don't need this yet. The 'principle use first' doctrine (why the vorbis
header must come first in an OggMIDI file) takes care of simple player selection and mimetype
issues. Anything sophisticated enough to understand that this file has both vorbis and midi data
can search the headers just like the players do. ogginfo could also be invoked to implement this
for scripts.

My primary concern is that we don't have a complete requirements list for this stream-description
metadata, and I don't want this 'future proofing' to turn into baggage. Defining everything
implicitly, as in this proposal, avoids the issue in a way that is elegantly compatible with
simpler implementations.

Requirements gathering could certainly be happening faster. The two main things I wanted to see
happen is a video player implementation that supports all the overlay and alternate track features
we've discussed (dvd done right) and some kind of support for structured text data (my infamous
xml proposal). Audio, Video, and Text are the three pillars of our grand unified theory of
multimedia, and I'd want us to have a experience and a plan for each of them before we write this
stream description format.

Now, anyone could slap an ogg packet format around the experimental tarkin or w3d codecs and start
playing with the video issues; that's ready anytime. Metadata I think needs to be connected to an
external database, so more thinking should happen there. Or we could settle on 'musicbrainz is
good enough' for now.

> One final concern: will hardware players using libvorbis 1.0 or even
> rc1 be able to play back these future Ogg streams, or at least
> intelligently skip past what they don't know about and just play the
> audio?

Yes, this is a real problem. Current libvorbisfile WILL choke on anything but a degenerate vorbis
stream and that's a bug! It will cause much grief if we don't fix it and we must make clear that
handling multiplexed streams is part of the ogg vorbis spec. Jack mentioned this in his post, and
that should be all the future proofing we need.

IMHO,
 -r

BTW, sorry about not mentioning the Ogg MNG code earlier. My normal mailhost is down and the
announcement got queued.

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.