[vorbis] xml stream formats

[John Morton]

On Fri, 07 Jul 2000 12:22:00 +1000 you wrote:

 > 
 > >You want to integrate the lyrics with time signitures with the stream? No
 > >problem:
 > >
 > ><?xml version="1.0"?>
 > ><!DOCTYPE vorbis-metadata SYSTEM "http://www.xiph.org/vorbis-lyrics.dtd">
 > ><vorbis::metadata>
 > >  <timestamp t="00:01.38"/> 
 > >  <chord symbol="Am" fingering="http://www.xiph.org/guitarcharts/Am.png"/>
 > >  <lyric>Pic</lyric>
 > > </vorbis::metadata>
 > >
 > >Shove each of these little XML lyric/subtitle/whatever packets in where
 > >you need them. Notice how small they are? Think of how tiny they'll be
 > >once you compress them with zlib (we're going to do this, right?).
 > >
 > 
 > Well, this isn't really what you were getting at, exactly - but this makes
 > a whole lot more sense. Basically, each packet would be an entire xml
 > document. This gets around my primary objection to XML. MUCH nicer.
 
I've leaped into this debate in mid-stream, so I'm not really sure why
anyone would want to try an interleve XML with the stream other than to 
spare including the header/footer repeatably. As far as I can see the
redundancy caused by separate docs is within acceptable tolerances.

 > I'm still not convinced that XML is neccesarily a good idea for the
 > purposes that are being proposed, but doing it like this will make it clean
 > and workable. I'll stop objecting, but I DO still want people to come
 > forward with a REAL, complete argument as to why XML is a good idea. 
 
I'm not really the best advocate for it as I'm not that well informed. I
have followed the development of ID3v2 and have looked into RSS, so I've
got some idea of the pitfalls to avoid and some of the things that XML
does a good job of. I've made a list below.
 
 > My main objection to XML now is about the complexity of XML - it's not THAT
 > bad, but an XML parser takes up a significant amount of space. For example,
 > the winamp plugin will almost certainly NOT ever support any metadata
 > format based on XML, because of this. We want to get the plugin into the
 > main winamp distribution, and they won't allow it in if it gets too big.
 > I'm guessing that adding an XML parser would double (at least) the size of
 > the compiled plugin. I don't WANT to reject XML because of this, but we
 > have to look at things realistically.
 
This is a good arguement against it. I'm hoping that XML parsing libraries
will be part of the operating system RSN, but I've got no idea if this the
case for win2k, BeOS or MacOS ATM. Some interim solutions could include
creating a customized XML parser that only handles what's necessary to
parse the DTD(s) that we eventually develop, or better yet, separate the
metadata processing into a separate plugin, and provide the hooks for
using it with other audio/video/whatever formats. 

 > Note that zlib won't compress this sort of thing all that much. It's just
 > too short, so the (fairly small) headers get significant. Realistically,
 > we'll probably have sections a little longer, but not all that much. Easy
 > to get around that, by making metadata packets not independent, but that's
 > certainly a bad idea. However, is this worth worrying about? Metadata won't
 > ever take up THAT much space in the stream.
 
Ok, so it's only worth compressing the metadata if it's bytes/second in
the stream is high, and the actual size of the document is above a
threshold where zlib's compression vs latency becomes useless. Handy for
the big gob of metadata that describes the whole track, not worth the
trouble for lyrics and subtitle dispersed through the stream unless the
bitrate and bandwith expectations are really low. I was really just
pre-emptively countering the 'XML is too big' argument :-)

Here's a list of assorted rambling about the topic, and why XML might be
good:

- It's human readable. Just run strings on the file and read!

- XML plus some DTDs is pretty easy to use, so hacking on the parsing end
will be fairly minimal; developers can concertrate on the actual display of the
data, or the GUI needed to enter and edit it. ID3v2 has pretty much bombed
because it's such a bastard to implement the parser. 

- XML parsers are already quite common on a lot of machines. In a lot of
cases, perhaps all in the future, we'll only need to link to the native
XML parser.

- XML parsers already enforce strict syntax adherence, reducing
interopenability problems.

- Backward compatability seems to be pretty eligant. To add more detailed
tags ("Doh! We forgot 'co-producer'!"), we just need to update the DTD to
include it. Older implimentations that use the older DTD will just ignore
tags they don't understand unless they aren't well formed  (Or at least
this is they way it works with Netscape's RDF Site Summary).

- Whilst many tags could be flat, we can still exploit XML's tree
structure in useful ways:

<copyright>
 <year>1995</year>
 <holder>EMI Records Ltd.</holder>
 <license>All Rights Reserved.</license>
</copyright>

- It can be manipulated by XML processors outside of the context of an ogg
player. If you're streaming audio for some radio station you can have a
small metadata packet with the title, artist and an url that points to the
full XML record for that song. If you really want to find out who mastered
the recording at what studio, you can grab the full record - and the site
could format it to be more readable or your XML browser could do the same.
Forget CDDB, imagine if it spat back a full XML record of metadata based
on a hash/track number pair rather than what it give you now.

- We can use different DTDs for different domains, reducing namespace
clashes and further simplifying implementation at the player end. A bit of
comunication between the developers would be useful to keep the sematics
of some of the overlaping tags consistent, but we don't have to develop a
DTD to handle video until we have a video codec or two. 

-----

Ok, so many of these reasons for using XML can just be design
considerations for some other format, and yeah, the size of the parser
will be a problem for plugins if we can't link to a native one. Personally
I'd rather not invent a wheel like ID3v2 when XML provides a pretty good
axel and hub to start with.

John

 

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.