[ogg-dev] oggfile, skeleton and vorbis tools

[Ian Malone]

illiminable wrote:
> 
> ----- Original Message ----- From: "Ian Malone" <ibmalone at gmail.com>
> To: <ogg-dev at xiph.org>
> Sent: Friday, March 10, 2006 8:42 AM
> Subject: Re: [ogg-dev] oggfile, skeleton and vorbis tools
> 
> 
>> Re-reading Monty's email: when you talk about vorbis-only vorbisfile and
>> the concurrent vorbis stream case do you mean vorbisfile will need to be
>> able to choose?  I'd been thinking (as above) about cases where there is
>> a Vorbis stream in with other stuff, the first two examples are
>> realities.  The obvious applications for multiple concurrent Vorbis
>> streams are the same as for DVD: multiple languages, commentaries,
>> scores (possibly needing mixing).  Deciding what to play needs
>> application involvement; how much should vorbisfile be able to do, and
>> how much should wait until oggfile (and how much should be left to media
>> players that understand all the complexities involved)?
>>
> 
> Re: multiple audio tracks, there are quite a few ogm files out there 
> with multiple vorbis streams, typically the default language is first. 
> So if you made vorbisfile drop unknown streams, these are potential 
> files people might try to play with it.
> 

I didn't realise there were files in the wild like this, that's
interesting.  So the question is, what should vorbisfile be expected
to do on encountering one?  The motivation is that players based on
vorbisfile will continue happily playing Oggs with annotation streams,
so you can start confidently rolling out things like skeleton in the
default tools without breaking players.  Once you have multiple
concurrent audio streams you've reached a much more complex situation
(my understanding: we've got an audio player looking through content;
if it finds one and only one audio stream we know what to do,
regardless of what else is there, if it finds more than one we are
here...).

> The other case, i have a few such files for testing, is two audio 
> streams designed to play concurrently. For example a dj type scenario... 
> intentionally playing two files over the top of each other. An even more 
> useful case is for example an online stream, where the music is in one 
> stream, and the dj/announcer can voiceover in the other stream... i've 
> got a few test files like this also. Vorbis+Vorbis and Speex+Vorbis 
> examples. I think somewhere i've got a file that plays four songs 
> simultaneously, even though this file is not particularly useful, it was 
> just a good test to see how my directshow filters would handle it. 
> Another scenario where this would be useful might be having different 
> instruments in each track.
> 

Actually, this is kind of what I meant by 'possibly needing mixing',
unlike a DVD which stores the soundtracks seperately you could have
the score as one stream, the dialogue (in multiple languages) in
another stream (similar to your dj voiceover example).  Put a third
commentary stream in and now you might want to dynamically change the
mixing.  You'd need CMML to do that as well as skeleton (I'm guessing
you could do this in CMML somehow).

(I'm not so sure about the seperate tracks per instrument thing though,
if you're mastering wouldn't you want a lossless format?  Maybe the
Flamming Lips might use it eventually.)

> Obviously neither vorbis file nor a player can tell whether the file 
> author intended for the streams to play at the same time or be 
> alternates... or even more complicated would be groups that are 
> alternates, ie either play these two together, other wise play these two 
> together... or always play this one with the music, and choose one from 
> voiceover streams to play concurrently with the music.
> 
> VLC for example will let you choose an audio stream, however there is no 
> option for playing them at the same time.
> 
> This is something we might want to extend skeleton to support, i know 
> we've talked about this kind of thing before. So that skeleton can 
> provide some kind of mapping as to how the various streams relate to 
> each other.
> 

You need player support to be able to collect the options and present
them to the user.  Without skeleton and something like a fully capable
oggfile you can't do it, as you've pointed out you can mix Speex and
Vorbis. (I recently, accidently, had a file which contained mp3 and
Vorbis.  Obviously no Xiph tool will ever handle or produce that.)  So,
with the aim that it's intended for simple audio only players (probably
ones already using it, or operating with limited resources), what
should vorbisfile be capable of?

-- 
imalone