[foms] Proposal: adaptive streaming using open codecs

[Chris Pearce]

On 9/11/2010 10:54 a.m., Christopher Blizzard wrote:
> On 11/8/2010 1:29 PM, Mark Watson wrote:
>> How you specify what you want in the appendVideo call, and how the browser loads it are somewhat independent. The browser could use small byte range requests or a single open-ended request: the instructions from the JS layer are about what to play, not exactly how to fetch it.
> OK, to put this in a set of requirements in order to contain scope:
>
> * The ability to pass already de-muxed but encoded audio&  video into
> the video element for playback.
> * That data can be from an HTTP download and exposed as an XHR result
> and then passed into the video element.  Data could also come from a
> WebSocket connection, or somewhere else.

In terms of implementing this in Firefox, our internal decoders assume 
they'll be passed muxed data, so you'd still need to push muxed A/V data 
to them (or at least data which is still in a valid Ogg/WebM container), 
otherwise we'd need to make significant changes to our internal decoder 
infrastructure.

Given our existing infrastructure, I think it's about the same amount of 
work for /us /to implement a "push raw chunks of data to the decoders" 
as it is to implement "append this url,time_range to the playback queue" 
approach which Jeroen proposed. The difference is Jeroen's proposal 
doesn't require messing with websockets, which means we need to write 
less JS to test it. We could do something relatively quickly, but I 
wouldn't try too hard to make it seamless, I'd reserve that effort for 
the final product. To get seamless stream switching (particularly for 
audio), we'd need to refactor our internal decoder infrastructure 
significantly, which is something we were planning on doing post FF4 
anyway, in order to implement seamless looping (for the @loop attribute, 
among other things).

> * The ability to mark some audio data as "might contain gaps" so the
> audio layer can do the right thing when skipping from one stream to another.

WebM can imply gaps by omitting Vorbis samples during encode, and I 
think that most browsers which support WebM already handle this. The 
players for other formats HTML5 formats may need to be updated for this 
though.

> This means that the JS code is required to actually figure out how to
> get the data, what the rate of the next chunk will be and which type of
> audio stream to use.  It also means that the JS code needs to know how
> to handle the index, how to seek, etc.  I think that it offers some
> great opportunities for fast start and to start experimenting with
> WebSockets for streaming as well.
>
> What Chris is proposing is different, and higher level, and makes
> assumptions about formats and how things will be chunked on the server.
> (Or at least how a custom server will translate byte requests into
> mappings into files.)
>

You don't need to chunk the files on the server, you can play time 
ranges of the file. If you could chunk the files into small segments, 
that would be slightly easier to get started with though.

I think it's about as much work for Firefox to implement an 
appendVideo(url,timeRange) as it is to implement appendVideo(rawChunk), 
so I think it's better to implement the former, as it'll be a simpler 
interface on the JS side.


Chris P.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.annodex.net/cgi-bin/mailman/private/foms/attachments/20101109/53a5e1e0/attachment-0001.htm