[foms] Proposal: adaptive streaming using open codecs

[Christopher Blizzard]

On 11/8/2010 1:29 PM, Mark Watson wrote:
> How you specify what you want in the appendVideo call, and how the browser loads it are somewhat independent. The browser could use small byte range requests or a single open-ended request: the instructions from the JS layer are about what to play, not exactly how to fetch it.

OK, to put this in a set of requirements in order to contain scope:

* The ability to pass already de-muxed but encoded audio & video into 
the video element for playback.
* That data can be from an HTTP download and exposed as an XHR result 
and then passed into the video element.  Data could also come from a 
WebSocket connection, or somewhere else.
* The ability to mark some audio data as "might contain gaps" so the 
audio layer can do the right thing when skipping from one stream to another.

This means that the JS code is required to actually figure out how to 
get the data, what the rate of the next chunk will be and which type of 
audio stream to use.  It also means that the JS code needs to know how 
to handle the index, how to seek, etc.  I think that it offers some 
great opportunities for fast start and to start experimenting with 
WebSockets for streaming as well.

What Chris is proposing is different, and higher level, and makes 
assumptions about formats and how things will be chunked on the server.  
(Or at least how a custom server will translate byte requests into 
mappings into files.)

The idea here is that once we have some best practices we can talk about 
a higher level format that specifies how a client could do the right 
thing about switching rates in the simple-for-the-developer case.  But I 
feel like we need to learn how things will work based on a lower-level 
API.  The use case of a standalone player matters to me, and I want to 
make sure we don't lose that over the long term.

But this is also worth doing because we want the lower-level API anyway.

Chris, this is pushing back on what you are suggesting, so I want to 
hear what you think.

--Chris