[foms] Proposal: adaptive streaming using open codecs

[Jeroen Wijering]

On Oct 18, 2010, at 10:00 PM, Christopher Blizzard wrote:

> One thing that's not in the proposal is what the accompanying API would contain, since that will have to be built in lock-step with the actual manifest & file bits.

The API is included, in the "scripting" section of the proposal. It follows e.g. the mozDroppedFrames / mozDecodedFrames enhancements. 

However, there's indeed no calls for a "Video data API" in the spec. While also very nice to have (for security, and all other reasons we talked about), I didn't think it should be part of the adaptive streaming spec. Going that way would require every publisher that wants to do adaptive streaming to build code for:

* retrieving chunks through websockets
* retrieving and parsing manifest files
* (possibly) demuxing container formats in javascript

However, again, having a Video.appendData(videoData) API call will be nice as well...

> Also one thing the file might want to contain is channel information for audio.  Our eyes might adjust easily to changes in quality, but changes in the quality of the audio might be something that a UA might want to try to avoid, especially on a regular basis.  You're much more sensitive to flipping back and forth on audio than you would be on video.

Yes, that could indeed be added. Another candidate IMO would be the video framerate, since that can be used to quickly decide whether to drop a stream level due to decoding (droppedFrames) issues. I left both out so we could remain 100% compatible with Apple's M3U8 format.

The M3U8 spec has nothing in this area, b/c Apple is not supporting different audio channel setups between stream levels. They might be interested in this, as Silverlight also just included multi-channel support [1]. 

- Jeroen

[1] http://www.reelseo.com/silverlight-srs-51-support-toolkit/