[foms] Proposal: adaptive streaming using open codecs

[Andy Berkheimer]

On Mon, Nov 1, 2010 at 7:46 PM, David R <videophool at hotmail.com> wrote:
>
> On Tue, 2 Nov 2010, Sylvia wrote:
> > I was merely referring to the need to support both models in a HTML5-only future with adaptive streaming.
> Supporting both models is fine, but they are very different use cases.  Maybe I misunderstood the original post, but it seems that the use cases are being mixed.  IMHO, the adaptive streaming spec should not address non-adaptive streaming use cases.

I _think_ we're talking about a small set of components which can be
added to browsers and composed to address most streaming use cases.
To me the fundamental principle is to think of media as a series of
resources rather than a single resource.  Everything else starts to
fall out from there.

* An interface to feed a stream into a media element as a series of
concatenated resources ('chunks') rather than a single resource.
Client runtime exposing buffering behavior controls.  Support changing
some parameters (clearly define what can and cannot change:
resolution, number of channels, etc) from one chunk to the next.

Re: Chris Blizzard's WebSocket proposal - quite interesting, though
I'm concerned about how that would work with third party CDN oriented
scenarios?

[ open question around: feeding video and audio into a media element
separately, with the element performing synchronization.  Desirable
and powerful, but very hard to splice into most existing
implementations without a significant revamp... ]

* An interface exposing performance metrics (e.g. dropped frame
counters, network read bandwidth) out of the appropriate elements.

* A manifest format describing the list of resources to be
concatenated along with any other metadata or bootstrap information
needed by the client application logic.  May include multiple lists of
resources at different bitrates representing the same presentation.

* A chunk format defining how the chunk resources are structured.
Ideally you have the property that 'cat chunk1 ... chunkX > allchunks'
reproduces the original media resource.

Also acceptable, each chunk can take a contiguous slice of an
unchunked media resource and prepend a small header.  This header
contains extra information to optimize processing in the client
application and/or make the chunk a valid media resource in its own
right.

With these simple blocks you can implement fully automatic adaptive
streaming, or make user-initiated stream changes (resolution,
language, captions, camera angle, etc) smoother, or non-adaptive
streaming with minimal buffering, or non-adaptive streaming with
aggressive buffering, etc.

-Andy