[foms] Proposal: adaptive streaming using open codecs

[Philip Jägenstedt]

I was waiting for this mail to appear on the FOMS list to reply, but it  
didn't, so:

On Thu, 21 Oct 2010 21:22:25 +0200, Christopher Blizzard  
<blizzard at mozilla.com> wrote:

> On 10/21/2010 12:43 AM, Philip Jägenstedt wrote:
>> On Wed, 20 Oct 2010 20:52:10 +0200, Christopher Blizzard  
>> <blizzard at mozilla.com> wrote:
>>
>>> On 10/20/2010 11:46 AM, Jeroen Wijering wrote:
>>>> On Oct 20, 2010, at 8:45 PM, Christopher Blizzard wrote:
>>>>
>>>>> On 10/20/2010 5:24 AM, Jeroen Wijering wrote:
>>>>>> Again, the proposal from Christopher on providing a "Manifest API"  
>>>>>> (basically a playlist of chunks) plus having some QOS metrics  
>>>>>> (bandwidth, framedrops) would already allow developers to build  
>>>>>> adaptive streaming on the javascript level. Far easier for a first  
>>>>>> implementation. I guess we swiftly need a proposal for the  
>>>>>> "Manifest API".
>>>>> Note that one of Philip's suggestion's (maybe not on the list? I  
>>>>> can't remember.) was that we do the API before we do the manifest  
>>>>> work.  This would allow us to iterate, test and figure out what  
>>>>> worked before figuring out what we needed in the manifest.
>>>> Yes, that was Philip's proposal as well. Makes a lot of sense.
>>>>
>>>> - Jeroen
>>>
>>> Also would allow us to test out switching algorithms that we might want
>>> to include in browsers by default.  And (*gasp*!) specify them.
>>>
>>> --Chris
>>
>> I support this message :)
>>
>> In some way or another, we need to achieve gapless playback. These are  
>> the options I know of so far:
>>
>> 1. A concatenation API (maybe Stream) to form a single stream from  
>> multiple URLs. This would basically be a byte concatentation API, and  
>> assumes that we either have the chunks be plain slices or that we  
>> support chained Ogg/WebM gaplessly. It has some similarity to a  
>> Manifest API in that it lists several URLs. The difference may be that  
>> the video element isn't aware of the multiple resources, that's all  
>> hidden in the URL, effectively made part of the network layer of the  
>> browser.
>>
>
> Basically an API that says "Play this chunk of video next"?  I think  
> that's what I've pushed for, but it's a decent amount of work.  I'm not  
> sure what the rules are for that esp. wrt sound sync.  Also I don't  
> think it has to be byte-concatination if we have decent support for  
> moving from one video to the next on a frame-by-frame basis.

The difference is if the concatenation is in terms of bytes or in terms of  
media resources. The manifest idea would seem to allow concatenating  
arbitrary media resources, which I think is far for difficult to  
implement, as it requires the media framework layer to be aware of each  
chunks, while byte concatenation allows it to treat it as an infinite  
stream.

>> 2. Have each chunk in its own <video> and add a synchronization API.  
>> The main use case for this is synchronizing external audio tracks, but  
>> as a side-effect one could allow synchronizing two clips at their  
>> edges. My assumption is that we will want such a sync API eventually  
>> anyway. However, it's not a terribly obvious way of thinking about  
>> gapless playback. Also, it would require switching the video elements  
>> at the exact right time, and <track> elements would have to be  
>> duplicated...
>>
>> Does anyone have opinions on either of these approaches? Are there  
>> others?
>>
>
> I feel like this is a separate thing entirely?

Implementation wise I think this is pretty much equivalent to switching on  
a frame-by-frame basis, as both involve syncing two different decoding  
pipelines and switching over.

-- 
Philip Jägenstedt
Core Developer
Opera Software