[foms] Proposal: adaptive streaming using open codecs

[Philip Jägenstedt]

On Thu, 18 Nov 2010 20:38:46 +0100, Silvia Pfeiffer  
<silviapfeiffer1 at gmail.com> wrote:

> On Fri, Nov 19, 2010 at 12:50 AM, Philip Jägenstedt <philipj at opera.com>  
> wrote:
>> On Thu, 18 Nov 2010 02:27:43 +0100, Silvia Pfeiffer
>> <silviapfeiffer1 at gmail.com> wrote:
>>
>>> On Thu, Nov 18, 2010 at 5:15 AM, Timothy B. Terriberry
>>> <tterriberry at mozilla.com> wrote:
>>>>> Can you explain what's a LSP extrapolation ? A quick google doesn't
>>>>> give anything.
>>>>
>>>> Probably because I meant LPC (linear prediction coefficients), not LSP
>>>> (line spectral pairs). The concepts are related.
>>>>
>>>> Vorbis, for example, already uses LPC extrapolation to pad the first  
>>>> and
>>>> last MDCT blocks, since blocks are overlapped, but there's no real  
>>>> data
>>>> to overlap with at the beginning and end of a track. Code starts at
>>>> https://trac.xiph.org/browser/trunk/vorbis/lib/block.c#L416
>>>
>>>
>>> Would a media framework typically expose that level of API? How would
>>> a decoder feed a player with audio data such that this works? Would it
>>> just make sure to hand over the next chunk of data or is there any
>>> decoding pipeline setup change required or something?
>>
>> I can speak for the two media frameworks I have experience with:  
>> GStreamer
>> and DirectShow. As far as I've seen, neither have any high-level support
>> for stitching together audio streams. Rather, you'll need to have
>> something like a joiner element in the decoding pipeline into which you
>> feed all audio streams. Usually it will do nothing, but at when one  
>> stream
>> ends and the other begins it'll stitch them. The audio sink would only  
>> see
>> one audio stream coming in (in other words, the joiner would also have  
>> to
>> resample). Not sure if it'll work well in practice, I haven't tried
>> implementing anything yet.
>
>
> That was what I thought. In theory the stitching would work well, but
> is not supported by media frameworks, so will be hard to implement by
> most media frameworks. It may be possible to hack it together with vlc
> or ffmpeg or a home-baked library, but anything the is built on a
> filter graph such as GStreamer or DirectShow or QuickTime would be
> difficult since often the pipeline cannot be reset with new decoding
> parameters without flushing out all the data - correct?

Any kind of intelligent stitching that tries to smooth over any gaps will  
be hard to implement, correct.

However, in the last week I've been happy to find that simple gapless  
playback of chained Ogg actually works very well in GStreamer and in  
Opera. I'm not sure how well it will work if the number of channels or the  
sampling rate changes, though, because then the audio sink will have to  
renegotiate the format and likely not keep sync very well.

-- 
Philip Jägenstedt
Core Developer
Opera Software