[foms] Adaptive streaming

Mon Nov 1 12:55:09 PDT 2010

Hi Tracey,

FFmpeg does write the CUES element at the end of the file.

Frank

On Mon, Nov 1, 2010 at 3:06 PM, Tracey Jaquith <tracey at archive.org> wrote:

> thanks, frank!
> I'll keep my eyes out for it if/when we head in that direction
> (though it's a ways out for us to transcode to WebM,
>   we lean heavily on ffmpeg for the decoding
>   and dump rawvideo either to x264 binary (for h.264) or
>   theora_encoder_example binary (for theora)
>   for the video transcode and remux the A/V...
> so hopefully ffmpeg either has CUES element generation
> or will by the time we embark on it (or use something like
>  vpxenc)
>
> --tracey
>
>
> On Nov 1, 2010, at 10:10 AM, Frank Galligan wrote:
>
> If you transcode to WebM I think most encoders will include a CUES element
> which contains the data you are talking about. That is how most players
> decide how to seek within a file.
>
> Frank
>
>
> On Fri, Oct 29, 2010 at 4:23 PM, Tracey Jaquith <tracey at archive.org>wrote:
>
>> btw, we are thinking of recoding a large portion of our videos here at
>> archive.org
>> (possibly all) to add some kind of file that describes the keyframes and
>> byte position and/or frame# and/or timecode in some manner.
>>
>> so if the group comes up with some "best practices" (and we certainly like
>> JSON here)
>> or format suggestions, we will try to go with that.
>>
>> -tracey
>>
>>
>> On Oct 29, 2010, at 1:17 PM, Mark Watson wrote:
>>
>> >
>> > On Oct 29, 2010, at 7:43 AM, Jeroen Wijering wrote:
>> >
>> >>
>> >> On Oct 27, 2010, at 5:51 PM, Mark Watson wrote:
>> >>
>> >>>> Do files with a compacted index still play on devices that do not
>> expect it? Think current mobile phones.
>> >>>
>> >>> In DASH its just a fragmented MP4 file with an extra box ("Segment
>> Index") near the beginning. Existing players would ignore the new box.
>> >>
>> >> Do you have an example video? I'm curious to see one and do a few
>> tests.
>> >>
>> >>
>> >>>> If WebM / Ogg by its nature already has much smaller headers, one of
>> the big drawbacks of range-request streaming would not be valid for these
>> containers.
>> >>>
>> >>> I need to read up on the WebM format (bit of a confession, being on
>> this list, sorry!). But I understood you have the framing information with
>> the samples and therefore smaller headers and probably no need for any kind
>> of formal fragmentation. So your "fragment" would just be a notional concept
>> of a group of samples spanning some time period. Your index would provide
>> time and byte offsets for these fragments. If you already have something
>> which provides time and byte offsets for Random Access Points (for seeking)
>> you could probably re-use that. (Re-using Movie Fragment Random Access box
>> in mp4 was discussed instead of creating the new Segment Index. Segment
>> Index was chosen for some slightly obscure technical reasons).
>> >>
>> >> Just did a quick test with a 15-minute WebM video (keyframes 2-6s):
>> >>
>> >> http://content.bitsontherun.com/videos/a95zAVN1-710492.webm
>> >>
>> >> The first Clusterheader appeared at 44k.
>> >>
>> >> That's pretty nice compared to an MP4 conversion with the same keyframe
>> interval, where the MOOV box is 480k:
>> >>
>> >> http://content.bitsontherun.com/videos/a95zAVN1-600332.mp4
>> >>
>> >> So it looks like my concern with range requests (startup delays due to
>> big headers) is not valid for WebM files....
>> >>
>> >> ----
>> >>
>> >> If the videoElement would expose a callback with valid range-requests
>> to javascript, the scripting layer would be aware of the valid ranges in a
>> quality level. For example something like "onSeekpoints" that returns:
>> >>
>> >> [
>> >> {position: 0,000,start: 8839, end:9957},
>> >> {position: 1,287,start: 9957, end:10994},
>> >> {position: 3,071,start: 10995, end:19840}
>> >> ]
>> >>
>> >> Next, this info can be used to do adaptive streaming using the
>> appendVideo function:
>> >>
>> >> videoElement.appendVideo("http://example.com/video_800.webm", 8839,
>> 9957);
>> >>
>> >> With this, plús some way to retrieve the bandwidth / bytesLoaded, it
>> should be possible to build a fully functional WebM adaptive streaming demo
>> in javascript - using existing WebM files.
>> >>
>> >> Would this work? Are there oversights? What do the FOMS browser
>> developers think of such a first step?
>> >
>> > The one additional thing I think is needed is that you need also to give
>> the player the file headers for each variant (Header/Segment/Track for WebM,
>> Movie Header and Segment Index for DASH mp4). Maybe also the Cueing Data in
>> WebM can be used as an index ?
>> >
>> > There are various ways to do this and those more expert in HTML5 than me
>> can suggest the best, but one way might be:
>> >
>> > tag = videoElement.initializeVideo( "http://example.com/video_800.webm"
>> )
>> >
>> > then onSeekpoints returns { tag: 1, seekpoints = [ { position: 0, start:
>> 8839, end: 9957 }, ... ] }
>> >
>> > then videoElement.appendVideo( tag, "http://example.com/video_800.webm",
>> 8839, 9957 )
>> >
>> > You are still appending to a single common playout buffer, the tag is
>> about where the information comes from, or rather how to understand it, not
>> where it is appended to.
>> >
>> > Same, independently, for audio.
>> >
>> > ...Mark
>> >
>> >
>> >>
>> >> Kind regards,
>> >>
>> >> Jeroen
>> >> _______________________________________________
>> >> foms mailing list
>> >> foms at lists.annodex.net
>> >> http://lists.annodex.net/cgi-bin/mailman/listinfo/foms
>> >>
>> >
>> > _______________________________________________
>> > foms mailing list
>> > foms at lists.annodex.net
>> > http://lists.annodex.net/cgi-bin/mailman/listinfo/foms
>>
>> _______________________________________________
>> foms mailing list
>> foms at lists.annodex.net
>> http://lists.annodex.net/cgi-bin/mailman/listinfo/foms
>>
>
> _______________________________________________
> foms mailing list
> foms at lists.annodex.net
> http://lists.annodex.net/cgi-bin/mailman/listinfo/foms
>
>
>
> _______________________________________________
> foms mailing list
> foms at lists.annodex.net
> http://lists.annodex.net/cgi-bin/mailman/listinfo/foms
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.annodex.net/cgi-bin/mailman/private/foms/attachments/20101101/4879d47c/attachment-0001.htm 