[theora] Indexing Ogg files for faster seeking

[Chris Pearce]

On 21/10/09 13:52, Silvia Pfeiffer wrote:
> I downloaded the specially compiled Minefield and played with the files a bit.
>    

Make sure you run each minefield instance with the --no-remote command 
line option, else it might just pop open a new window of an already 
running instance of Firefox, rather than starting a new instance of the 
program.

The index-capable builds dump a whole lot of output to stdout, so if 
you're seeing that, you're running the index-capable build.

> I ran http://pearce.org.nz/video/arctic_giant.ogg in current Minefiled
> (3.7a1pre) and in your special Minefield.
> Interestingly, I found current Minefield to take longer in loading the
> file initially, but then being faster in offset seeking.
> Is this because current Minefiled buffers the whole file first and
> then seeking is local and fast, while your Minefield does seeking over
> the network still?
>    

The buffering behaviour is not changed by the index-seek code. The 
buffered areas show up as a lighter shade of gray in the progress bar, 
and seeking inside buffered video should be almost instantaneous, 
regardless of whether you're seeking in indexed or non-indexed files.

> I found the same with the comparison in
> http://pearce.org.nz/video/indexed-seek-demo.html - seeking on the
> file with the index is slower. I am a bit puzzled. Would you know
> what's happening?
>    

That's really weird. Are you running the index-capable build with 
--no-remote? You should see debug output on stdout containing 
"get_seek_point" etc?

You also could be suffering from random network latency spikes and 
variances in network caches in between you and my server. I have a 
custom rate limited webserver which I run locally to test performance, 
to eliminate these variables. The index-capable builds seek about 60-70% 
faster in indexed files served from my local server.


Chris P.

> Thanks,
> Silvia.
>
>
> On Wed, Oct 21, 2009 at 7:42 AM, Chris Pearce<chris at pearce.org.nz>  wrote:
>    
>> Back by popular demand, a new version of OggIndex, which encapsulates the
>> keyframe index in the skeleton track. Available here:
>> http://github.com/cpearce/OggIndex/tree/skeleton-index-per-stream
>>
>> I added a few fields to the skeleton ident header, and added "index" packets
>> after the "fisbone" packets. I increased the skeleton version field to 3.1.
>> Below is the skeleton track specification I used.
>>
>>
>> Skeleton 3.1 Spec - Includes one "index" packet per stream.
>>
>> Skeleton header packet:
>>
>> 1.  Identifier: 8 bytes, "fishead\0".
>> 2.  Version major: 2 Byte unsigned integer signifying the major (3)
>> 3.  Version minor: 2 Byte unsigned integer signifying the minor (1)
>> 4.  Presentationtime numerator: 8 Byte signed integer
>> 5.  Presentationtime denominator: 8 Byte signed integer
>> 6.  Basetime numerator: 8 Byte signed integer
>> 7.  Basetime denominator: 8 Byte signed integer
>> 8.  UTC [ISO8601]: a 20 Byte string containing a UTC time
>> 9.  [NEW] Start time, the presentation time in milliseconds of the
>>      first sample in the media. 8 byte signed integer, -1 if unknown.
>>      Note that samples between the Start time and the Presentationtime
>>      are not supposed to be shown.
>> 10. [NEW] End time, the end time of the last sample in the media.
>>      8 byte signed integer, -1 if unknown.
>> 11. [NEW] The length of the segment, in bytes: 8 byte signed integer,
>>      -1 if unknown.
>>
>> Skeleton 'fisbone\0' packets as per skeleton track 3.0, one per stream.
>>
>> [NEW] Skeleton index packets. There should be one per content stream,
>> coming after the fisbone packets, before the skeleton eos packet:
>>
>> 1. Identifier 6 bytes: "index\0"
>> 2. The serialno of the stream as a 4 byte field.
>> 3. The number of keypoints in the index packet, 'n' as a 4 byte
>>     unsigned integer. This can be 0.
>> 4. 'n' key points, each of which contain, in the following order:
>>      - the page's byte offset as an 8 byte unsigned integer, followed by
>>      - the checksum of the page found at the offset, as a 4 byte field,
>>        followed by
>>      - the presentation time in milliseconds of the key point, as an 8
>>        byte unsigned integer.
>>
>> Existing player compatibility with index-in-skeleton and
>> index-in-index-track files:
>>
>> Firefox 3.5.3 plays the index-in-skeleton files, purely by luck, whereas it
>> can't get the duration of index-in-index-track files due to a bug in
>> liboggz. Firefox3.5 uses liboggplay, which assumes any skeleton packet which
>> doesn't have "fishead" magic bytes is a "fisbone" packet. It's possible that
>> a skeleton index packet would break FF3.5 due to being parsed as a fisbone
>> packet. So regardless of which approach we took, we'd have to patch Firefox
>> 3.5.x.
>> The ogg DirectShow codecs work ok with index-in-skeleton files, but seeks in
>> index-in-index-track files result in a seek to 0.
>> VLC - plays both index-in-skeleton and index-in-index-track files,  but
>> seeks to 0 in index-in-skeleton files result in the video window
>> disappearing, but the media can still be played again after this happens.
>> XiphQT - plays and seeks (inside buffered ranges) with both types of files
>> Totem/gstreamer - with both index-in-skeleton and index-in-index-track
>> files, Totem/gstreamer prompts for a missing plugin on load, but will still
>> play the file. All seeks just reset playback position to 0. I assume
>> gstreamer must be checking the skeleton version field, and/or prompting when
>> it reads the packets/tracks with unrecognized magic bytes.
>>
>> Firefox index-in-skeleton capable builds available for download here:
>> http://build.mozilla.org/tryserver-builds/cpearce@mozilla.com-try-b7b626658548
>>
>> Indexed videos for testing available here:
>> http://pearce.org.nz/video/
>>
>>
>> Chris P.
>>
>>
>> _______________________________________________
>> theora mailing list
>> theora at xiph.org
>> http://lists.xiph.org/mailman/listinfo/theora
>>
>>
>>      
>