[theora] Indexing Ogg files for faster seeking

[Gregory Maxwell]

On Fri, Jan 22, 2010 at 12:47 PM, Jason Self <jason.self at gmail.com> wrote:
> On Thu, Jan 21, 2010 at 3:46 PM, Chris Pearce <chris at pearce.org.nz> wrote:
>> I have been experimenting with compressing the keyframe indexes.
>
> I think you are awesome and this is GREAT!
>
> I've been playing around with the daily builds of ffmpeg2theora and it
> seems to work well overall. There's still the issue that it can't
> index files where the container doesn't have a duration.
>
> I've inserted a previous discussion with Chris. I've been meaning to
> ask this question but never got around to it. This seems like as good
> of a time as any.
>
> Chris says that the guys on #theora were vehement that the output not
> be rewritten to index it. My question is: What's the harm? It'd only
> need to do this in cases where the duration is not known. OggIndex
> does this, so what's the harm? If it's okay for the OggIndex to do it,
> why not ffmpeg2theora?

Because it doubles the storage requirement temporarily and/or creates
a potential for corruption.

It also creates a lot of unneeded disk I/O.   It's also totally
unnecessary:  The correct thing to do is to reserve some space at the
start, then go and fill it in at the end.  If you over-estimated the
needed space, thats no harm, if you under-estimated you simply drop
some index points.  Since the index is already lossy applications
already need to cope with that.

I thought ffmpeg2theora was doing this already. The message you're
quoting from is old, are you sure this is still the case?


>>> [1] http://lists.xiph.org/pipermail/theora/2009-November/003025.html
>>>
>>
>> I asked Greg on the list exactly what information you can use to reliably
>> guesstimate the duration, and he never responded, which I took to mean
>> "d'oh, there is no way to reliably guesstimate the duration". ;)

Reliably, no. E.g. what happens when the input is a non-seekable
stream (as is the case when encoding live from a camera)?

Even knowing the duration you can only roughly size the index because
you don't know exactly how many seekpoints there will be.

But thats okay, just reserve some space and make use of what you got.
If you do have a more reliable source of information, you can make use
of it, but there are lots of situations where it won't be available.

I do wonder if the theora two-pass API should make mode decision
information available, because in two pass that would give you a more
reliable indication of the number/location of seekpoints.