[Theora-dev] A/V sync strategies

[Ralph Giles]

On Sat, Mar 05, 2005 at 12:10:49AM -0500, Keenan Pepper wrote:

> Theora and MPEG2 handle A/V sync in different ways. Theora has an 
> absolutely fixed frame rate and the time at which a frame should be 
> displayed can be determined by multiplying the frame rate by the number 
> of frames in the stream so far. That's why the header has an exact 
> rational frame rate. MPEG video's frame rate can vary slightly because 
> each frame has an associated PTS ("presentation timestamp"), which do 
> not form an exact arithmetical sequence.

That's true for theora's part. I can't comment on the DVD-Video half (or 
some other mpeg2-based format).

> So it seems to me the only way to keep A/V sync when transcoding from 
> MPEG or any timestamp-based format into Theora is to drop frames or 
> alter audio samples when the A/V offset gets too high. Neither the 
> encoder_example nor ffmpeg2theora currently does this, so the sync drifts.

If the input is in general variable framerate (or variable samplerate 
for vorbis/flac/speex audio) then the theora encoder must account for 
this and add/drop/interpolate frames (or audio samples) to create a 
fixed-rate stream.

If DVD-Video in fact works as you suggest, then all dvd players must 
have some similar add/drop frame logic to handle playback on their usual 
target, which is a fixed-framerate television. The only difference is 
that this logic would have been moved to the encoder in theora.

 -r