[Theora-dev] A/V sync strategies
keenanpepper at gmail.com
Fri Mar 4 21:10:49 PST 2005
I tried to encode some video from a DVD with encoder_example and
ffmpeg2theora, and it looked great, except that with both encoders the
A/V sync would drift noticeably after about 10 minutes, and by the end
of a two hour movie it would be a couple seconds off. So instead of just
whining about this and waiting for someone else to fix it, I decided to
try to learn about what's causing the problem and maybe even fix it
myself. Being a newbie, I'll just explain my understanding of it and you
experts can point out any misunderstandings and enlighten me.
Theora and MPEG2 handle A/V sync in different ways. Theora has an
absolutely fixed frame rate and the time at which a frame should be
displayed can be determined by multiplying the frame rate by the number
of frames in the stream so far. That's why the header has an exact
rational frame rate. MPEG video's frame rate can vary slightly because
each frame has an associated PTS ("presentation timestamp"), which do
not form an exact arithmetical sequence.
Is this right so far?
So it seems to me the only way to keep A/V sync when transcoding from
MPEG or any timestamp-based format into Theora is to drop frames or
alter audio samples when the A/V offset gets too high. Neither the
encoder_example nor ffmpeg2theora currently does this, so the sync drifts.
If this is a correct description of the situation, I'll start hacking on
ffmpeg2theora to implement some kind of sync-adjusting method.
(Obviously it would be impossible with encoder_example because that only
accepts raw audio and video with no sync information such as PTSs.)
More information about the Theora-dev