[theora-dev] A/V sync in Theora

[Dan Miller]

this is the same problem you get sometimes trying to convert Quicktime to AVI -- QT has timestamps in millisec, AVI just has a global frame-per-sec field and a framenumber for each frame.  Ogg is more like AVI in that respect.

For the record, MPEG2 only has timestamps that are in effect frame numbers.  Ralph is correct that if the original file is properly constructed, the audio and video should relate in a mathematically perfect fashion.  The problem comes up in this context: many cheaper multimedia systems do not have perfect, frame-accurate A/V sync on input (for instance, the video is usually locked to the incoming signal, but the audio is resampled by the computer's internal audio board using a different clock).  Material captured from such systems will not have a 'standard' framerate, and can even drift around within the file.  AVI files (and, by extension, Ogg streams) can only deal with this in a global way, by slightly changing the frames-per-sec number (that's why we have so much precision in the numerator/denominator).  This usually works (at least for short files -- try to watch a 2-hour AVI file captured on a home system; you'll often have up to 10 frames out of sync in the middle of the movie).  To see how this works, look at your favorite DivX AVI pRon clip using riffwalk, and check out the frames per sec -- it's probably something like 23.92364.  Quicktime is more sophisticated; it can actually stamp each frame with the exact time it was captured.  Therefore, there may be no way to map a Quicktime movie onto AVI (or Ogg/Theora), maintaining perfect sync, without duplicating or dropping frames.

These issues frequently plague anyone who has tried to convert Quicktime files to another format.  We debated going with a timestamp approach, but the framecount won out (for now).

 ___  Dan Miller
(++,) Founder, On2 Technologies

<p>> -----Original Message-----
> From: Ralph Giles [mailto:giles at xiph.org]
> Sent: Thursday, June 12, 2003 11:53 AM
> To: theora-dev at xiph.org
> Subject: Re: [theora-dev] A/V sync in Theora
> 
> 
> On Thursday, June 12, 2003, at 05:30 pm, David Kuehling wrote:
> 
> > Is ist valid for Vorbis packets to drift the granulepos to 
> compensate
> > for A/V sync?  That would mean that granulepos does not 
> represent the
> > number of actually played samples any more.  Seems like an 
> ugly hack.
> 
> Yes it does. Counter to spec too, so I don't recommend it.
> 
> I guess I'm perplexed from the other end. The vorbis granulepos is in 
> number of samples, so multiply by the sample rate to get a timestamp. 
> The theora granulepos can be converted to the frame index, which you 
> multiply by the framerate to get a timestamp. Those timestamps should 
> align at playback. It's up to the player to work out how to make that 
> happen.
> 
>   -r
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 
> 'theora-dev-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No 
> subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
> 
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.