[theora-dev] A/V sync in Theora

[David Kuehling]

Forget my previous mail, it just starts getting somewhat clearer.  But I
think, that at least mplayer employs some kind of A/V sync strategy for
AVI which takes information about the interleaving of audio and video
packets into account (according to tech/formats.txt).  I frequently use
mencoder to record from my v4l tuner card to avi.  Sound comes from my
soundcard and is thus guaranteed to not be accurately synced with video.
I never encountered A/V desyncs even for long recordings.

So judging by usability, I get more and more concerned, that I'll never
be able to properly transition to Theora/Vorbis.  I thought Ogg/Theora
would be somewhat revolutionary new technology, but the more I learn
about it, the more I'm disappointed.

David

>>>>> "Dan" == Dan Miller <dan at on2.com> writes:

> this is the same problem you get sometimes trying to convert Quicktime
> to AVI -- QT has timestamps in millisec, AVI just has a global
> frame-per-sec field and a framenumber for each frame.  Ogg is more
> like AVI in that respect.  For the record, MPEG2 only has timestamps
> that are in effect frame numbers.  Ralph is correct that if the
> original file is properly constructed, the audio and video should
> relate in a mathematically perfect fashion.  The problem comes up in
> this context: many cheaper multimedia systems do not have perfect,
> frame-accurate A/V sync on input (for instance, the video is usually
> locked to the incoming signal, but the audio is resampled by the
> computer's internal audio board using a different clock).  Material
> captured from such systems will not have a 'standard' framerate, and
> can even drift around within the file.  AVI files (and, by extension,
> Ogg streams) can only deal with this in a global way, by slightly
> changing the frames-per-sec number (that's why we have so much
> precision in the numerator/denominator).  This usually works (at least
> for short files -- try to watch a 2-hour AVI file captured on a home
> system; you'll often have up to 10 frames out of sync in the middle of
> the movie).  To see how this works, look at your favorite DivX AVI
> pRon clip using riffwalk, and check out the frames per sec -- it's
> probably something like 23.92364.  Quicktime is more sophisticated; it
> can actually stamp each frame with the exact time it was captured.
> Therefore, there may be no way to map a Quicktime movie onto AVI (or
> Ogg/Theora), maintaining perfect sync, without duplicating or dropping
> frames.

> These issues frequently plague anyone who has tried to convert
> Quicktime files to another format.  We debated going with a timestamp
> approach, but the framecount won out (for now).

>  ___ Dan Miller (++,) Founder, On2 Technologies

<p>>> -----Original Message----- From: Ralph Giles [mailto:giles at xiph.org]
>> Sent: Thursday, June 12, 2003 11:53 AM To: theora-dev at xiph.org
>> Subject: Re: [theora-dev] A/V sync in Theora
>> 
>> 
>> On Thursday, June 12, 2003, at 05:30 pm, David Kuehling wrote:
>> 
>> > Is ist valid for Vorbis packets to drift the granulepos to
>> compensate > for A/V sync?  That would mean that granulepos does not
>> represent the > number of actually played samples any more.  Seems
>> like an ugly hack.
>> 
>> Yes it does. Counter to spec too, so I don't recommend it.
>> 
>> I guess I'm perplexed from the other end. The vorbis granulepos is in
>> number of samples, so multiply by the sample rate to get a timestamp.
>> The theora granulepos can be converted to the frame index, which you
>> multiply by the framerate to get a timestamp. Those timestamps should
>> align at playback. It's up to the player to work out how to make that
>> happen.
>> 
>> -r
>> 
>> --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project
>> homepage: http://www.xiph.org/ogg/ To unsubscribe from this list,
>> send a message to 'theora-dev-request at xiph.org' containing only the
>> word 'unsubscribe' in the body.  No subject is needed.  Unsubscribe
>> messages sent to the list will be ignored/filtered.
>> 
> --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project
> homepage: http://www.xiph.org/ogg/ To unsubscribe from this list, send
> a message to 'theora-dev-request at xiph.org' containing only the word
> 'unsubscribe' in the body.  No subject is needed.  Unsubscribe
> messages sent to the list will be ignored/filtered.


-- 
GnuPG public key: http://user.cs.tu-berlin.de/~dvdkhlng/dk.gpg
Fingerprint: B17A DC95 D293 657B 4205  D016 7DEF 5323 C174 7D40

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.