[theora-dev] [Patch] Drop Frames in Cortado

Benjamin M. Schwartz bmschwar at fas.harvard.edu
Sat Oct 31 15:28:21 PDT 2009

Hash: SHA1

The attached patch allows frames to be dropped in Cortado:

Currently, frames cannot truly be dropped in Cortado.  If the decoded YUV
arrives too late at the videosink, it will not be converted to RGB, but
every frame must still be decoded.  On slow systems, once the queue of
undecoded Theora packets fills up, the video decoder will block the ogg
demuxer, with the result that audio will not be decoded in real time.
Once the audio gets sufficiently late, it is rejected by the audio sink,
resulting in the appearance of a total lockup.  In fact, both video and
audio are being decoded, but too slowly to be worth displaying.

This patch is a slavish transcription of the "leakiness" framework from
Gstreamer's Queue element.  A leaky queue drops buffers if it fills up,
rather than blocking entirely.  By making the queue of undecoded video
buffers leaky, we can ensure that audio continues to play correctly even
when there is not enough CPU for video.  Intriguingly, jst's TheoraDec
element was already correctly wired to notice the discontinuity introduced
by a dropped buffer and skip all frames until the next keyframe.

The effect of this patch is approximately to make Cortado fall back to
keyframe-only decode when it runs out of CPU.

I tested the modified code in Sun JDK 1.1.8, running in Wine with JIT
disabled (i.e. pure slow interpreted mode).  Without this patch, the
player runs for two seconds before appearing to hang, churning 100% CPU.
With this patch, I was able to watch a 720p Theora/Vorbis file with
uninterrupted audio and keyframe-only video.

- --Ben

P.S.  Tuning max-buffers and max-lateness is welcome.
Version: GnuPG v2.0.11 (GNU/Linux)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: drop_frames.patch
Url: http://lists.xiph.org/pipermail/theora-dev/attachments/20091031/21691b5a/attachment.txt 

More information about the theora-dev mailing list