[theora-dev] [Patch] Drop Frames in Cortado

[Benjamin M. Schwartz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The attached patch allows frames to be dropped in Cortado:

Why:
Currently, frames cannot truly be dropped in Cortado.  If the decoded YUV
arrives too late at the videosink, it will not be converted to RGB, but
every frame must still be decoded.  On slow systems, once the queue of
undecoded Theora packets fills up, the video decoder will block the ogg
demuxer, with the result that audio will not be decoded in real time.
Once the audio gets sufficiently late, it is rejected by the audio sink,
resulting in the appearance of a total lockup.  In fact, both video and
audio are being decoded, but too slowly to be worth displaying.

How:
This patch is a slavish transcription of the "leakiness" framework from
Gstreamer's Queue element.  A leaky queue drops buffers if it fills up,
rather than blocking entirely.  By making the queue of undecoded video
buffers leaky, we can ensure that audio continues to play correctly even
when there is not enough CPU for video.  Intriguingly, jst's TheoraDec
element was already correctly wired to notice the discontinuity introduced
by a dropped buffer and skip all frames until the next keyframe.

What:
The effect of this patch is approximately to make Cortado fall back to
keyframe-only decode when it runs out of CPU.

Where:
I tested the modified code in Sun JDK 1.1.8, running in Wine with JIT
disabled (i.e. pure slow interpreted mode).  Without this patch, the
player runs for two seconds before appearing to hang, churning 100% CPU.
With this patch, I was able to watch a 720p Theora/Vorbis file with
uninterrupted audio and keyframe-only video.

- --Ben

P.S.  Tuning max-buffers and max-lateness is welcome.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkrsugUACgkQUJT6e6HFtqT/KQCffkdFpm0K8Ht78zFKc6Qrh9iS
35kAn3tIxYZQcXZR+NUYTq6gygFhoiyk
=8Qv3
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: drop_frames.patch
Url: http://lists.xiph.org/pipermail/theora-dev/attachments/20091031/21691b5a/attachment.txt