[theora] Getting only frame deltas?

[Ralph Giles]

On Thu, Aug 13, 2009 at 7:04 AM, Gabriel
Gambetta<gabriel at mysterystudio.com> wrote:

> I was thinking - since videos are encoded as delta frames, is it possible to
> get Theora to give me the rects modified since the last frame and upload
> only that? I'm stuck with theora_decode_YUVout which decodes a full frame
> (yep, old API, but th_decode_ycbcr_out in the new one seems to be similar)

libtheora doesn't expose this, but you could of course hack it up to
do so. Start with the list of block coding modes. Unless your video
has completely static backgrounds I'm not sure it would help much
though. If you look, for example, at the telemetry images toward to
bottom of http://web.mit.edu/xiphmont/Public/theora/demo3.html, even
when most of the coded blocks are in groups there are still some
outliners which make it hard to find efficient dirty rectangles
without just breaking the image into hundreds of 16x16 tiles, which
will be slow in the other direction.

As you mention, doing yuv2rgb in simd will help a lot. Also, many
video cards have support for doing the conversion on the GPU, either
through texture extensions like Shayne suggested, or directly with a
shader program. This can help a lot because the yuv data is subsampled
in the chroma planes and is half the size of the rgb version, so
there's less memory bandwidth needed to upload it to texture memory.
Also, if you're doing flat playback even very old cards often have
support for yuv2rgb and scaling as a 2d graphics 'overlay' feature.

HTH,
 -r