[theora] Getting only frame deltas?

[Shayne Wissler]

On Thu, Aug 13, 2009 at 8:04 AM, Gabriel
Gambetta<gabriel at mysterystudio.com> wrote:
> Hi, first post on this list. I run a small game development company
> specializing in casual Windows/Mac games.
>
> We've been using Theora for video playback inside our engine for a while,
> but we always run into performance issues. I've tracked them down to two
> parts : YUV to RGB decoding (done via software) and uploading the new pixel
> data to the video card as a texture.

On the Mac, I use these calls:

  glTexParameteri(Target, GL_TEXTURE_STORAGE_HINT_APPLE,
GL_STORAGE_SHARED_APPLE);

  glTexImage2D(Target, 0, GL_RGBA, W, H, 0,
      GL_YCBCR_422_APPLE,GL_UNSIGNED_SHORT_8_8_REV_APPLE, pixels);

I can run 1080p at full frame rates on an iMac with these calls and an
efficient cropping/422 packing algorithm

Linux is problematic, I have not found an equivalent API that works. I
haven't looked yet for Windows, but if you find one please let me
know. It is ridiculous that in this day and age, OpenGL does not have
standard YUV -> RGB conversion functions.

On OSX, be sure to use Shark.

> For the YUV to RGB issue, I'll rewrite it in assembler. Uplading the whole
> 1024x768 frame is proving problematic, though - this is the operation that
> causes delays in one of the test machines, and it's pretty much unavoidable.

Look at liboggplay, there are some fast conversion routines there, but
on the Mac you shouldn't need them, and I'd hope that on Windows you
wouldn't either.

> I was thinking - since videos are encoded as delta frames, is it possible to
> get Theora to give me the rects modified since the last frame and upload
> only that? I'm stuck with theora_decode_YUVout which decodes a full frame
> (yep, old API, but th_decode_ycbcr_out in the new one seems to be similar)

Sounds like a cool idea, but also like an R&D project that has a good
chance of failing.


Shayne Wissler