[theora-dev] [theora in hardware] Contents of YUV_BUFFER_ENTRY

[Timothy B. Terriberry]

Arthur Valadares wrote:
> 1) What i understand: recon_pixel_index_table(i) returns the absolute 
> location of a relative location. Example: if i do 
> recon_pixel_Index_table(0), i would get the first pixel of the frame 
> (although i dont quite get the visual idea of what is the first pixel. 
> Is it in that black area around the frame? is it the first real pixel? 
> If i do recon_pixel_index_table(YPlaneFragments) do i get the first 
> pixel in U Plane?)

recon_pixel_index_table gives the offset for the pixel in the 
lower-right of the given fragment (remember Theora stores the image from 
bottom to top internally). Be careful, as the fragments are not arranged 
in row-major order as one might expect, but "Hilbert-curve order". This 
is explained in the spec in section 2.3. If what I just said doesn't 
make any sense, I recommend reading all of Chapter 2. frinit.c is also a 
good section of code to read, as this is what sets up all of these offsets.

The "black area" around the frame is added solely for video whose 
dimensions are not multiples of 16. Internally Theora only deals with 
images that are a multiple of 16, so the "first pixel" will indeed be 
inside this area, and not the first "real" pixel. However, the decoder 
shouldn't care whether a pixel is "real" or not... it must operate on 
the full, padded frame until it is time for display, when it is supposed 
to crop it for display (the cropped data can be used to predict future 
frames, though, and so _cannot_ be ignored to save a few bytes).

> 2) What i dont understand: i expected to YUV_BUFFER_ENTRY to contain 
> only the pixels I'ld be using. So in my mind, if i did LastFrameRecon(0) 
> to LastFrameRecon(YPlaneFragments), for example, i'ld get all the Y 
> pixels. I realized after careful study of the recon_pixel_index_table 
> that if i wanted the first pixel of the Y Plane it would not be 0, but 
> in fact, ReconYDataOffset.
> Now, why is there such an offset? What is in between 0 and 
> ReconYDataOffset? Is there any more "unexpected" data mixed in the 
> pixels im using? Where am i going to be needing these pixels? It would 
> help a bunch if i could eliminate them in hardware, since we're short on 
> memory

This makes more sense if you look up at ExpandBlock(), in particular the 
part where it computes MVOffset. When motion compensation is used, a 
fragment is reconstructed from an _offset_ version of pixels in one of 
the reconstruction buffers. This offset can quite legally point outside 
of the image proper, so an extra "border" is added all around the image 
to give it some valid data to point to. This is the "unexpected" data 
you describe.

If you don't want to add this border, you will be required to clamp the 
indices of the pixels you're reading to lie within the image, for 
_every_ pixel you read (see, e.g., Section 7.9.1 of the spec). In C, 
it's obviously much faster to copy the edge pixels into this border and 
avoid the clamping.