[theora-dev] [OT] Just saying hi!

[Christoph Lampert]

On Wed, 26 Feb 2003, Mike Melanson wrote:
> > I myself am most involved into motion estimation, so as final questions
> > before letting you get back to work: Is the general method for ME
> > similar to MPEG? I-frames (golden... I like that... ;-) , P-frames,
> > B-frames? Motion estimation is done blockwise? Is there support for
> > global motion? 
> 
> 	I haven't gotten too much into the motion stuff (getting close to
> decoding keyframes right now). And I'm still a little fuzzy on motion
> esitmation/compensation in general. Can you give a brief synopsis of how
> ME/MC stuff operates in MPEG variants, particularly where half-pel and
> quarter-pel prediction figure in and how SIMD averaging instructions help
> with that?

;-) Hey, I thought _I_ was the one to be asking questions...

I am not 100% sure about other MPEG variants (because MPEG-4 also contains
H263 stuff), but in general it's like this: 

P-frames: 
For every 16x16 macroblock a motion vector is determined. As a special
case it's also possible to save 4 vector, one for each 8x8 luminance block
(chroma vector is averaged from those). Vectors are stored in bitstream
relative to a median prediction (left, top, top-right). Maximum range is
controlled by a "fcode" parameter, between -16/+16 and -128/+128.

The vector points to a position in the reference frame (previous, I or
P) and the corresponding block there is used as basis for the block in the
new frame. In the bitstream the (encoded, quantized, etc.) residue is
coded and added to the reference image. 
The motion vector(s) can be in half- or quarterpel resolution. 

Halfpel: For positions that corresponds to fullpel positions (both vector 
components are even), the block is simply copied from the reference
frame. 
For positions that correspond to halfpel positions (odd components) 
a simple bilinear interpolation is used to determine the values of the
block to be used. 

Quarterpel is more difficult: Fulpel is again just copied, For halfpel
positions there is a complicated (and slow and ugly) process of filtering 
and "real" quarterpel is calculated from the filtered halfpel positions
by bilear interpolation again. 

<p>SIMD instruction are helpful for the bilinear parts, of course. In fact,
for encoding we calculate 3 full images of horizontally, vertically and
horizontally&vertically bilinear filtered images and decode based on 
even/odd values of vector components from which to simply copy the block
(or from which to calcualte SUM-of-absolute-errors). 
For Quarterpel this is more difficult, I don't now of any way to really
use SIMD instructions (would of course be possible, but difficult and
maybe not much speedup). 

<p>B-frames are more difficult again. 
Every block can be encoded like in a P-frame, with a vector pointing
backwards, or with a vector pointing forwards, or with two vectors, one to
front, one to back, and the average of both blocks is used (SIMD!).
The last and most efficient mode is "direct mode", there the motion
vector is calculated from the vector of a co-located P-frame (scaled), 
and only a differential vector is save. The compensation process is
similar again, halfpel&quarterpel etc.  

<p>> 	I can tell you that VP3 has I-frames that have to persist until
> the next I-frame is decoded. Every successive P-frame can use information
> from either the previous P-frame or from the most recent I-frame. I can
> also tell you that the maximum range of a motion vector appears to be
> +/-31, according to my old notes on the decoder:
>   http://www.pcisys.net/~melanson/codecs/vp3-notes.txt
> Also, the decoder has code to support sub-pixel motion compensation, but I
> have not gotten to investigate that too deeply yet (which is why I was
> hoping you will be able to fill me in on the MPEG methods).

In general, halfpel is very efficient and not very slow (for MMX CPUs). 
Efficiency of Quarterpel depends on the input material, and the MPEG-4
rules for it really s*ck. Maybe bilinear interpolation would have been
better, again...

<p>Christoph

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.