[theora-dev] [OT] Just saying hi!

[Dan Miller]

quick points --

on the coeff layout, both patent and datarate considerations exist.  The original VP3 was intended as a low-resolution, low-datarate codec, so the performance issues were not paramount.  On modern PC's, even a 320x240 image can mostly be kept in cache, so it's not too bad a hit.  It helps datarate given that the entropy coder is not so sophisticated.  And it definitely avoids some of the most difficult patents.  Downside is playback performance on higher-res material is probably not as great as we would like.  However machines are pretty fast these days, and other codecs have other deficiencies that slow them down, so it's probably a wash.

motion: VP3 uses only full-pel and 1/2 pel bilinear MV's.  The wierd case is when X and Y vectors are both 1/2 pel aligned; in this case we interpolate along the diagonal (look at the code to find out exactly how that works).  It's very fast with SIMD.

VP3 only has I and P frames, no B frames.  B frames traditionally don't help much on datarate per se; they are there mostly to avoid a 'pulsing' that can occur when keyframes come after alot of interframes (theoretically they could help with dissolves, but that's very encoder-specific).  However good rate controls and smart encoding help to avoid the pulsing problem.  B-frames are a huge religious issue among video coder geeks.  We (at On2) are in the "B-frames are stupid and unnecessary" camp.  They certainly make encoders & decoders much more complicated.  And they kill latency.

this just in:

NUM_HUFF_TABLES = 80, MAX_ENTROPY_TOKENS = 32, tot = 2560

> -----Original Message-----
> From: Christoph Lampert [mailto:chl at math.uni-bonn.de]
> Sent: Wednesday, February 26, 2003 2:48 PM
> To: theora-dev at xiph.org
> Subject: Re: [theora-dev] [OT] Just saying hi!
> 
> 
> On Wed, 26 Feb 2003, Mike Melanson wrote:
> > > I myself am most involved into motion estimation, so as 
> final questions
> > > before letting you get back to work: Is the general method for ME
> > > similar to MPEG? I-frames (golden... I like that... ;-) , 
> P-frames,
> > > B-frames? Motion estimation is done blockwise? Is there 
> support for
> > > global motion? 
> > 
> > 	I haven't gotten too much into the motion stuff 
> (getting close to
> > decoding keyframes right now). And I'm still a little fuzzy 
> on motion
> > esitmation/compensation in general. Can you give a brief 
> synopsis of how
> > ME/MC stuff operates in MPEG variants, particularly where 
> half-pel and
> > quarter-pel prediction figure in and how SIMD averaging 
> instructions help
> > with that?
> 
> ;-) Hey, I thought _I_ was the one to be asking questions...
> 
> I am not 100% sure about other MPEG variants (because MPEG-4 
> also contains
> H263 stuff), but in general it's like this: 
> 
> P-frames: 
> For every 16x16 macroblock a motion vector is determined. As a special
> case it's also possible to save 4 vector, one for each 8x8 
> luminance block
> (chroma vector is averaged from those). Vectors are stored in 
> bitstream
> relative to a median prediction (left, top, top-right). 
> Maximum range is
> controlled by a "fcode" parameter, between -16/+16 and -128/+128.
> 
> The vector points to a position in the reference frame (previous, I or
> P) and the corresponding block there is used as basis for the 
> block in the
> new frame. In the bitstream the (encoded, quantized, etc.) residue is
> coded and added to the reference image. 
> The motion vector(s) can be in half- or quarterpel resolution. 
> 
> Halfpel: For positions that corresponds to fullpel positions 
> (both vector 
> components are even), the block is simply copied from the reference
> frame. 
> For positions that correspond to halfpel positions (odd components) 
> a simple bilinear interpolation is used to determine the values of the
> block to be used. 
> 
> Quarterpel is more difficult: Fulpel is again just copied, For halfpel
> positions there is a complicated (and slow and ugly) process 
> of filtering 
> and "real" quarterpel is calculated from the filtered halfpel 
> positions
> by bilear interpolation again. 
> 
> 
> SIMD instruction are helpful for the bilinear parts, of 
> course. In fact,
> for encoding we calculate 3 full images of horizontally, 
> vertically and
> horizontally&vertically bilinear filtered images and decode based on 
> even/odd values of vector components from which to simply 
> copy the block
> (or from which to calcualte SUM-of-absolute-errors). 
> For Quarterpel this is more difficult, I don't now of any way 
> to really
> use SIMD instructions (would of course be possible, but difficult and
> maybe not much speedup). 
> 
> 
> B-frames are more difficult again. 
> Every block can be encoded like in a P-frame, with a vector pointing
> backwards, or with a vector pointing forwards, or with two 
> vectors, one to
> front, one to back, and the average of both blocks is used (SIMD!).
> The last and most efficient mode is "direct mode", there the motion
> vector is calculated from the vector of a co-located P-frame 
> (scaled), 
> and only a differential vector is save. The compensation process is
> similar again, halfpel&quarterpel etc.  
> 
> 
> > 	I can tell you that VP3 has I-frames that have to persist until
> > the next I-frame is decoded. Every successive P-frame can 
> use information
> > from either the previous P-frame or from the most recent 
> I-frame. I can
> > also tell you that the maximum range of a motion vector 
> appears to be
> > +/-31, according to my old notes on the decoder:
> >   http://www.pcisys.net/~melanson/codecs/vp3-notes.txt
> > Also, the decoder has code to support sub-pixel motion 
> compensation, but I
> > have not gotten to investigate that too deeply yet (which 
> is why I was
> > hoping you will be able to fill me in on the MPEG methods).
> 
> In general, halfpel is very efficient and not very slow (for 
> MMX CPUs). 
> Efficiency of Quarterpel depends on the input material, and the MPEG-4
> rules for it really s*ck. Maybe bilinear interpolation would have been
> better, again...
> 
> 
> Christoph
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 
> 'theora-dev-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No 
> subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
> 
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.