[theora] A comparison of VP3, and two MPEG-4 variants

[Dan Miller]

a couple points before I look at this too hard:

"For VP3, since you can't do a constant quality encode"

what?  that should not be the case.  Could be a QT-specific bug. Theora at any rate is certainly capable of doing constant-quality encode.  This probably explains your standard deviation complaint -- VP3 is running a rate control alg, where the other codecs are just shooting for consistent PSNR.  Apples to bears.

"VP3 preferred rgb"

very suspicious, as YUV 12 (ie planar YUV with U and V each subsampled at 1/2 resolution in both directions) is the native colorspace.  These sorts of conversions are a huge source of innacuracies on PSNR tests.

To be frank, it's great that you went to the trouble of doing this, but the only way to get accurate results would be to work with code modules that are provably devoid of any color conversion, rate control, non-standard quality enhancements (sharpening, gamma), etc. and so on.  In this sort of situation, where you have a huge architecture (QT) between you and the codec, I think you would be much better off relying on subjective measurements rather than PSNR.

And as a final note, PSNR is really a terrible way to judge codec quality.  Among many other sins, it completely fails to penalize codecs that stomp on frequency response rather than try to approximate the spectral nature of an image. (TBH, the present VP3 encoder is guilty of this, though a future encoder could be much smarter).

I could go on but it's late.

<p> ___  Dan Miller
(++,) Founder, CTO, On2 Technologies 

> -----Original Message-----
> From: Colin Mckellar [mailto:c.mckellar at student.murdoch.edu.au]
> Sent: Sunday, March 23, 2003 12:20 PM
> To: theora at xiph.org
> Subject: Re: [theora] A comparison of VP3, and two MPEG-4 variants
> 
> 
> 
> On Monday, March 24, 2003, at 01:14  AM, Christoph Lampert wrote:
> 
> > On Mon, 24 Mar 2003, Colin Mckellar wrote:
> >
> >> I recently did a test of different settings of VP3, and two MPEG-4
> >> variants: 3ivx, and Apple MPEG-4. My results are at the 
> following URL.
> >>
> >> I am interested in people's thoughts, and comments on it.
> >>
> >> Colin.
> >
> > It is just me or did you really forget to give the URL ?
> 
> woop.
> 
> In my defense, it is late, and I am tired :-)
> 
> <http://mornmist.2y.net/~blibbler/codecpsnr.html>
> 
> Colin.
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 
> 'theora-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No 
> subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
> 
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.