[theora] A comparison of VP3, and two MPEG-4 variants

[Dan Miller]

...
[dan:]
> > And as a final note, PSNR is really a terrible way to judge codec 
> > quality.  Among many other sins, it completely fails to penalize 
> > codecs that stomp on frequency response rather than try to 
> approximate 
> > the spectral nature of an image. (TBH, the present VP3 encoder is 
> > guilty of this, though a future encoder could be much smarter).
>
[colin:] 
> I guess that is true. I have not done any studies of PSNR... 
> I am aware 
> that you know more about this than I will ever know. Do you 
> know of any 
> test that that a computer do that take those into account? I would be 
> very happy if you could point me in the direction of a better test.

ee http://www.its.bldrdoc.gov/n3/video/vqmsoftware.htm

the ITS solution is a front-runner for standardization by ITU/VQEG.  It does substantially better than PSNR, especially at lower datarates where PSNR really breaks down.

Their software can be downloaded free but only for eval and research (ie 'non-commercial' products).  It is also unfortunately heavily patented.  Therefore, while Xiph will probably not use it as part of their core development, it seems perfectly OK for some independent individual to use it to do an evaluation and report the results.  Their tool is PC or Linux only, and imports AVI's but not QT files.

There are several competing systems, the best 2 or 3 of which apparently work according to similar principles and have shown very similar results in comparisons with extensive subjective tests on a wide range of materials.  I suspect that once this stuff is standardized, PSNR will finally be relegated to a useful but clearly insufficient tool for objective quality measurement.

> 
> > I could go on but it's late.
> 
> Please do (if you have time) I am interested in your comments. I have 
> played around with video encoding a lot over the last 2-3 
> years, but I 
> have never written a codec, or done any other low level stuff with 
> encoding video. I am interested in your thoughts on this matter.
> 
> Colin.
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 
> 'theora-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No 
> subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
> 
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.