[theora] A comparison of VP3, and two MPEG-4 variants

[Colin Mckellar]

On Monday, March 24, 2003, at 10:24  AM, Dan Miller wrote:

> a couple points before I look at this too hard:
>
> "For VP3, since you can't do a constant quality encode"
>
> what?  that should not be the case.  Could be a QT-specific bug. 
> Theora at any rate is certainly capable of doing constant-quality 
> encode.  This probably explains your standard deviation complaint -- 
> VP3 is running a rate control alg, where the other codecs are just 
> shooting for consistent PSNR.  Apples to bears.

This has been the case with every VP3 quicktime codec. In the very 
first release, (3.2.0.1 beta) the quality slider was enabled, but it 
did not vary the quality, as much as it determined how far the actual 
bitrate could vary from the given bitrate. Since that release, the 
quality slider has been disabled altogether.
If I could encode with a constant quantizer using Quicktime into VP3, I 
would be very very happy. I have long suspected that it was just 
overlooked in the quicktime release.

> "VP3 preferred rgb"
>
> very suspicious, as YUV 12 (ie planar YUV with U and V each subsampled 
> at 1/2 resolution in both directions) is the native colorspace.  These 
> sorts of conversions are a huge source of innacuracies on PSNR tests.

I was surprised too. The difference between the psnr of the rgb and the 
yuv was only about 1 or 2, though. I am interested in whether the 
quicktime codec accepts YUV video directly, and outputs YUV video 
directly (opposed from doing the conversion internally)

> To be frank, it's great that you went to the trouble of doing this, 
> but the only way to get accurate results would be to work with code 
> modules that are provably devoid of any color conversion, rate 
> control, non-standard quality enhancements (sharpening, gamma), etc. 
> and so on.  In this sort of situation, where you have a huge 
> architecture (QT) between you and the codec, I think you would be much 
> better off relying on subjective measurements rather than PSNR.

Fair enough. The problem with subjective measurements is that they 
are... well, subjective. I have seen many codec comparisons that look 
at 5 frames from a clip, and comment on what the author thought as the 
best quality. PSNR is one method of "objectively" testing. I tried to 
not include any results that were obviously broken (such as sorenson 2 
and 3 codecs.) It would be good if I could test VP3 without the 
pre-processing.

It would be good to do a test of the quality of encodes before a bunch 
of random people... but I don't really have the resources or time (not 
to mention, it is much more difficult to do the pretty graphs)

> And as a final note, PSNR is really a terrible way to judge codec 
> quality.  Among many other sins, it completely fails to penalize 
> codecs that stomp on frequency response rather than try to approximate 
> the spectral nature of an image. (TBH, the present VP3 encoder is 
> guilty of this, though a future encoder could be much smarter).

I guess that is true. I have not done any studies of PSNR... I am aware 
that you know more about this than I will ever know. Do you know of any 
test that that a computer do that take those into account? I would be 
very happy if you could point me in the direction of a better test.

> I could go on but it's late.

Please do (if you have time) I am interested in your comments. I have 
played around with video encoding a lot over the last 2-3 years, but I 
have never written a codec, or done any other low level stuff with 
encoding video. I am interested in your thoughts on this matter.

Colin.

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.