[theora] A comparison of VP3, and two MPEG-4 variants

[Colin Mckellar]

On Monday, March 24, 2003, at 03:14  AM, Freun Laven wrote:

> Colin;
>
> Very, VERY interesting.
>
> I've wondered for a long time about the actual quality but never could
> find any good information and didn't know how to do it myself.  And my
> eyesight isn't good enough to be able to make decent subjective quality
> testings.

Cool.

> I would, however, suggest expanding your tests.  Formats like:
>
> 1) mpeg-1  Yes, it's antique, but it's still often used because codecs
> are readily available.  This would also show people how much smaller
> files can really be, etc.
 >
> 2) mpeg-2.  mpeg-4 always gets compared to it, but nobody ever really
> says how much better it is etc.

In my experience, these vary widely between the quality of the 
encoders. A poor quality mpeg1 encoder will look not that much better 
than cinepak, but a good quality one will be stunning. I might try 
Cleaner 5's MPEG encoders. Apparently they are quite high quality.

> 3) The classic AVI codecs in Windows.  I know, I know... but people are
> going to be curious.

There is a limitation with this, unfortunately. The program I am using 
to test the psnr works on QuickTime movies, under OSX. The Indeo codecs 
are available for QuickTime, but only under OS9. I will ask the author 
of the program if he could release it for OS9, so I will be able to 
test them.

> 4) The standard codecs in Quicktime.  The Apple people are going to be
> interested.

I might include cinepak, H261, H263, MJPEG, and JPEG 2000. As I said 
though, Sorenson 2 and 3 both change the brightness of the video when 
encoding. This can actually improve how it looks, but it substantially 
drops the psnr. (in my tests, a good quality Sorenson 3 movie was given 
a psnr of about 20-25, which is much too low.) Beyond those codecs, 
they are all too big/inefficient to rest, imo.

> 5) Several compression levels of Windows Media and Real player.  One
> version of Real's compression is based on VP3, so it would be
> interesting to see a comparision.  I don't know which version, though.
> Probably the G2, but I don't actually know.

I think that VP3 was included as an optional codec, rather than 
actually being used as a basis for one of their codecs.
The problem with this is I have no way of testing the PSNR of these 
codecs. I am trying to do a completely objective test. I could test 
these codecs subjectively, but subjective comparisons always fall 
though, as people interpret things differently, and prefer different 
artifacts differently.

> 6) And finally, it should be compared against XVid (which is 
> opensource)
> and *especially* DivX PRO 5.04.  (You could throw in the free version 
> of
> DivX, but I think the Pro might be more interesting.)

I don't have a PC at the moment, but I will be borrowing one soon. I 
will try to test divx then.
I may be able to test xvid before then... but I am not entirely sure if 
it will work.

> I realize that would take some time, but since you seem to have a way 
> to
> actually make the tests, so....

The time isn't a problem.. I always work it around my schedule... which 
is fairly open.

> Finally, one last suggestions.... Could you use some other video clip?
> Perhaps something that originates in digital video, rather than a mpeg2
> format?  That might reduce any mpeg-2 compression artifacts to begin
> with.  (And allow you to do a mpeg-2 comparison.)  And if you use some
> other (non copyrighted) clip, perhaps you could talk the Theora people
> into hosting / posting a few small video clips that actually show the
> various qualities.  That way people can download it themselves and see
> what it looks like.

I would like to do this, but I have a few reservations:
1) I could go out and film some stuff with a DV camera (I actually have 
a bunch of footage around) but I think that the film would be too jerky 
(because I don't have the propper training in camera technique) so it 
might be unrealistic footage.
2) copyright footage would be very cool to play with, but I don't know 
where to get it (beyond, perhaps, DVDs of old film) but I suspect that 
the old video would be very noisy... which would also be a problem.
3) MPEG2, and Motion JPEG artifacts are a problem, but I think that 
they are fairly negligible... in any case, all the codecs have to deal 
with the same artifacts. I doubt that any codec is particularly 
susceptible to them, so no codecs are given a special advantage or 
special disadvantage....
just to be clear: I would like to get video without any artifacts to 
start with, but I have no way of obtaining realistic video like this, 
and I don't think that it makes too much of a difference in the long 
run.

I will see.

Colin.

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.