[theora] video codec comparisons

[Gregory Maxwell]

On Mon, Mar 15, 2010 at 2:04 PM, John Kintree <jkintree at swbell.net> wrote:
>
> Reviewers who have published comparisons of video codecs recently tend not to publish their source video clip, nor encoded video clips.  It might help promote theora if there was a link we could provide to a nice source clip, only one or two minutes long with a mix of action and talking head spots, with several versions of it compressed in theora format; maybe 250 kbps, 500 kbps, 750 kbps, 1 mbps, 1.5 mbps, and 2 mbps.  Then if anyone wants to do a comparison of theora with alternatives, the set of theora clips with source clip would already be available for that purpose.  If this already exists, a link would be appreciated.

Xiph.Org maintains a lossless media archive for this sort of purpose:
http://media.xiph.org/

Files from this archive can be combined to produces mixtures like you suggest.

We're interested in taking additional test material, providing that it
is content that we can legally distribute and use it, and that its
available in a lossless format.  (If we want to test from previously
compressed data we could compress the lossless material, but the other
way around isn't possible).

I think pre-generated Theora examples may suffer problems with the
many opportunities for tool-chain differences.  E.g. your tools may
perform resampling or color-space conversion differently than ours,
resulting in results which are not suitable for direct comparison.

Testing well is really hard.  No amount of prefabbed files is going to
make it easy,