[theora] Sampling encoder timings between libtheora-1.0 & libtheora-1.1beta3

[Basil Mohamed Gohar]

As requested by Ralph in my introduction e-mail to this list [1], I've
gone ahead and done some sample test timings for encoding between
libtheora-1.0 and libtheora-1.1beta3, as available from the Theora
website [2].

So far, I've tested with 4 different files that I generated with content
I already have.  The files, in brief, are the following:

   1. planet-earth-360x240.yuv4mpeg - A clip from the Discovery/BBC
      series Planet Earth from DVD 1, the "Mountains" chapter - the
      first 5000 frames (I dropped half the fields to get non-interlaced
      video, then rescaled)
   2. kuala-lumpur-downtime-timelapse.yuv4mpeg - A time-lapse video clip
      I created from 307 individual frames of the downtown skyline of
      Kuala Lumpur, Malaysia I took while staying there.  The resolution
      is "hd720" in FFmpeg parlance, or 1280x720, a.k.a. 720p.
   3. kuala-lumpur-afternoon-1-vga-30fps.yuv4mpeg - Another time-lapse
      video clip, this time from 1500 individual frames, taken from the
      balcony of the apartment I stayed in, again in Kuala Lumpur.  The
      resolution is 640x480.
   4. canon-powershot-a550.yuv4mpeg - A video clip taken with my Canon
      PowerShot A550 of the Gong Badak stadium in Kuala Terengganu
      shortly after it happened [3].  The video is 640x480.  The format
      was originally MJPEG with the quality boosted by the CHDK firmware
      add-on to about 49,386 kb/s.

All clips were converted to yuv4mpeg format and then copied to /dev/shm
to remove, as much as possible, any potential outside influences on
encoder performance.  This limited me to working with clips whose size
in yuv4mpeg format would be under approximately 750MB.  Also, no sound
was included, thus limiting this test solely to Theora encoding.

Commands used were of the form:

time ~/packages/libtheora-1.0/examples/encoder_example --output
/dev/null --video-quality 7 canon-powershot-A550.yuv4mpeg

I am presenting the best time out of three trials for each combination
of input file, video quality setting, and encoder version.  Time taken
is the first value returned by the "time" command, which is total
elapsed time from command start to command end.

All tests were run under Fedora 11, 32 bit, booted into single user mode
[4].

The hardware used was a Dell Inspiron 8600 with a 2GHz Pentium M
processor and 1,544,420 kB (according to /proc/meminfo) of RAM.

Okay okay!  Enough pedantism!  On to the results!

1. planet-earth-360x240.yuv4mpeg --video-quality 10

libtheora-1.0 - 1m11.486s
libtheora-1.1beta3 - 1m39.108s

2. planet-earth-360x240.yuv4mpeg --video-quality 7

libtheora-1.0 - 0m53.652s
libtheora-1.1beta3 - 1m21.661s

3. kuala-lumpur-downtown-timelapse-yuv4mpeg

libtheora-1.0 - 36.566s
libtheora-1.1beta3 - 48.982s

4. kuala-lumpur-balcony-afternoon-1-vga-30fps.yuv4mpeg

libtheora-1.0 - 0m33.185s
libtheora-1.1beta3 - 1m7.169s

5. canon-powershot-a550.yuv4mpeg

libtheora-1.0 - 1m17.176s
libtheora-1.1beta3 - 1m32.746s

I apologize for not presenting these in the best possible way - I simple
don't know what that would be!  Suggestions are welcome, though.  Also,
if anyone would like the samples I used (aside from the first file,
which I cannot legally share...yet), I am happy to do so.

Interesting to note are tests 4 & 5.  I don't know why the spread is so
great in #4 and so little in #5.  #4, on first appearances, is a lot
like #3, in that the bottom is mostly static and the top consists of
clouds moving.  #5 is taken from a tripod and consists of some close-up
and zoomed-out pans of the collapsed stadium.  Picture quality on #5 is
pretty good as it was quite a sunny day and the viewing conditions were
quite clear.

If there's a lot of interest in this testing methodology, I might put
together some scripts to make automating, timing, & reporting more
systematic.  These runs were all basically done manually on the
command-line.  I even hand-wrote the results down.  ;)

I intended to put up more of an analysis on my blog, but I am not sure
if I'll do that yet, as I'm planning to migrate my site to a new server,
so I prefer not to have new uploads just yet.

Constructive feedback will be most appreciated!

[1] http://lists.xiph.org/pipermail/theora/2009-August/002678.html
[2] http://www.theora.org/downloads/
[3]
http://www.basilgohar.com/blog/2009/06/05/sultan-mizan-zainal-abidin-stadium-aka-gong-badak-stadium-collapses
[4]
http://docs.fedoraproject.org/install-guide/f11/en-US/html/s1-rescuemode-booting-single.html

-- 
      Basil Mohamed Gohar
abu_hurayrah at hidayahonline.org
http://www.basilgohar.com/blog
basilgohar on irc.freenode.net
GPG Key Fingerprint:  5AF4B362