[theora] Sampling encoder timings between libtheora-1.0 & libtheora-1.1beta3

[John Kintree]

Thanks for your testing and reporting.  It would be helpful for me to know, in 
addition to the time it took to encode each sample, the time it takes to play 
each sample, the size of each sample prior to encoding, and the sizes of 
resulting files after encoding.
Regards,
John Kintree

On Saturday 29 August 2009 09:11 pm, Basil Mohamed Gohar wrote:
> As requested by Ralph in my introduction e-mail to this list [1], I've
> gone ahead and done some sample test timings for encoding between
> libtheora-1.0 and libtheora-1.1beta3, as available from the Theora
> website [2].
>
> So far, I've tested with 4 different files that I generated with content
> I already have.  The files, in brief, are the following:
>
>    1. planet-earth-360x240.yuv4mpeg - A clip from the Discovery/BBC
>       series Planet Earth from DVD 1, the "Mountains" chapter - the
>       first 5000 frames (I dropped half the fields to get non-interlaced
>       video, then rescaled)
>    2. kuala-lumpur-downtime-timelapse.yuv4mpeg - A time-lapse video clip
>       I created from 307 individual frames of the downtown skyline of
>       Kuala Lumpur, Malaysia I took while staying there.  The resolution
>       is "hd720" in FFmpeg parlance, or 1280x720, a.k.a. 720p.
>    3. kuala-lumpur-afternoon-1-vga-30fps.yuv4mpeg - Another time-lapse
>       video clip, this time from 1500 individual frames, taken from the
>       balcony of the apartment I stayed in, again in Kuala Lumpur.  The
>       resolution is 640x480.
>    4. canon-powershot-a550.yuv4mpeg - A video clip taken with my Canon
>       PowerShot A550 of the Gong Badak stadium in Kuala Terengganu
>       shortly after it happened [3].  The video is 640x480.  The format
>       was originally MJPEG with the quality boosted by the CHDK firmware
>       add-on to about 49,386 kb/s.
>
> All clips were converted to yuv4mpeg format and then copied to /dev/shm
> to remove, as much as possible, any potential outside influences on
> encoder performance.  This limited me to working with clips whose size
> in yuv4mpeg format would be under approximately 750MB.  Also, no sound
> was included, thus limiting this test solely to Theora encoding.
>
> Commands used were of the form:
>
> time ~/packages/libtheora-1.0/examples/encoder_example --output
> /dev/null --video-quality 7 canon-powershot-A550.yuv4mpeg
>
> I am presenting the best time out of three trials for each combination
> of input file, video quality setting, and encoder version.  Time taken
> is the first value returned by the "time" command, which is total
> elapsed time from command start to command end.
>
> All tests were run under Fedora 11, 32 bit, booted into single user mode
> [4].
>
> The hardware used was a Dell Inspiron 8600 with a 2GHz Pentium M
> processor and 1,544,420 kB (according to /proc/meminfo) of RAM.
>
> Okay okay!  Enough pedantism!  On to the results!
>
> 1. planet-earth-360x240.yuv4mpeg --video-quality 10
>
> libtheora-1.0 - 1m11.486s
> libtheora-1.1beta3 - 1m39.108s
>
> 2. planet-earth-360x240.yuv4mpeg --video-quality 7
>
> libtheora-1.0 - 0m53.652s
> libtheora-1.1beta3 - 1m21.661s
>
> 3. kuala-lumpur-downtown-timelapse-yuv4mpeg
>
> libtheora-1.0 - 36.566s
> libtheora-1.1beta3 - 48.982s
>
> 4. kuala-lumpur-balcony-afternoon-1-vga-30fps.yuv4mpeg
>
> libtheora-1.0 - 0m33.185s
> libtheora-1.1beta3 - 1m7.169s
>
> 5. canon-powershot-a550.yuv4mpeg
>
> libtheora-1.0 - 1m17.176s
> libtheora-1.1beta3 - 1m32.746s
>
> I apologize for not presenting these in the best possible way - I simple
> don't know what that would be!  Suggestions are welcome, though.  Also,
> if anyone would like the samples I used (aside from the first file,
> which I cannot legally share...yet), I am happy to do so.
>
> Interesting to note are tests 4 & 5.  I don't know why the spread is so
> great in #4 and so little in #5.  #4, on first appearances, is a lot
> like #3, in that the bottom is mostly static and the top consists of
> clouds moving.  #5 is taken from a tripod and consists of some close-up
> and zoomed-out pans of the collapsed stadium.  Picture quality on #5 is
> pretty good as it was quite a sunny day and the viewing conditions were
> quite clear.
>
> If there's a lot of interest in this testing methodology, I might put
> together some scripts to make automating, timing, & reporting more
> systematic.  These runs were all basically done manually on the
> command-line.  I even hand-wrote the results down.  ;)
>
> I intended to put up more of an analysis on my blog, but I am not sure
> if I'll do that yet, as I'm planning to migrate my site to a new server,
> so I prefer not to have new uploads just yet.
>
> Constructive feedback will be most appreciated!
>
> [1] http://lists.xiph.org/pipermail/theora/2009-August/002678.html
> [2] http://www.theora.org/downloads/
> [3]
> http://www.basilgohar.com/blog/2009/06/05/sultan-mizan-zainal-abidin-stadiu
>m-aka-gong-badak-stadium-collapses [4]
> http://docs.fedoraproject.org/install-guide/f11/en-US/html/s1-rescuemode-bo
>oting-single.html