[opus] Generation loss test

[Rowan Lewis]

Hi everyone,

I've just finished doing a generation loss test using ffmpeg on the
following codecs at 96kbps:

- AAC (libfdk_aac)
- Opus (libopus)
- Vorbis (libvorbis)

I wanted to see what which codec is best to use for sites like YouTube were
content is often uploaded and downloaded, edited, then uploaded again in
cycles. The results are mildly interesting:

AAC was still passable by 10 generations, in subjective terms it just
sounds like most YouTube music videos uploaded before HD. Otherwise it took
roughly the same amount of time for each decode/encode run. However the
resulting filesize steadily grew from about 530 to 580KB, I'm not sure what
caused this, but when I attempted the test with a larger example after just
15 runs the filesize had grown from 4MB to 18MB.

Opus did subjectively better than either AAC and Vorbis, the quality
remained listenable all the way to 100 generations, while it was getting
noisy I don't perceive it as being any worse or better than some popular
YouTube videos. Everything else is as you might expect, it took less time
on average for each encode/decode ctyle and the filesize remained a fairly
steady 500KB.

Vorbis unfortunately did an appalling job, after just 3 runs there was a
noticeable high frequency hiss that grew to unbearable levels by run 5. The
filesize was quite literally all over the place, the first encoding was
526KB, the 5th was 436KB and then the 33rd was 510KB. I'm not quite sure
was was going on there.

You can look at my results here:
https://docs.google.com/a/nbsp.io/spreadsheet/ccc?key=0AgWJk3lK40SgdFhLaWhWOE14ekdYRXM0S2JYUjdzRXc&usp=sharing

And you can download my source file, scripts and some samples here:
https://drive.google.com/a/nbsp.io/file/d/0BwWJk3lK40SgZlduZDlYQW93RDg/edit?usp=sharing

It's also available on Github as a gist:
https://gist.github.com/rowan-lewis/8104933

Please take a look over my scripts (one for each codec), a few extra pairs
of eyes over them would be good, because I'm not sure if the options I've
used are optimal.

All of these tests were run using ffmpeg from the Arch Linux ffmpeg-full
AUR package built on the 1st of December. I'm not sure exactly how to tell
what version of each codec it is using, but hopefully it's representative
of what is available to normal users.

Any thoughts are welcome,
Rowan Lewis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.xiph.org/pipermail/opus/attachments/20131224/86030958/attachment.htm