<div dir="ltr">I wanted to roughly benchmark how the different version of libopus performed at each complexity level for a 6kbit/s output opus file. This was conducted on a Raspberry Pi so it is a constant hardware platform. This was done on an early Pi so only 256MB RAM but it was never used up so should not make a difference.<div>
<br></div><div>I compiled the three final versions of each major release of libopus so that was 0.9.14, 1.0.3 & 1.1. These were all compiled natively on the machine using the current repo version of gcc 4.6.3 and with these optimisation flags:</div>
<div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>-O2 -pipe -march=armv6j -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard<br></div></blockquote></div><div><br></div><div>These were compiled with floating point enabled. I will look at the fixed point version separately later.</div>
<div><br></div><div>I used a clip of speech from a librevox recording which was resampled from 44.1khz to 48khz within audacity. The clip is 2 minutes long. I wrote a simple bash script that ran the encode at each complexity level and repeated 10 times to try and get a good average.</div>
<div><br></div><div>The results can be seen in this graph <a href="http://ubuntuone.com/2gOdUG3h3MyjLY7gSYseRN">http://ubuntuone.com/2gOdUG3h3MyjLY7gSYseRN</a> </div><div><br></div><div><img src="cid:ii_1430c080d17f16db" alt="Inline images 1" width="564" height="572"><br>
</div><div><br></div><div>This clearly shows what I had discovered in what appears to be a regression for complexity 7,8,9 and 10. From what Gregory said earlier then in fact this is because these levels are producing more quality than they did before. It is still good to know this profile though if you only have a little CPU to play with such as in embedded applications. The thing I cannot graph on this is encode quality. My ears are not good enough to hear the difference and unless there is an automated way to score it we will just have to assume that each complexity level does increase the quality.</div>
<div><br></div><div>The graph also suggests that on this platform at least complexity level 9 is pointless as it was slower than 10 and presumably produces worse results. This could of course have been some background task kicking in on the OS when this ran and the error bars are quite large so I will see if this maintained over other runs.</div>
<div><br></div><div>All these speeds were taken from opusenc outputs and I used version 0.1.2 of opus-tools which was compatible with all three versions of the library. I am running another test using 0.1.8 at the moment but it will only work with libopus 1.0.3 and 1.1. I think I observed that it was slightly slower but we will see if the results will bear that out.</div>
<div><br></div><div>I also will run some tests at different bit rates and with music instead of voice as well and share the charts here. If anyone wants I can share the OpenOffice spreadsheet with the raw numbers and the bash script I used (though you have to do all the compiling yourself).</div>
<div><br></div><div>Hope this is helpful.</div>
<div class="gmail_extra"><br clear="all"><div><div>Best Regards,</div><div><br></div><div>Stuart Marsden</div><div><br></div><div>Tactical Communications Consultant</div><div>FinMars Consulting Ltd</div><div>UK: <a href="tel:%2B441865589833" value="+441865589833" target="_blank">+441865589833</a>
</div><div>Finland: <a href="tel:%2B358453046287" value="+358453046287" target="_blank">+358453046287</a></div></div>
<br><br><div class="gmail_quote">On 18 December 2013 00:14, Stuart Marsden <span dir="ltr"><<a href="mailto:stuartmarsden@finmars.co.uk" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=stuartmarsden@finmars.co.uk&cc=&bcc=&su=&body=','_blank');return false;">stuartmarsden@finmars.co.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Gregory,<div><br></div><div>That is good to know and if therefore the true apples to apples comparison is 0.9.14 at comp 10 and 1.1 at comp 5 then things are fine. My ears are not good enough to hear the difference so for speed I would target comp 5 or lower.</div>
<div><br></div><div>I just did a quick test and 0.9.14 at comp 10 was 3.872</div><div>1.1 at comp 5 was 5.218</div><div><br></div><div>So if the output is comparable then we do in fact see a speed improvement.</div><div>
<br>
</div><div>Thanks for pointing this out. Is it documented? I admit I have only read some of the documentation.</div><div><br></div><div class="gmail_extra"><br clear="all"><div><div>Best Regards,</div><div><br></div><div>
Stuart Marsden</div><div><br></div></div><div><div>
<br><br><div class="gmail_quote">On 17 December 2013 23:50, Gregory Maxwell <span dir="ltr"><<a href="mailto:gmaxwell@gmail.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=gmaxwell@gmail.com&cc=&bcc=&su=&body=','_blank');return false;">gmaxwell@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div>On Mon, Dec 16, 2013 at 5:03 AM, Stuart Marsden<br>
<<a href="mailto:stuartmarsden@finmars.co.uk" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=stuartmarsden@finmars.co.uk&cc=&bcc=&su=&body=','_blank');return false;">stuartmarsden@finmars.co.uk</a>> wrote:<br>
> I have just started trying Opus with a view to using it in a project. I am<br>
> interested in embedded hardware and tried it on the Raspberry Pi using the<br>
> raspbian distro.<br>
><br>
> The version of libopus in the repos is 0.9.14. I installed this and tried<br>
> encoding 2 minutes of speech from a librevox recording. It managed this at a<br>
> respectable pace for complexity 10:<br>
<br>
</div>Complexity 10 is new analysis code that didn't exist in prior<br>
versions, setting complexity 5 gets you basically the same analysis<br>
that the 1.0 version had.<br>
<br>
On x86 and modern arm cores with fast FPUs the other speedups are<br>
enough that complexity 10 is about the same speed in the old software<br>
or the new software (but with much higher and consistent VBR quality).<br>
But on chips with slow FPUs the new analysis code is much slower, in<br>
particular because it has not been entirely converted to fixed point<br>
(e.g. in the fixed point builds) which is what I believe you're seeing<br>
here.<br>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div></div>