[Speex-dev] Testing Help

[Steve Underwood]

Jean-Marc Valin wrote:
>> What commercial VOIP test products can I turn to in order to objectively
>> evaluate your CODEC; in order to analyze audio streams for MOS scores and
>> performance parameters?
>>     
>
> The MOS evaluation is subjective, not objective and actually involves
> getting (many) real people to listen to the audio, not just buying a
> product. Of course, there's tools like PESQ that attempt to guess MOS
> scores, but it's really just a guess. You can find these kinds of
> results for Speex on the comparison page: http://www.speex.org/comparison/
>   
While MOS is based on the subjective evaluation of a number of people, 
the final result can be said to be reasonably objective. Provided the 
people overseeing the tests are non-partisan, a MOS test is quite 
repeatable. Many measurements considered objective are statistically 
based, and a properly conducted MOS test is such a test.

PESQ is just something to keep dumb managers of the "if I can't measure 
something meaningful, just give me any old numbers I can put in reports" 
school happy.

Steve