[Speex-dev] Results of Automated Batch Tests

[Nic Roets]

I realize that hearing tests are far superior for calibrating codec
 internals, particularly the perceptual enhancement. The tester is for
 finding gross mistakes like overflows.

 The test program includes code to subsample the 32Khz source down to 8 and
 18 Khz.

 I've broadend the tests to try all the possibilities like VBR, perceptual
 enhancement, complexity and quality. The results are at
 www.rational.co.za:81/speex.csv

> ----- Original Message ----- 
> From: "Jean-Marc Valin" <Jean-Marc.Valin at USherbrooke.ca>
> To: "Nic Roets" <speex at rational.co.za>
> Cc: <>
> Sent: Thursday, September 22, 2005 2:12 PM
> Subject: Re: Fw: [Speex-dev] Results of Automated Batch Tests
>
>
> Hi Nic,
>
> While your tool may be useful to detect regressions (I'd include
> fixed-point tests too while you're at it), it cannot be used to really
> estimate quality. You could obtain a much better SNR from Speex, but the
> sound quality would be much poorer. As for narrowband, it's designed for
> 8 kHz, so if you send it 16 kHz audio, it'll still think it's 8 kHz and
> put twice the nominal bit-rate in it.
>
> Jean-Marc
>
> Le jeudi 22 septembre 2005 à 10:41 +0200, Nic Roets a écrit :
>>  The results are at www.rational.co.za/speex.csv
>>
>>  Each of the 11 quality settings is tested 3 times (narrow, wide and 
>> ultra
>>  wide band). Strangely narrow band quality 11 outperforms all wide band
>>  tests, but it can be due to my subsampling or some other inaudible 
>> effect
>>  like delaying.
>>
>>  You can import it into Excel and sort it by SNR or other value. Divide
>> the
>>  bits by 24 to get the bps.
>>
>>  The patch is at www.rational.co.za/speexBatchTest.patch
>>  The complete source is at www.rational.co.za/speex-1.1.10a.tar.gz on a
>> slow
>>  server.
>>
>>
>>  ----- Original Message ----- Tom Grandgent
>> > Hmm, why not just make the URL of the reference sample available in the
>>
>>  OK. If you run the program and the file was not downloaded, it tells you
>>  where to get it.
>>
>> > Ah, so you need a high resolution timer.  IMO, it's better to use
>> > timeGetTime() on Windows, which is very simple to use and offers
>> > reliable
>>
>>  I've added #ifdefs that you can fill in.
>>
>> _______________________________________________
>> Speex-dev mailing list
>> Speex-dev at xiph.org
>> http://lists.xiph.org/mailman/listinfo/speex-dev
>>
> -- 
> Jean-Marc Valin <Jean-Marc.Valin at USherbrooke.ca>
> Université de Sherbrooke
>