[opus] Questions Regarding Opus Test Vectors

[Wang, Chris]

Timothy ,

I appreciate your feedback.  So It looks like there are no standardized test vectors that I could used to verify whether my encoder implementation is bit-exact with the open source standard version.

To verify decoder I could use testvectorxx.bit as input to my decoder and see whether the output would match testvectorxx.dec accordingly.

Chris

-----Original Message-----
From: opus-bounces at xiph.org [mailto:opus-bounces at xiph.org] On Behalf Of Timothy B. Terriberry
Sent: Sunday, November 10, 2013 5:33 PM
To: opus at xiph.org
Subject: Re: [opus] Questions Regarding Opus Test Vectors

Wang, Chris wrote:
> Benjamin,
>
> Thanks for the prompt response.  Are there other recommended methods 
> to verify encoder implementations?

There are few if any requirements placed on the encoder by the actual RFC, however there are a number of things you can do which provide various amounts of assurance your encoder is working as expected.

The first test I would run is to write out the final range coder state of each packet, as is done in the file format used by opus_demo, and then use opus_demo to decode the files. It will check to make sure the final range coder state of the decoder matches. With high probability this will tell you if the decoder is decoding the same bitstream symbols the encoder thought it encoded.

Slightly more complicated is to have the encoder resynthesize the coded audio from its internal state as it encodes, and compare that with what the decoder produces. In libopus this behavior can be enabled via the RESYNTH #define, at least for the CELT layer.

http://www.ietf.org/proceedings/80/slides/codec-4.pdf covers a number of additional software tests we run. Determining the quality of a correctly-functioning encoder requires subjective (listening tests) and/or objective metrics (PEAQ, PESQ, etc.).
_______________________________________________
opus mailing list
opus at xiph.org
http://lists.xiph.org/mailman/listinfo/opus