[Speex-dev] PESQ calculated MoS-Values for Speex

[Rasmussen, Lloyd]

What do your ears tell you about the performance of these codecs?  If you play some of the test signals back at half-speed, can you hear some frequencies that were not in the original signal?  Are all files running in mono?  If you have time, you might want to be testing the Opus codec, which has undergone considerable development since Speex.

I did not try these things under Cygwin.  And I have never run the Speex bitrate up to 80 or 96 kbps.  But using the pre-compiled Windows binaries under Windows 7, I could never get the resampler to work.  If I wanted the encoder to see a clean signal, I resampled it in SoundForge first.  This made a huge difference in audio quality, when working with signals that were initially recorded at 44.1 KHz sampling rate.  If aliasing is occuring, it will become more evident sometimes when the signal is compressed to a higher bitrate.

Resampling seems to work correctly in the pre-compiled Opus encoder and decoder available at
   www.opus-codec.org
.

Lloyd Rasmussen, Senior Project Engineer
National Library Service for the Blind and Physically Handicapped
Library of Congress   202-707-0535
http://www.loc.gov/nls
The preceding opinions are my own and do not necessarily reflect those of the Library of Congress, NLS.


From: speex-dev-bounces at xiph.org [mailto:speex-dev-bounces at xiph.org] On Behalf Of Oliver Martin
Sent: Wednesday, January 09, 2013 7:00 AM
To: speex-dev at xiph.org
Subject: [Speex-dev] PESQ calculated MoS-Values for Speex

Hello,

I just signed up to this mailing-list (note: my first mailing list at all), because I'm having some problems related to speex.
Let me just introduce you to what I'm doing.

I am writing a short (really short) paper about VoIP techniques, especially audio codecs for speech.
I pointed out basic technologies behind audio codecs; vector quantization, lpc, long-term prediction and some more.

I planned to compare G711(.1 eventually) and speex with reference to filesize
(which is a bit weird, since the encoded g711-files are 6 times larger than the original wave files - but this is another matter), MoS value,
and a subjective opinion, wether I can still understand the file, or not.
The MoS value is (or rather should be) calculated by PESQ (ITU P.862: http://www.itu.int/rec/T-REC-P.862/en).
(I know the pesq scale ranges from -0,5 - 4,5, where MoS is usually from 0 - 5)

Pesq Settings:
R2a: Sampling Rate: 8kHz, Overall bit rate: 80kbit/s
R3: Sampling Rate: 16kHz, Overall bit rate: 96kbit/s

Pesq Results:
R2a (mode 2) of G711 returned Mos-Values from 0.656 to 3.33 (parameter +8000 in pesq)
R3 (mode 4) of G711 returned Mos-Values from 3.981 to 4.269 (parameter +16000 in pesq)

Speex Settings:
To get comparable results I decided to use settings that conform with the G711 ones. Command-Line Parameters:
"mode 2": "-n --comp 10 --quality 10 --bitrate 80000"
"mode 4": "-w --comp 10 --quality 10 --bitrate 96000"

Speex Results (this is where it starts to be weird):
mode 2: Mos Values from 2.541 to 3.211 (parameter +8000 in pesq)
mode 4: Mos Values from 2.045 to 3.355 (parameter +16000 in pesq)

I tried to get better values, at least in mode4, for like 2 hours, but I did not get something that corresponds to this:
http://speex.org/comparison/nb_codecs_small.png (Speex.org Website).
The fact, that a bitrate of like 25kbit/s results in a score of nearly 4,5 is confusing me like hell. (compared to my results)

First thing I tried was just "speexenc origfile.wav destfile.spx", but the result did not satisfy me.

The used wavefiles are created in Cubase5.1, 16bit and 16kHz.
Im using 1.2rc1, compiled with cygwin, running on Windows 8.
The 1.2beta3 kept crashing on Win8 and Win7 using the same audiofiles.
I also tried using 1.0.5, but the (MoS)results were not satisfying.

I have no clue why the score is at such a low level (compared to the graphic below).

Let me already thank you for your help.

Greetings
Olli

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.xiph.org/pipermail/speex-dev/attachments/20130109/960c939d/attachment.htm