[Speex-dev] PESQ calculated MoS-Values for Speex

Wed Jan 9 12:15:48 PST 2013

OK.  Different mailing lists are set up differently.  This list is unusual because your answers only go to the person who replied to you.  So if you want the other people on the listserv to see your answer, you should make sure that Speex-dev at xiph.org<mailto:Speex-dev at xiph.org> is added to the TO: field of your outgoing message.  Hopefully someone else will also attempt to answer your questions.  I am sending this reply, which includes your answer, to the listserv.

Lloyd Rasmussen, Senior Project Engineer
National Library Service for the Blind and Physically Handicapped
Library of Congress   202-707-0535
http://www.loc.gov/nls
The preceding opinions are my own and do not necessarily reflect those of the Library of Congress, NLS.

From: Oliver Martin [mailto:scholl3ss93 at gmail.com]
Sent: Wednesday, January 09, 2013 2:04 PM
To: Rasmussen, Lloyd
Subject: Re: [Speex-dev] PESQ calculated MoS-Values for Speex

Thanks for your fast answer Lloyd.

I would rate mode4 of speex with 4 and mode2 with 4. (MoS)
I can't hear a huge difference between both, except that mode4 files have more bass-parts (I guess because they are wideband encoded).

I recorded the files on a RME RayDat with 48kHz per second and sampled it down with cubase (at the export).
I'm not sure if this can make the difference, but I will try to record it with a samplerate of 16khz (I'm not sure if my RayDat even supports this?!)

No, I don't hear unusual frequencies (though it's sounding really weird).
Btw, I'm listening with EV Zx1-90's, which are basically PA-Speakers, but are as linear as a usual HiFi-Loudspeaker. These should not be the problem.

It bugs me, that im using speex with 96kbps and I'm getting even worse results than on the website with like 25 kbps.

I will check-out Opus tomorrow, and use pesq again.
Maybe it's a pesq problem, since im using P862, which - at least the documentation says so - supports only narrowband.

As I said, this is my first time on a mailing list, and I'm not sure where I should answer now.
Is this right?

Thanks for your help again!

Greetings,
Olli

2013/1/9 Rasmussen, Lloyd <lras at loc.gov<mailto:lras at loc.gov>>
What do your ears tell you about the performance of these codecs?  If you play some of the test signals back at half-speed, can you hear some frequencies that were not in the original signal?  Are all files running in mono?  If you have time, you might want to be testing the Opus codec, which has undergone considerable development since Speex.

I did not try these things under Cygwin.  And I have never run the Speex bitrate up to 80 or 96 kbps.  But using the pre-compiled Windows binaries under Windows 7, I could never get the resampler to work.  If I wanted the encoder to see a clean signal, I resampled it in SoundForge first.  This made a huge difference in audio quality, when working with signals that were initially recorded at 44.1 KHz sampling rate.  If aliasing is occuring, it will become more evident sometimes when the signal is compressed to a higher bitrate.

Resampling seems to work correctly in the pre-compiled Opus encoder and decoder available at
   www.opus-codec.org<http://www.opus-codec.org>
.

Lloyd Rasmussen, Senior Project Engineer
National Library Service for the Blind and Physically Handicapped
Library of Congress   202-707-0535<tel:202-707-0535>
http://www.loc.gov/nls
The preceding opinions are my own and do not necessarily reflect those of the Library of Congress, NLS.

From: speex-dev-bounces at xiph.org<mailto:speex-dev-bounces at xiph.org> [mailto:speex-dev-bounces at xiph.org<mailto:speex-dev-bounces at xiph.org>] On Behalf Of Oliver Martin
Sent: Wednesday, January 09, 2013 7:00 AM
To: speex-dev at xiph.org<mailto:speex-dev at xiph.org>
Subject: [Speex-dev] PESQ calculated MoS-Values for Speex

Hello,

I just signed up to this mailing-list (note: my first mailing list at all), because I'm having some problems related to speex.
Let me just introduce you to what I'm doing.

I am writing a short (really short) paper about VoIP techniques, especially audio codecs for speech.
I pointed out basic technologies behind audio codecs; vector quantization, lpc, long-term prediction and some more.

I planned to compare G711(.1 eventually) and speex with reference to filesize
(which is a bit weird, since the encoded g711-files are 6 times larger than the original wave files - but this is another matter), MoS value,
and a subjective opinion, wether I can still understand the file, or not.
The MoS value is (or rather should be) calculated by PESQ (ITU P.862: http://www.itu.int/rec/T-REC-P.862/en).
(I know the pesq scale ranges from -0,5 - 4,5, where MoS is usually from 0 - 5)

Pesq Settings:
R2a: Sampling Rate: 8kHz, Overall bit rate: 80kbit/s
R3: Sampling Rate: 16kHz, Overall bit rate: 96kbit/s

Pesq Results:
R2a (mode 2) of G711 returned Mos-Values from 0.656 to 3.33 (parameter +8000 in pesq)
R3 (mode 4) of G711 returned Mos-Values from 3.981 to 4.269 (parameter +16000 in pesq)

Speex Settings:
To get comparable results I decided to use settings that conform with the G711 ones. Command-Line Parameters:
"mode 2": "-n --comp 10 --quality 10 --bitrate 80000"
"mode 4": "-w --comp 10 --quality 10 --bitrate 96000"

Speex Results (this is where it starts to be weird):
mode 2: Mos Values from 2.541 to 3.211 (parameter +8000 in pesq)
mode 4: Mos Values from 2.045 to 3.355 (parameter +16000 in pesq)

I tried to get better values, at least in mode4, for like 2 hours, but I did not get something that corresponds to this:
http://speex.org/comparison/nb_codecs_small.png (Speex.org Website).
The fact, that a bitrate of like 25kbit/s results in a score of nearly 4,5 is confusing me like hell. (compared to my results)

First thing I tried was just "speexenc origfile.wav destfile.spx", but the result did not satisfy me.

The used wavefiles are created in Cubase5.1, 16bit and 16kHz.
Im using 1.2rc1, compiled with cygwin, running on Windows 8.
The 1.2beta3 kept crashing on Win8 and Win7 using the same audiofiles.
I also tried using 1.0.5, but the (MoS)results were not satisfying.

I have no clue why the score is at such a low level (compared to the graphic below).

Let me already thank you for your help.

Greetings
Olli

_______________________________________________
Speex-dev mailing list
Speex-dev at xiph.org<mailto:Speex-dev at xiph.org>
http://lists.xiph.org/mailman/listinfo/speex-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.xiph.org/pipermail/speex-dev/attachments/20130109/5d5ad3e8/attachment.htm 