[speex-dev] Re: Preprocessing and Echo Cancellation Notes.

[Steve Kann]

On Nov 9, 2003, at 1:00 AM, Jean-Marc Valin wrote:

>> 2) VAD: I never had a good VAD implementation in the library;  I had a
>> user-configurable audio energy threshold that did this, plus, I had a
>> hokey algorithm where I did a pretty naive estimate of the noise 
>> floor,
>> and then considered anything 5dB above that to be speech.  This worked
>> OK, but since I never updated my "noise floor" estimate, it was easily
>> broken if there was additional noise added at any time (i.e. the user
>> raised their microphone gain).  Here, I have gone in and adjusted some
>> knobs here:
>>
>>
>> /*      if (st->speech_prob> .35 || (st->last_speech < 20 &&
>> st->speech_prob>.1)) */
>>        if (st->speech_prob> .30 || (st->last_speech < 20 &&
>> st->speech_prob>.07))
>
> Well, the tuning always depends on what you're trying to achieve.
> Currently, the VAD is mostly tuned to make sure it doesn't start
> transmitting noise.

Right.  I'm actually planning on using it in two places:

1) In the VoIP client, where it is used mostly to lower upstream 
bandwidth.  (which is generally more expensive than downstream).

2) in the input to a conferencing mixer.  In this case, some clients 
will be coming in via VoIP, and so I will not double-process them (i.e. 
they'll run VAD at the client), but some clients will be coming to us 
via the PSTN, in which case we will do VAD on them.  The VAD is helpful 
here, because (a) the job of the conferencing mixer is greatly 
simplified if we only have to mix signals from actual speakers, and (b) 
noise is additive, so if we can just eliminate signals from 
non-speakers, we have less noise sent to everyone.

In either case, though, I'd rather err on the side of getting false 
positives, rather than false negatives.
>

>> to make it more sensitive, because I was getting some missed speech,
>> and some dropouts.  The dropouts were especially troubling, because
>> they caused a big degradation in speech in some cases.  The second
>> parameter helped a bit in this case, but I think there might be a
>> smarter implementation yet  -- like immediately lowering the threshold
>> once speech is detected, and then raising it gradually based on the
>> previous probabilities?
>
> There's probably lots of improvements that can be done...

Yep.  Also, since all three preprocessing functions are integrated, it 
means they can take advantage of each-others improvements.

>
>> I had also experimented with the 3GPP AMR VAD code (which is, of
>> course, copyrighted) to see how it compares, and it was still better
>> than speex, but speex is still pretty good.
>
> Well, if this VAD was able to beat the 3GPP VAD, then some people would
> probably lose their job :)

Hmm, or maybe they'd just use your VAD instead of theirs, and then get 
big bonuses for getting such great work done in such a short period of 
time!
>
>>     a) The most interesting thing it does is sometimes it also 
>> de-voices
>> speech.  I.e. if you say "aaaaaaa" into the filter, after about 3
>> seconds, you're voice just disappears :).  I thought this was
>> interesting, and I wanted to see how smart it was, so instead of a
>> single vowel sound, I tried repeating vowel-consonant pairs, like
>> "badumpbadumpbadump", and If I was consistent enough with that, I 
>> could
>> make them mostly disappear as well.  This was lots of fun.  What it
>> points out, though, is that denoising and, say, singing, won't go 
>> along
>> very well at all!  I'm also wondering if it could be used to cancel 
>> out
>> a boring speaker :)
>
> Well, what you observe is the effect of noise adaptation. If (in
> general) a signal is stationary, there's no real way to differentiate 
> it
> from noise... On easy way to solve the problem though is simply to
> increase the time over which the signal needs to be stationary to be
> considered as noise.

Right.  I understood that much.  I think that it might be possible also 
to tune the bands in which denoising happens; i.e. don't remove (or 
completely remove) signals likely to be vowels..  (maybe a range from 
500hz-1500hz or something?  It might be something interesting to play 
with.
>
>>      b) There are some "musical" artifacts left over.  They're not 
>> huge,
>> but I did notice them as voices faded out, etc.  I'm guessing this is
>> de-noising, but I was using denoise + AGC at the time, so I'm not 
>> sure;
>>   if AGC is just scaling, then I guess it must be the denoise.  I'll
>> probably add options to my UI to individually control the different
>> filters, which will make evaluation easier.
>
> Musical noise is something that most (all?) denoisers have at different
> degrees.

Yes, that's what I've read, which is why I at least knew the correct 
term for it :) I guess I should read the techniques involved in 
reducing their perception, and see what can be done.
>
>> Finally, echo cancellation.  I haven't actually been able to get the
>> echo canceller to do anything really useful for me.  I'm currently
>> using it something like this:
>>
>>      ec = speex_echo_state_init(160, 500); /* in ms */
>
> Actually, the second parameter is in samples, since there's no way to
> tell the sampling rate.
>> I've also tried to use it the same way, but scaling my short samples
>> into the range -1< n < 1 (dividing/multiplying by 32767).
>
> The right range is +- 32768. Actually in the CVS version, all inputs 
> and
> outputs are now short, so it solves the problem.
>
>> 1) How should I call the echo canceller with frames of short samples?
>
> Not sure I understand the question?

You've already answered it, I think, with the range answer above..
>
>> 2) Could the apparent "no effect" be due to also later using the
>> preprocessor on the frames?  I.e. if the echo canceller is only
>> reducing the echo by -20 db or something, the AGC will later bring it
>> right back.  Is this the reason for the noise array?  Should it work 
>> at
>> all without that code (that I've read isn't quite complete yet?).  [I
>> haven't tried to use that yet, because the library architecture
>> currently has the echo canceller down in the audio driver, where it
>> gets well-correlated input/output buffers, and the preprocessing is
>> much higher, in the audio-device independent layer, where it only has
>> input buffers -- so it will be a bit of work to try this out].
>
> I'm not sure what's the problem. First, you need to know that the echo
> canceller is still in experimental state. The theory of echo
> cancellation is rather simple, but the implementation is not. For
> example, in order to get good results, you need a good crosstalk
> detector. The current one kind of sucks. One thing too. In your 
> example,
> you have a 500 sample (not ms) filter length. However, if the
> input/output offset introduced by your card is larger than that (or in
> the same order), then you won't have any cancellation at all.

I'll play with it some more, using the correct parameters, and let you 
know how I fare.

Thanks for your quick response, even on the weekend!

-SteveK

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'speex-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.