<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi Jean-Marc,</p>
<p>Touching base as it's been awhile since our last correspondence. Did you need any additional information from us?</p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols;">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">Thanks,
<div style="font-size:13px">Peter</div>
<div style="font-size:13px"><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Freshman, Peter<br>
<b>Sent:</b> Tuesday, June 20, 2017 1:08:19 PM<br>
<b>To:</b> Jean-Marc Valin; Jean-Marc Valin; opus@xiph.org; Nadeau, Benoit<br>
<b>Subject:</b> Re: [opus] [EXTERNAL] Re: Submitting a patch that exposes VAD voiced/unvoiced signal type</font>
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi Jean-Marc,</p>
<p>We're exposing the opus_internal_flags data structure so that we can access the value assigned to prevSignalType. Here's a snippet of our code:</p>
<p><br>
</p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> error = opus_encoder_get_internal_flags(vad->opus, &internalflags);</span></p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> if (error != OPUS_OK)</span></p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> {</span></p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> return OPUSVAD_OPUS_ERROR;</span></p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> }</span></p>
<p><br>
</p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> cur_signal_type = internalflags.prevSignalType;</span></p>
<p><br>
</p>
<p><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;"> if ((vad->cur_state == STATE_NO_STATE) && (cur_signal_type == TYPE_UNVOICED || cur_signal_type == TYPE_VOICED) && (vad->prev_signal_type == </span><span style="font-family: Consolas, Courier, monospace; font-size: 9pt;">TYPE_NO_VOICE_ACTIVITY))
{</span></p>
<div><span style="font-family: Consolas, Courier, monospace; font-size: 12px;"> </span><span style="font-family: Consolas, Courier, monospace; font-size: 12px;"> </span>...</div>
<div><br>
</div>
<div>Our library uses this information to apply end pointing on voice-based audio. For example, we work with many customers implementing speech-enabled TV set-top-box solutions. We offer a small library that does some start and end of speech detection on the
audio so they can understand whether or not someone is actually speaking into the remote control.</div>
<div><br>
</div>
<div>Does this help make things more clear?</div>
<meta content="text/html; charset=UTF-8">
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols;">
<p><br>
</p>
<p><br>
</p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols;">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">
<div style="font-family:Tahoma">Thanks,
<div style="font-size:13px">Peter</div>
<div style="font-size:13px"><br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Jean-Marc Valin <jmvalin@jmvalin.ca><br>
<b>Sent:</b> Friday, June 16, 2017 2:27:01 PM<br>
<b>To:</b> Freshman, Peter; Jean-Marc Valin; opus@xiph.org; Nadeau, Benoit<br>
<b>Subject:</b> Re: [opus] [EXTERNAL] Re: Submitting a patch that exposes VAD voiced/unvoiced signal type</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi Peter,<br>
<br>
Can you say a little bit more about what you're doing exactly with the<br>
information you're exposing and how? unfortunately, I don't have a<br>
concrete proposal in mind right now. That's in part because I don't<br>
quite understand the use case, but also because it's really hard to<br>
expose this kind of information in a way that both avoids breaking<br>
application with new versions and doesn't prevent future improvements to<br>
Opus.<br>
<br>
Cheers,<br>
<br>
Jean-Marc<br>
<br>
On 08/06/17 08:20 AM, Freshman, Peter wrote:<br>
> Hi Jean-Marc,<br>
> <br>
> Thank you for the valuable feedback. You're correct in that we focused<br>
> on enabling this just for SILK. Because our solutions are focused on<br>
> voice, we did not explore doing the same in CELT mode, but we can<br>
> certainly look into the details of analysis.c.<br>
> <br>
> <br>
> Regarding the concern of exposing internals, do you have a specific<br>
> proposal in mind?<br>
> <br>
> <br>
> We've been sharing this patch with our customers over the last several<br>
> months, and the preference obviously would be to have it in the public<br>
> domain. We're interested in any opportunity to accelerate this.<br>
> <br>
> <br>
> Thanks,<br>
> Peter<br>
> <br>
> ------------------------------------------------------------------------<br>
> *From:* Jean-Marc Valin <jmvalin@mozilla.com><br>
> *Sent:* Wednesday, June 7, 2017 2:46:52 AM<br>
> *To:* Freshman, Peter; opus@xiph.org<br>
> *Subject:* [EXTERNAL] Re: [opus] Submitting a patch that exposes VAD<br>
> voiced/unvoiced signal type<br>
> <br>
> Hi Peter,<br>
> <br>
> There's two main issues with a patch like the one you're proposing.<br>
> First, the data is only valid when SILK is being used and is essentially<br>
> undefined in CELT mode. The second issue is that by exposing internals,<br>
> it makes it impossible to improve these algorithms since it would break<br>
> API compatibility. I'm not fundamentally against trying to expose some<br>
> information, but there would have to be a way to address those two issues.<br>
> <br>
> On a slightly different topic, have you looked at the VAD probability<br>
> that's computed in analysis.c (along with the speech/music probability)?<br>
> <br>
> Cheers,<br>
> <br>
> Jean-Marc<br>
> <br>
> <br>
>> I'm reaching out because we'd like to contribute back to the project<br>
>> a patch that exposes the signal type of the audio packet when<br>
>> encoding the PCM audio to OPUS. We've found the Opus VAD algorithm to<br>
>> be exceptional in this regard and have written a library that<br>
>> leverages this information for audio end-pointing. Attached is the<br>
>> patch. Please let us know if you'd be willing to accept it, or if<br>
>> you'd prefer we fork libopus or recommend some other option.<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> opus mailing list<br>
> opus@xiph.org<br>
> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.xiph.org_mailman_listinfo_opus&d=DwICaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=WPCM474fpTKKLKdP8t4H4wmfeR2M_bMLGQolTVoLb_c&m=rz_WLh1GC8llHGw1p_zxxyNuaBDfmeqbwjCOZRqu4cg&s=qCUgcbAdWcKbN8KlEp-WDAPekhb1nSBQiFWaur7Z6CU&e=" id="LPlnk876481" previewremoved="true">
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.xiph.org_mailman_listinfo_opus&d=DwICaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=WPCM474fpTKKLKdP8t4H4wmfeR2M_bMLGQolTVoLb_c&m=rz_WLh1GC8llHGw1p_zxxyNuaBDfmeqbwjCOZRqu4cg&s=qCUgcbAdWcKbN8KlEp-WDAPekhb1nSBQiFWaur7Z6CU&e=</a>
<br>
> <br>
</div>
</span></font></div>
</div>
</body>
</html>