[opus] Opus for ASR - update and questions

[Gregory Maxwell]

On Fri, Dec 7, 2012 at 6:01 PM, Young, Milan <Milan.Young at nuance.com> wrote:
> I'm assuming that by file pattern, you are referring to Ross' method of capturing actual lossey transmissions, and mapping those errors onto the decoded waveform.  If so, I see a few problems:

A file input can be just a simple interface for getting data from a
model into the application.

Also, a while back the mumble folks were kind enough to provide us
with a very large amount of traces from real users which could also be
used with a file input.

> Unquantified - What loss and delay level are we simulating?  It needs to have a name so it can be compared to other levels.  You can map from a name into a set of error patterns, but you can't map from an error pattern into a single name.

Average the results from a random sampling. Characterize the
collection by its average loss rate or higher order parameters
(probability of 2,3,4, etc sequential loss).

> Non trivial - Maybe I'm behind the times, but serializing loss and delays in live traffic sounds difficult.  This problem is compounded by the relative infrequency of loss events.  (Most of what I'm interested in are worst case scenarios north of 5% errors).

Mumble added instrumentation to just have their client software to get
traces from hundreds of thousands of users...

> Unrepresentative - We need a variety of error patterns at each error level to ensure statistically significant results.  For my large scale measurements, were talking in the thousands.

The biggest problem wrt representativeness in collections like that
from live inputs is that there may be systemic bias like mumble users
being more likely than a randomly selected member of the public to be
playing a game or downloading a movie while chatting. And that I have
no answer for... but you do run into the same kind of exposure if you
try to pick model parameters reflecting typical usage.

> Sounds like the queuing theory suggestion is the right way to proceed.  If you have any particular pointers within that domain, your advice would be appreciated.

Everything I've seen for producing queuing traces involved e.g running
NS2 simulations of whole networks with many network stacks. Kinda
overkill for this sort of thing. Perhaps Ross has some good citations.