[vorbis] Graceful degradation of signal

[Eric Scheirer]

Hi Johan and al;,

There are two ideas mixed together in your email, I'll
try to separate them and give some suggestions.

1. Scalability

"Scalability" is the name that MPEG uses for the notion of
having a bitstream that can be unpeeled.  That is, given
a 128 kbps bitstream, you can easily extract from it a
96 kbps bitstream that has a minimal degradation in quality.
MP3, for example, does not have this property because you
have to requantize all the scalefactors at once--essentially
decode, then re-encode--in order to scale back the bitstream.

MPEG-4 Audio is the first audio standard to enable 
significant scalable functionality.  It is still a 
rather new topic in audio coding (unlike graphics,
in which "progressive" techniques have been studied
for some time).  As best we can tell, it is not possible
to have a 128 kbps scalable bitstream that gives
quality as good as a 128 non-scalable bitstream (some 
bits must be used to describe how the scalability 
happens).  The way MPEG-4 does it, undoubtedly patent-
pending, is to represent the stream as a "base layer"
plus one or more "scalability layers."  The layers
are not each complete sound signals, but represent 
extra quantization--like coding the residual--of the
original sound.  A similar technique can be used for
mono/stereo scalability, so that a mono bitstream 
can be extracted from less than half the bandwidth of
a stereo bitstream.

There are formal listening-test results for MPEG-4 Audio
using scalability in two public documents on the MPEG
Audio web page:

   [1] C. Colomes, C. Jacobson, E. Scheirer.  "Report on
       the MPEG-4 Audio NADIB verification tests."  
       ISO-MPEG Document N2276, Dublin, Jul. 1998.

   [2] E. Scheirer, S.-W. Kim, M. Dietz.  "MPEG-4 Audio
       verification test results: Audio on Internet."
       ISO-MPEG Document N2425, Atlantic City, Oct. 1998.

Both are available at 
  http://sound.media.mit.edu/mpeg4/audio/public/ .

2. Error resiliance

"Error resiliance" is the name that MPEG uses for the notion
of having a bitstream in which if packets are corrupted
(via bit-flipping with uniform bit-error-rate) or vanish,
the quality of the sound is minimally degraded.  MPEG uses
more-or-less well-known information-theory techniques,
such as using self-correcting codes for bit-stuffing rather
than Huffman coding, plus some rearrangement of the format,
to provide error resiliance.  I don't know a lot about
the technical details.  In MPEG-4 (the first MPEG audio 
standard to provide error resiliance), you pay about 25%
penalty to get statistically indistinguishable quality
from the uncorrupted signal at 1e-3 bit error rate. 

There is a formal listening test on this style of coding
as well:

   [3] R. Sperschneider, F. Feige, S. Quackenbush.  "Report
       on the MPEG-4 Audio Version 2 Verification Test."
       ISO-MPEG Document N3075, Maui, Dec. 1999.

I don't think MPEG has paid a lot of attention to error
concealment, which is what you have to do when you have
packet loss.  Private techniques (RealAudio) probably do
this better.

The techniques for combining scalability and error-resiliance
haven't been studied much as far as I know.  I suspect
that doing the obvious thing doesn't work well (probably
because you'd want to provide unequal error protection for
the base layer and the scalability layers), otherwise 
MPEG would have tested and reported it.

Best,

 -- Eric

+-----------------+
|  Eric Scheirer  |A-7b5 D7b9|G-7 C7|Cb   C-7b5 F7#9|Bb  |B-7 E7|
|eds at media.mit.edu|      < http://sound.media.mit.edu/~eds >
|  617 253 1750   |A A/G# F#-7 F#-/E|Eb-7b5 D7b5|Db|C7b5 B7b5|Bb|
+-----------------+

-----Original Message-----
From: Johan Ovlinger <johan at ccs.neu.edu>
To: vorbis at xiph.org <vorbis at xiph.org>
Date: Monday, May 15, 2000 6:00 PM
Subject: [vorbis] Graceful degradation of signal

>Hello all. 
>
>In the shower the other day (where most of this sort of musing gets
>done, eh?) I was thinking about graceful degradation of audio signals.
>Let me apologise in advance if these are elementary concepts or if I
>demonstrate a complete lack of insight -- I don't rate even a dabbler
>status in the area of audio codecs.
>
>Anyway:
>
>If we have a 128kbs signal coming down a *udp* channel with close to
>25% packet loss, it would be really nice if it sounded like a 96kbs
>signal, not a choppy 128kbs.
>
>Short question: is this possible and feasible?
>
>Extended question/scenario: 
>
>It is of course easy to achieve if we are unconcerned about the
>absolute quality. Just define a 128kbs stream to be a 2-redundant
>64kbs stream, and voila, we have graceful degradation. 
>That's none to nice tho. That's still an all-or-nothing affair. 
>
>Instead, we'd like to have the property of the codec that it be
>possible to separate the encoded signal into "quality shells" (to
>invent a term). A necessary property of such a shell is that they be
>additive onto the next lower rung. (sorry for no doubt misusing
>terminology) Ie:
>
>a 128kbs shell is equiv. to the 64kbs shell + 64kbs extra information
>
>Now we can acheive graceful degradation by sending the various shells
>with varying degrees of redundancy. For example, 
>the 0->32kbs shell might  be sent 3/2 (ie 1 parity bit for 2 data) and 
>the 32->64kbs shell might be sent 5/4 and 
>the 64kbs->128kbs shell      sent 9/8.
>
>that adds up to sending 17/14 which is something like 155kbs instead
>of the 128kbs data stream. However, we are now able to accept a 11%
>drop rate before noticing any degradation, up to 20% with slight
>degradation, and a 33% rate for extreme degradation.
>
>*However, the sound will still be continuous, although with decreasing
>quality* 
>
>Users accept drops in quality rather than glitches (source: pulled out
>of a hat).
> 
>So now that I've demostrated my ignorance, I invite ya'll to tear the
>idea to shreds. I imagine that the feasability of producing a codec
>with the quality shells will be the most glaring flaw.
>
>This is of course only one approach. If any others (or indeed this
>one) have been investigated, please point me in the right direction.
>
>Johan
>
>
>--- >8 ----
>List archives:  http://www.xiph.org/archives/
>Ogg project homepage: http://www.xiph.org/ogg/
>To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
>containing only the word 'unsubscribe' in the body.  No subject is needed.
>Unsubscribe messages sent to the list will be ignored/filtered.
>

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.