[theora-dev] Analogue artifact estimation

Sun Sep 1 11:09:08 PDT 2002

On Sun, 1 Sep 2002 jbradford at dial.pipex.com wrote:

> > I think a lot these kind of artifacts are better handled through
> > pre-processing.
>
> Yeah, that occurred to me, too, after I'd posted it :-).
>
> > Something like a temporal smoother can pretty effectively limit pixel
> > crawl and the like, and it's a well known fact that simply blurring a
> > video makes it compress more efficiently.
>
> Also, interestingly, adding a small amount of noise to a blurred video, (or still image), makes a lot of people percieve it as being sharper.  Of course, the noise-enhanced video is then practically useless for any further processing, but it's an interesting side-effect.
>
> > In fact, many have said that VP3 already pre- and post- filters too
> > much, which can lead to blurry video.
>
> Perhaps an option to add noise after decompression might satisfy some
> people!?  Sounds odd, but if it works...
>
hmm...  Ok, I'm the one who said we should focus on the bitstream, but
since the subject of quality has come up I feel compelled to add my 2
cents:

I've recently done quite a bit of research into the whole issue of
perceived vs. objectively measurable quality of compression algorithms.
It turns out that the usual metric, PSNR (Peak Signal-to-Noise Ratio) is
extremely limited for a number of reasons.  In particular, it gives little
useful information about the frequency domain.  For example, an algorithm
can improve PSNR by making the low-frequency information more accurate at
the expense of high-frequency data.  This does little for perceived
quality, and can actually make the result look worse.  Similarly, any
noise at all reduces PSNR, whereas the human psycho-visual system is very
good at extracting information from a noisy channel; this leads to codecs
that avoid any noise at all costs, even when it would be an appropriate
approximation of the original image (for instance, a textured wall will
look better noisy than filtered).

The point being, most algorithms including VP3 have been developed with
PSNR being the major point of reference by which improvements are
quantified and judged.  This has led to a generation of codecs that tend
to blur out high frequency detail rather than allow any noise at all into
the signal.

I'm in the process of developing an alternative metric to PSNR that
provides useful information about the frequency response of a given codec.
It will also use an information-theoretical perspective to detect when
signals are present within a noisy channel, rather than defining any noise
at all as a negative.  These techniques could conceivably be used
'in-band' as well, ie we could rework the encoder to make various
decisions (which motion vector to use, how to quantize DCT, etc) in a way
that will better track perceived visual quality, rather than striving only
to minimize the input/output error term regardless of frequency and noise
characteristics.  Interestingly, most of this work tracks what is already
done in audio; for some reason, the video crowd has shied away from any
serious psycho-visual approach to compression.

This approach could conceivably allow us to make substantial improvements
in the VP3 encoder without having to make major changes in the bitstream
definition.  We may even want to set up a separate discussion group,
theora-encoder at xiph.org say, to focus on the issues of encoder quality as
opposed to the more general task of defining the bitstream (in MPEG-land,
this distinction is referred to as 'normative' vs. 'non-normative' work;
normative meaning anything that affects the bitstream definition.)

Please respond with your level of interest in this subject so we can
calibrate the level of support we might have for such a project.

-dbm

> John.

 ___  Dan Miller
(++,) CTO and founder, On2 Technologies

>
> > ----- Original Message -----
> > From: "Daniel B. Miller" <dan at on2.com>
> > Date: Sat, 31 Aug 2002 18:31:43 -0400 (EDT)
> > To: <theora-dev at xiph.org>
> > Subject: Re: [theora-dev] Analogue artifact estimation
> >
> >
> > > hi --
> > >
> > > Interesting points you bring up.  Realistically, theora is going to be
> > > focused more on the definition of the bitstream and issues of
> > > audio/video integration into the OGG format.  Advanced thoughts about
> > > new codec ideas might be well received at the tarkin list
> > > http://www.xiph.org/ogg/vmail.html.
> > >
> > > Not to discourage you from thinking about these things as they relate to
> > > VP3;  however it's important to note that our primary objective
> > > will be to define the bitstream before tackling major improvements in the
> > > encoder.
> > >
> > >  ___  Dan Miller
> > > (++,) CTO and founder, On2 Technologies
> > >
> > > On Wed, 28 Aug 2002 jbradford at dial.pipex.com wrote:
> > >
> > > > Hi List,
> > > >
> > > > Just batting a few ideas around here, but would it be possible to
> > > > include in to the codec, estimation for common video artifacts that
> > > > occur in the analogue world?
> > > >
> > > > For example, anything that's gone through a composite stage will
> > > > likely have dot-crawl and false colour - if we can recognise this
> > > > effect in the encoder, we can treat it as a special case.
> > > >
> > > > Other artifacts that come to mind are:
> > > >
> > > > * 3-2 Pull down
> > > >
> > > > * 30/25 and 25/30 FPS conversion artifacts
> > > >
> > > > * Dropouts, (both noisy, a-la VHS, and solid bars, a-la BetaSP, (which
> > > > I believe halves the chroma resolution - can we possibly take this in
> > > > to account as well?) )
> > > >
> > > > * Stobe effect of a CRT filmed by a different scan-rate camera
> > > >
> > > > * Colour changes in multi-layered painted cel animation, (where a
> > > > character is, say, talking, and they repeatedly add and remove a cel
> > > > from the top layer, their face flashes bright and dark :-) )
> > > >
> > > > Infact, existing video compression codecs do a *terrible* job of
> > > > encoding hand painted animation, especially anime, despite claims by
> > > > well known studios that it is not an issue.  Good motion estimation to
> > > > compensate for poor registration of animation would be worth
> > > > investigating.
> > > >
> > > > I think that the problem with all codecs to date is that they are
> > > > designed to compress perfect material originating directly from a
> > > > camera.  In my opinion, the above *analogue* artifacts are what give
> > > > the digital codecs a hard time.
> > > >
> > > > I'd be very interested on any comments on these points.
> > > >
> > > > John. --- >8 ---- List archives:  http://www.xiph.org/archives/ Ogg
> > > > project homepage: http://www.xiph.org/ogg/ To unsubscribe from this
> > > > list, send a message to 'theora-dev-request at xiph.org' containing only
> > > > the word 'unsubscribe' in the body.  No subject is needed. Unsubscribe
> > > > messages sent to the list will be ignored/filtered.
> > > >
> > >
> > > --- >8 ----
> > > List archives:  http://www.xiph.org/archives/
> > > Ogg project homepage: http://www.xiph.org/ogg/
> > > To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> > > containing only the word 'unsubscribe' in the body.  No subject is needed.
> > > Unsubscribe messages sent to the list will be ignored/filtered.
> > >
> > >
> >
> >
> > --
> > _______________________________________________
> > Get your free email from http://mymail.operamail.com
> >
> > Powered by Outblaze
> > --- >8 ----
> > List archives:  http://www.xiph.org/archives/
> > Ogg project homepage: http://www.xiph.org/ogg/
> > To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> > containing only the word 'unsubscribe' in the body.  No subject is needed.
> > Unsubscribe messages sent to the list will be ignored/filtered.
> >
>
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
>

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.