[theora-dev] Analogue artifact estimation

Sun Sep 1 16:30:56 PDT 2002

There actually has been some work done on psycho-visual stuff. The XVID video codec has a "luminance-masking" function that applies more compression to parts of the picture that is darker, and RealVideo 9 seems to have some functions that allow it to compress the background of a picture more than the foreground. MPEG-4 also seems to have some specs for "GMC", which supposedly provides for some basic psycho-visual stuff.

Unfortunately, in this business, research generally means patents. We may want to proceed with caution if we really want to get into this kind of thing.

-Henry

----- Original Message -----
From: "Daniel B. Miller" <dan at on2.com>
Date: Sun, 1 Sep 2002 14:09:08 -0400 (EDT)
To: <theora-dev at xiph.org>
Subject: Re: [theora-dev] Analogue artifact estimation

<p>> On Sun, 1 Sep 2002 jbradford at dial.pipex.com wrote:
> 
> > > I think a lot these kind of artifacts are better handled through
> > > pre-processing.
> >
> > Yeah, that occurred to me, too, after I'd posted it :-).
> >
> > > Something like a temporal smoother can pretty effectively limit pixel
> > > crawl and the like, and it's a well known fact that simply blurring a
> > > video makes it compress more efficiently.
> >
> > Also, interestingly, adding a small amount of noise to a blurred video, (or still image), makes a lot of people percieve it as being sharper.  Of course, the noise-enhanced video is then practically useless for any further processing, but it's an interesting side-effect.
> >
> > > In fact, many have said that VP3 already pre- and post- filters too
> > > much, which can lead to blurry video.
> >
> > Perhaps an option to add noise after decompression might satisfy some
> > people!?  Sounds odd, but if it works...
> >
> hmm...  Ok, I'm the one who said we should focus on the bitstream, but
> since the subject of quality has come up I feel compelled to add my 2
> cents:
> 
> I've recently done quite a bit of research into the whole issue of
> perceived vs. objectively measurable quality of compression algorithms.
> It turns out that the usual metric, PSNR (Peak Signal-to-Noise Ratio) is
> extremely limited for a number of reasons.  In particular, it gives little
> useful information about the frequency domain.  For example, an algorithm
> can improve PSNR by making the low-frequency information more accurate at
> the expense of high-frequency data.  This does little for perceived
> quality, and can actually make the result look worse.  Similarly, any
> noise at all reduces PSNR, whereas the human psycho-visual system is very
> good at extracting information from a noisy channel; this leads to codecs
> that avoid any noise at all costs, even when it would be an appropriate
> approximation of the original image (for instance, a textured wall will
> look better noisy than filtered).
> 
> The point being, most algorithms including VP3 have been developed with
> PSNR being the major point of reference by which improvements are
> quantified and judged.  This has led to a generation of codecs that tend
> to blur out high frequency detail rather than allow any noise at all into
> the signal.
> 
> I'm in the process of developing an alternative metric to PSNR that
> provides useful information about the frequency response of a given codec.
> It will also use an information-theoretical perspective to detect when
> signals are present within a noisy channel, rather than defining any noise
> at all as a negative.  These techniques could conceivably be used
> 'in-band' as well, ie we could rework the encoder to make various
> decisions (which motion vector to use, how to quantize DCT, etc) in a way
> that will better track perceived visual quality, rather than striving only
> to minimize the input/output error term regardless of frequency and noise
> characteristics.  Interestingly, most of this work tracks what is already
> done in audio; for some reason, the video crowd has shied away from any
> serious psycho-visual approach to compression.
> 
> This approach could conceivably allow us to make substantial improvements
> in the VP3 encoder without having to make major changes in the bitstream
> definition.  We may even want to set up a separate discussion group,
> theora-encoder at xiph.org say, to focus on the issues of encoder quality as
> opposed to the more general task of defining the bitstream (in MPEG-land,
> this distinction is referred to as 'normative' vs. 'non-normative' work;
> normative meaning anything that affects the bitstream definition.)
> 
> Please respond with your level of interest in this subject so we can
> calibrate the level of support we might have for such a project.
> 
> -dbm
> 
> > John.
> 
>  ___  Dan Miller
> (++,) CTO and founder, On2 Technologies
> 
> >
> > > ----- Original Message -----
> > > From: "Daniel B. Miller" <dan at on2.com>
> > > Date: Sat, 31 Aug 2002 18:31:43 -0400 (EDT)
> > > To: <theora-dev at xiph.org>
> > > Subject: Re: [theora-dev] Analogue artifact estimation
> > >
> > >
> > > > hi --
> > > >
> > > > Interesting points you bring up.  Realistically, theora is going to be
> > > > focused more on the definition of the bitstream and issues of
> > > > audio/video integration into the OGG format.  Advanced thoughts about
> > > > new codec ideas might be well received at the tarkin list
> > > > http://www.xiph.org/ogg/vmail.html.
> > > >
> > > > Not to discourage you from thinking about these things as they relate to
> > > > VP3;  however it's important to note that our primary objective
> > > > will be to define the bitstream before tackling major improvements in the
> > > > encoder.
> > > >
> > > >  ___  Dan Miller
> > > > (++,) CTO and founder, On2 Technologies
> > > >
> > > > On Wed, 28 Aug 2002 jbradford at dial.pipex.com wrote:
> > > >
> > > > > Hi List,
> > > > >
> > > > > Just batting a few ideas around here, but would it be possible to
> > > > > include in to the codec, estimation for common video artifacts that
> > > > > occur in the analogue world?
> > > > >
> > > > > For example, anything that's gone through a composite stage will
> > > > > likely have dot-crawl and false colour - if we can recognise this
> > > > > effect in the encoder, we can treat it as a special case.
> > > > >
> > > > > Other artifacts that come to mind are:
> > > > >
> > > > > * 3-2 Pull down
> > > > >
> > > > > * 30/25 and 25/30 FPS conversion artifacts
> > > > >
> > > > > * Dropouts, (both noisy, a-la VHS, and solid bars, a-la BetaSP, (which
> > > > > I believe halves the chroma resolution - can we possibly take this in
> > > > > to account as well?) )
> > > > >
> > > > > * Stobe effect of a CRT filmed by a different scan-rate camera
> > > > >
> > > > > * Colour changes in multi-layered painted cel animation, (where a
> > > > > character is, say, talking, and they repeatedly add and remove a cel
> > > > > from the top layer, their face flashes bright and dark :-) )
> > > > >
> > > > > Infact, existing video compression codecs do a *terrible* job of
> > > > > encoding hand painted animation, especially anime, despite claims by
> > > > > well known studios that it is not an issue.  Good motion estimation to
> > > > > compensate for poor registration of animation would be worth
> > > > > investigating.
> > > > >
> > > > > I think that the problem with all codecs to date is that they are
> > > > > designed to compress perfect material originating directly from a
> > > > > camera.  In my opinion, the above *analogue* artifacts are what give
> > > > > the digital codecs a hard time.
> > > > >
> > > > > I'd be very interested on any comments on these points.
> > > > >
> > > > > John. --- >8 ---- List archives:  http://www.xiph.org/archives/ Ogg
> > > > > project homepage: http://www.xiph.org/ogg/ To unsubscribe from this
> > > > > list, send a message to 'theora-dev-request at xiph.org' containing only
> > > > > the word 'unsubscribe' in the body.  No subject is needed. Unsubscribe
> > > > > messages sent to the list will be ignored/filtered.
> > > > >
> > > >
> > > > --- >8 ----
> > > > List archives:  http://www.xiph.org/archives/
> > > > Ogg project homepage: http://www.xiph.org/ogg/
> > > > To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> > > > containing only the word 'unsubscribe' in the body.  No subject is needed.
> > > > Unsubscribe messages sent to the list will be ignored/filtered.
> > > >
> > > >
> > >
> > >
> > > --
> > > _______________________________________________
> > > Get your free email from http://mymail.operamail.com
> > >
> > > Powered by Outblaze
> > > --- >8 ----
> > > List archives:  http://www.xiph.org/archives/
> > > Ogg project homepage: http://www.xiph.org/ogg/
> > > To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> > > containing only the word 'unsubscribe' in the body.  No subject is needed.
> > > Unsubscribe messages sent to the list will be ignored/filtered.
> > >
> >
> > --- >8 ----
> > List archives:  http://www.xiph.org/archives/
> > Ogg project homepage: http://www.xiph.org/ogg/
> > To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> > containing only the word 'unsubscribe' in the body.  No subject is needed.
> > Unsubscribe messages sent to the list will be ignored/filtered.
> >
> 
> --- >8 ----
> List archives:  http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
> containing only the word 'unsubscribe' in the body.  No subject is needed.
> Unsubscribe messages sent to the list will be ignored/filtered.
> 
> 

-- 
_______________________________________________
Get your free email from http://mymail.operamail.com

Powered by Outblaze
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'theora-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.