[daala] HDR is coming

[HuBandiT@gmail.com]

HDR and higher bit-depth seem to be coming:

http://www.dvinfo.net/article/misc/science_n_technology/hpa-tech-retreat-2014-day-4.html 
section "Better Pixels: Best Bang for the Buck?"

http://www.dvinfo.net/article/misc/science_n_technology/hpa-tech-retreat-2014-day-1-poynton-watkinson.html

  * industry seems to use 12-14 bits today, consensus seems to be at
    least 12 bits of luma is needed soon even for consumers; prosumer
    camcorders (e.g. Sony PXW-X70 - $2000) are doing 10-bit 4:2:2
    1080p59.94 today, and anything above $2500-3000 seems to be 12 bit
    or above
  * looks like 13 bits would be sufficient with a simple log curve,
    Dolby is proposing 12 bits with their "Perceptual Quantization" curve
  * some armchair thinking (just my pebbles to throw into the thinking
    pool):
      o log encoding would have the benefit that the traditional color
        difference signals derived from log encoded RGB components would
        eliminate (white-balanced) intensity changes (e.g. shadows,
        fades/dips to-from black) from color channels
      o with intensity decoupled from color:
          + considerably lower color precision could be sufficient
            (since cosine falloff from object curvature, lens
            vignetting, primary light shadows no longer leak into
            chroma, no longer forcing it to have comparable precision to
            eliminate banding on final result)
          + maybe replace color differences with more preceptual
            metrics: some kind of saturation and hue
              # could allow heavier quantization still or even lower
                color precision outright (on the assumption that hue and
                saturation changes much less in well-lit, well-exposed,
                real life scenes)
              # think of it like one aspect of reverse Phong shading:
                shiny sphere in vacuum under white light only ever has
                it's own hue - only intensity and saturation changes
                (cosine falloff: towards black, highlight: towards
                white; hue channel is quasi constant, flat; real world
                will be messier e.g. hue will be pulled away by light
                reflected from surrounding objects - but see below on
                illumination/object color decomposition)
      o once chroma/color precision is lowered, it might make sense to
        go 4:4:4 all the time and just don't bother with chroma
        down/upsampling at all
      o establish the scene/discussion for scene decomposition: e.g.
        separately coding albedo (object reflectance) and illuminance
          + the first step could be a separate
            illuminance/intensity/gain channel, that factors (multiply
            in linear light = addition in log light) into the final
            intensity of the output pixels
          + encoders unwilling to utilize this can leave this channel
            blank at 0dB/0EV/1x
          + simplistic encoders could benefit:
              # dips to/from black could preserve full color in main
                channels, and only adjust this channel
              # crossfades could ramp up/down this channel while
                referencing main channels at the two frames at both ends
                of the crossfade (weighed prediction in linear light
                conceptually)
          + advanced encoders: separately encoding high amplitude scene
            illuminance variations from lower ampltude object
            reflectance/texture might provide coding gains, especially
            in the case of HDR
              # scene illuminance: higher amplitude, but less details
                (mostly "broad strokes" - different statistics than main
                channels)
              # object reflectance/texture (main channels): smaller
                amplitude, but more details
              # separate prediction/motion compensation for these two
              # ideally, scene illuminance should be color as well, to
                predict coloured lighting (e.g. illuminance in single
                off-white or multiple lightsource cases)
              # use it as hints for HDR tonemapping tool (see still
                photography research)
          + next step could be to add a highlight layer (kinda like
            specular highlights in reverse Phong shading - gradually
            blending away the area around the highlight position into
            some color of the light source, whether it's transform
            coding or some kind of shape based parametric modelling),
            there exists machine color vision research in these directions
          + doesn't need to be perfect (it's just prediction after all)
            or even cover many cases - just go for some low-hanging
            fruits, enough to spark industry discussion/experimentation




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.xiph.org/pipermail/daala/attachments/20150816/35d9853d/attachment.htm