[vorbis-dev] Calculation of floor1_inverse_dB_table

[John Ripley]

I can't seem to figure out exactly how floor1_inverse_dB_table is
calculated. The table in "vorbis/lib/floor1.c" appears to be roughly
generated by the equation:

table[x] = pow(10, ((x - 255) * (140 / 256) / 20))

Basically, 256 entries represents 140dB. The problem is this doesn't
generate exactly the same numbers. For example, table[0] with this
formula (using "bc") is:

1.0649856353504289818e-7

But the "official" table says:

1.0649863e-7

This is a relatively small error and probably wouldn't show up in the
decode. I assume that either my equation isn't quite right, or the
official table has an accumulated error in it. This is bug 323 at
bugs.xiph.org.

- John Ripley.
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.