[tremor] Re: [PATCH] new mdct code, redundant tables eliminated

[Nicolas Pitre]

On Fri, 13 Sep 2002, timmy brolin wrote:

> I noticed the not-identical output from Nicolas last patch (the one that only took care about the negation).
> So I agree that that's likely the problem, but it must be examined more closeley.

That can be easily explained.

Suppose you have the following 64 bit values:	0xffffffffffffffff
Negating it would give you:			0x0000000000000001

Now if you truncate those values, preserving the top 32 bits, you obtain 
0xffffffff and 0x00000000 respectively.  Negating them at that again at that 
point will give you 0x00000001 and 0x00000000.  We are now up to a 
difference of 1.  Remember that we started with the same value but integer 
truncation combined with multiple negations produced this error.

Therefore, by removing extra value negations in the code, I might actually 
have improved the accuracy of the final output at some point.

> But, there is another problem with this latest patch... When the 8192 or
> 4096 tables are required it will probably fail.

It won't fail.  The output will just not be perfectly accurate.  See my 
previous post where I explained the whole issue.

<p>Nicolas

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'tremor-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.