[tremor] Re: [PATCH] new mdct code, redundant tables eliminated

[timmy brolin]

I noticed the not-identical output from Nicolas last patch (the one that only took care about the negation).
So I agree that that's likely the problem, but it must be examined more closeley.
But, there is another problem with this latest patch... When the 8192 or 4096 tables are required it will probably fail.
Examine this code snipet:
***************
/* rotate + window */
step>>=2;
{
DATA_TYPE *oX1=out+n2+n4;
DATA_TYPE *oX2=out+n2+n4;
DATA_TYPE *iX =out;
T =sin_lookup-(step>>1);
V =sin_lookup+2048+(step>>1);

do{
oX1-=4;

T += step;
V -= step;
oX1[3] = MULT31 (iX[0] , *T) - MULT31(iX[1] , *V);
oX2[0] =-(MULT31 (iX[0] , *V) + MULT31(iX[1] , *T));
*********
When table size 8192 is required step will be initialized to 2, at the second line in the above code snippet step is shifted right two steps. 2>>2==0 Not a very good thing when you read the rest of the above code. :)
Even the 4096 table size will fail. Step is initialized to 4 when the table is 4096. First, step will be shifted right two steps, leaving only 1.
Then, check this row:
T =sin_lookup-(step>>1);
When the required tablesize i 4096, the code will not fail, but it will return the wrong sine values (They will be 0.5*PI/4096 degrees off).

I was working on the very same patch when Nicolas posted this, I have gotten rid of the second tables, but I have not found a solution to the last tables due to the above problem.
In order to use the table quarter_sin to replace the last tables, the resolution of quarter_sin must be increased by four, and that would result in a increase of table sizes, instead of a decrease.

Nicolas, please check that your code produce bit-identical PCM outputs before posting it.. If it doesn't produce identical outputs and you can't find the problem, then please note that in your post.

/
Timmy Brolin

-----Original Message-----
From: Monty <xiphmont at xiph.org>
To: Nicolas Pitre <nico at cam.org>
Date: Fri, 13 Sep 2002 11:35:45 -0400
Subject: Re: [tremor] Re: [PATCH] new mdct code, redundant tables eliminated

Patch is on mainline.

It is not bit-identical decode to the original, but upon review, the
negation shuffling would cause that.  This appears to be simple
integer rounding behavior.

Monty
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'tremor-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.

<p><p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'tremor-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.