[vorbis] MDCT input and output data blocks

[Govind Kharbanda]

Hi,

I'm trying to determine how the frequency data that is passed to the
MDCT, and output time domain data obtained from the MDCT, is changed
when setting #define MDCT_INTEGERIZED

For instance, the trig array, for a size 256 window begins
1.000000
0.000000
0.998795
-0.049068
0.995185
-0.098017

if INTEGERIZED is set, then it begins
16384
0
16364
-803
16305
-1605

which is simply a left shift of TRIGBITS (14 in this case), or a
multiply by 16384 if you prefer to think of it like that.  So far, so
good.

However if I display the data blocks (*in and *out) before and after
transformation:

mapping0.c:    mdct_backward(b->transform[vb->W][0],pcm,pcm);

in floats:

'in' begins (before transformation)
-0.003689
-0.017678
-0.041495
-0.044482
-0.011444  (although it goes a bit odd later on, with long strings of
zeros and then long strings of -431602080.000000)

'out' begins (after transformation)
0.060559
0.046839
0.023448
-0.003603
-0.032965

But in fixed point, if I set INTEGERIZED I get

'in' begins (before transformation)
-1150172552
-1131359692
-1121323368
-1120521410
-1136950943

which clearly isn't the floating point version shifted left by TRIGBITS

and for completeness, 'out' begins (after transformation)
-23137
119197
-124861
43611
42769

which looks better, although note the signs don't agree with the float
version.

Should I perform a left shift of the data in mapping0.c BEFORE it calls
mdct_backward?  (Is that not taken care of).

Source file was keoki_passiton.ogg obtained from the Vorbis.com web site
some time ago (I don't think it is there any more).  I'm using Version
1.0 Win 32 distribution under Visual C++.  The blocks shown above are
the first audio block in the file (size 256).

Cheers
Gov


-- 
Govind S Kharbanda                Institute for System Level Integration
Hardware Perceptual Audio Decoding     Alba Campus, Livingston, EH54 7EG
http://www.sli-institute.ac.uk/~gk/                    Tel: 01506 469339
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.