[CELT-dev] get_required_bits32() and alloca() -> corrupted stack

[Timothy B. Terriberry]

Bjoern Rasmussen wrote:
> HiI'm trying to run CELT on Win32 but I'm running into some corrupted stack errors when using
> alloca(). When get_required_bits32() (in file cwrs.c:308) is called
from get_required_bits()
> (in file cwrs.c:328) the 'K' parameter is 1 which means that the
uint32 which is allocated on
> the stack only has 3 bytes instead of 4. This results in a corrupted
stack in the following
> call to log2_frac().Shouldn't the line
cwrs.c:313:ALLOC(u,K+2,celt_uint32_t);Instead
> say:ALLOC(u,max(K+2,sizeof(celt_uint32_t)),celt_uint32_t);?-- Bjoern

If you are really using alloca, then ALLOC should be #define'd to
(stack_alloc.h:99):

var = ((type*)alloca(sizeof(type)*(size)))

So ALLOC(u,K+2,celt_uint32_t) should allocate 12 bytes when K=1. The
actual cause appears to be a micro-optimization violating one of our
assumptions in unext32() when called from ncwrs_u32 (cwrs.c:220). Try
the following patch and let me know if it fixes your problem:

diff --git a/libcelt/cwrs.c b/libcelt/cwrs.c
index f44fca1..aa2a732 100644
--- a/libcelt/cwrs.c
+++ b/libcelt/cwrs.c
@@ -217,7 +217,7 @@ celt_uint32_t ncwrs_u32(int _n,int _m,celt_uint32_t
*_u){
     k=2;
     do _u[k]=(k<<1)-1;
     while(++k<len);
-    for(k=2;k<_n;k++)unext32(_u+2,_m,(k<<1)+1);
+    for(k=2;k<_n;k++)unext32(_u+1,_m+1,1);
   }
   else{
     celt_uint32_t um1;