[xiph-commits] r17543 - trunk/Tremor

[xiphmont at svn.xiph.org]

Author: xiphmont
Date: 2010-10-18 02:52:45 -0700 (Mon, 18 Oct 2010)
New Revision: 17543

Modified:
   trunk/Tremor/floor1.c
Log:
If fuzzing swaps in a codebook that allows values outside the circular
range of the piecewise representation, it can overflow the lookup.
Proper fix here is just a simple clamp.




Modified: trunk/Tremor/floor1.c
===================================================================
--- trunk/Tremor/floor1.c	2010-10-18 09:52:01 UTC (rev 17542)
+++ trunk/Tremor/floor1.c	2010-10-18 09:52:45 UTC (rev 17543)
@@ -394,7 +394,7 @@
 	  }
 	}
 
-	fit_value[i]=val+predicted;
+	fit_value[i]=(val+predicted)&0x7fff;;
 	fit_value[look->loneighbor[i-2]]&=0x7fff;
 	fit_value[look->hineighbor[i-2]]&=0x7fff;
 
@@ -425,14 +425,20 @@
     int hx=0;
     int lx=0;
     int ly=fit_value[0]*info->mult;
+    /* guard lookup against out-of-rage values */
+    ly=(ly<0?0:ly>255?255:ly);
+
     for(j=1;j<look->posts;j++){
       int current=look->forward_index[j];
       int hy=fit_value[current]&0x7fff;
       if(hy==fit_value[current]){
 	
+	hx=info->postlist[current];
 	hy*=info->mult;
-	hx=info->postlist[current];
-	
+        /* guard lookup against out-of-rage values */
+        hy=(hy<0?0:hy>255?255:hy);
+
+
 	render_line(n,lx,hx,ly,hy,out);
 	
 	lx=hx;