[flac-dev] [PATCH] stream_encoder : Improve selection of residual accumulator width

[Miroslav Lichvar]

On Fri, Jun 20, 2014 at 01:21:03PM +0400, lvqcl wrote:
> Miroslav Lichvar цкщеу:
> 
> > +/*
> > + * This is used to avoid overflow with unusual signals in 32-bit
> > + * accumulator in the *precompute_partition_info_sums_* functions.
> > + */
> > +#define FLAC__MAX_EXTRA_RESIDUAL_BPS 4
> 
> > +		/* WATCHOUT: "+ bps + FLAC__MAX_EXTRA_RESIDUAL_BPS" is the maximum
> > +		 * assumed size of the average residual magnitude */
> > +		if(FLAC__bitmath_ilog2(default_partition_samples) + bps + FLAC__MAX_EXTRA_RESIDUAL_BPS < 32) {
> 
>  From FLAC__fixed_compute_residual:
>      residual[i] = data[i] - 4*data[i-1] + 6*data[i-2] - 4*data[i-3] + data[i-4];
> 
> so max(residual[i]) == 16 * max(data[j]), or:  max_bps(residual[]) == 4 + max_bps(data[]).
> 
> Am I right that it's the reason why FLAC__MAX_EXTRA_RESIDUAL_BPS is equal to 4?

Not really, it's just a guess. With LPC the maximum possible residual
could be much larger than with the fixed predictor if the coefficients
were chosen randomly, but the autocorrelation routine should keep them
more reasonable. The snippet6.wav file needed 2, so I made it slightly
larger to have some extra room.

As overflow in the accumulator won't result in a data loss, I think
this is good enough until someone can figure out a better approach.

-- 
Miroslav Lichvar