[flac-dev] [PATCH] stream_encoder : Improve selection of residual accumulator width

[Miroslav Lichvar]

On Thu, Jun 19, 2014 at 05:04:23PM +0200, Miroslav Lichvar wrote:
> Yes, it's the same check. Assuming residual can be at most
> FLAC__MAX_EXTRA_RESIDUAL_BPS bits wider than subframe_bps, I think it
> should be:
> 
> if(subframe_bps + qlp_coeff_precision + FLAC__bitmath_ilog2(order) + FLAC__MAX_EXTRA_RESIDUAL_BPS - 1 <= 32)
> 	if(subframe_bps + FLAC__MAX_EXTRA_RESIDUAL_BPS <= 16 && qlp_coeff_precision <= 16)

On second thought, this is probably not necessary. The residual seems
to be always treated as 32-bit in the LPC encoding and decoding
functions, even in the MMX assembly if I'm reading it correctly. It
doesn't matter if it's wider than the original signal.

-- 
Miroslav Lichvar