[flac-dev] exhaustive-model-search issue results in multi-gigabyte FLAC file

[lvqcl]

Erik de Castro Lopo wrote:

> I have committed an improvement on the above fix.
>
>     https://git.xiph.org/?p=flac.git;a=commit;h=f34f31dac0032887887b5bbcb0944de055b757d0
>
> that reverts to the use of a FLAC_uint32 accumulator for files of less
> than 24 bits per sample.
>
> I still have no proof that this overflow cannot occur for 16 bit files.
>
> Erik


If the least significant bit in all samples of a 24-bit WAV file is set to 0, the encoder sets 'bps' variable to 23 and the description of this patch -

"This fix [...] restores the use of a FLAC_uint32 accumulator for 16 (and less) bit files"

- is not correct: this fix restores the use of a FLAC_uint32 accumulator for 23 (and less) bit files. I slightly modified snippet6.wav and the current version hangs on it when I add -b 8192 --lax options.


The code to enable 32-bit accumulator only for 16 (and less) bit files should be:

"if(bps <= 16 && FLAC__bitmath_ilog2(default_partition_samples) + bps < 32)"

...and it seems that default_partition_samples must be less or equal to 65535; this means that FLAC__bitmath_ilog2(default_partition_samples) is less or equal to 15 and the code above is equivalent to:

"if(bps <= 16)"