[flac-dev] 64-bit residuals

[Martijn van Beurden]

Op vr 1 apr. 2022 om 09:19 schreef Martijn van Beurden <mvanb1 at gmail.com>:
> I'll start working on a patch to include stereo decorrelation and
> handling fixed subframes for 32-bit FLAC in both decoder and encoder
> to do some more testing. It seems that those two changes are worth the
> extra encoding and decoding complexity, while adding handling for >
> 32-bit residuals (which need new bitreader and bitwriter functions)
> does not.

A still rough (but fully functional) patch to libFLAC to enable 32-bit
decoding and encoding is here:
https://github.com/ktmf01/flac/tree/32bit-encoder-decoder

This implements encoding and decoding to and from all 4 subframe
types, with stereo decorrelation (i.e. 33 bps subframes). The decoder
rejects residuals exceeding 32-bit int (still work in progress) and
the encoder takes care it doesn't produce residuals exceeding 32-bit
int.

I compared encodes with and without stereo decorrelation, and with
even lower residual limits. Comparison was done with the same corpus
as mentioned earlier in this thread. Results are as follows:

With stereo decorrelation, residual limited to 32-bit int: 1161MB
(0.01% verbatim)
Without stereo decorrelation, residual limited to 32-bit int: 1180MB
(0.04% verbatim)
With stereo decorrelation, residual limited to "31-bit int": 1162MB
(0.6% verbatim)
Without stereo decorrelation, residual limited to "31-bit int": 1183MB
(1.7% verbatim)
With stereo decorrelation, residual limited to "30-bit int": 1172MB
(6.9% verbatim)
Without stereo decorrelation, residual limited to "30-bit int": 1197MB
(10% verbatim)

Extrapolating from these results, it seems unlikely that increasing
the residual limit beyond a 32-bit int would bring significant
compression improvement for music sources. Enabling stereo
decorrelation does bring quite some benefit, but at the cost of
increased encoder and decoder complexity (see patch for details).

Kind regards, Martijn van Beurden