[Flac-dev] [OT] primer on differences between floating point and integer audio formats

[Brian Willoughby]

Dan,

It's hard to say.  I'm sure there are many reasons for using float.   
Historically, a lot of research software used float because their  
software was producing and processing float anyway, so that's the  
format they used.  None of those tools were real-time anyway, so CPU  
efficiency was not the primary concern in choosing a format.  Much of  
this software is older than any of the programs you see today.  Newer  
software came about when real-time was possible, and then size became  
an issue, as well as the fact that early audio software ran on  
computers with poor float performance.  Now that floating point  
performance is on par with integer, software is using float again  
where its advantages can be realized.

As far as psychoacoustics is concerned, I've heard some criticisms of  
float.  Fixed point integer formats have a constant digital  
quantization noise floor, and a fixed step size per increase in  
sample value.  An interesting thing about integer is that the number  
of significant bits changes with amplitude.  Only a full-scale sample  
has all 24 bits of precision because there are never any fractional  
amount allowed.  6 dB down is 23 bits, 12 dB down is 22 bits, etc.

The weird thing about float is that all samples have 24 bits of  
precision.  A sample at -12 dBFS has 22 bits of precision with the 24- 
bit integer range, but then 2 more bits of fractional precision.  As  
audio samples get closer to zero, they can take smaller and smaller  
steps, although you'll never be able to hear these small steps with  
even the latest DACs.  In the opposite direction, exceeding the  
standard +/- 1.0 value in floating point samples means that sample  
jump 2 codes at a time, exceeding +/- 2.0 jumps 4 codes at a time.   
Because of this, the quantization noise level floats up and down  
depending upon the signal level.  Not the average signal level, but  
the instantaneous sample value.  The critics of floating point are  
complaining about this.  However, it probably isn't an issue if the  
float values are properly dithered when converted to 24-bit integer  
values, and also if audio exceeding +/- 1.0 is brought down in level  
before the conversion.  If anyone has links to psychoacoustic  
research regarding floating point audio, I would be very interested  
in learning more.

I think the biggest reason for float usage today is to avoid clipping  
in intermediate processing.  Pro Tools plugins almost all have  
clipping built-in to clamp samples to +/- 1.0, making it crucial that  
you pay attention to gain staging in Pro Tools session, but float  
does not strictly require this.

I'm not trying to lay down a definite comparison of floating point  
vs. integer.  I just don't know of any primer on the subject.   
Personally, I'm quite interested to know if there are any floating  
point DACs, and also whether psychoacoustic research can been done on  
the format differences.  In the early days of CD Players, some  
manufacturers used a 14-bit DAC with shifting precision, such that  
loud samples were converted with 14-bit precision (hoping that the  
quantization noise would be masked by the loud passages), signals at  
-6 dB were converted with 15-bit precision, and signals at -12 dB  
were converted with effective 16-bit precision.  These DACs had to  
switch precision without creating clicks in the audio.  It seems like  
similar technology could be used to create a floating point DAC, but  
it's really hard to create an analog circuit which maintains the 144  
dB dynamic range of 24-bit audio, and I imagine it would be almost  
fruitless to attempt to exceed that dynamic range.

Brian Willoughby
Sound Consulting


On Mar 29, 2007, at 13:26, Dan Pritts wrote:

> On Thu, Mar 29, 2007 at 01:15:12PM -0700, Brian Willoughby wrote:
>> that it supports.  I'm sorry that I confused things in my earlier
>> message by pointing out that you can convert 32-bit float to a format
>> that FLAC supports or that you can write your own encoder/decoder for
>> 32-bit integer FLAC.  You really need to understand floating point
>> numbers and what kind of audio data you have before trying to analyze
>> FLAC this way.
>
> can you suggest a good primer on the differences between floating
> point and integer audio formats?   I see from googling that perhaps
> the motivation for using floating-point formats is that modern x86
> CPUs are better at floats than ints, is there more to it than that?
>
> danno
> --
> dan pritts
> danno at umich.edu