[Flac-dev] Non-audio applications

[Daniel O'Connor]

Hi,
I work for a company which makes meteor and wind radar
(http://www.gsoft.com.au).

On occasion (ie during meteor showers such as the Leonids) we configure
the system to save raw data as it comes out of the acquisition system,
the data rate for this varies (depends on acquisition parameters and
number of coherent integrations etc), but usually it is around
600kb/sec.

The data consists of 16 bit samples, one for each digital channel (in
phase and quad for each receiver) at each sampled height. Typically
there are 10 channels and 30 range gates, so for a single transmitter
pulse tick you get 10 * 30 = 300 16 bit samples.

In an effort to compress the data I looked at lossless audio compressors
(as well as gzip etc) and I ended up trying flac and shorten. 


Program       Compressed Size         Compression time        Decompression time
gzip           358 Mb (84.9%)          147.96 sec              21.28 sec
bzip2          330 Mb (78.2%)          589.01 sec             232.13 sec
shorten -c 10  326 Mb (77.4%)           91.39 sec              55.89 sec
shorten -c 300 313 Mb (74.3%)          102.08 sec              53.04 sec
flac 5chn      328 Mb (77.8%)          131.22 sec              66.54 sec
flac 8chn      332 Mb (78.7%)          139.57 sec              66.74 sec
flac 10chn     327 Mb (77.5%)          126.32 sec              n/a sec

Note the 10 channel version was done by me changing the format.h file
and recompiling, but alas I couldn't get decompression to go :(
The command line I used for flac was
/usr/bin/time flac --lax --endian=little --channels=5 --bps=16 --sample-rate=64320  --sign=signed --force-raw-format reallyraw -o reallyraw.flac-5chn

I needed the --lax otherwise it generated a FLAC__STREAM_ENCODER_NOT_STREAMABLE error.

The shorten command line was
/usr/bin/time shorten -c 10 -ts16lh reallyraw reallyraw.short.10

I am basically letting you guys know about a possibly more interesting
than normal application for your code and soliciting any tips you may
have :)

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 9A8C 569F 685A D928 5140  AE4B 319B 41F4 5D17 FDD5