[Flac-dev] Flac-dev Digest, Vol 45, Issue 2

[Frederick Akalin]

On Tue, May 6, 2008 at 4:52 PM, Christopher Peplin
<chris.peplin at rhubarbtech.com> wrote:
> Along the same line as Frederick, myself and another university student were able to implement a multi threaded FLAC
>  encoder, but using Intel's Threading Building Blocks (TBB) package. We saw similar near-linear speedup.

Great!  Your approach is better in that it bounds the memory usage of
the encoding, which is nice, and it also is amenable to streaming; a
similar pipelined approach would probably be best for the production
encoder.  Using mmap() for the input file like I did might simplify
your first pipeline stage and reduce the number of modifications you
need to make to libFLAC (as well as improve I/O performance) at the
cost of having to write callbacks yourself.  Using aio for the output,
too, might improve performance but it might not be worth the extra
trickiness.

>
>  I agree with Frederick in that the existing encoding API was obviously designed for serial execution, and any internal
>  threading support would be messy. We had to come up with quite a few tricks to modify as little of libFLAC as possible.

It looks like implementing parallel-friendly APIs is indeed the
logical next step.

-- 
Frederick Akalin
http://www.akalin.cx