[vorbis-dev] More on parallelism

[Jeff Squyres]

Many of you probably remember the flurry of posts that I created a few
weeks ago about writing a parallel vorbis encoder.  Here's what I have
been doing since...

Someone (I can't remember who offhand -- might have been Greg) mentioned
using threads instead of MPI for parallelism on the argument that more
people have SMPs than who have (and know how to use) an MPI
implementation.

Good point.

It was fairly striaghtforward to write a multi-threaded encoder, but I
found out the hard way that libvorbis.a is not thread safe.  :-( Monty
tells me that it is on the to-do list, but not currently at the top.  
Hence, someday a plain-vanilla multi-threaded encoder will be possible.  
But not today.

Bummer.

But it got me thinking -- what about mixing MPI and threads?  For those of
you who don't know anything about MPI, this is a relatively unexplored
area, mainly because of the lack of thread-safe MPI implementations
(including mine.  Oops!).  

But the more that I thought about it, the more interesting it got, because
there's actually a lot of deliciously intracate real-world issues
involved.  So in between real work (i.e., while waiting for long compiles
and the like), I devoted a fair number of cycles to thinking about this.  
On and off, I've spent the past few weeks writing down many of my thoughts
on the topic in a white paper, complete with oodles of pseudocode
(somewhat of an oddity, I know, but IMHO, a code snipit = a thousand
words).

It turned into a generalized white paper (i.e., not specific to vorbis)
because of the parallel issues that are raised.  However, it *exactly*
fits the model of the vorbis library calls (imagine that :-), mainly
because vorbis exactly fits a convenient generalized manager/worker model.

So to anyone on this list who is interested, I'd appreciate any feedback,
comments, suggestions, corrections, etc.  Here's the URL where to get the
paper:

        http://www.mpi.nd.edu/~jsquyres/vorbis/

Thanks.

{+} Jeff Squyres
{+} squyres at cse.nd.edu
{+} Perpetual Obsessive Notre Dame Student Craving Utter Madness
{+} "I came to ND for 4 years and ended up staying for a decade"

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.