[Flac-dev] Re: multiple core support

[Harry Sack]

it's really not complicated I think: only api changes to write on any
position in the file if that's not possible already with existing
function.

I'm not sure if decoding can have multi-core support: you need an api
for writing pcm files in different parts then and this is maybe more
difficult to check if it's valid pcm data since the decoder can only
check for valid flac streams , not pcm streams i think.

you can make recording also support multi cores: just let cpu 1 fill a
pcm data buffer while cpu 2 encodes it. In this case you have a more
reliable encoding with less cpu usage of your system (it's spread
among 2 cpu's)


2007/9/7, Brian Willoughby <brianw at sounds.wa.com>:
> Hmm,
>
> You're actually correct, when you put it that way.  Because the audio
> blocks are coded independently, you could speed things by having the
> encoder (or decoder) do a little up-front processing on the metadata,
> then use the overall settings to divide the file into sections and
> run the FLAC process in parallel.  However, I think it is generally
> frowned upon in some circles to create one thread per processor,
> rather than making the division dependent upon some other factor that
> is actually independent of the number of processors.  Apart from that
> caveat, someone could certainly develop an encoder or decoder which
> uses the FLAC library API to implement a threaded encoder/decoder -
> well, assuming that the API can write to subsets of a file, instead
> of requiring that the entire file be written from start to finish -
> and also assuming that the API can run in multiple threads and write
> to the same file.
>
> By your method, decoding would be faster, too.  Not everybody decodes
> just to listen.  Some of us keep original masters in FLAC format for
> archival purposes, but need the whole thing decoded before we can mix
> and master.  Anything which speeds up decoding would help get work
> done faster.
>
> On the flip side, another thing to consider is that not all encoding
> is done with files that already exist.  If you are recording live to
> FLAC, you cannot multi-thread the encoder because you don't have any
> audio besides the current block.
>
> So, decoding for playback and encoding for recording are two examples
> which cannot be multi-threaded.  But file format conversion could be
> developed to take advantage of multiple processors.  It would require
> a lot of tweaks to the library, perhaps.
>
> Note: PC games are multi-threaded because they're already doing
> several different things at once.  Sometimes the complexity is
> actually reduced by dedicating a thread to each part of the game
> logic.  And precisely because there are so many different things
> going on, multi-threading can speed things up when they overlap in
> unpredictable ways.  Games would be harder to write in a single-
> threaded way.  Meanwhile, encoding/decoding FLAC is easier to write
> in a single-threaded design.
>
> Brian W.
>
>
> On Sep 6, 2007, at 16:42, Harry Sack wrote:
>
> yes, i totally agree but I'm talking about the ENcoder :)
> You can perfectly 'cut' the pcm stream 'in half' and let 1 cpu encode
> one part , the other 2nd part (or with 4 cpu's in 4 parts, ...),
> because it's just a stream one frame after the other,  as far as i
> know (correct me if i'm wrong).
> Many pc games synchronise multiple threads on 2 (even 4!) cores (much
> More complicated then reading frames, compressing, writing compressed
> frames like the encoder does), so i think making the encoder support 2
> cpu's is perfectly possible.
>
> of course adding multi-core support to the decoder is silly because
> it's not necessary (any single core cpu is fast enough to play 1 file
> and it would only slow it down)!
>
> 2007/9/7, Brian Willoughby <brianw at sounds.wa.com>:
> > Any software which supports multiple processors must be multi-
> > threaded.  The process of designing multi-threaded code adds
> > complexity to the software, so there must be a good reason to go
> > through all the trouble.  The procedure for decoding a single file
> > could only benefit marginally from being multi-threaded, because most
> > operations would need to wait until the previous operation was
> > completed.  One section of the decoder might be able to process
> > several blocks, but the overall result would still need to wait until
> > all blocks are completed.  Besides, managing dynamic buffers between
> > each step of the decoding process would actually require the decoder
> > to do more, meaning it would slow down to an extent.  There would
> > pretty much be no difference in the total time, whether single
> > processor or multiple processors.
> >
> > The best way to take advantage of multiple processors is to decode
> > multiple FLAC files at the same time.  This will take full advantage
> > of your system, provided that the disk bandwidth and memory bandwidth
> > can keep up.
> >
> > Brian W.
>
>