[Flac-dev] Re: multiple core support

[Harry Sack]

2007/9/7, Brian Willoughby <brianw at sounds.wa.com>:
>
> I really should have just said that it will require some testing to
> make sure the FLAC API can handle writing the same file from multiple
> threads.  It may not turn out to be complicated at all.
>
> The FLAC decoder has its own code for writing PCM files already.
> Tweaking this to support multi-threaded writes would not be too
> difficult.  It's simply a matter of building the PCM file, setting
> aside the data area for the PCM samples, and then providing each
> thread with information needed to write at different offsets.  FLAC
> never needs to "check for valid PCM streams" since it is creating the
> PCM stream from scratch every time.



But it's not 100% lossless if you don't check it when written using
different threads. Now there is only 1 thread that writes at the back of the
file in a lineair way. So the chance there is an error when decoding is
almost non existing because it just write the new block at the end of the
file till it's finished.

If you write the file in parts at different locations inside the pcm file on
even different cpu's, there are certainly things that can go wrong. And you
say there is no need to check those resulting pcm streams ... So this makes
no sense: what if there was something wrong with 1 thread on 1 cpu (maybe a
synchronisation problem, ...)? Then there would be missing pcm blocks or
even corrupted ones (because they are written at different locations in the
file while running different threads on differents cpu's). So making
decoding to support multiple cores is not easy as it sounds. There really
has to be a 'checking' phase at the end to verify if the pcm stream was
build up correctly using the different threads. If you don't check here, you
risk getting non-lossless stream because of missing/corrupted blocks and
this is exactly what a lossless codec doesn't want!

At the encoding end there you can automatically check the resulting flac
file using the verify option. So after running the multiple threads on the
pcm stream that results in the flac file, you just check that resulting flac
file using verify to see if the md5 checksum matches. So if something went
wrong in some thread and there were missing blocks/corrupted ones, you will
get a different md5 and you know exactly something went wrong and you just
repeat the encoding process till everything is correct. At the decoding end
you can't see this when using multiple threads because there is no way to
check if the pcm stream was built up correctly (at least: as far as I know)



Recording always involves a PCM data buffer.  You're correct that
> there might be a 2 CPU split, automatically, depending upon the
> system.  However, I was simply stating that going to an arbitrary
> number of threads would not be possible when recording, since you're
> only getting one block at a time, and the block is where the division
> between CPUs would be made.



yes of course: but recording has nothing do to with the encoder itself. The
recording part happens in other software. You just get raw pcm data to
encode, just like when encoding an existing file. There is really no
difference because you can let the recording software say write x blocks of
pcm data and you just encode these using multiple threads to flac. So it's
perfectly possible to do it here too, but it's not really needed. It's only
useful for encoding existing files.

Harry

Brian W.
>
>
> On Sep 6, 2007, at 17:24, Harry Sack wrote:
>
> it's really not complicated I think: only api changes to write on any
> position in the file if that's not possible already with existing
> function.
>
> I'm not sure if decoding can have multi-core support: you need an api
> for writing pcm files in different parts then and this is maybe more
> difficult to check if it's valid pcm data since the decoder can only
> check for valid flac streams , not pcm streams i think.
>
> you can make recording also support multi cores: just let cpu 1 fill a
> pcm data buffer while cpu 2 encodes it. In this case you have a more
> reliable encoding with less cpu usage of your system (it's spread
> among 2 cpu's)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.xiph.org/pipermail/flac-dev/attachments/20070907/2ee26a59/attachment-0001.html