[vorbis-dev] Re: Possible fix for bug #14 (ogg123 -b memory leak)

[Kenneth Arnold]

On Mon, May 14, 2001 at 09:04:48AM -0400, Jeff Squyres wrote:
> IMHO, threads make certain types of tasks easier.  Of course, threads
> introduce their own difficulties -- there's no such thing as a free lunch,
> after all.  But these are typically application-dependant issues.  So
> you'd need to compare in terms of what ogg123 specifically does (I'm not
> really familiar with the ogg123 internals).

The part of ogg123 in question is just a buffer; the only reason
processes are split in this case is to allow for optimal non-blocking
I/O; it may be possible to implement a buffer in a single process but
all of my efforts with non-blocking I/O would skip on any hint of
system load, just because the individual Vorbisfile library calls take
too long and the overhead of watching a file descriptor while doing
heavy math work. That was before libao also; that extra library would
make a single-process method all that much harder. So I just decided a
reader/writer process pair would be the most efficient way to go,
with a single shared memory segment to share the buffer between the
two processes.

> Here's some questions to ask:
> 
> - What happens now if [one of] the writer child process[es] die?  Do they
>   all die?  Is this acceptable?  (because if one thread dies, they will
>   all die)

Especially if I can get threads (with all memory shared) to update the
appropriate data structures more easily, the writer process can keep
itself running indefinately, and if it dies there should be no reason
for any of the rest of ogg123 to be running anyway.

> - What IPC is ogg123 doing, exactly?  e.g.:
>   - Can it easily be mapped to thread-to-thread queues?

Possibly, but since we know more about the data than simply that it
needs to get passed along, a circular buffer works better. Maybe a
smart queue implentation would work, but it would come out looking
like a buffer anyway.

>   - Will you cut down on overhead by eliminating memory copies by passing
>     pointers instead of copying in/out of the shared memory space?  Or
>     is the data so small that the extra memory copies don't matter?

My current method doesn't even pass pointers; the pointers are all
stored in the shared memory area, since there really are only two of
them that are significant and each process owns one.

>   - Can you take advantage of mutexes/semaphores/condition variables to
>     utiulize blocking instead of spinning?  Do you want to?  (there is
>     more initial overhead with these, but if you're going to wait for a
>     long time, it costs less than spinning, particularly if the machine
>     is not otherwise idle)

I am considering using a semaphore on the chunks, but ideally we can
calculate an estimated time delay to sleep (and wake up on signal or
some other method) that will time the usage better than a semaphore
would.

> - Does the parent process have to manage its children processes?  i.e.,
>   waitpid() and the like?  (that *may* be able to disappear, or it may
>   just get replaced with the main thread doing essentially the same thing)

See above.

> - Would ogg123 benefit from thread scheduling rather than process
>   scheduling? (excluding Linux, of course)

Not really; the only reason we're using threads is for I/O purposes.

> - Are pthreads portable to mac? (or has the community decided not to
>   support anything < OS X, which does have pthreads?  I haven't been
>   following if any such decisions were made...)
> - Does your debugger support threads?  You're already dealing with
>   debugging in parallel (which can be painful); moving them to threads
>   potentially allows the ability to use a single debugger (although gdb's
>   support for threads is kinda crappy).

Yes I've noticed about gdb, but I've been able to work around it
(processes even hairier than threads...)

> Of course, you can take threads out to the Nth degree and revamp the
> application to fully utilize mulitple points of execution within a single
> program (for example, I'm a big fan of hiding latency with threads;
> whenever you need to do something that will block [such as fetching I/O],
> just let some sub-thread do it and let the rest of the main program
> continue -- or even have a thread pre-emtively do it so that the main
> program never has to wait), but I'm guessing that that's not the goal
> here.

I considered having an input thread also, but realized that data in
gets processed almost immediately and input rate is not of real
concern (kernel buffers network input anyway, and disk input is
similarly cached). Though having one output thread for each output
device is a possibility I want to look into (might be cleaner than
maintaining my linked devices list between the threads).

Thanks for input.


-- 
Kenneth Arnold <ken at arnoldnet.net> / kcarnold / Linux user #180115
http://arnoldnet.net/~kcarnold/



<HR NOSHADE>
<UL>
<LI>application/pgp-signature attachment: stored
</UL>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: part
Type: application/octet-stream
Size: 233 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/vorbis-dev/attachments/20010514/6db9f582/part-0001.obj