[Icecast-dev] Crash in Icecast-2.3.1 (in source_recheck_mounts)

[Klaas Jan Wierenga]

Hi Karl,

I've not been able to duplicate it (yet). Assuming that brendan's 
snapshot is a daily build from icecast/trunk I've compared icecast_2_3_1 
and trunk code for src/source.c. The most likely change that fixes the 
problem is the introduction of avl_tree_rlock (and matching unlock) just 
below "Applying mount information for". Another lock that might be 
related is the avl_tree_wlock that is introduced in source_clear_source.

Diff command used:
svn diff http://svn.xiph.org/icecast/tags/icecast_2_3_1/src/source.c 
http://svn.xiph.org/icecast/trunk/icecast/src/source.c

I guess I'll have to patch my code with these changes to fix the 
problem. I'm reluctant to use daily snapshot in my production systems.

Will there be a 2_3_2 release any time soon? If you have any suggestions 
on how to best deal with these situations I would appreciate your advise.

KJ

Karl Heyes wrote:
> Klaas Jan Wierenga wrote:
>> Hi all,
>>
>> I'm experiencing a crash in Icecast 2.3.1 when a lot of sources (300) 
>> are disconnecting at more-or-less the same time.
>>
>> I run a test where I set Icecast configured with 300 mount points on 
>> one machine and attach 300 ezstream source clients from another 
>> machine serving a low-bitrate stream (24 kbit/sec). When I 'killall 
>> ezstream' to stop all sources at the same time Icecast crashes with 
>> the following stacktrace:
>>
>> #0  avl_get_by_key (tree=0x0, key=0xb6a01308, value_address=0xb6a01310)
>>    at ../../../src/avl/avl.c:319
>> #1  0x08060213 in httpp_getvar (parser=0x8cd2100, name=0x8064c56 
>> "ice-name")
>>    at ../../../src/httpp/httpp.c:518
>> #2  0x08052f9e in source_update_settings (config=0x8068bc0, 
>> source=0x8cd9d78,
>>    mountinfo=0x8cbd618) at ../../src/source.c:1021
>> #3  0x08053114 in source_recheck_mounts () at ../../src/source.c:1395
>> #4  0x0805408c in source_client_thread (arg=0x8dad7f0)
>>    at ../../src/source.c:1244
>> #5  0x0805fe27 in _start_routine (arg=0x8e18be8)
>>    at ../../../src/thread/thread.c:655
>> #6  0x00341bd4 in start_thread () from /lib/libpthread.so.0
>> #7  0x00d1e4fe in clone () from /lib/libc.so.6
>>
>> This is running on an AMD Opteron with FC4.
>>
>> It appears that there is some sort of race-condition or maybe 
>> source_recheck_mounts needs to acquire an additional lock that it is 
>> not currently acquiring.
>> Anyone experiencing the same problem or do you have any idea how I 
>> can solve this problem, I would very much appreciate the help.
>>
>> I'm working on a test-bench for Icecast with which I can test Icecast 
>> by generating configuration files for Icecast and ezstream and some 
>> scripts to start large numbers of sources and even larger numbers of 
>> listeners for each source. The test-bench found the problem mentioned 
>> above. If anyone is interested I can get you a copy of the test-bench.
>
> Can you duplicate this with the current svn trunk code and/or my 
> branch work?
>
> trunk snapshot on
>
> http://people.xiph.org/~brendan/snapshots/icecast/
>
> my branch work on
>
> http://karl.mediacast1.com/
>
>
> karl.
>
>
>