[Icecast-dev] listen backlog patch

[Stephan Leemburg]

Hi Thomas,

I don't know if you like top or bottom quoting. That seems to be a
big-little endian thing ;-)

So, I will top quote and inline quote.

Please see my comments inline, below.

Kind regards,
Stephan


On 02/19/2015 04:18 PM, "Thomas B. Rücker" wrote:
> Hi,
> 
> On 02/19/2015 03:07 PM, Stephan Leemburg wrote:
>> Hello Icecast-dev,
>>
>> I am new to this list.
> 
> Welcome!

Thank you.

> 
>> I am working for the NPO, the Dutch Public Broadcasting agency.
>> We do a lot of icecast streaming. We run at least 20 icecast server
>> instances on our media streaming cluster.
> 
> That's very nice to hear.
> 
>> We ran into an issue that clients which where connecting to our streams
>> seemed to be 'hanging' on the connection setup frequently. The client
>> 'thinks' it is connected, but no data.
>>
>> People suggested that it probably had to do with the poll() call. So, I
>> looked into that.
>>
>> I found that the issue was actually caused by the very low listen
>> backlog (5).
>>
>> On our clusters, we typically set this to 8192. Yes it is high, but we
>> do a _lot_ of streaming and host very high volume websites.
> 
> I'm not very familiar with socket programming, so will let Philipp
> comment on this. Interesting enough this issue hasn't come up so far as
> far as I can tell and there are some pretty high load deployments out there.

We often have 'bursts' of new connections. Due to something said on a
website, radio, tv, some top-2000 end of year event, etc. And when we
get a lot of simultanious connection requests, this becomes an issue.

I wrote a small server to simulate it. And the linux backlog acts a
little bit different then expected.

> 
> If you can share that info, what sort of concurrent listener load are we
> talking about?

Sure I can share, we are a national broadcasting agency funded by tax
money. So no secrets here ;-)

I just asked the media streaming guys.. 70k icecast connections on a
regular day and 150k around special broadcasts (like top 2000 around new
year).

> 
> 
>> Currently we are using icecast 2.3. We are migrating to 2.4.
>> So, I have written patches for 2.3 and 2.4, but also for the current 2.5
>> git tree.
>>
>> Unfortunately, I am a newby when it comes to git (sorry). But I do have
>> unified diff patch files for the 2.3, 2.4 and 2.5 source trees.
>>
>> The patched 2.4 icecast was tested by our media streaming team and they
>> confirmed that their issue was solved by it.
>>
>> Can I submit them (and how)?
> 
> Just send them as attachments to this list, or open a ticket over at
> https://trac.xiph.org.
> If trac is naughty and thinks you're a spammer, please let me know.
> 
> If there are differences between them, then please the 2.5 and 2.4
> patches. Otherwise 2.5 will do just fine.
> 

I have attached the 2.4 and 2.5 patches.

> 
> Thanks a lot for taking the time to reach out to us!

Thank you for your Open Source contributions and efforts!
And as it is Open Source, we can fix problems ourself and share.

> 
> 
> Cheers
> 
> Thomas
> 
> _______________________________________________
> Icecast-dev mailing list
> Icecast-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast-dev
> 


-- 
Met vriendelijke groet,

Stephan Leemburg, internetbeheerder
NPO ICT Internet Services
Bart de Graaffweg 2, 1217 ZL Hilversum
servicedesk at omroep.nl, 035-6773555
-------------- next part --------------
A non-text attachment was scrubbed...
Name: icecast-2.4.1-somaxconn.patch
Type: text/x-patch
Size: 4507 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20150219/0a602615/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: icecast-2.5-somaxconn.patch
Type: text/x-patch
Size: 7524 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20150219/0a602615/attachment-0004.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: icecast-2.3.3-somaxconn.patch
Type: text/x-patch
Size: 4025 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20150219/0a602615/attachment-0005.bin