[icecast] Icecast2 memory requirements

[Michael Smith]

On Monday 10 March 2003 11:45, Ross Levis wrote:
> I mentioned the other day about a new company I'm involved with setting
> up 1000's of streams using Vorbis.  This appears to be the favoured
> format now so that is good news.  They are also investigating hosting
> the IceCast server(s) at their own premices rather than using a stream
> hosting provider.
>
> It was mentioned that 1000's of streams could likely be processed by
> IceCast2 on the same PC (2Ghz or more) and the bottleneck would be the
> Internet bandwidth required for the number of listeners.

Thousands of clients. Thousands of streams was something we said would require 
some work (but should be possible).

>
> I would like estimates of the amount of RAM required on the PC per
> stream and per listener.  The streams will all average 32kb/s.  Any
> ideas?

RAM requirements are low. I haven't done any real measurement, and memory 
usage hasn't been an optimisation goal so far - but I'd say worst case would 
on the order of 20 kB per client. Significantly less for the typical case.

>
> If we take an example of 1000 streams with 30 listeners each = 30000
> individual streams @ 32kb/s = 960Mb/s.  That doesn't sound like too
> much.  One T1 connection will handle that easily.  But will IceCast and
> one CPU handle that?

960 Mb/s is a... lot. That's more than gigabit ethernet will cope with (once 
you factor in all the protocol overheads), and a T1 is way underpowered for 
that.

Right now, icecast wouldn't handle this load anyway. 30000 listeners might be 
possibly on a high-end cpu (and there's some very low-hanging optimisation 
fruit if it didn't), but 1000 streams is less likely to be easy - whilst the 
design should scale up to hundreds of streams, there are some optimisations 
which would likely be required (again, easy ones - but they'd still be 
neccesary), scaling to 1000 would be difficult (at this load level, you'll be 
seriously stressing out most unixes - solaris might be ok, but I don't know, 
the win32 icecast2 port has some limits to total clients at the moment, linux 
would have serious trouble with that many threads (that's apparently fixed in 
2.5 kernels, but that's for the future...).

The optimisations needed for most of these things are not in themselves 
difficult - but for the most part they only manifest under extreme loads. 
This makes it difficult to test effectively (I have fairly low-end hardware 
for my development, and certainly nothing exciting for network connectivity).

Mike

<p><p>--- >8 ----
List archives:  http://www.xiph.org/archives/
icecast project homepage: http://www.icecast.org/
To unsubscribe from this list, send a message to 'icecast-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.