[Icecast] Burst and queue-size vs slow_listeners tally

[Philipp Schafft]

Good morning,

On Fri, 2024-02-02 at 10:04 +0000, Marek Dziembowski wrote:
> Greetings ice-casters!
>  
> I’m hoping to get some clarity and guidance on understanding the
> relation between queue-size and burst-size settings, and how this
> affects the slow_listener count shown on the admin pages.

it's not that complicated, so let's dig in a little:
Icecast has a queue for the data. As soon as a source pushes new data
that is added to this queue and if the queue then overshoot the queue
size it's tail is cut to fit again. So the queue-size just gives the
size of that data queue.

Once a listener is connected they "swim" in this queue. Ideally they
are right at the head of it getting the data as it comes in.

However as clients have a internal buffer placing them right at the
head of the queue would mean that they would first take take to fill
their buffer.

the burst size is how far behind the head of the queue we put them.
This means this is the amount of data that is instantly available to
them to pre-fill their buffer.

What does that mean:
 * The burst size is always smaller or equal to the queue size as it is
   just part of it.
 * The burst size should match the listener's pre-buffer setting. It
   doesn't have any effects on connection quality or anything else.

Practical considerations:
 * It is safe for the burst size to be too small (even 0 is valid). It
   would just increase the listeners pre-buffer time.
 * If the value is too big you over-fill the clients buffer to the
   point it will add extra latency (because the listener first needs to
   play out some of it).
 * Listener clients don't tell Icecast what they want. So this value is
   always a guess.
 * I would suggest to have it set to a few seconds worth at your
   average bitrate. The current default is 64kByte which is about 4.7s
   on 112kBit/s. If your bitrate is very different just scale it
   accordingly.
 * Lower is always "safe" so better go with a too low than with a too
   big value.
 * It is totally fine to play with this and see how long it takes for a
   client to start playing. But that is the only goal here. Not
   connected to the rest of your e-mail. (So an easy one!)


> We are having constant issues with listeners eventually being
> disconnected at varying timespans, and I’ve tried adjusting burst-
> size and queue-size to find the right balance for avoiding long
> duration disconnects, but have had no joy yet.

Here the queue-size is important.
As said above the listener "swims" in the queue. It is placed there and
tries to keep up with the flow (from head to tail). The queue allows
the listener to swim a bit faster or a bit slower as long as on average
keeps up with the flow. This compensates for things like network
jitters and player scheduling.

A bigger queue size means that a listener is less likely to fall behind
and hit the tail end. However in reality too high of a value here just
means that you hide the cases of clients that actually died.

So similar as above: consider how far a client may fall behind "live"
(our head) and then make a number up. for example if we use a burst
size of 4 seconds the queue might be 8 or 12 seconds long.

But I wouldn't go with much more as a listener that can for example not
read from Icecast in 12 seconds likely will not sustain a stable stream
even if you give it more time. 


> What exactly does the slow_listener count in relation to queue -size
> indicate? I’ve seen the slow_listeners number grow well beyond the
> number of connected listeners:

When a listener hist the tail it is considered slow and removed.
The infamous "Client [...] has fallen too far behind, removing" error.

This just means that they could not keep up.

So the value you see is the counter of how often that happened. It is
the sum and expected to only grow. The speed of it growing is what is
interesting not the value.

And even that value needs to be put in context: e.g. when you have many
mobile listeners the value is expected to grow faster. When you have
listeners all in the same building as your Icecast server you should
hardly see the number grow.

So if at all you compare that number directly to something compare it
to your total amount of listeners, not the current value. :)

 
> Right now I am seeing:
> listeners
> 508
> queue_size
> 2191778
> slow_listeners
> 1501

If you're having 508 listeners constantly and your server is running
for some time 1501 slow listeners wouldn't be too much of an alarm to
me. :)


> How do I use this data to find the best queue-size and burst-size
> settings to keep listeners connected without dropout?

I hope my hints help you a bit. :)


With best regards,

PS: I just realised that a fish and swim example is most appropriate
for a Xiph.Org Foundation project.

-- 
Philipp Schafft (CEO/Geschäftsführer) Telephone:           +49.3535 490
17 92 Website:             https://www.loewenfelsen.net/ Follow
us:           https://www.linkedin.com/company/loewenfelsen/
Geschäftsführer/CEO: Philipp Schafft Löwenfelsen UG
(haftungsbeschränkt)     Registration number: Bickinger Straße
21                     HRB 12308 CB 04916 Herzberg
(Elster)                 VATIN/USt-ID:
Germany                                 DE305133015