[Icecast] help with "too many clients awaiting authentication"

[subscription at nextdial.com.br]

Hello guys,

Today we faced an unique problem.

We have about 120 stations and we are reaching 40k simultaneous listeners monday/friday (~ 11 AM GMT -3).

We also use url auth to log, audit and build audience reports.

But today, at the end of the day we were about 10k simultaneous and listeners start get errors (403, busy, please try again later), and after digging, delete logs and restart icecast we found this:

[2020-11-15  22:09:38] INFO auth/queue_auth_client auth on (default mount) has 101 pending
[2020-11-15  22:09:38] WARN auth/auth_add_listener too many clients awaiting authentication

Before those errors our auth api was logging ~60 rps in ~150ms (during the error the api was not getting requests).

Has anyone else has already faced this problem? Any tips?

Best,
Thiago
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xiph.org/pipermail/icecast/attachments/20201115/0311f293/attachment.htm>