<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"><div>Hello,</div>
<div> </div>
<div>In order to identify if was some issue regard to the auth api we include some trace logs and we move it to other machine.</div>
<div><br />
Bellow is what trace we are generating (date, result, elapsed time in milleseconds, request):</div>
<div>
<div>
<div style="margin-left: 40px;">[8/29/18 11:49:00 AM] 200 0.3721 action=listener_add&server=r22&port=443&client=2616&mount=/alphafmsp&user=&pass=&ip=177.223.XX&agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br />
[8/29/18 11:49:00 AM] 200 0.6181 action=listener_remove&server=r22&port=443&client=673&mount=/aradiorock&user=&pass=&duration=983&ip=191.206.XX&agent=NxApp/1.18.07.10 (Xamarin; Android; 27; Mi A2 Lite; Xiaomi)<br />
[8/29/18 11:49:00 AM] 200 0.4407 action=listener_add&server=r22&port=443&client=2618&mount=/alphafmsp&user=&pass=&ip=177.138.XX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br />
[8/29/18 11:49:00 AM] 200 0.3872 action=listener_add&server=r20&port=443&client=3040&mount=/aradiorock&user=&pass=&ip=189.8.XX&agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36</div>
<div> </div>
<div>Even with responses quickly responses we still getting:</div>
<div> </div>
<div style="margin-left: 40px;">[2018-08-29 11:49:08] INFO auth_url/url_add_listener client auth (http://10.68.0.136:8001/listener/add) failed with ""</div>
<div> </div>
<div>And sometimes the icecast process just shutdown! Bellow is the last second logged:</div>
<div> </div>
<div style="margin-left: 40px;">AUTH ACCESS.LOG</div>
<div style="margin-left: 40px;"> </div>
<div style="margin-left: 80px;">[8/29/18 11:49:10 AM] 200 0.3864 action=listener_add&server=r20&port=443&client=3071&mount=/amorfm&user=&pass=&ip=191.39.XXX&agent=App/2.4.2 (Linux;Android 7.0) ExoPlayerLib/2.5.4<br />
[8/29/18 11:49:10 AM] 200 0.204 action=listener_add&server=r20&port=443&client=3070&mount=/alphafmsp&user=&pass=&ip=201.81.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br />
[8/29/18 11:49:10 AM] 200 0.2598 action=listener_remove&server=r22&port=443&client=2625&mount=/alphafmsp&user=&pass=&duration=6&ip=177.95.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br />
[8/29/18 11:49:10 AM] 200 0.1067 action=listener_remove&server=r20&port=443&client=512&mount=/aradiorock&user=&pass=&duration=1087&ip=179.127.XXX&agent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36<br />
[8/29/18 11:49:10 AM] 200 0.4311 action=listener_add&server=r20&port=443&client=3072&mount=/alphafmsp&user=&pass=&ip=201.52.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36<br />
[8/29/18 11:49:10 AM] 200 0.3617 action=listener_add&server=r22&port=443&client=2638&mount=/aradiorock&user=&pass=&ip=179.241.XXX&agent=NxApp/1.18.07.10 (Xamarin; Android; 26; moto x4; motorola)</div>
<div style="margin-left: 40px;"> </div>
<div style="margin-left: 40px;">ICECAST ERROR.LOG (machine named "r20")</div>
<div> </div>
<div style="margin-left: 80px;">[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /amorfm<br />
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp<br />
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /amorfm is -1 (cur 6)<br />
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source Added client to /amorfm<br />
[2018-08-29 11:49:10] DBUG auth/add_authenticated_listener client authenticated, passed to source<br />
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /alphafmsp is 750 (cur 750)<br />
[2018-08-29 11:49:10] DBUG fserve/fserve_add_client Adding client to file serving engine<br />
[2018-08-29 11:49:10] DBUG fserve/fserve_add_pending fserve handler waking up<br />
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /alphafmsp is now 751<br />
[2018-08-29 11:49:10] INFO auth/auth_new_listener client 3070 failed<br />
[2018-08-29 11:49:10] DBUG fserve/fserv_thread_function fserve handler exit<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2719)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2720)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br />
[2018-08-29 11:49:10] DBUG client/client_read_bytes reading from connection has failed<br />
[2018-08-29 11:49:10] INFO source/send_to_listener Client 512 (179.127.XXX) has fallen too far behind, removing<br />
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /aradiorock is now 502<br />
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /aradiorock has 1 pending<br />
[2018-08-29 11:49:10] DBUG source/source_main Client removed<br />
[2018-08-29 11:49:10] INFO source/source_main listener count on /aradiorock now 500<br />
[2018-08-29 11:49:10] DBUG source/source_main Client added for mountpoint (/amorfm)<br />
[2018-08-29 11:49:10] INFO source/source_main listener count on /amorfm now 7<br />
[2018-08-29 11:49:10] INFO auth/auth_add_listener adding client for authentication<br />
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /alphafmsp is now 752<br />
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /alphafmsp has 1 pending<br />
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /aradiorock<br />
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /aradiorock is now 501<br />
[2018-08-29 11:49:10] DBUG format/format_check_http_buffer processing pending client headers<br />
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp<br />
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /alphafmsp is 750 (cur 750)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3046)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/aradiorock" slow_listeners (63)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listeners (1317)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/aradiorock" listeners (500)<br />
[2018-08-29 11:49:10] DBUG fserve/fserve_add_client Adding client to file serving engine<br />
[2018-08-29 11:49:10] DBUG fserve/fserve_add_pending fserve handler waking up<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3047)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/amorfm" listener_peak (7)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/amorfm" listeners (7)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2721)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listeners (1318)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listener_connections (2347)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br />
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3048)<br />
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /alphafmsp is now 751<br />
[2018-08-29 11:49:10] INFO auth/auth_new_listener client 3072 failed<br />
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /alphafmsp is now 751<br />
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /alphafmsp has 1 pending<br />
[2018-08-29 11:49:10] DBUG fserve/fserv_thread_function fserve handler exit<br />
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp</div>
<div> </div>
<div>I can't figure out why icecast is shutting down and why sometimes it can't authenticate the request.</div>
<div> </div>
<div>It happens with different loads. Sometimes with about 2500, others with 3000 and others with about 4200 listeners.</div>
<div> </div>
<div>Other useful info: we have a hearbeat api that do a "status-json.xsl" request at every 4s.</div>
<div> </div>
<div>Any ideas?</div>
<div> </div>
</div>
<div> </div>
<div> </div>
<div>Best,</div>
<div>Thiago</div>
<div style="-webkit-touch-callout: none; -webkit-user-select: none; -khtml-user-select: none;-moz-user-select: none;-ms-user-select: none;-o-user-select: none;user-select: none;"> </div>
<div> </div>
<hr align="center" size="2" width="100%" />
<div><span style="font-family: tahoma,arial,sans-serif; font-size: 10pt;"><b>De</b>: "subscription@nextdial.com.br" <subscription@nextdial.com.br><br />
<b>Enviado</b>: sexta-feira, 24 de agosto de 2018 20:51<br />
<b>Para</b>: icecast@xiph.org<br />
<b>Assunto</b>: Client auth failed with ""</span>
<div> </div>
<div>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">Hello,</span></div>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">We are using url authentication and getting the item bellow at the log:</span></div>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span></div>
<div style="margin-left: 40px;"><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">[2018-08-24 23:26:05] INFO auth_url/url_add_listener client auth (http://127.0.0.1:8001/listener/add) failed with ""</span></div>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span></div>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">I put try/catch and reviewed all my code ensure always return a proper status and header error code, so what trigger this kind of error?</span></div>
</div>
<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span></div>
<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span></div>
<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">Regards,</span></div>
<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span>
<div><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px">Thiago</span></div>
<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span>
<div style="-webkit-touch-callout: none; -webkit-user-select: none; -khtml-user-select: none;-moz-user-select: none;-ms-user-select: none;-o-user-select: none;user-select: none;"><span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"> </span></div>
</div>
</div></span>