<div dir="auto">Hi all<div dir="auto"><br></div><div dir="auto">I still have to analyze my problem comma but I'll let you know if it is exactly the same.</div><div dir="auto"><br></div><div dir="auto">Luckily my server is not in production yet. I am still testing.</div><div dir="auto"><br></div><div dir="auto">Running Ubuntu 16.04 and Ice cast 2.4.2.</div><div dir="auto"><br></div><div dir="auto">Is there anyone out there that has also had a similar problem of the service coming to an abrupt stop. </div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 29 Aug 2018, 14:45 <a href="mailto:subscription@nextdial.com.br">subscription@nextdial.com.br</a>, <<a href="mailto:subscription@nextdial.com.br">subscription@nextdial.com.br</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"><div>Hello,</div>
<div> </div>
<div>In order to identify if was some issue regard to the auth api we include some trace logs and we move it to other machine.</div>
<div><br>
Bellow is what trace we are generating (date, result, elapsed time in milleseconds, request):</div>
<div>
<div>
<div style="margin-left:40px">[8/29/18 11:49:00 AM] 200 0.3721 action=listener_add&server=r22&port=443&client=2616&mount=/alphafmsp&user=&pass=&ip=177.223.XX&agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br>
[8/29/18 11:49:00 AM] 200 0.6181 action=listener_remove&server=r22&port=443&client=673&mount=/aradiorock&user=&pass=&duration=983&ip=191.206.XX&agent=NxApp/<a href="http://1.18.07.10" target="_blank" rel="noreferrer">1.18.07.10</a> (Xamarin; Android; 27; Mi A2 Lite; Xiaomi)<br>
[8/29/18 11:49:00 AM] 200 0.4407 action=listener_add&server=r22&port=443&client=2618&mount=/alphafmsp&user=&pass=&ip=177.138.XX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br>
[8/29/18 11:49:00 AM] 200 0.3872 action=listener_add&server=r20&port=443&client=3040&mount=/aradiorock&user=&pass=&ip=189.8.XX&agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36</div>
<div> </div>
<div>Even with responses quickly responses we still getting:</div>
<div> </div>
<div style="margin-left:40px">[2018-08-29 11:49:08] INFO auth_url/url_add_listener client auth (<a href="http://10.68.0.136:8001/listener/add" target="_blank" rel="noreferrer">http://10.68.0.136:8001/listener/add</a>) failed with ""</div>
<div> </div>
<div>And sometimes the icecast process just shutdown! Bellow is the last second logged:</div>
<div> </div>
<div style="margin-left:40px">AUTH ACCESS.LOG</div>
<div style="margin-left:40px"> </div>
<div style="margin-left:80px">[8/29/18 11:49:10 AM] 200 0.3864 action=listener_add&server=r20&port=443&client=3071&mount=/amorfm&user=&pass=&ip=191.39.XXX&agent=App/2.4.2 (Linux;Android 7.0) ExoPlayerLib/2.5.4<br>
[8/29/18 11:49:10 AM] 200 0.204 action=listener_add&server=r20&port=443&client=3070&mount=/alphafmsp&user=&pass=&ip=201.81.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br>
[8/29/18 11:49:10 AM] 200 0.2598 action=listener_remove&server=r22&port=443&client=2625&mount=/alphafmsp&user=&pass=&duration=6&ip=177.95.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36<br>
[8/29/18 11:49:10 AM] 200 0.1067 action=listener_remove&server=r20&port=443&client=512&mount=/aradiorock&user=&pass=&duration=1087&ip=179.127.XXX&agent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36<br>
[8/29/18 11:49:10 AM] 200 0.4311 action=listener_add&server=r20&port=443&client=3072&mount=/alphafmsp&user=&pass=&ip=201.52.XXX&agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36<br>
[8/29/18 11:49:10 AM] 200 0.3617 action=listener_add&server=r22&port=443&client=2638&mount=/aradiorock&user=&pass=&ip=179.241.XXX&agent=NxApp/<a href="http://1.18.07.10" target="_blank" rel="noreferrer">1.18.07.10</a> (Xamarin; Android; 26; moto x4; motorola)</div>
<div style="margin-left:40px"> </div>
<div style="margin-left:40px">ICECAST ERROR.LOG (machine named "r20")</div>
<div> </div>
<div style="margin-left:80px">[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /amorfm<br>
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp<br>
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /amorfm is -1 (cur 6)<br>
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source Added client to /amorfm<br>
[2018-08-29 11:49:10] DBUG auth/add_authenticated_listener client authenticated, passed to source<br>
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /alphafmsp is 750 (cur 750)<br>
[2018-08-29 11:49:10] DBUG fserve/fserve_add_client Adding client to file serving engine<br>
[2018-08-29 11:49:10] DBUG fserve/fserve_add_pending fserve handler waking up<br>
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /alphafmsp is now 751<br>
[2018-08-29 11:49:10] INFO auth/auth_new_listener client 3070 failed<br>
[2018-08-29 11:49:10] DBUG fserve/fserv_thread_function fserve handler exit<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2719)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2720)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br>
[2018-08-29 11:49:10] DBUG client/client_read_bytes reading from connection has failed<br>
[2018-08-29 11:49:10] INFO source/send_to_listener Client 512 (179.127.XXX) has fallen too far behind, removing<br>
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /aradiorock is now 502<br>
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /aradiorock has 1 pending<br>
[2018-08-29 11:49:10] DBUG source/source_main Client removed<br>
[2018-08-29 11:49:10] INFO source/source_main listener count on /aradiorock now 500<br>
[2018-08-29 11:49:10] DBUG source/source_main Client added for mountpoint (/amorfm)<br>
[2018-08-29 11:49:10] INFO source/source_main listener count on /amorfm now 7<br>
[2018-08-29 11:49:10] INFO auth/auth_add_listener adding client for authentication<br>
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /alphafmsp is now 752<br>
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /alphafmsp has 1 pending<br>
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /aradiorock<br>
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /aradiorock is now 501<br>
[2018-08-29 11:49:10] DBUG format/format_check_http_buffer processing pending client headers<br>
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp<br>
[2018-08-29 11:49:10] DBUG auth/add_listener_to_source max on /alphafmsp is 750 (cur 750)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3046)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/aradiorock" slow_listeners (63)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listeners (1317)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/aradiorock" listeners (500)<br>
[2018-08-29 11:49:10] DBUG fserve/fserve_add_client Adding client to file serving engine<br>
[2018-08-29 11:49:10] DBUG fserve/fserve_add_pending fserve handler waking up<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3047)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/amorfm" listener_peak (7)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update "/amorfm" listeners (7)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global client_connections (2721)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1331)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listeners (1318)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global listener_connections (2347)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global clients (1332)<br>
[2018-08-29 11:49:10] DBUG stats/modify_node_event update global connections (3048)<br>
[2018-08-29 11:49:10] DBUG auth/auth_release ...refcount on auth_t /alphafmsp is now 751<br>
[2018-08-29 11:49:10] INFO auth/auth_new_listener client 3072 failed<br>
[2018-08-29 11:49:10] DBUG auth/queue_auth_client ...refcount on auth_t /alphafmsp is now 751<br>
[2018-08-29 11:49:10] INFO auth/queue_auth_client auth on /alphafmsp has 1 pending<br>
[2018-08-29 11:49:10] DBUG fserve/fserv_thread_function fserve handler exit<br>
[2018-08-29 11:49:10] DBUG auth/auth_run_thread 1 client(s) pending on /alphafmsp</div>
<div> </div>
<div>I can't figure out why icecast is shutting down and why sometimes it can't authenticate the request.</div>
<div> </div>
<div>It happens with different loads. Sometimes with about 2500, others with 3000 and others with about 4200 listeners.</div>
<div> </div>
<div>Other useful info: we have a hearbeat api that do a "status-json.xsl" request at every 4s.</div>
<div> </div>
<div>Any ideas?</div>
<div> </div>
</div>
<div> </div>
<div> </div>
<div>Best,</div>
<div>Thiago</div>
<div> </div>
<div> </div>
<hr align="center" size="2" width="100%">
<div><span style="font-family:tahoma,arial,sans-serif;font-size:10pt"><b>De</b>: "<a href="mailto:subscription@nextdial.com.br" target="_blank" rel="noreferrer">subscription@nextdial.com.br</a>" <<a href="mailto:subscription@nextdial.com.br" target="_blank" rel="noreferrer">subscription@nextdial.com.br</a>><br>
<b>Enviado</b>: sexta-feira, 24 de agosto de 2018 20:51<br>
<b>Para</b>: <a href="mailto:icecast@xiph.org" target="_blank" rel="noreferrer">icecast@xiph.org</a><br>
<b>Assunto</b>: Client auth failed with ""</span>
<div> </div>
<div>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">Hello,</span></div>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">We are using url authentication and getting the item bellow at the log:</span></div>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span></div>
<div style="margin-left:40px"><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">[2018-08-24 23:26:05] INFO auth_url/url_add_listener client auth (<a href="http://127.0.0.1:8001/listener/add" target="_blank" rel="noreferrer">http://127.0.0.1:8001/listener/add</a>) failed with ""</span></div>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span></div>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">I put try/catch and reviewed all my code ensure always return a proper status and header error code, so what trigger this kind of error?</span></div>
</div>
<span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span></div>
<span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span></div>
<span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">Regards,</span></div>
<span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px">Thiago</span></div>
<span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span>
<div><span style="font-family:Arial,Helvetica,Sans-Serif;font-size:12px"> </span></div>
</div>
</div></span>
_______________________________________________<br>
Icecast mailing list<br>
<a href="mailto:Icecast@xiph.org" target="_blank" rel="noreferrer">Icecast@xiph.org</a><br>
<a href="http://lists.xiph.org/mailman/listinfo/icecast" rel="noreferrer noreferrer" target="_blank">http://lists.xiph.org/mailman/listinfo/icecast</a><br>
</blockquote></div>