[Icecast] falling back

Sat May 22 09:07:26 UTC 2021

Thanks for the comments Petr and Marvin.

I know Icecast was not specifically designed for this, but I am sure that I am not asking it to do anything (apart from triggering scripts) that it cannot do. I understand about the different mounts, but each is independent of the other and as I have found, Icecast does not care that it is relaying from itself. That works. If I start the transcode process to BBCR2.aac first, I can then connect/listen to the BBCR2 mount. This works perfectly, so Icecast has no problem with this setup. The reason for multiple mounts like this is because a <mount type="normal”> can only trigger the on-connect script when the source connects, but the script IS the source, so a chicken and egg problem there. However, I figured a relay type mount might trigger when it starts pulling from the relayed server as that is as close to the source connecting as a relay can get. More on that later.

As Petr points out, BBCR2.aac is only required to stream to a single client, whereas Icecast is designed to stream to multiple clients. However how can this be a problem. Icecast cares not how many listeners, as long as it is below the maximum and in the case of a relay, only to determine when to connect to and disconnect from the relayed server mount. Having just a single listener does not mean Icecast will malfunction in any way. What it possibly means though is more resources being used than is strictly necessary. However this is actually irrelevant since Icecast is already running anyway, so an additional mount for a single listener has to consume less resources than a completely independent process just for the purpose of streaming to a single client. I’m certainly not against that however, but trying to set up an on-demand transcode process that streamed to Icecast turned out to be far harder (I’ve failed so far) than simply interposing an additional mount in Icecast.

The BBCR2 mount is not specified twice as I understand it. A <relay> mount block specifically configures the relay and nothing else. If you need any other configuration (like authorisation etc.) you HAVE to set up a <mount type="normal”> block of the same name to contain all its other required parameters, as that stuff cannot be held within the <relay> block. So unless anyone can explain how the docs are incorrect, I am sure that this is correct.

The problem I have with this Icecast setup is as I said, triggering the transcode script and the streaming fallback file.

The latter seems likely to be as suggested and my next test will involve a file of some sound(s) that will not compress so much and hopefully not fill up the buffers. However it is interesting that I have read many mentions of the use of a file of silence specifically for this exact task, even though it clearly has problems. Anyway, I can try an alternative file and see how that goes. Something is required however as without the fallback file configured, any attempt to listen to the relay is immediately rejected (with a 404 I seem to recall).

That leaves the on-(dis)connect issue. This is where my plan might completely fail. I am hoping that a relay mount can trigger these, however I have seen no direct evidence that it will. The standard explanation is that on-connect is triggered when the source connects, but maybe that only works when a source is pushed to a normal Icecast mountpoint and not when a relay sees the mount from which it is supposed to relay and so streams from it. In my case, the stream will actually be the fallback file, but it should still count as a source. The problem that occurs to me is that the relay mount code may simply not recognise being able to pull from the relayed mount (internal or external) as a source connect event and hence will never trigger the script. 

This last has always been in my mind, but no-one has yet been able to confirm how Icecast actually functions in this respect. Hence why I’ve been testing, but the absence of success so far does not confirm absolutely that it cannot work and may only indicate I have something else wrong, although I am coming to the realisation that a relay mount will not work as I require.

In which case, back to the idea of the transcode process as an http streamer. This however has to be very low ‘footprint’ as multiple streams would mean multiple http servers, one for each transcoded stream. Better to have a single server that can handle multiple listeners and http streams. And we’re back to Icecast that is already running…

If anyone can actually suggest a solution to this conundrum, please let me know as I’m all ears.

Ken  G i l l e t t

_/_/_/_/_/_/_/_/

> On 21 May 2021, at 20:50, Petr Pisar <petr.pisar at atlas.cz> wrote:
> 
> V Fri, May 21, 2021 at 06:28:42PM +0100, Ken Gillett napsal(a):
>> As I mentioned, I need to transcode and want Icecast to start and stop
>> according to listeners. So this is what I’m trying:-
>> 
>> 	<relay>
>> 		<server>127.0.0.1</server>
>> 		<port>8127</port>
>> 		<mount>/BBCR2.aac</mount>
>> 		<local-mount>/BBCR2</local-mount>
>> 	</relay>
>> 	<mount type="normal">
>> 		<mount-name>/BBCR2</mount-name>
>> 		<public>0</public>
>> 		<on-connect>/usr/local/bin/transtart</on-connect>
>> 		<on-disconnect>/usr/local/bin/transtop</on-disconnect>
>> 	</mount>
>> 	
>> 	<mount type="normal">
>> 		<mount-name>/BBCR2.aac</mount-name>
>> 		<public>0</public>
>> 		<fallback-mount>/silence.aac</fallback-mount>
>>    	<fallback-override>1</fallback-override>
>> 	</mount>
>> 
>> So the first 2 blocks are the relay mount that ‘pulls’ from the mount
>> configured in the 3rd block. Yes, this does work, Icecast CAN relay from
>> itself. The reason why I do this is to take advantage of the on-(dis)connect
>> scripts and the fact that an Icecast relay only ‘pulls’ when there are
>> listeners for that mountpoint.
>> 
> In my opinion you cannot achieve what you want with Icecast because it goes
> against its design.
> 
> Icecast has two types of mount points: source-mounts which content is pushed
> into Icecast and relay-mounts which content is pulled by Icacast.
> A transmission of the content which is pulled can only be initiated by Icecast.
> On the other hand, the pushed content cannot be initiated by Icecast. The
> pushed content is always initated by an external stream source (transcode in
> your case).
> 
> Basically, Icecast is a streaming proxy of which task is to multiply an external
> source to multiple clients at the same time. The external source is either
> pushed or pulled content. Streaming local files is only a trivial tool for
> providing a fallback when the external source is unavailable.
> 
> What you need is an transcoder which waits on a connection from an ICY client
> and then it opens a tunner and starts transcoding. And similarly it will close
> the tunner after the client disconnects. The transcoder does not need and
> should not be able to handle multiple clients. It should be a trivial program
> for one tunner and one client. After having this setup, you can place Icecast
> between the transcoder and the (thousands of) clients.
> 
> I cannot recommend you any ready-made transcoder because I have no experience
> in this area. But I believe there exist plenty of transcoders which can work
> as a source-client for Icecast and thus it should be easy to enhance them to
> implement a one-client ICY server (the pull model from Icecast point of view)
> in addition.
> 
> (Technically it could be possible to introduce a third type of a mount type
> to Icecast: It would execute an external command for the first client,
> streamed its standard output and terminated the command with the last client.
> But handling execution of possibly blocking commands adds more complexity and
> from security point of view its not wise to give Icecast server a permission
> to access various hardware (e.g. tunners) directly.)
> 
>> However, about 35 seconds after this first starts (i.e. listener connects to
>> BBCR2), Icecast logs this:-
>> 
>> [2021-05-21  17:21:28] INFO source/send_to_listener Client 377 (xxx.xxx.xxx.xxx) has fallen too far behind, removing
>> [2021-05-21  17:21:28] INFO source/source_main listener count on /BBCR2 now 0
>> [2021-05-21  17:21:28] INFO source/source_shutdown Source from 127.0.0.1 at "/BBCR2" exiting
>> [2021-05-21  17:21:28] INFO source/source_main listener count on /silence.aac now 0
>> 
>> So Icecast kills everything, but the listener (ffplay) doesn’t know this and
>> continues running, obviously still silent so I cannot tell what it actually
>> thinks it is playing.  Why does Icecast say the Client (ffplay on my Mac at
>> xxx.xxx.xxx.xxx) is too far behind?
> 
> I think this message means that the client does not consume the stream. So an
> output buffer on the server has been filling up and after 35 seconds reached
> a limit and thus the server disconnected the client.
> 
> I recommend you either to replace the silent stream with something audiable,
> or better use a network analyzer tool (e.g. tcpdump) to see whether the stream
> flows from the server to the client or not.
> 
> I suspect that either the client misunderstands the stream, or the server
> misunderstands the /silence.aac file.
> 
>> Why are the scripts not run? It seems to me that when the relay receives
>> anything, whether from the relayed mount (BBCR2.aac in this case) or its
>> fallback file, that event is the source connecting and should trigger the
>> on-connect script. But it does not. The script writes a log and I can see it
>> is simply not being run at all (unless I run it manually).
>> 
> Does something change if you remove the second block (mount at /BBCR2)?
> 
> I think it does not call the scripts because you defined /BBCR2 twice. Maybe
> Icecast parser does not expect having them twice, ignores the second
> block and uses only the first one.
> 
> -- Petr
> 
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast