[Icecast] Load balancing Icecast - aggregated logs

[Philipp Schafft]

Good afternoon,

On Mon, 2020-01-13 at 13:41 +0000, Chip wrote:
> Good afternoon Philipp
> 
> Many thanks for your reply.

> Sorry for not being clear.
> 
> I think the problem I have would be in the implementation.

Ok.


> How do I run two versions of Icecast on two servers, with load balancing
> between the two (perhaps using RR-DNS),

Generally I recommend RR-DNS as it is a very good solution for most
problems. Even if something more advanced is used it is a layer that
should be added.

If there are no special requirements, it's the way to go.


> but I present my client with one unified log file for the audience
> statistics?

> Should the two servers write their logs via a network file share to a
> third logging server, for example?

I would recommend against that. It just adds another single point of
failure. Most software I know for that kind of analysis can in fact read
multiple files anyway. (e.g. some wants to import the data into it's own
database first, other just want to support log rotation.) So it depends
on what software you want to use. Maybe it can already read multiple log
files.

If not you could join the files. Check if the software will be happy
with a file that is basically 'cat server-A.log server-B.log >
both.log'. That should be the case as the log lines are not guaranteed
to be in chronological order anyway (to support multithreading).

If it does not work this way some more advanced solution must be found.
But I think this should be possible.

You could implement cat-ing the logs together as part of logrotate:
rotate the log and send it to a common server for arching and
processing. That way the servers themself could also keep a copy in case
that processing server can not be reached (e.g. for maintenance).

For live statistics consider using the STATS interface.

With best regards,



> On Mon, 13 Jan 2020 at 13:36, Philipp Schafft <phschafft at de.loewenfelsen.net>
> wrote:
> 
> > Good afternoon,
> >
> > On Mon, 2020-01-13 at 13:30 +0000, Chip wrote:
> > > Hi
> > >
> > > I have a potential project for which my client requests that we load
> > > balance the streaming service.
> > >
> > > Of course, the Icecast server scales very well.
> > >
> > >    - http://icecast.org/loadtest/
> > >
> > > However, the client requests high-availability and, due to the scale of
> > the
> > > potential project, we would like to load balance the service over two or
> > > more servers.
> > >
> > > I think the load balancing aspect is not my major issue at this stage but
> > > how to aggregate the listener logs in one place so that my client has an
> > > accurate representation of the audience stats.
> > >
> > > Hope someone can help and many thanks in advance
> >
> > I hardly see any question in your mail so it is a bit hard to answer.
> > What exactly is the problem you face? Is it finding a good architecture
> > for your setup or is it in actually implementing it?
> >
> > Please feel free to reply to me off-list for professional Icecast
> > support.


-- 
Philipp Schafft (CEO/Geschäftsführer) 
Telephon: +49.3535 490 17 92

Löwenfelsen UG (haftungsbeschränkt)     Registration number:
Bickinger Straße 21                     HRB 12308 CB
04916 Herzberg (Elster)                 VATIN/USt-ID:
Germany                                 DE305133015
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part
URL: <http://lists.xiph.org/pipermail/icecast/attachments/20200113/0a3a3b6a/attachment.sig>