[Icecast] Re-Assembling SongData in Icecast Streams..

[oddsock]

At 06:19 PM 11/22/2004, you wrote:
>Like any other major radio station we output our music from automation 
>software via analog audio through a mix board in a studio, where we insert 
>other stuff like live DeeJays, etc. only to have that stream re-encoded by 
>hardware MP3 encoders for distribution to our network of IceCast servers.
>
>The chain of song information gets broken as soon as we output from the 
>Automation software, although nowadays most of the software such as RCS 
>and Dalet can output raw text or XML.  They had to do this for guys like 
>XM and Sirius.
>
>While we re-unite song data with the stream via our Flash-based player at 
>www.pulverradio.com but you can always tell such radio stations on Live365 
>or Shoutcast.com because they don't pump out song data with their stream, 
>which makes listening to the streams on dedicated hardware or desktop 
>clients less interesting.
First off, from your description is seems that you might be a bit confused 
as to how metadata (song titles, etc) is handled and managed so I'll 
provide a little background.

I am going to assume that you are dealing only with MP3, as metadata is 
handled differently (for instance) with vorbis.  For MP3 streams, all 
metadata is updated in the streaming server via various administrative 
interfaces.  Shoutcast has one 
(admin.cgi?mode=updinfo&pass=password&song=Booga) and Icecast has a 
different one (it also supports the Shoutcast style now too).  Either way, 
the metadata is always sent (normally by the source client - i.e. the 
application sending the stream to the server - but not necessarily) to the 
streaming server.  A common misconception is that the streaming servers 
somehow pull out ID3 tags embedded in the stream, they do not.  For mp3 
streams they all take in metadata via the various admin interfaces I 
described above.  For delivering the metadata to listening clients, there 
is really only one supported mechanism, and that is the 
"shoutcast-metadata-protocol".  A really great explanation of this is here 
http://www.smackfu.com/stuff/programming/shoutcast.html.   Most mp3 clients 
support this protocol but not all do (Windows Media Player does NOT, and 
the various Flash players that you are using do NOT).  Most other media 
players that people tend to use (Winamp, Foobar, XMMS) all do.

So, enough background, to your specific problem.  If you are looking to get 
metadata into WMP, or Flash (or other clients that do not support the 
shoutcast-metadata-protocol) you are pretty much out of luck.  Outside of 
developing your own custom version (as you did with your flash 
player).  For all other clients, you should be able to support metadata in 
the way you want, but you are just missing the piece that extracts the 
metadata from your repository (via XML or flatfile) and updates the 
streaming server via the well-defined admin interface.  This should not 
really be the job of icecast or shoutcast (it's job is to primarily stream 
to clients), so you are probably looking at a fairly straightforward python 
script that parses an XML file, pulls out the metadata and then sends it to 
icecast/shoutcast.

sorry for rambling on about this, but this question seems to come up quite 
a bit so I thought I would try to expound upon the topic a bit.  Anyway, 
Vorbis streaming is quite different than MP3 (in many ways).    Many 
streaming codecs (definitely all the ones supported by Shoutcast, which 
includes AAC, MP3, and NSV (which is technically a container)) are streamed 
in "passthru" fashion, in that the streaming server really need not know 
what is being streamed (outside of a content-type).  What this means is 
that a listener can pretty much start at any point in the stream and begin 
listening, think of it as a big pipe with water flowing out of it, if you 
are thirsty, you can pretty much take a drink of water any time you 
want.  As an aside, in the trunk of icecast we are now supporting this 
"passthru" mode, and as such now support the streaming of AAC/AACPlus as 
well as NSV.  The vorbis spec has a small oddity that causes a bit of 
trouble at first, and thats the fact that each streaming server MUST cache 
the vorbis header packets that are sent at the beginning of the vorbis 
stream.  This requirement means that the streaming server MUST know that 
the data coming in is a vorbis stream, and must somewhat intelligently 
interpret it (in order to pull out the vorbis header packets).  It must 
save off these header packets because the spec says that the header packets 
must be the *first* thing sent to connecting listeners.  So, in vorbis' 
case, you can just take a drink whenever you want, you have to first pick 
up the header packets before taking a sip of the water coming from the pipe 
(bad analogy I know, but lets go with it)...So what does this have to do 
with metadata ? Well, in these header packets is the metadata (stored as 
vorbis comments, which are name-value pairs of string.  By convention, 
TITLE=X and ARTIST=Y are used by most media players to display "what's 
playing" in the player.  So, in this case, it is still the responsibility 
of the source client to set the metadata, but it does it by embedding it 
into the actual vorbis stream (and it does it in a much better, more 
efficient, and more elegant way).  So how do you update metadata in a 
vorbis stream ? Well, via some vorbis magic calls, you tear down the 
logical vorbis stream (not socket!) and rebuild a new vorbis stream with 
new header packets - which the streaming server must detect and cache (if 
you were following me above).  The downside to this is that it falls apart 
when your metadata is not accessible by your source client (such is your 
specific case).  If anyone ever wondered why you can't update the metadata 
in a vorbis stream via the admin interface, it's because in that case 
icecast would need to take this update, and rebuild it's cached header 
packets, which can be a bit tricky in all cases.   Karl's branch does it, 
and I'm trying to get it integrated into the trunk right now.

ok, I think I've gone on enough about how metadata is handled in a few 
different places...The only reason I'm being so in depth is that I don't 
think it's terribly common knowledge, and I tend to answer questions on it 
all the time...

oddsock