[icecast-dev] BUG: page-aligned data in libshout

[Arc]

I've spent three days tracking this bug down.. I have yet to find how to
fix it in the code, tho I do have a workaround which proves the nature
of the bug.  Sorry for the somewhat-overkill nature of explaining this
bug, but some people have been doubting it so I'm going to give the
evidence nessesary to prove the nature of it.

If you send page-aligned data to libshout (via shout_send) it segfaults
on the first page containing vorbis data within the vorbis_packet_blocksize
function.  Gdb traceback as follows:

#0  0x282da38a in vorbis_packet_blocksize () from /usr/local/lib/libvorbis.so.2
#1  0x28419827 in blocksize (vd=0x8234400, p=0xbfbff54c) at vorbis.c:61
#2  0x28419aa4 in send_vorbis (self=0x81c6a80, data=0x827d014 "OggS", len=8153) at vorbis.c:124
#3  0x2841732d in shout_send (self=0x81c6a80, data=0x827d014 "OggS", len=8153) at shout.c:173
#4  0x28411486 in pshoutobj_send () from /usr/local/lib/python2.2/site-packages/shout.so
#5  0x080b0c53 in PyCFunction_Call ()

I waded through libshout for more than a few hours trying to figure out
what could cause this, then went to look in libvorbis (and quickly gave
up seeing the complexity of it).

It was very predictable, segfaulting while sending pageno 2 no matter
what bitstream it tried to send.  I verified the data by having it save
to disk vs send through libshout, the page spliting function is working
great.  Then I tried to duplicate it on another machine..

Same OS I compiled identical versions of libogg, libvorbis, libshout,
py-ogg (my extended version), py-vorbis, py-shout (by brendan), and
python just to be on the safe side.  Then I ran brendan's example.py (vs
attempting to run the exact same python program), worked fine.  Then I
tried his example.py on the first machine, it also worked fine.

I then went back to the buggy code, still segfaulting, this is the block
of code that was generating the segfault.. note this does more than
split the pages, it's purpose is to "clean up" a physical bitstream
where logical chained bitstreams may be unterminated (no EOS page) or
where duplicate logical bitstreams are chained sequentially which
confuses players because they have the same serialno... :

class shoutogg :
   def __init__(self) :
      self.ogg = OggSyncState()
      self.page = 0
      self.serial = 0
      self.shout = Shout()
      self.shout.host = stream.host
      self.shout.port = stream.port
      self.shout.user = stream.user
      self.shout.password = stream.password
      self.shout.mount = stream.mount
      self.shout.format = 'vorbis'
      self.shout.protocol = 'http'
      self.shout.open()
   def __call__(self, data) :
      self.ogg.bytesin(data)
      while 1:
         newpage = self.ogg.pageseek()
         if newpage == None :
            break
         if self.page == 0 :
            self.page = newpage
            continue
         if self.page.serialno != newpage.serialno : self.page.eos = 1
         if newpage.pageno==0 : self.page.eos = 1
         self.page.serialno=self.serial
         if self.page.eos == 1 :
            if self.serial<65536 : self.serial = self.serial + 1
            else : self.serial = 0
         print 'syncing'
         self.shout.sync()
         out=self.page.header+self.page.body
         print 'sending page %d' % self.page.pageno
         self.shout.send(out)
         print 'sent'
         self.page=newpage
      return

So I changed the end of it to this:

   def __call__(self, data) :
      self.ogg.bytesin(data)
      buffer=''
      while 1:
         newpage = self.ogg.pageseek()
         if newpage == None :
            break
         if self.page == 0 :
            self.page = newpage
            continue
         if self.page.serialno != newpage.serialno : self.page.eos = 1
         if newpage.pageno==0 : self.page.eos = 1
         self.page.serialno=self.serial
         if self.page.eos == 1 :
            if self.serial<65536 : self.serial = self.serial + 1
            else : self.serial = 0
         out=self.page.header+self.page.body
         print 'buffering page %d' % self.page.pageno
         buffer=buffer+out
         self.page=newpage
      if buffer!='' :
         print 'sending %d bytes' % len(buffer)
         self.shout.sync()
         self.shout.send(buffer)
      return

It still crashed.  I increased the size of the data being sent to the
function to 20k, and it continued to segfault in the same way even tho
it was sending multiple pages (vorbis headers plus data in first piece)
at once, it was still page-aligned. then I changed it like this:

   def __call__(self, data) :
      self.ogg.bytesin(data)
      while 1:
         newpage = self.ogg.pageseek()
         if newpage == None :
            break
         if self.page == 0 :
            self.page = newpage
            continue
         if self.page.serialno != newpage.serialno : self.page.eos = 1
         if newpage.pageno==0 : self.page.eos = 1
         self.page.serialno=self.serial
         if self.page.eos == 1 :
            if self.serial<65536 : self.serial = self.serial + 1
            else : self.serial = 0
         out=self.page.header+self.page.body
         print 'looked at page %d (%d bytes)' % (self.page.pageno,len(out))
         self.page=newpage
      self.shout.sync()
      self.shout.send(data)
      return

So it still processed the pages the same as it always was, but ignored
them and sent the non-page aligned input data anyways.  Guess what?
  - It worked -

The last test, and the way the code currently stands, proves that the
bug is in sending libshout page-aligned data:

[...]
      self.shout.open()
      self.shout.send('OggS')
   def __call__(self, data) :
[...]
         out=self.page.header[3:]+self.page.body+'OggS'
         print 'Sending Page %d' % self.page.pageno
         self.shout.sync()
         self.shout.send(data)
         self.page=newpage
      return

As you can see it de-page-aligns the data by an offset of four, and this
code works beautifully, both fixing the ogg stream before sending it to
libshout and also doing so in a way that doesnt cause it to die. :-)

I assume the bug causes problems with normal streams, likely one out of
4k pages result in a segfault?  I can only guess.  Anyways I hope I have
provided enough information to track this down, it's too complex for me.


--
------------------------------------------------------------------------

Use GNU Privacy Guard to protect your email, see [http://www.gnupg.org/]

The attachement to this and every email I send is my GPG signature which
is used to verify that I am the sender and it is unmodified by any third
party.  You need GNU Privacy Guard installed to verify my GPG signature.

This was important even before our government's war on civil liberties,
but with the USA PAT RIOT Act signed into law it's especially important
to secure our right to communicate freely without federal surveillance.

If you have GNU Privacy Guard, please use it to encrypt and sign any and
all mail you send me.  I can help you in setting up and using encryption
to protect your email as well as host workshops for teaching your group.

------------------------------------------------------------------------


-------------- next part --------------
A non-text attachment was scrubbed...
Name: part
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20030517/336c099d/part.pgp