[theora-dev] [Cortado] How to support seeking in on-the-fly generated Theora stream?

[Tom Sparks]

I think byte-range serving[1] would be easy to do
or use Chunked transfer encoding[2] 

[1] http://en.wikipedia.org/wiki/Byte_serving
[2] http://en.wikipedia.org/wiki/Chunked_transfer_encoding
--
tom_a_sparks "It's a nerdy thing I like to do"
Please use ISO approved file formats excluding Office Open XML - http://www.gnu.org/philosophy/no-word-attachments.html
3 x (x)Ubuntu 10.04, Amiga A1200 WB 3.1, UAE AF 2006 WB 3.X, Sam440 AOS 4.1


--- On Mon, 23/5/11, David Schueler <david.schueler at wapkamera.de> wrote:

> From: David Schueler <david.schueler at wapkamera.de>
> Subject: [theora-dev] [Cortado] How to support seeking in on-the-fly generated Theora stream?
> To: theora-dev at xiph.org
> Received: Monday, 23 May, 2011, 8:02 PM
> Hello all!
> 
> i want to use Theora in a little video portal, because its
> free and open 
> source and i want to contribute some code to get the Java
> Theora player 
> (Cortado) more feature-rich.
> But now I'm stuck and hope that someone can point me into
> the right 
> direction.
> 
> I have videos stored in several formats (mostly H.264 or
> MPEG4) and use 
> ffmpeg2theora to recode them on the fly. The recoded output
> is send via 
> HTTP stream directly to the Cotrado player which stores the
> data and 
> begins playing if enough data is buffered.
> In practice this looks like this is a PHP file:
> passthru("ffmpeg2theora -o - ".$source." 2>
> /dev/null");
> 
> I recoded the Queue.java (Queue class) that the queue array
> is not a dump 
> FIFO anymore and called it QueueSeek.java.
> Now the Buffer array is filled at the end and a variable
> points to the 
> current play position which gets increased on every loop
> and sends the 
> data to the next sink pad (at OggDemux).
> 
> QueueSeek Buffer:
> 
>            +--# data
> added at the end
> +0-1-2-3-4-v-------
> |#|#|#|#|#| | | | -> unlimited capacity
> +----|-------------
>      v
> current play position
> data will be send to demuxer
> but not removed from the FIFO
> 
> Now i managed that the fill level of the buffer is the
> difference between 
> the size of the data in the FIFO and the current play
> position. So the 
> play begins if the data is enough and stops if the
> "watermark" gets below 
> the lowest level.
> 
>      + current play position
> +0-1-V-3-4-5-6-----
> |#|#|#|#|#|#|#| | 
> +----|-------|-----
>      |buffer-|
>       level=4
> 
> Now i want to seek. And thats where the problems start to
> begin. I 
> understood how the demuxer works. It gets the data from the
> Queue, stores 
> it in its own buffer and seeks for a "OggS" pattern which
> marks the start 
> of a new Theora page. Then is checks if that whole page is
> present in the 
> buffer and calculates a checksum. If all went well the page
> is processed 
> in a way i did not look about any further.
> But if i seek in my new Queue - which means just setting
> the play position 
> to any value - then the demuxer stumbles, because the
> checksum and the 
> length of the page  are not correct.
> To get around this i did a little trick:
> On my new FIFO i seek though the incoming data and look for
> the OggS 
> pattern. If it is present the data will be split into two
> segments and 
> both will be inserted info the FIFO. Now every page starts
> at the 
> beginning of a FIFO buffer:
> 
> +------0-------------1-------------2------------3-----
> | OggS...data | OggS...data.|....data    |
> OggS...
> +-----------------------------------------------------
> 
> As you can see, sometimes the data is lager than the 4096
> byte of one 
> buffer, so it takes more than one buffer for a page. I've
> taken care of 
> that in the seeking method by rewinding the play position
> until the buffer 
> starts with an OggS pattern.
> But now a second problem exists: The seeking should start
> if one page is 
> completely submittet. And thats where i'm stucking now.
> 
> The pads of the elements are separate threads which work
> independently 
> from each other. 
> 
> ----+        +-----------[QueueSeek
> element]---------+     
>    +----
> prev|        |     
>                
>              
>    |   to    | sink
> pad
> src O--push->O sink pad --> [FIFO magic] --> src
> pad O--next-->O of the
> pad |        |     
>                
>              
>    | element | demuxer
> ----+       
> +---------------------------------------+   
>      +----
> 
> Thats why many code blocks are synchronized on the queue
> object, that only 
> one thread can access the queue at one time. Now, i want so
> synchronize 
> the sink- and sourcepad threads of the QueueSeek class on
> another object, 
> which is called "permissionToSeek". The sinkpad should wait
> until it gets 
> a permission to seek.
> I tried to achieve this with wait() and notifyAll().
> the sinkpad has a method which gets called if a seek
> request arrives. It 
> synchronizes on the permission object and waits for the
> srcpad to grant 
> it:
> 
> private void doSeek(Event event) {
>     // wait until we get the permission to seek
>     synchronized (permissionToSeek) {
>     try {
>         Debug.info("[BUFFER] sink ->
> doSeek() waiting for 
> permissionToSeek");
>         // wait until we are allowed to
> seek
>         permissionToSeek.wait();
>     } catch (InterruptedException e) {
>         e.printStackTrace();
>     }
>     /* ... code to seek ... */
> }
> 
> The sinkpad checks if the next buffer begins with OggS
> pattern and calls 
> the notifyAll() method:
> 
> protected void taskFunc() {
>     boolean runSeekIfWanted = false;
>     synchronized (queue) {
>         /* ... */
>         if (playPosition+1 <
> queue.size()) {
>             obj =
> queue.elementAt(playPosition+1);
>             if (obj
> instanceof Buffer) {
>                
> Buffer bb = (Buffer) obj;
>                 if
> (bb.data[0] == 'O' &&
>                
>     bb.data[1] == 'g' &&
>                
>     bb.data[2] == 'g' &&
>                
>     bb.data[3] == 'S') {
>                
>     runSeekIfWanted = true;
>                
>     Debug.info("[BUFFER] src setting
> permissionToSeek");
>                 }
>             }
>         }
>         /* ... */
>     }
>     /* ... main method code ... */
>     synchronized (permissionToSeek) {
>         if (runSeekIfWanted) {
>            
> Debug.info("[BUFFER] src granting permissionToSeek");
>            
> permissionToSeek.notifyAll();
>         }
>     }
> }
> 
> The problem is that the applet stops in the doSeek() method
> at the wait() 
> and waits forever. The whole applet does not respond
> anymore:
> 
> [INFO] [BUFFER] playPosition = 324, queue.size = 669
> Elements, size = 
> 1359872 bytePlayPosition = 678325 -> 64% full
> [INFO] [BUFFER] src granting permissionToSeek
> [INFO] [BUFFER] playPosition = 325, queue.size = 669
> Elements, size = 
> 1359872 bytePlayPosition = 679936 -> 64% full
> [INFO] [BUFFER] playPosition = 326, queue.size = 669
> Elements, size = 
> 1359872 bytePlayPosition = 679962 -> 64% full
> [INFO] [BUFFER] src setting permissionToSeek
> [INFO] doSeek(): aPos=0.05273069679849341
> [INFO] doSeek(): Element: [pipeline].sendEvent [Event]
> type: SEEK, format: 
> 5, position: 52730
> [INFO] Element: [pipeline] sendEvent(): [Event] type: SEEK,
> format: 5, 
> position: 52730
> [INFO] Element: [pipeline] doSeek(): [Event] type: SEEK,
> format: 5, 
> position: 52730
> [INFO] Element: [pipeline] doSendEvent(): [Event] type:
> SEEK, format: 5, 
> position: 52730 to Pad: httpsrc:src
> [INFO] [BUFFER] sink -> doSeek() waiting for
> permissionToSeek
> 
> If i do a wait for 5 seconds to get around this issue, i
> get discontinue 
> messages in the console, and i don't know why:
> 
> [INFO] theora: got discont
> [INFO] theora: got discont
> [INFO] theora: got discont
> 
> So my first question is:
> Is my whole thinking right with providing full pages to the
> demuxer? Is 
> that the right way for seeking in a Theora video stream?
> Why does my applet wait forever? I suppose the srcpad does
> not fire the 
> notifyAll() method but i have no idea why or where the
> srcpad gets stopped 
> or interrupted.
> 
> I will provide the whole code project in a .tar.bz here, so
> everyone can 
> use and experiment with my code changes:
> http://www.wapkamera.de/cortado.tar.bz
> 
> Hopefully someone is able to help me, because i just
> started in coding 
> Java and i don't know the Theora video stream structure in
> detail.
> 
> Best regards to everyone.
> 
> David
> _______________________________________________
> theora-dev mailing list
> theora-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/theora-dev
>