[Theora-dev] Problems with Theora DirectShow filters

[illiminable]

----- Original Message ----- 
From: "Robert Huitl" <theora-dev at huitl.de>
To: <theora-dev at xiph.org>
Sent: Friday, September 17, 2004 1:47 AM
Subject: Re: [Theora-dev] Problems with Theora DirectShow filters

>First of all, thank you very much for the help. I could achieve some good
>progress.

No problem... thanks for such detailed feedback, it's invaluable to me 
improving it.

>On Wednesday 15 September 2004 10:40, Illiminable wrote:
>> You should build a graph something like this if you want to capture...
>>
>> Audio Input Source --> Speex Encoder -->  }
>>                                                                      } 
>> Ogg
>> Mux Filter
>> Video Input Source --> Theora Encoder -->  }
>
>I built my application to use a filter graph like that to capture A/V:
>http://www.huitl.de/avcapenc.png
>
>
>The problem is, I cannot play the recorded file. A bug in my application is
>unlikely, because the same symptoms come up when I use GraphBuilder to
>capture A/V.

Hmmm.... it's hard to say exactly what is causing this... could you perhaps 
send me the created file ?

If it's large just break it arbitrarily at about the 3-5 meg mark. If 
mailing attachments that large is a problem, send me a mail off-list and 
i'll arrange somewhere you can upload it to.

>This is how I play the stream: http://www.huitl.de/avcapdec.png

>OGG Mux (Speex+Theora):
>BSPlayer: black window, doesn't advance
>- theora decoder and speex decoder => black video window, doesn't advance,
>timeout when trying to stop (cancel => stopped)

This usually means a thread deadlock... i'd have to check out the file to be 
certain... one thing i did do in this release was reduce the number of 
buffers for each stream... so perhaps i made it a bit small... the problem 
arises because if there are a lot of one stream in a row, then the graph 
will block waiting for the first sample of each stream to reach a renderer, 
which it never does.

>- only speex decoder, theora demux pin unattached => plays, sound is 
>correct,
>crashes at end

This could be lots of things... but it's most likely it tries to do 
something on that unatached pin and doesn't check whether some variable is 
initialised.

>- only speex decoder, theora demux pin attached to null renderer => doesn't
>advance, timeout when trying to stop (cancel => stopped)

Hmmm... this sounds liek deadlock too... one thing thats better than the 
null renderer for testing is the Dump Filter (you have to build it yourself 
from the DX9 SDK)... it's non-blocking unlike the null renderer... but it 
does dump the output to the disk.

>- only theora decoder, speex demux pin unattached => video window opens,
>graphedit crashes

Probably same as above... i probably don't check something, and just assume 
the pins will be connected.

>- only theora decoder, speex demux pin attached to null renderer => black
>video window, doesn't advance, timeout when trying to stop (cancel =>
>stopped)

Deadlock again ! :)

>As I didn't know whether it's okay to have a demux pin unattached, I 
>attached
>the null renderer. It looks as though it's not supposed to be used this
>way :)

Yeah... the null renderer i've found isn't good for testing... try the dump 
filter.

>OGG Mux (only theora stream):
>- BSPlayer shows first frame, doesn't advance, each time play is clicked, 
>the
>image gets more distorted (screenshot below)
>- graphedit: ogg demux => theora decode => video renderer works, although 
>much
>too fast. timeout when video stops (cancel => stopped)

Sounds like the samples aren't timestamped... either coming from the capture 
source or after passing through the ffdshow filter.

>OGG Mux (only speex stream):
>- plays with BSPlayer, sound is correct
>- graphedit: ogg demux => speex decode => default directsound device: 
>works,
>sound is correct

At least something works ! :)

>Is this what "Cannot handle any chained multiplexed files or streams" on

This mainly relates to things like icecast streams.

>But then there's still the playback which is too fast. Could this be 
>related
>to "When transcoding theora, if a media decoder (WMV usually) doesn't
>advertise its framerate, output may be time distorted"?

Very likely.

>This is the timeout that appears quite often: 
>http://www.huitl.de/timeout.png

Yeah this is from deadlock.

>Distorted video in BSPlayer: http://www.huitl.de/bsplayer_theoraonly.jpg

It could be two things... either the ffdshow filter is making some transform 
incorrectly, or it's just the result of showing interframes without their 
keyframes (if this is from just stop/play/stop/play)

>The sample grabber filter looks suitable for what I want to do. I didn't 
>test
>it, this is the next step, but it should allow me to extract data at the 
>end
>of the filter chains (can I feed it to the chain's beginning, too?)

Yes... but you'll need a custom filter, it needs to output the same type as 
the encoder inputs.

>audio capture => encode => sample grabber
>video capture => encode => sample grabber
>and
>sample grabber (/feeder?) => decode => audio out
>sample grabber (/feeder?) => decode => video out

Yeah... this is how i would expect an application that transports over a 
network would do it... you'll need to write the feeder specific to the 
application... that's another one of those things i'll get around to 
eventually, at least for the generic case.

>As these would be two separate filter chains (in the same graph), is there
>some facility for A/V-sync? If I could extract packets this way, I would 
>need
>timestamps to synchronize both streams during playback.

In a directshow graph... there is only one refernce clock... any filter 
chains within the graph will all use the same refernce clock (by default the 
clock on the audio renderer... that's the little yellow clock you see on the 
audio renderer filter)... so this shouldn't be a problem.

>What is the difference between packet and page level?

This is more an ogg thing than a directshow thing... directshow doesn't 
really make the distinction... it calls everything "samples"... it's up to 
the filters to define what a sample means to it.

But in ogg packets and bundled together into pages (all of the same 
stream)... and then the pages from all streams are interleaved. Only pages 
have timestamps in ogg... and it's only an end timestamp.

What comes out of the demux pins are packets... same as what comes out of 
the encoder pins... so if you are transport packets over the network... by 
grabbing them as they come out of the encoder and then at the other end 
feeding them back into the decoder. Transporting pages over the network... 
would be "easier" but probably not very effective as they are probably too 
granular for this type of application.

Hope that's of some help... if you have any more questions, just ask.

Also one other thing...    what is the resolution of the video coming out of 
the capture device ?

Cheers,

Zen.