[Speex-dev] speex with jitter buffer

[nomprenom]

Hello, so I finally managed to make it work.
I answer here, maybe it could help someone in the future (I have also a
small question at the end too)

I don't use the speex_jitter_buffer.[ch] from the speex_client demo, I use
directly the jitter.[ch] api instead. (There are few bugs in the
implementation)

I checked all the steps in my flow by saving the pcm or the speex data and
listening to them before to continue.

One of the problems was the high cpu usage (85%+)  The process didn't have
time to read on the pcm device sometimes : [pcm_read(40ms) + speex
encoding] > 40 ms
So, only with the encoding part, I started to lose frames...
I found that passing the --enable-fixed-point to the compilation reduce the
cpu usage from 85% to 15% (cpuinfo says there is an FPU on the platform,
maybe it's not working well...)

I was not using mutex arround the put and get of the jitter buffer like it
is suggested in the doc. i fixed this too.

My question is about the *jitter_buffer_tick()* function : when to call it
on the player side ?
I understood from the doc that it is the timer tick of the jitter buffer.

- I tried without calling it too, there is no problem, why ?
- I call it only if jitter_buffer_get() was OK. I thought, this way, if
there is nothing in the jitter buffer, the timer will be "in pause" until
something new will arrive, Is it the right way ?

Thanks for your help.




On Sun, 24 Jan 2021 at 18:08, nomprenom <nomprenom at gmail.com> wrote:

> I did some changes:
> - instead of "pcm size" units, i'm back to ms units
> - I put in the buffer 40 ms directly instead of 2*20
> speex_jitter_put(&m_jitter, (char *)RFMessage->Data, SPEEX_SIZE_40MS,
> CPSeqNumber * 40);
>
> On the player thread, I call 2 times speex_jitter_get, the first one with
> a ts of 0, the 2nd one with 20ms timestamp offset - then I play the pcm
> buffer.
> I now wait for the first jitter_put before to do a jitter_get.
>
> So it seams to work now.
>
> The only problem is there are many warnings such as :
>
> warning: jitter buffer sees negative buffering, your code might be broken.
> Value is  -80
> warning: jitter buffer sees negative buffering, your code might be broken.
> Value is  -120
> warning: jitter buffer sees negative buffering, your code might be broken.
> Value is  -80
> warning: jitter buffer sees negative buffering, your code might be broken.
> Value is  -80
> etc..
>
> How to fix this ?
>
> thanks
>
> On Sun, 24 Jan 2021 at 16:59, nomprenom <nomprenom at gmail.com> wrote:
>
>> Hi,
>> I can't understand how to implement my encoder/decoder with the jitter
>> buffer. I hope someone can help me:
>> I have 2 threads.
>> Because I don't have yet the full environment (with the devices etc..),
>> - I'm reading from thread1,
>> - sending to the main thread,
>> - write to a jitter buff,
>> - from th2 : read from the jitter and play the data.
>> (Tomorrow, this is a device which will send the main thread the speex
>> data via RF - that's why I need to handle a jitter).
>>
>> thread 1:
>> 1) read 640 bytes from pcm (=40ms) (my settings are 8khz/16bits le)
>> 2) encode to 40 bytes of speex
>> 3) send to main thread 40 bytes of speex
>>
>> main thread:
>> 1) speex_jitter_put(20 first bytes of speex, counter * 640) // timestamp
>> = 0, 640, 1280, etc..
>> 2) speex_jitter_put(20 last bytes of speex, 320 + counter * 640) //
>> timestamp = 320, 960, 1600, etc..
>>
>> thread 2:
>> 1) speex_jitter_get(buffer, NULL)
>> 2)  speex_jitter_get(buffer + 320, timestamp=320)
>> 3) pcm_write(buffer, 640)
>>
>> Actually I can't hear anything. I tried many things without success. It's
>> the most "stable" flow I found without seeing many warnings from the api
>> (underrun, negative buffer, etc..)
>> - in the main thread, I think speex_jitter_put always puts 20 ms, right ?
>> (I initialized the buffer with an audio sampling rate of 8khz)
>> - in the main thread, I don't know if it's ok to put directly 40ms of
>> speex or if I need to put 2 * 20ms with a different timestamp for the 2nd
>> - I don't know too well what the timestamp parameter is : is it a
>> timestamp in ms ? I use now, after searching on the net, "pcm units" (320 =
>> 20 ms/ 640 = 40ms)
>> - in thread 2, I can't pcm_write 320 only else I see many "underrun"
>> errors. I thought, at the beginning, to simply read from the jitter and
>> play (320) and to do this 2 times.
>>
>> Any help will be appreciated.
>>
>> Thanks.
>>
>>
>>
>>
>> On Mon, 18 Jan 2021 at 21:30, nomprenom <nomprenom at gmail.com> wrote:
>>
>>> Thank you Tristan for your answer.
>>>
>>> For now I took jitter.c and .h from dsp and added them to the makefile
>>> of speex.
>>> I also added speex_jitter_buffer.c and h from the sample into the lib.
>>>
>>>
>>>
>>> On Mon, 18 Jan 2021 at 20:30, Tristan Matthews <tmatth at videolan.org>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> On Mon, Jan 18, 2021 at 12:29 PM nomprenom <nomprenom at gmail.com> wrote:
>>>> >
>>>> > Hi,
>>>> > I'd like to use the speex encode/decoder library with the speex
>>>> jitter buffer.
>>>> > I took, first, the project "speex" from
>>>> https://gitlab.xiph.org/xiph/speex
>>>> > There is a nice sample in the directory speex_client/ which is using
>>>> speex_jitter_buffer.h which itself is using <speex/speex_jitter.h>
>>>> > I couldn't find this include file in the project.
>>>> >
>>>> > I checked in the project speexDSP and found it.
>>>> > I thought, maybe, speexDsp is the new "speex" because it's deprecated.
>>>> >
>>>> > The problem is, now, with speexDsp I can't find the "speex.h" header
>>>> and can't find either the speex_decode_init(), speex_decoder_ctl() APIs.
>>>> > It's only in the sampledec.c in the doc/ directory of speexdsp
>>>> >
>>>> > How can I use the speex decoder/encoder with the speex_jitter api
>>>> together ?
>>>>
>>>> The Speex project has been split in 2, speex "the codec" (where you'll
>>>> find the decode_init and decoder_ctl API) and speexdsp "the DSP
>>>> library" where you'll find the jitter buffer API. The motivation was
>>>> that users want to be able to use purely the codec without the DSP
>>>> stuff and vice-versa.
>>>>
>>>> N.B. The most active/used project in speexdsp is the resampler, you
>>>> may find jitterbuffer implementations elsewhere that are a better fit
>>>> for your project.
>>>>
>>>> See:
>>>> https://gitlab.xiph.org/xiph/speex
>>>> https://gitlab.xiph.org/xiph/speexdsp
>>>>
>>>> Best,
>>>> Tristan
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xiph.org/pipermail/speex-dev/attachments/20210125/a5317178/attachment.html>