<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Hi,</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">I'm trying to use Speex for real time encoding in an iPhone project but I'm having latency problems. I appreciate any tips you may provide for speeding things up.</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">The recording callback is providing 1024 samples (mono, frequency: 44100 Hz, of type short) every 23 ms (1024.0 / 44100.0). I'm saving the samples in a buffer and use them in the encoding thread (set for mode: wide band - 16 Khz), but for every frame of 320 samples it takes ~40 ms, so soon the buffer runs out of space and it'll start missing samples. </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">I've tried lowering the quality and setting the recorder frequency to 16 Khz but didn't worked.</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Is there any way to optimize the encoding so it can keep up with the recording?</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica">Below you can find the code I'm using. Maybe you notice some obvious problems.</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">- (<span style="color: #be299d">void</span>)initSpeexEncoder {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #30595d"><span style="color: #000000"> </span>speex_bits_init<span style="color: #000000">(&</span><span style="color: #4d8186">speexBits</span><span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span>speexEncoderState<span style="color: #000000"> = </span><span style="color: #30595d">speex_encoder_init</span><span style="color: #000000">(&</span>speex_wb_mode<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// get the frame size (expressed in samples)</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span>speexFrameSize<span style="color: #000000"> = </span><span style="color: #2f2fd1">0</span><span style="color: #000000">;</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span><span style="color: #30595d">speex_encoder_ctl</span><span style="color: #000000">(</span>speexEncoderState<span style="color: #000000">, </span><span style="color: #79482e">SPEEX_GET_FRAME_SIZE</span><span style="color: #000000">, &</span>speexFrameSize<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// set quality parameter that controls the quality vs bit-rate tradeoff</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span><span style="color: #733ea4">UInt8</span><span style="color: #000000"> speexEncodingQuality = </span><span style="color: #2f2fd1">10</span><span style="color: #000000">; </span>// values from 0 to 10 (inclusively)</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #30595d">speex_encoder_ctl</span>(<span style="color: #4d8186">speexEncoderState</span>, <span style="color: #79482e">SPEEX_SET_QUALITY</span>, &speexEncodingQuality);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// alloc the buffer</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span>_compressedSpeexBufferSize<span style="color: #000000"> = </span><span style="color: #2f2fd1">1024</span><span style="color: #000000">;</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span>_compressedSpeexBuffer<span style="color: #000000"> = </span><span style="color: #401f7d">malloc</span><span style="color: #000000">(</span>_compressedSpeexBufferSize<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">}</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">- (<span style="color: #be299d">void</span>)freeSpeexEncoder {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #30595d"><span style="color: #000000"> </span>speex_bits_destroy<span style="color: #000000">(&</span><span style="color: #4d8186">speexBits</span><span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #30595d"><span style="color: #000000"> </span>speex_encoder_destroy<span style="color: #000000">(</span><span style="color: #4d8186">speexEncoderState</span><span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span><span style="color: #401f7d">free</span><span style="color: #000000">(</span>_compressedSpeexBuffer<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span>_compressedSpeexBufferSize<span style="color: #000000"> = </span><span style="color: #2f2fd1">0</span><span style="color: #000000">;</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">}</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><br></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">- (<span style="color: #be299d">void</span>)encodeThread {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// alloc buffer to hold the specified amount of short type samples</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">short</span> *input = <span style="color: #401f7d">malloc</span>(<span style="color: #4d8186">speexFrameSize</span> * <span style="color: #be299d">sizeof</span>(<span style="color: #be299d">short</span>));</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">while</span> (<span style="color: #4d8186">recording</span>) {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// proceed only if we have enough samples for a frame</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">if</span> (<span style="color: #4d8186">_toCompressBuffer</span>.<span style="color: #4d8186">size</span> >= <span style="color: #4d8186">speexFrameSize</span> * <span style="color: #be299d">sizeof</span>(<span style="color: #be299d">short</span>)) { </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// fill frame with samples</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>//memcpy(input, _toCompressBuffer.data, speexFrameSize * sizeof(short));</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// process one frame at a time</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #401f7d">NSLog</span>(<span style="color: #d42722">@"Start encode"</span>);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #30595d">speex_bits_reset</span>(&<span style="color: #4d8186">speexBits</span>);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>//speex_encode_int(speexEncoderState, input, &speexBits);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span><span style="color: #30595d">speex_encode_int</span><span style="color: #000000">(</span>speexEncoderState<span style="color: #000000">, </span>_toCompressBuffer<span style="color: #000000">.</span>data<span style="color: #000000">, &</span>speexBits<span style="color: #000000">); </span><span style="color: #008423">// without using extra buffer</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> </span><span style="color: #733ea4">UInt16</span><span style="color: #000000"> nbBytes = </span><span style="color: #30595d">speex_bits_write</span><span style="color: #000000">(&</span>speexBits<span style="color: #000000">, </span>_compressedSpeexBuffer<span style="color: #000000">, </span>_compressedSpeexBufferSize<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #d42722"><span style="color: #000000"><span class="Apple-tab-span" style="white-space:pre"> </span></span><span style="color: #401f7d">NSLog</span><span style="color: #000000">(</span>@"Ended encode"<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">static</span> <span style="color: #be299d">int</span> processedBytes = <span style="color: #2f2fd1">0</span>;</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> processedBytes += <span style="color: #4d8186">speexFrameSize</span> * <span style="color: #be299d">sizeof</span>(<span style="color: #be299d">short</span>);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #401f7d">NSLog</span>(<span style="color: #d42722">@"Processed bytes: %d"</span>, processedBytes);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">if</span> (nbBytes) {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #008423">// write to file</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// first write the header (2 bytes)</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// the header must contain the number of compressed bytes for a frame</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">static</span> <span style="color: #be299d">bool</span> once = <span style="color: #be299d">NO</span>;</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #be299d">if</span> (!once) {</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> [file <span style="color: #401f7d">writeData</span>: [<span style="color: #733ea4">NSData</span> <span style="color: #401f7d">dataWithBytesNoCopy</span>: &nbBytes <span style="color: #401f7d">length</span>: <span style="color: #be299d">sizeof</span>(nbBytes)]];</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> once = <span style="color: #be299d">YES</span>;</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> }</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// write encoded data</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> [file <span style="color: #401f7d">writeData</span>: [<span style="color: #733ea4">NSData</span> <span style="color: #401f7d">dataWithBytesNoCopy</span>: <span style="color: #4d8186">_compressedSpeexBuffer</span> <span style="color: #401f7d">length</span>: nbBytes]];</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008423"><span style="color: #000000"> </span>// remove from buffer -- one frame </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4d8186"><span style="color: #000000"> [</span>_toCompressBuffer<span style="color: #000000"> </span><span style="color: #30595d">removeFirstBytes</span><span style="color: #000000">: </span>speexFrameSize<span style="color: #000000"> * </span><span style="color: #be299d">sizeof</span><span style="color: #000000">(</span><span style="color: #be299d">short</span><span style="color: #000000">)];</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #d42722"><span style="color: #000000"> </span><span style="color: #401f7d">NSLog</span><span style="color: #000000">(</span>@"Stop encode frame"<span style="color: #000000">);</span></p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> }</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> }</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> }</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px"> </p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"> <span style="color: #401f7d">free</span>(input);</p>
<p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo">}</p><p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"><br></p><p style="margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo"><span class="Apple-style-span" style="font-family: arial; font-size: small; "></span></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">Thanks,</p><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Helvetica; ">
Alex Bumbu</p><p></p>