<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hello, guys! </span><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I'm a novice at programming, learning about the speex and its applications now. Specifically, my current goal is to encode microphone input on iOS device in real-time, saving result to a file frame-by-frame. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">So far, I've managed to:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div>
<div style="font-family:arial,sans-serif;font-size:13px">1) compile speex static library (it works, at least for iPhone/iPad, but not on iOS simulator; this is fine for now);</div><div style="font-family:arial,sans-serif;font-size:13px">
2) encode pcm data from microphone into 'something' which resembles speex (in fact it expected to be speex file, but not actually; see below for details).</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">To check validity of the result file, I use speexdec script, as following:</div><div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"><br>
</span></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"> ...$ speexdec encodingResultSpeexFile decodedSpeexFile<br></font></div><div style="font-family:arial,sans-serif;font-size:13px">
<font face="courier new, monospace"> This</font><span style="font-family:'courier new',monospace"> doesn't look like a Speex file</span></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div>
<div style="font-family:arial,sans-serif;font-size:13px">So I assume that the file, which my program creates, is invalid for some reason. Is some particular kind of header required? What should it be (ogg, rtp or something else) and how should I add it?</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Could you guide me in right direction, please?</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">---</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">To give you fuller picture, I'll attach single audio input buffer along with piece of the code to encode bytes from such buffers frame-by-frame:</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">a) Audio input buffer (abridged a bit):</div><br style="font-family:arial,sans-serif;font-size:13px">
<font face="courier new, monospace" style="font-size:13px"> CMSampleBuffer 0x6ead20 retainCount: 1 allocator: 0x33bdf213<br> invalid = NO<br> ...</font><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"> formatDescription = <CMAudioFormatDescription 0x6e84b0 [0x3f7cd650]> {<br>
mediaType:'soun' <br> mediaSubType:'lpcm' <br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">mediaSpecific: {<br> ASBD: {<br>
</font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">mSampleRate: 44100.000000 <br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">mFormatID: 'lpcm' <br>
</font><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mFormatFlags: 0xc </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mBytesPerPacket: 2 </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mFramesPerPacket: 1 </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mBytesPerFrame: 2 </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mChannelsPerFrame: 1 </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">mBitsPerChannel: 16 </span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">} ...</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">} </span><span style="font-family:'courier new',monospace">...</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">}<br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">sbufToTrackReadiness = 0x0<br>
</font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">numSamples = 375<br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">sampleTimingArray[1] = {<br>
</font><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">{PTS = {82626949766094/1000000000 = 82626.950, rounded}, DTS = {INVALID}, duration = {1/44100 = 0.000}},</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">}<br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">sampleSizeArray[1] = {<br>
</font><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace">sampleSize = 2,</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">}<br></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace">dataBuffer = 0x6e7a40</font><div>
<br><div><br></div><div>b) Actual data which is kept in aforementioned buffer (for the sake of completeness):</div><div><br></div><div><font face="courier new, monospace">0f001400 0b000d00 04000a00 f7ff1300 0a000300 f9ff1c00 1b001c00 f9ffc0ff</font></div>
<div><font face="courier new, monospace">bfffadff b7ff86ff 7cff5fff 63ff67ff 84ffb5ff bbffd4ff e4ff1500 1c001000</font></div><div><font face="courier new, monospace">04001400 39002500 27005000 44002000 e7ffe5ff c7ffd7ff bbffcbff d1ffc8ff</font></div>
<div><font face="courier new, monospace">c7ffc4ff e8fff6ff 2f003a00 5c005400 66006600 71006700 5e006000 4a003300</font></div><div><font face="courier new, monospace">0a002300 4a004c00 47001c00 fbfff6ff 0a00b1ff 8dff85ff 9bff87ff 93ff7fff</font></div>
<div><font face="courier new, monospace">84ff74ff 6fff75ff 91ffa2ff c1ff9aff 86ff7eff 7aff59ff 53ff6aff 71ffb3ff </font></div><div><font face="courier new, monospace">c5ffe1ff 0900faff e9fff3ff 1d003100 34001400 0600fcff d5ffecff 0d002200 </font></div>
<div><font face="courier new, monospace">60006500 6f006c00 61002b00 0700d8ff ccffcbff baff7aff 65ff61ff 94ffccff </font></div><div><font face="courier new, monospace">0e002800 56006400 9800ad00 ea00c700 cb00ab00 af00bc00 ab00be00 a700ad00 </font></div>
<div><font face="courier new, monospace">76006d00 34003100 0c00e7ff c2ffcbff c6ffcdff acffadff a2ff88ff 86ff4aff </font></div><div><font face="courier new, monospace">5eff19ff 3eff26ff 66ff8bff bcffe9ff d8ff3300 1e006a00 7600a700 7e009f00 </font></div>
<div><font face="courier new, monospace">b2000801 15011501 d100d900 92008f00 86007b00 62004900 6d006e00 52002f00 </font></div><div><font face="courier new, monospace">f0ff0000 b6ffebff 96ffb0ff 51ff76ff 45ff61ff 18ff2aff 12ff29ff 2fff70ff </font></div>
<div><font face="courier new, monospace">6aff9fff b6ffdeff c3ffd9ff a8ffc0ff b7fffbff 00001e00 c4ffe4ff dbffe1ff </font></div><div><font face="courier new, monospace">d5ff0300 0e001c00 03000800 0b001f00 d9ff1a00 d7ffe3ff bdff0300 f6ff1b00 </font></div>
<div><font face="courier new, monospace">c9ffecff 06005e00 50009100 60005f00 05002000 ffff1200 dfffecff d9ff0100 </font></div><div><font face="courier new, monospace">ccfff3ff b2ffceff 9bffd4ff c5ffe2ff b1ffecff e0ff0000 e6fffbff e2ff1c00 </font></div>
<div><font face="courier new, monospace">11002a00 11003900 4c009800 8a00b100 63008800 66008d00 65008600 69006700 </font></div><div><font face="courier new, monospace">5e006200 74006f00 5f006700 62006700 4f007100 75007800 7a006f00 65005200 </font></div>
<div><font face="courier new, monospace">1f00d7ff caffb8ff aaffb4ff 9aff6eff 61ff6bff 5dff6cff 75ff75ff 77ff95ff </font></div><div><font face="courier new, monospace">adffd3ff d9ffd7ff f6ff0700 1a002f00 40001e00 1b000e00 1a001400 25002800 </font></div>
<div><font face="courier new, monospace">38002900 1d002000 0000e6ff effff2ff 0100dbff daffb9ff efffe6ff e4ffd5ff </font></div><div><font face="courier new, monospace">dbffb5ff 89ff8dff a6ffc1ff e3ffd5ff f7ff0500 0e00e9ff 0f001200 2f002c00 </font></div>
<div><font face="courier new, monospace">37004e00 46002500 d3ffbbff b0ff</font></div><div><br></div><div>c) The code (here is the piece to deal with single input buffer, which example is give above; so it's same for all input buffers):</div>
<font face="courier new, monospace"><br> int writtenBytesNumber;<br> SpeexBits bits;<br> int samplesPerFrame;</font></div><div><span style="font-family:'courier new',monospace"> int quality = 6;</span><font face="courier new, monospace"><br>
</font></div><div><span style="font-family:'courier new',monospace"> char compressedBits[200];</span><span style="font-family:'courier new',monospace"><br></span></div><div><font face="courier new, monospace"><br>
speex_bits_init(&bits);</font></div><div><span style="font-family:'courier new',monospace"> memset(compressedBits, 0, 200);</span></div><div><font face="courier new, monospace"><br> void *encoderState = speex_encoder_init(&speex_wb_mode); <font color="#999999">// setting wideband mode</font><br>
speex_encoder_ctl(encoderState, SPEEX_GET_FRAME_SIZE, &samplesPerFrame); </font><span style="color:rgb(153,153,153);font-family:'courier new',monospace">// 320 samples/frame</span><font face="courier new, monospace"><br>
</font></div><div><span style="font-family:'courier new',monospace"> speex_encoder_ctl(encoderState, SPEEX_SET_QUALITY, &quality);</span><div><p><font face="courier new, monospace"> CMItemCount numberOfSamples = CMSampleBufferGetNumSamples(sampleBuffer); <font color="#999999">// 375 samples</font><br>
size_t sampleSize = CMSampleBufferGetSampleSize(sampleBuffer, 0); <font color="#999999">// 2 bytes</font><br><br> CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBuffer); <font color="#999999">// see attachment (b) above</font><br>
size_t blockBufferDataLength = CMBlockBufferGetDataLength(blockBufferRef);<font color="#999999"> // 750 bytes</font><br><br> NSMutableData *allSamplesMutableData = [NSMutableData dataWithLength:blockBufferDataLength];<br>
CMBlockBufferCopyDataBytes(blockBufferRef, 0, blockBufferDataLength, allSamplesMutableData.mutableBytes);<font color="#999999"> // copying bytes to object</font><br><br> int nextSampleStartPosition = 0;<br><br> for (int i = 0; i < numberOfSamples; i++) {<br>
<br> NSData *singleSampleData = [NSData dataWithBytes:[[allSamplesMutableData subdataWithRange:NSMakeRange(</font><span style="font-family:'courier new',monospace">nextSampleStartPosition</span><font face="courier new, monospace">, sampleSize)] bytes] length:sampleSize];<br>
<br> </font><span style="font-family:'courier new',monospace">nextSampleStartPosition</span><font face="courier new, monospace"> += sampleSize;<br> [_intermediateSampleBuffer addObject:singleSampleData];<font color="#e06666"> </font><font color="#999999">// we put samples here until it's enough for the frame</font><br>
<br> if (_intermediateSampleBuffer.count == samplesPerFrame) {<br><br> speex_bits_reset(&bits);<br> short samplesArray[samplesPerFrame]; <font color="#999999">// c-array which will get samples from </font></font><span style="font-family:'courier new',monospace"><font color="#999999">_intermediateSampleBuffer</font></span><font face="courier new, monospace"><br>
<br> for (int j = 0; j < samplesPerFrame; j++) {<br> samplesArray[j] = (short)[(NSData *)[_intermediateSampleBuffer objectAtIndex:j] bytes];<br> }<br><br> [_intermediateSampleBuffer removeAllObjects];<font color="#999999"> // flushing </font></font><span style="font-family:'courier new',monospace"><font color="#999999">_intermediateSampleBuffer to make it ready for next samples</font></span><font face="courier new, monospace"><br>
<br> speex_encode_int(encoderState, samplesArray, &bits);<br> writtenBytesNumber = speex_bits_write(&bits, compressedBits, samplesPerFrame); <font color="#999999">// usually 52 bytes for wideband and quality=6</font><br>
<br> fwrite(&writtenBytesNumber, sizeof(int), 1, _speexFile);<font color="#999999"> // following the example in sampleenc.c we put size of compressedBits first</font><br></font><span style="font-family:'courier new',monospace"> fwrite(compressedBits, sizeof(char), writtenBytesNumber, _speexFile);<font color="#999999"> // putting encoded bits themselves</font></span><font face="courier new, monospace"><br>
<br> memset(compressedBits, 0, 200);<br> }<br> }<br><br> speex_bits_destroy(&bits);<br> speex_encoder_destroy(encoderState);</font></p><p>p.s. I understand that code is far from optimal, but it's not final version. Though advice regarding code (if any) quality is also appreciated.</p>
</div></div></div></div>