[opus] Opus FEC/loss concealment on multiple packets ?

Philip-Dylan Gleonec philip-dylan.gleonec at savoirfairelinux.com
Mon Jan 11 16:55:52 UTC 2021


Hello,

I'm currently working on enabling opus FEC/loss concealment in Jami (GNU 
VoIP application) through ffmpeg. We have integrated this patch 
(https://patchwork.ffmpeg.org/project/ffmpeg/patch/E1fIUGT-0003tH-Ql@pannekake.samfundet.no/) 
to enable FEC on the decoder.

I traced the codec execution while debugging and noticed that, when 
multiple consecutive packets are lost, the 'opus_multistream_decode' 
function is called with a frame size equal to the total number of 
samples lost, instead of just the size of the last packet lost. What 
happens in this case ? From my understanding, FEC can only be used when 
a single packet is missing. However, is it possible to do loss 
concealment on multiple consecutive packets this way ?

For the record, here is below a commented trace of ffmpeg using opus:

     [sdp @ 0x7f72a4001000] max delay reached. need to consume packet
     [sdp @ 0x7f72a4001000] RTP: missed 1 packets
     [libopus @ 0x7f72a4006640] FEC: calculating audio mismatch : 251472 
| 252432
     [libopus @ 0x7f72a4006640] FEC: calculating nb_lost_samples : 960
     [libopus @ 0x7f72a4006640] FEC: calculated lost samples :  960
     [libopus @ 0x7f72a4006640] FEC: lost 960 samples
     // Here, opus_multistream_decode is called with frame_size = 960, 
as expected
     [libopus @ 0x7f72a4006640] FEC: Recovered 960 samples
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [sdp @ 0x7f72a4001000] max delay reached. need to consume packet
     [sdp @ 0x7f72a4001000] RTP: missed 2 packets
     [libopus @ 0x7f72a4006640] FEC: calculating audio mismatch : 259152 
| 261072
     [libopus @ 0x7f72a4006640] FEC: calculating nb_lost_samples : 1920
     [libopus @ 0x7f72a4006640] FEC: calculated lost samples : 1920
     [libopus @ 0x7f72a4006640] FEC: lost 1920 samples
     // Here, opus_multistream_decode is called with frame_size = 1920. 
This seems functional and seems to work, but I'm not sure how/why
     [libopus @ 0x7f72a4006640] FEC: Recovered 1920 samples <- Is it 
possible to recover 1920 samples through loss concealment when a packet 
is 960 samples ?
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected
     [libopus @ 0x7f72a4006640] FEC: no mismatch detected

Thanks a lot for your time and assistance (and for the codec, of course).

Best regards,

Philip-Dylan Gleonec

-- 
Logo Savoir-faire Linux inc.
Philip-Dylan Gleonec
Consultant en logiciel libre / Ingénieur systèmes embarqués | Rennes, Fr
Bureau
(+33) 9 72 46 89 80 <tel:+14185257354>
Logo Jami sfl-pgleonec
new-year
Site web <https://www.savoirfairelinux.com> | Blog 
<https://blog.savoirfairelinux.com/fr-ca/> | Jami <https://jami.net>
Suivez-nous Facebook <https://www.facebook.com/savoirfairelinux/> 
LinkedIn <https://www.linkedin.com/company/savoir-faire-linux> Twitter 
<https://twitter.com/sflinux> gitLab <https://gitlab.savoirfairelinux.com>
Messages de confidentialité : Ce courriel (de même que les fichiers 
joints) est strictement réservé à l'usage de la personne ou de l'entité 
à qui il est adressé et peut contenir de l'information privilégiée et 
confidentielle. Toute divulgation, distribution ou copie de ce courriel 
est strictement prohibée. Si vous avez reçu ce courriel par erreur, 
veuillez nous en aviser sur-le-champ, détruire toutes les copies et le 
supprimer de votre système informatique.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xiph.org/pipermail/opus/attachments/20210111/d7306a9e/attachment.html>


More information about the opus mailing list