[opus] How to get started with opus and C++/JavaScript/WebAsm
Andreas Stöckel
astoecke at uwaterloo.ca
Thu Dec 3 18:20:28 UTC 2020
Hi Juan,
raw Opus packets can be thought of the minimal unit processed by the
Opus encoder/decoder. They do contain all side-channel information
necessary to decode a chunk of audio and potentially error correction
data, but nothing more.
Encapsulating Opus packets in an Ogg stream is simply a standardised
way for adding more metadata to the stream, such as an arbitrary
metadata header and timestamps for each packet. Also, Ogg can be used
to multiplex multiple streams.
Regarding your previous questions, I had similar questions a while
back [1]. I wanted to (independently) pre-encode small chunks of audio
data and splice them together in the browser for playback. This turned
out to be a little bit more complicated.
Long story short, for proper gap-less concatenation of *independently*
encoded Opus chunks you need to add a lead-in/lead-out region to each
chunk and interpolate between the chunk boundaries. See my C++/JS code
here:
https://github.com/astoeckel/opus_gapless
This code has no dependencies except for libopus and ffmpeg being
installed as an executable. It also comes with its own Ogg muxer. I'm
using WebAudio for decoding the opus packets on the client-side.
Note that using DASH as suggested by Emily is the correct solution if
you can pre-encode the entire stream up-front---which wasn't an option
in my case, since I generate the Opus chunks on-demand; i.e., the user
requests the chunk for one hour thirty minutes into the stream, and I
can neither pre-encode the entire stream, nor live with the latency of
encoding the entire file first once the request arrives.
I hope this helps!
All the best,
Andreas
[1] http://lists.xiph.org/pipermail/opus/2017-November/004031.html
On 03.12.20 12:56, Juan Gerardo Ruelas Jr wrote:
> Sorry, forgot to cc the opus mailing list...
>
> What is the difference between Ogg-encapsulated Opus packets vs
> Raw packets?
>
>
>
>
>
> How does one generate one or the other programmatically?
>
>
> - Juan
>
> On Thu, Dec 3, 2020 at 9:30 AM Juan Gerardo Ruelas Jr
> <jruel006 at ucr.edu <mailto:jruel006 at ucr.edu>> wrote:
>
> What is the difference between Ogg-encapsulated Opus packets vs
> Raw packets?
>
> How does one generate one or the other programmatically?
>
> - Juan
>
> On Thu, Dec 3, 2020 at 2:10 AM Emily Bowman
> <silverbacknet at gmail.com <mailto:silverbacknet at gmail.com>> wrote:
>
> Small correction, I just realized while re-reading this that
> where I said raw Opus packets, I should have said
> Ogg-encapsulated Opus packets, which is at least a much
> simpler format than MP4. Raw packets are useful in some
> contexts but more awkward to work with.
>
> - Em
>
> On Thu, Dec 3, 2020 at 12:15 AM Emily Bowman
> <silverbacknet at gmail.com <mailto:silverbacknet at gmail.com>> wrote:
>
> Since everything goes back to the C library anyway (or a
> variant of it, if you use, say, emscripten), you're free
> to use any language you want to interface with the API.
> Third-parties have created wrappers for a number of other
> languages, like Python, since C is a simple and universal
> binding.
>
> While most web streams use chunking via DASH these days,
> which involves encapsulating packets in ISO-BMFF (MP4),
> you're perfectly free to just send raw Opus packets sliced
> up and the decoder will handle them gracefully. Even if
> packets are lost and you have no error correction, the
> decoding will resynchronize within one packet, and sound a
> lot less ugly than the chirps you get with many other
> codecs. That was a specific design goal of the format. The
> format on disk is exactly identical to the streamed
> format, so just concatenating the received packets is a
> valid and simple way to save to disk.
>
> For a basic example of using the API from C/C++, check out
> https://opus-codec.org/docs/opus_api-1.3.1/group__opus__encoder.html
> <https://opus-codec.org/docs/opus_api-1.3.1/group__opus__encoder.html>
> . There's a bit of boilerplate to initialize it, but not
> too much, and from there you can broadcast the encoded
> packets however you like. DASH is much more complex,
> though if you have a library for your platforms of choice
> it'll either support Opus natively or be fairly simple to
> retrofit in.
>
> -Em
>
> On Wed, Dec 2, 2020 at 5:36 PM Juan Gerardo Ruelas Jr
> <jruel006 at ucr.edu <mailto:jruel006 at ucr.edu>> wrote:
>
> I am interested in using the opus codec to stream
> audio files to a mobile app.
>
> I have a couple of questions about how to get started?
>
> 1. How do I generate opus files programmatically with
> C++? What inputs does it need? Is there a special
> file format because all the examples I see only
> have .wav files as inputs.
> 2. Does opus support chunking? For example, breaking
> a 24 hour long audio into a series of sequential
> smaller files (say 5 seconds long) to stream them
> down to a user?
> 3. Does opus support stitching together opus files?
> Say, after a user receives the 5 second long
> stream, can I programmatically stitch the files
> together as they come so as to save the entire 24
> hour long audio file for offline use at a later time?
> 4. How do you stream these files programmatically?
> What is the general approach. Can you do it with
> node.js? C++? Python? What is most efficient?
>
> p.s. (when I say audio files I generally mean just
> voice audio like in audio books)
>
> Thank you for your help, I am honestly confused as to
> how to get started.
>
> - Juan Gerardo Ruelas Jr.
> _______________________________________________
> opus mailing list
> opus at xiph.org <mailto:opus at xiph.org>
> http://lists.xiph.org/mailman/listinfo/opus
> <http://lists.xiph.org/mailman/listinfo/opus>
>
>
> _______________________________________________
> opus mailing list
> opus at xiph.org
> http://lists.xiph.org/mailman/listinfo/opus
>
More information about the opus
mailing list