[Vorbis-dev] Fwd: Object Audio - Misusing Vorbis?

[xiphmont at xiph.org]

Oops, had meant to reply on-list as well.

Monty

---------- Forwarded message ----------
From:  <xiphmont at xiph.org>
Date: Tue, Nov 11, 2014 at 12:57 AM
Subject: Re: [Vorbis-dev] Object Audio - Misusing Vorbis?
To: Richard Furse <richard at muse440.com>


Hello Richard,

On Fri, Oct 31, 2014 at 3:50 AM, Richard Furse <richard at muse440.com> wrote:
> Hi there
>
> I'm trying to use Vorbis in a slightly unusual way, and I'm a little unclear
> if I'm going to be able to do what I want to do nicely, or at all. Many
> apologies if I'm on the wrong list, or if this is a FAQ, or off-topic. I'd
> be very happy to be re-routed!

If Vorbis can't do what you need, perhaps Opus can. In any case, let's
get down to it...

> I'm doing a little experimental work on a block-based bitstream format for
> object audio streaming. This format allows any number of audio objects to
> start or stop on block boundaries, and each audio object might be mono or
> multichannel. Blocks may vary in length within the stream, from 1 to 65535
> samples, and each block is encoded as a single "chunk" of data containing
> all audio for all audio objects for that block of time. There's a bunch of
> other data in the chunk, for instance spatalisation information. The
> bitstream format is already (experimentally) operational using a very simple
> codec that is lossless for all this data, including the audio.

...yup, already thinking Opus might be a better fit because it can
transition without new headers...

> What I was hoping to do next is a lossy version, and thought Vorbis might be
> suitable for the underlying audio coding, but I'm struggling a little to
> understand exactly what I could/should be using. I thought initially I could
> use just the vorbis/vorbisenc calls to encode the audio for each audio
> object each as an embedded bitstream, and then write the bytes returned in
> ogg_packet structures from vorbis_analysis_headerout() and vorbis_analysis
> () within my packets, but I'm not sure if this is supported behaviour - and
> for decoding, I'm not sure how I should populate the reconstructed
> ogg_packet structures correctly.

What you would likely do in this case is write self-contained Ogg
bitstreams.  The code will be happy to do it.  The same would be true
of Opus, though it's easier to omit the high-level container in the
case of Opus, as you can predict packet size much more easily.


> I could use the whole Ogg binding, which would clearly take me back into
> standard API space. Then, for each block of time, my bitstream chunk would
> contain, for each audio object, a number of Ogg packets, which would in turn
> contain Vorbis data. There's clearly quite a bit of redundancy here.

Right.  Nothing to add, you understand this well already.

> One of the things I'm unclear about is whether or not I'm guaranteed to get
> the right data into my chunk if I do this. If I submit, say, 4800 samples of
> audio, and call all the relevant vorbis/ogg calls to fully flush the
> resulting bytes, am I guaranteed to get exactly the ogg/vorbis packets that
> are required to resynthesize those 4800 samples of audio?

Yes. You could do it without 'complete' bitstream construction, though
that would be the most strightforward way.

> During
> resynthesis, no ogg/vorbis packets from the previous or next block/chunk
> will be available (except any internal state the vorbis decoders might be
> keeping).

Vorbis packets are lapped, however the encoder also goes to some
trouble to minimize dicontinuities at the boundaries between
self-contained streams, as individual streams are not lapped, but you
often want seamless playback boundaries regardless.  The seamlessness
is not perfect, but it's pretty good.

> Another thing I'm not too sure about is how I should handle lost chunks.
> Because the ogg/vorbis streams for new audio objects can start up at any
> point in time, it's possible that the new ogg/vorbis header packets are
> within the chunk that is lost. Is there then a way I can recover this audio
> object?

So long as you have the header matching a chunk, you can begin
playback anywhere.  If the header requirement is an issue, Opus will
get around it (as it does not have Vorbis-style headers and can
transition between many playback states without restarting).

>I periodically send "sync" packets which are designed to allow
> reconstruction of all state, so some more header data could go in there, but
> I'm not too sure what should be used. Maybe all the ogg packets relating to
> vorbis_analysis_headerout? Can I reset all the decoders with this header
> data and then start feeding the (much later) analysis packets?

Yes.

> Is there a
> lighter way to do this (I don't really want to be resending all the Ogg
> headers every second)?

I don't blame you :-)

For any given Vorbis setup (number of channels, rate, quality), the
headers for that stream will be the same for our encoder, and you can
count on that.  If you're always using the same encoder, you can just
assume the given headers.
Or you could stick the headres that will be used somewhere once if you
want to proof against encoder updates.

And, again, Opus sidesteps the issue by not having the heavyweight
headers to begin with.  But it sounds like what you want can indeed be
done with Vorbis.

> PS, some audio objects will be carrying HOA data, so will have potentially
> LOTS of channels (64?). How many channels can Vorbis handle in coupled mode
> these days?

Specialized coupling goes to 5/7 channels.  You can do pairwise
coupling up to 254 channels and independent up to 255.  You could also
design a custom coupling if the project warrants it; that's in the
headers.  That's not quite as scary as it sounds.

Monty