[vorbis] Re: Website for Ogg/Vorbis support for MPEG4IP

[Peter Maersk-Moller]

Hi Christian

For others, this is mostly about IT-politics and perspectives. Maybe
you prefer doing other things than actually reading this long email.

> Hi Peter,
> i really dont want to start a flamewar here or such, believe me, in fact i
> guess every contribution to opensource community is precious,

Nahh, it's ok, I just didn't imagine that the first response to the web
site (and its work) would generate such rather harsh critique. I guess when
one stick out one must be prepared for an occasional slap in the face.

Anyway, let me try first to explain the reasoning behind my work. Then
maybe you can be convinced that it is not all that bad.

My main interest is in streaming audio and video. Analyzing that
interest I'm seeing a clear trend where Microsoft is rapidly
gaining supremacy at the expense of Real in an all too familiar
way resembling the browser war; and Real, not unusual for such a
large company facing such a threat, is doing too little and
probably too late.

Now where is OSS in all this ? The fact is that OSS wrt. streaming
is immature for mainstream streaming and basically not there yet.
What is worse, OSS may never get there sufficiently because
it is being cornered and locked by software patents. And
should it actually get there, it'll find a world dominated
by a single supplier. A battle then will be much harder so why not
try to level the fight a bit by starting earlier ? At least that
is the strategy I'm supporting.

Now at the moment, or rather when I started out on what has become a
journey, the only real serious contender to Microsoft proprietary
streaming I could find was MPEG-4 based streaming or to be more precise,
ISMA compliant streaming.

Now ISMA is a subset of MPEG-4 streaming and you could say that war
creates strange bedfellows, but then you may be failing to see
what ISMA really is. From a strategic point of view, ISMA is
the most serious threat to Microsoft streaming. Some of the
reasons are

  a) Its here (VERY important)
  b) It's a defined public available standard based on
     the public available MPEG-4 standard.
  c) It has support for most of the important companies within the
     streaming business with the exception of Microsoft and to
     some extend Real, though the latter will change I'm sure.

The bad side is of course that many aspects of MPEG-4 is covered by
patents that may be enforceable in some countries (though not all).

Now from a technical point of view ISMA may be quite interesting too.
If you try to break down ISMA to see what it actually is, then
it is a couple of codecs and a streaming framework. Now the codecs
may be covered by patents that may be enforceable in some countries,
but the streaming framework consists mostly of IETF RFCs and as
such probably not covered by royalty claiming patents.

Now if it wasn't for the original royalty claims for MPEG-4, ISMA
could for reasons listed above, be the most powerful tool if not
the only effective tool for fighting Microsoft supremacy in the
streaming AV market. However ISMA was burden with an unrealistic
high entry cost in terms of royalties needed to be paid to
our common *friend* MPEGLA. Now luckily major vendors forced
MPEGLA to come up with a more realistic and better royalty scheme.
Please note that I'm not writing if this is good, just that it is
better than the original one. I'm also not saying it will last.

Now the new royalty scheme for using ISMA for streaming may exempt you
from having to pay royalty (under certain circumstances) for streaming video.
The reason why I write ISMA rather than MPEG-4 is because the new
royalty scheme only covers a subset of MPEG-4 (or more or less  what ISMA
covers). This sounds good until one realize that it doesn't exempt you
from having to pay royalty for streaming audio - and most video
streaming is dead boring without audio.

My little hack enables you to stream Vorbis audio
or MPEG-4 video combined with Vorbis audio. These two things
are very appealing to many due to obvious reasons.

BTW, when I write streaming, I mean either RTP and RTSP based
streaming or Microsoft streaming based on Microsoft's proprietary
closed methods. For me HTTP based streaming is evil and should be
avoided, but this discussion belongs in another forum.

What I have done (and probably as the first) is that I have
written and released some OSS code that among other things
enables you (in combination with video if needed) to

  a) live encode audio using Vorbis
  b) stream it using RTP/RTSP and unicast or multicast
  c) play the stream.

To me and others, this is very important because we now
have a full featured implementation of streaming Vorbis
using RTP/RTSP and unicast/multicast. This was by no
means easy and I believe it is very important to the Xiph/Vorbis
community to be able to show an OSS perhaps reference
implementation of Vorbis streaming based on RTP/RTSP.

Anyway, to skip to conclusion before commenting your individual
remarks, then MPEG-4 streaming as ISMA defines it, is actually
an open royalty free streaming framework plus codecs claimed to
be covered by patents requiring royalties. If you replace
the codecs with true OSS alternatives, then you have something
really cool. So my advice would be to embrace ISMA with OSS
and make it an allied rather considering it a competitor.

Now back to your comments. The way I read them, you seem mostly to
object to me using the MP4 file format, but reading between the
lines, I get the feeling that you consider it to be an abomination
to be involved in developing software for MPEG-4.

Let me first say, that out of the total code and test work I did,
the work related to the MP4 file format may be in the range of
2-3%. Hey, it's just a file format for saving information.

> but i am
> failing to see the sense behind doing this ? Are you hoping to raise
> people's awareness to Vorbis if it is supported in mp4 container ?

As stated in too many lines above, the intention is much more ambitious
than this, but I'm always up for raising peoples awareness about Vorbis
and for removing all reasons for not using Vorbis.

> I find it more than questionable to 'hack' support for codecs that are not
> in MPEG4 specs into the MP4 container. Whats the sense behind it ?

Well, the intention is to embrace and digest. Actually as I se it, the
fact that the MPEG-4 specs doesn't covers Vorbis, is a VERY good reason for
me to demonstrate how MPEG-4 could cover Vorbis.

> Again, i am not at all questioning MPEG4IP .. there has to be an opensource
> possibility to create valid MPEG4 files, no doubt .... but why at all use
> this limited, proprietary container for anything outside MPEG4 specs at all ?

I'm not religious about container format. I'll use whatever suits the purpose.
And for now, the MP4 file format suits its purpose very fine thank you.

One remark though about the MP4 file format. The MP4 file format is close to
95% identical to the Quicktime .mov format with a few extra features. Where
I live, no royalty covering patents covers the MP4 file format (i've checked -
and it was really really really boring to do so). Neither is it likely there
will be. The reason is not that I live in a banana republic, but rather that
no patent request has been filed with the required time frame from existence
of prior art. So from that perspective, I can't see why people would object
to me using a MP4 file formats. It's just a container and has only minor
importance to the whole package I released. Do you have a royalty problem
with the MP4 file format ? Then I'd beglad to hear about it.

> Both Ogg and MCF are much better suited for this, Ogg is excellent for
> streaming also and MCF is targeted for video editing, also there will be (
> free ) conversion tools to allow transmuxing of contents from Ogg to MCF and
> vice versa .....

No, Ogg and MCF IS NOT better suited for streaming, or it at least depends
on your context. Ogg and MCF may very well be better suite done day, that is true,
but better suited is not all about technical features, but also about
other programs able to use it.

Let me give you an example. I heavily depends on the OSS Darwin Streaming
Server (DSS). One thing Ogg and MCF is missing, is DSS support for Ogg and
MCF. DSS doesn't support Vorbis either, but DSS does understand a
container format like MP4. Now putting Vorbis into a MP4 file doesn't
make it streamable by DSS that doesn't know about Vorbis,
but DSS has a RTP stream module that can stream all kind of data if
the MP4 file also contains information about how to do this. This
is popular being referred to a hinting. All you have to do is to
create a hint track associated with what want to stream and DSS
will happily stream it without knowing anything about what it streams.
This is a very powerful feature. Now you may be able to develop
such a feature for Ogg and MCF, but without wide popular acceptance
and support, you are not even half the way.

> The only real existing advantage MP4 container has to both, is that hardware
> support is much more likely to come one day. In any case, those people
> dreaming their XviD/DivX5 rips with MP3 or AAC sound will really be playable
> in hardware devices one day, without changes, are dreaming IMHO. MP3 is not
> even in official MPEG4 specs, and the way MPEG4IP people hacked suport for
> it into MP4 container will certainly not be the one supported by MPEG-LA (
> and such by hardware players ), you can bet on that !! If AAC is used, there
> are no real free encoders ( in terms of paid licenses to MPEG-LA ), FAAC
> cant yet compare to Vorbis at all qualitywise and PsyTel isnt really free.

I don't know of any hardware players actually reading container formats. They
all have the container peeled of by software wrappers before sending it to the
decoder. A HW decoder is not about container formats, but rather about codecs.
I'm replacing proprietary royalty requiring codecs with an OSS alternative.
A part from that, I agree with your views on MP3, AAC, but can't claim that
I'm an expert on quality.

> Another very important point about missing hardware support for existing
> .MP4 files is video resolution ! Of course, modern MPEG decoders get more
> and more flexible ( just browse SIGMA for their latest decoders ), but by no
> means will they be able to support every resolution dividable by 8 ( or even
> 2 in DivX5 case ), so in the end only a couple of resolutions will make
> their way into official MPEG hardware specs, leaving 99% of all existing
> MPEG4 files useless !!

Probably. I can't see what this has to do with me.

> Only MCF ( and Tobias' unofficial Ogg implementation known as .ogm ) has a
> Dshow/VfW compatibility mode, allowing you to use every existing VfW and ACM
> codec in your movies. Only MCF has a Quicktime compatibility mode to allow
> same for elder QT codecs.

Probably. I can't see what this has to do with me.

> Please, try to explain that to me :
> Whats the hassle about the MP4 container ?

It fits a purpose, but I can use most container formats. However if I prepare for
streaming of recorded material, for now I need a format I can hint and that
is understood by the streaming servers available.

> Whats the major advantage of it compared to Ogg and MCF ?

Well, its there (in the MPEG4IP project) It fits a purpose, but I can use
most container formats. However if I prepare for streaming of recorded material,
for now I need a format I can hint and that is understood by the streaming
servers available. So why shouldn't I be using it ?

<p>> Why is so much energy invested to hack support of non-MPEG standard codecs
> into it ?

To MP4 or MPEG-4 ? Well, mostly because it is not there. I happen to think
it is a problem for Vorbis not being part of MPEG-4. SOme agrees, others
like you seems to disagree, but hey, we ought to respect that.

> Both MCF and Ogg are opensource and belong to the community, while MP4
> belongs to f..in, greedy, rich MPEG-LA !!!!

Nope, the MPEG-4 frame work uses the MP4 file format and MPEGLA
administers royalties for patents probably covering some of the
codecs being a subset of the MPEG-4, but I have no evidence that
the MP4 file format in it self requires a MPEG-4 license from
MPEGLA. So my best guess is that you can use the MP4 file format
free of royalties in many countries, but if you uses the MPEG-4
codecs covered by royalty claiming patents, then you're in the
mercy of our beloved *friends*.

<p>> You like the licensing terms of
> MPEG4 they did ?

No.

> I like them as well !!! They are fair, as they are now !!

As I explained above, this is a much more ambitious effort. For it to
really succeed, I need within a year or two a royalty free video
codec comparable to what Microsoft will have in a year or two.

> How long you think they will stay like this ? Wonna know ? I tell you how
> long .....

Yes, I agree, but I have to fight with what I have right now (or rather yesterday).
I can't fight today's war with tomorrow's weapons or for that matter today's weapon
if the ammo won't fit. So good luck with your MFC. When its finish and widely deployed,
I probably be happy to use it if it offers advantages to me over MP4.

> .... they will just wait until everybody uses MPEG4 and then they will bring
> new licensing terms

Of course they will

> and MPEG4IP will do good to make their CVS disappear
> from surface

Of course they will

> ( same goes to XviD BTW ) or they will risk to pay licensing
> costs until the end of their lifetime !!!!

Of course they will

> The current existing MPEG4
> licensing can be changed any time they like to !!!

Of course it can.

But in the mean time, a OSS streaming community may have been built and
both you and I are planning and helping for when that time comes.

> Why am i writing all of this ???
>
> I'd like you to consider stoping your project

I'm always open to ideas. I hope you are too.

<p>> and to invite you to join
> either Xiph or MCF team, to invest your precious engery into projects that
> belong to us, the opensource community ..... not to companies launching
> proprietary standards and charging the 'free' world for them !!!

I'm trying hard to get Xiph to cooperate to get RTP/RTSP based streaming
implemented and Michael from the Xiph/Vorbis team has been a great help
on basic Vorbis things. When it comes to actually get Vorbis RTP/RTSP
streaming moved from draft to RFC, Emmett has unfortunately been not so
helpful, but I guess he's a busy man. I thought once he was dead, but
he appears to still be breathing, so lets see.

In principle you and I are working for the same goal, luckily we use
different approaches. That's healthy and why OSS thrives.

Kind regards

Peter Maersk-Moller
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.