[vorbis-dev] clarifications on comments spec

[Beni Cherniavsky]

Scott Wheeler wrote on 2003-06-29:

> I've been hacking on an audio metadata library (creatively called TagLib)
> which we'll be using in KDE in the next release ( == I need to get around to
> finishing it).  I've got the Vorbis comment implementation working, but just
> wanted to clarify a couple of things:
>
> *) The vendor ID -- "vendor" is ambiguous here.  Is this the "vendor" of the
> standard?  The encoder?  (Seems to be implied by the value mentioned in the
> spec.)  The tool writing the comment?  In my current implementation, I've
> assumed the second and as such don't modify the vendor ID, but I didn't know
> if I should stuff something in there mentioning TagLib.
>
Currently AFAIK all ogg files in the world are have a vendor string
string identifying some the version of libvorbis that encoded them.
So de-facto, it's the encoder.  I can't say whether officially it's
intended for anything else but frankly, identifying the encoder is
more useful than identifying the last application that edited the
tags...  And if you do want to say something important about the
source of the tags, it's probably better put it in a comment tag.
The vendor string seems to be not intended as uneditable by end-users
(though not protected in any DRM sense), so anything a human might
want to say belongs in comment tags.

> *) There seems to be no provision for "padding" (to use the ID3v2 word).
> Since the comment is very near to the beginning of a Vorbis stream, assuming
> a file based implementation, the overwhelming majority of the file's contents
> will have to be moved when the comment size changes.  Is this correct?
>
Ogg tells you the size of the comment packet.  However, the packet
itself contains the number of the comment strings and each has a
length count so you can know where the useful packet content ends from
the data itself.  Therefore, you can pad the comments by putting
garbage (preferably zeros, to allow compression on network lines and
make sure pieces of old coments don't remain there (the MS word
sindrom of embarassing partial erasure)) at the end of the packet,
after the useful data.  AFAIK that's the "blessed" way to do padding.
I'm not aware of anybody implementing this to-date and I'm not 100%
sure quick-and dirty programs will ignore this padding but if not the
are broken.

> *) There's no location to read the complete length of the header
> from.  This makes parsing in a streaming situation, where data is
> being pushed rather than pulled, more complicated.
>
Indeed.  The same is true of all ogg/vorbis packets.  As far as Ogg is
concerned, packets might be arbitrarily long and it might make sense
to do stream processing of a single packet (if you are ready to deal
with partial packet corruption.

> *) Rerendering the full Ogg page(s) seems to be a requirement of the current
> scheme.  This isn't particularly difficult, but could be simplified.
>
You mean that you want to avoid a full implemnattion of the Ogg
framing?  You are free to use libogg to save yourself the trouble ;-).
Handling of Vorbis comments is designed to be easy on top of aa full
Ogg implementation; making the comments any kind of a special case
outside Ogg framing would make it easier on comment-only programs ubt
complicate things for full Vorbis implmentations.  Note also that
Vorbis does not have to be wrapped in Ogg (e.g.  there is a draft
Vorbis-over-RTP standard).  Conceptually, all Vorbis depends upon is
marking of packet boundaries.

> *) A comment is allowed to span multiple Ogg pages.  This is a pain in the
> butt.  ;-)  Despite the Ogg spec saying "large packets are forseens as being
> useful for initialization data at the beginning of a logical bitstream", it
> wasn't stipulated in the Vorbis comment spec that this is one such
> application.  Especially when you're just tagging -- and not encoding at the
> same time -- you're never going to want to split the tag into multiple pages
> because then you have to go through and rewrite the page number for every
> following page.
>
Yes, this is a problem.  It can't be helped because pages are limited
to 64KB and comments can in theory exceed that.  Rewriting the pages
is not a lot of work if you are copying the file anyway; padding will
solve both issues in the common case.

> I was thinking about these issues, and while they weren't connected in the
> issues that they raise, I think I've come up with a way that they could all
> be solved in a backwards compatible way:
>
> If it was required, as is true with the Vorbis identification header, that
> future implementations put the comment in an Ogg stream page by itself this
> could solve the above issues.  It would then be possible to deduce the total
> size of the comment from the Ogg page header.  Though I haven't tried it yet,
> I also presume that empty space at the end of an Ogg page would simply be
> ignored, thus providing a mechanism for padding.  The size of the padding
> would then be the size of the comment page minus the comment size.  Also this
> would simplify reading and rendering the comment since it wouldn't be required
> to account for additional Vorbis data that comes later in the Ogg page or
> deal with comments that span multiple pages.
>
Again, this won't help much because the comments won't  always fit in
a single page.

> This seems to have the benefit that it would not confuse current Vorbis
> implementations, but would solve the issues mentioned above.
>
Not really, because there are a lot of existing files out there that
don't close the page between the comments and codebooks packets.

> Anyway -- let me know your thoughts, and please CC me since I'm not on the
> list.
>


-- 
Beni Cherniavsky <cben at tx.technion.ac.il>
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.