[ogg-dev] Fixing ogg vorbis corruption caused by bad metadata

[Conrad Parker]

Hi,

(I was discussing this with Adam earlier today, and he has also mailed
vorbis at xiph.org about this issue in March).

This sounds like it needs a new tool specifically for fixing broken
Ogg files. Suggestions so far (tried by Adam, or suggested to him in
email/IRC):

  * oggz-sort, hogg sort

These rewrite the file, sorting the pages by presentation time. This
usually fixes muxing problems, but will not change the actual page
contents, so are of little use here where header data is corrupted.

  * hogg reconstruct

This reconstructs a file "by doing a full packet demux". Here, hogg
actually keeps track of page boundaries, ie. what segments each packet
consists of. The reconstruction then involves writing new pages with
the same segmentation. This will fix some paging issues (sequence
numbering, granulepos) and re-calculate checksums; this may have fixed
some files where some bad metadata pages were randomly inserted at the
start.

  * oggz-dump, [edit], oggz-dump -r

This is a fun hack, but doesn't preserve any segmentation info so is
unlikely to help here.

  * oggz-chop -s 0

(a clever suggestion I saw from ogg.k.ogg.k ;-) This will usually
rewrite the headers and flush them correctly, so it can fix some
header segmentation problems, but like oggz-sort it doesn't modify the
contents.

So basically it looks like these files are corrupted in a new and
interesting way! The current tools won't work, and I'm unlikely to
have time to write a fixer for these in the short term. This sounds
like the perfect challenge for someone who wants to dive deeper into
Ogg hacking (or a good tool for the recent C++ libraries :-)

If you want to try with liboggz, a good place to start would be the
files in src/examples, such as fix-eos.c or rewrite-pages.c. I could
also help anyone who wants to hack on hogg, where it's probably just a
matter of writing a short helper along the lines of:

sort :: [OggPage] -> [OggPage]
sort = sortHeaders . listMerge . demux

Anyway, an interesting problem :-)

Conrad.

2009/6/18 Adam Rosi-Kessel <adam at rosi-kessel.org>:
> I have over 1,000 ogg vorbis files that are no longer playable or
> editable. I believe the main cause was a MediaMonkey plugin that
> automatically fetched album art and added it to the header. The plugin
> itself just uses the generic MediaMonkey metadata facility, but
> something happened -- perhaps when the album art bitmap image was too
> large -- that caused the files to no longer work. Unfortunately, I don't
> have backups.
>
> I have noticed a handful of unresolved threads about ogg corruption on
> various message boards, so I suspect I am not alone in having, for one
> reason or another, unplayable oggs.
>
> I have tried rewriting the files with oggz-sort (both last release and
> latest from git) and also with hogg reconstruct (including latest
> version built from source). I have also tried dumping the files with
> oggz-dump and swapping in either packetno 1 or 1,2 or 1,2,3 from a known
> good ogg file, replacing the serialnos, and then recreating the ogg with
> oggz-dump -r. None of these changes work. vlc will "play" the file, but
> all I hear is silence.
>
> In a few instances, I have had ogg files in this state where they played
> but I couldn't edit the metadata with any tool. hogg reconstruct fixed
> those files so they seem to be fully working again, but most of my
> corrupted files don't fall into this category.
>
> oggz-validate reports on an example file:
>
> serialno 1225743615: Terminal header page contains non-header packet
>
> after attempting to splice in good headers, on the same file:
>
> serialno 1225743615: Terminal header page has non-zero granulepos
>
> I have posted a couple of sample files, one before and one after running
> through oggz-sort, here:
>
> http://adam.rosi-kessel.org/bugs/liboggz/484/
>
> I'm not sure if all of my files have the same problem, so I can share
> others if it would be helpful.
>
> I'd appreciate any assistance/suggestions in fixing these files. I don't
> see any reason why the corruption should extend beyond just the metadata
> part of the file, but I could of course be wrong.
> _______________________________________________
> ogg-dev mailing list
> ogg-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/ogg-dev
>
>