[vorbis] Various Ogg Vorbis largefile notes and/or patches
Barry Bouwsma
bugs
Tue Jun 8 06:13:07 PDT 2004
[this address is IPv6-only, and usually doesn't work, so don't
reply directly to me. I'll catch the archives next time online]
I slobbered the following all into my keyboard:
> Anyway, I've been working with ogg123/oggenc and large files as
> both source and destination. (Large files being those several GB
> in size, for which 64-bits are needed.) This has exposed some
> weaknesses that I've tried to hack around. I'll explain in messages
> to follow, each problem that I'm addressing along with brutish hacks
Howdy. My next problem was encountered when trying to play an ogg
vorbis file I had encoded, whose size was more than 4GB (nearly 100
hours worth of continuous .. um, music, I guess).
There seem to be a heap of fseek() and ftell() calls that use longs
and can't handle 64-bit-sized files. FreeBSD (on which I've done these
hacks) has an off_t-capable version of these calls (ftello() and its
ilk), while other OSen may have something like fseek64(). Thus, I
suspect a bit of configure magic as well as a much more comprehensive
hack than my BSD-specific hack is needed in order to handle >4GB size
ogg vorbis files.
First of all, in order to do any sort of seeking to the end of the
file, I needed to replace several fseek() and ftell() with the off_t
versions. (Corresponding hacks were needed in the http transport to
match.) Without this, I'd get into a loop lseek()ing in the file.
It seems that part of the vorbisfile library is 64-bit ready; only
the OS-specific calls need to be hacked. As always, these hacks are
not necessarily applicable to a non-BSD system -- please review!
But to make things easier, I'm waiting with these hacks until the
end of the message, since there's a few sets, then I'll hit you
with them all at once.
Next, giving a time-seek parameter in the file seemed to degenerate
quickly past a certain point. This seems to be a problem in the
algorithm used to find the value of bisect.
This algorithm uses 64-bit math already; however, when one is
more-or-less squaring a value that doesn't fit into 32 bits, the
64-bit math overflows. I converted to a float/double, and the
algorithm quickly found a suitable bisect value. Yay.
My question, though, is: does it matter that the calculated value
of bisect is no longer a clean 64-bit integer? Someone with more
Clue than I may wish to see if that is a problem.
--- /stand/work/vorbis/vorbis/lib/vorbisfile.c-DIST Wed Sep 17 04:17:35 2003
+++ /stand/work/vorbis/vorbis/lib/vorbisfile.c Fri May 7 00:59:52 2004
@@ -1123,18 +1133,24 @@
ogg_int64_t best=begin;
ogg_page og;
+/* XXX */
+/* XXX printf("end = %qd, begin = %qd, begintime = %qd,\n endtime = %qd, target = %qd, best = %qd\n", end, begin, begintime, endtime, target, best); */
+
while(begin<end){
- ogg_int64_t bisect;
+ /* XXX ogg_int64_t bisect; */
+ double bisect;
if(end-begin<CHUNKSIZE){
bisect=begin;
}else{
/* take a (pretty decent) guess. */
bisect=begin +
- (target-begintime)*(end-begin)/(endtime-begintime) - CHUNKSIZE;
+ (double)(target-begintime)*(double)(end-begin)/(double)(endtime-begintime) - CHUNKSIZE;
if(bisect<=begin)
bisect=begin+1;
}
+/* XXX printf("bisect = %f\n", bisect); */
+/* XXX sleep(1); */
_seek_helper(vf,bisect);
With this hack, I can apparently seek at will within the first
2GB or so of the ogg vorbis file. However, seeking to a time
value past this point results in only a fraction of a second of
audio playback before EOS is reached. Likewise, seeking to
just before this point results in several seconds of playback
but then EOS is reached at the point where normal playback
ends.
I'm looking into this now. Or rather, I had been thinking about
looking into it a while ago -- while oggenc had no problem blatting
encoded data onto the file, I wonder if perhaps there's something
being written in the ogg vorbis data including a 32-bit representation
of the file position, or if it's purely something to do with the
playback application (ogg123), before I dig around in the source
code and/or description of the format.
Here are the various hacks I have so far for 64-bit files, as
I threatened above. As you have certainly noted, there is a
lot of debugging crap and stuff left in these diffs, so if I'm
doing the right thing, you'll want to redo this from scratch.
I think this is complete, but I could be overlooking something.
--- /stand/work/vorbis/vorbis/include/vorbis/vorbisfile.h-DIST Thu Sep 4 03:50:02 2003
+++ /stand/work/vorbis/vorbis/include/vorbis/vorbisfile.h Thu May 6 22:56:42 2004
@@ -40,7 +40,7 @@
size_t (*read_func) (void *ptr, size_t size, size_t nmemb, void *datasource);
int (*seek_func) (void *datasource, ogg_int64_t offset, int whence);
int (*close_func) (void *datasource);
- long (*tell_func) (void *datasource);
+ off_t (*tell_func) (void *datasource);
} ov_callbacks;
#define NOTOPEN 0
--- /stand/work/vorbis/vorbis/lib/vorbisfile.c-DIST Wed Sep 17 04:17:35 2003
+++ /stand/work/vorbis/vorbis/lib/vorbisfile.c Fri May 7 00:59:52 2004
@@ -738,7 +746,7 @@
(size_t (*)(void *, size_t, size_t, void *)) fread,
(int (*)(void *, ogg_int64_t, int)) _fseek64_wrap,
(int (*)(void *)) fclose,
- (long (*)(void *)) ftell
+ (off_t (*)(void *)) ftell
};
return ov_open_callbacks((void *)f, vf, initial, ibytes, callbacks);
@@ -788,7 +796,7 @@
(size_t (*)(void *, size_t, size_t, void *)) fread,
(int (*)(void *, ogg_int64_t, int)) _fseek64_wrap,
(int (*)(void *)) fclose,
- (long (*)(void *)) ftell
+ (off_t (*)(void *)) ftell
};
return ov_test_callbacks((void *)f, vf, initial, ibytes, callbacks);
--- /stand/work/vorbis/vorbis-tools/ogg123/transport.h-DIST Wed Dec 19 03:52:54 2001
+++ /stand/work/vorbis/vorbis-tools/ogg123/transport.h Thu May 6 19:50:27 2004
@@ -48,9 +48,11 @@
data_source_t* (* open) (char *source_string, ogg123_options_t *ogg123_opts);
int (* peek) (data_source_t *source, void *ptr, size_t size, size_t nmemb);
int (* read) (data_source_t *source, void *ptr, size_t size, size_t nmemb);
- int (* seek) (data_source_t *source, long offset, int whence);
+/* XXX int (* seek) (data_source_t *source, long offset, int whence); */
+ int (* seek) (data_source_t *source, off_t offset, int whence);
data_source_stats_t * (* statistics) (data_source_t *source);
- long (* tell) (data_source_t *source);
+/* XXX long (* tell) (data_source_t *source); */
+ off_t (* tell) (data_source_t *source);
void (* close) (data_source_t *source);
} transport_t;
--- /stand/work/vorbis/vorbis-tools/ogg123/file_transport.c-DIST Sat Jan 26 12:06:37 2002
+++ /stand/work/vorbis/vorbis-tools/ogg123/file_transport.c Thu May 6 23:03:28 2004
@@ -77,20 +77,22 @@
}
+/* XXX HACK off_t ... */
int file_peek (data_source_t *source, void *ptr, size_t size, size_t nmemb)
{
file_private_t *private = source->private;
FILE *fp = private->fp;
int items;
- long start;
+ off_t start;
+/* XXX fprintf(stderr, _("Called file_peek.....\n")); */
/* Record where we are */
- start = ftell(fp);
+ start = ftello(fp);
items = fread(ptr, size, nmemb, fp);
/* Now go back so we maintain the peek() semantics */
- if (fseek(fp, start, SEEK_SET) != 0)
+ if (fseeko(fp, start, SEEK_SET) != 0)
items = 0; /* Flag error condition since we couldn't seek back to
the beginning */
@@ -113,12 +115,14 @@
}
-int file_seek (data_source_t *source, long offset, int whence)
+/* XXX HACK NEEDS TO BE MADE 64-bit-safe off_t ... */
+int file_seek (data_source_t *source, off_t offset, int whence)
{
file_private_t *private = source->private;
FILE *fp = private->fp;
- return fseek(fp, offset, whence);
+/* XXX fprintf(stderr, _("Called UNHACKED file_seek offset %qd whence %d.....\n"), offset, whence); */
+ return fseeko(fp, offset, whence);
}
@@ -130,12 +134,17 @@
}
-long file_tell (data_source_t *source)
+/* XXX HACK NEEDS TO BE MADE 64-bit-safe off_t ... */
+off_t file_tell (data_source_t *source)
{
file_private_t *private = source->private;
FILE *fp = private->fp;
+ ogg_int64_t pos;
- return ftell(fp);
+ /* XXX return ftello(fp); */
+ pos = ftello(fp);
+/* XXX fprintf(stderr, _("Called UNHACKED file_tell position %qd.....\n"), pos); */
+ return pos;
}
--- /stand/work/vorbis/vorbis-tools/ogg123/http_transport.c-DIST Thu Sep 4 03:53:13 2003
+++ /stand/work/vorbis/vorbis-tools/ogg123/http_transport.c Thu May 6 19:52:56 2004
@@ -285,7 +285,7 @@
}
-int http_seek (data_source_t *source, long offset, int whence)
+int http_seek (data_source_t *source, off_t offset, int whence)
{
return -1;
}
@@ -309,7 +309,7 @@
}
-long http_tell (data_source_t *source)
+off_t http_tell (data_source_t *source)
{
return 0;
}
--- /stand/work/vorbis/vorbis-tools/ogg123/oggvorbis_format.c-DIST Thu Sep 4 03:53:14 2003
+++ /stand/work/vorbis/vorbis-tools/ogg123/oggvorbis_format.c Fri May 7 00:47:21 2004
@@ -271,11 +277,15 @@
return 1; /* Ignore close request so transport can be closed later */
}
-long vorbisfile_cb_tell (void *arg)
+off_t vorbisfile_cb_tell (void *arg)
{
decoder_t *decoder = arg;
+ ogg_int64_t pos;
- return decoder->source->transport->tell(decoder->source);
+ /* return decoder->source->transport->tell(decoder->source); */
+ pos = decoder->source->transport->tell(decoder->source);
+/* XXX printf("vorbisfile_cb_tell returned %qd\n", pos); */
+ return pos;
}
thanks
barry bouwsma
More information about the Vorbis
mailing list