[vorbis-dev] ogg vorbis dynamic memory allocation.

[Segher Boessenkool]

Howdy,

Firelight Multimedia wrote:
> 
> > > Hi,
> > > I finally got around to overriding ogg_malloc etc and running
> through my
> > > memory manager.  Thinking of limited memory consoles like PS2, I was
> >
> > How much RAM does a PSX2 have?
> 
> Not enough.. its got 32mb but for audio the programmer usually gets
> either 0k (they are usually told to only use the IOP subprocessor which
> has its own little bit of memory, but its not powerful enough to decode
> vorbis)..

Are you sure about this?  If it can deceode mp3, it can decode vorbis in
about the same amount of cpu.  At least, in theory...  Any public spec
of that IOP?

> A generous game team might give 512k to 1mb for audio processing on the
> main CPU, which also has to include all other memory allocs used by the
> audio engine..
> 
> > > somewhat alarmed at what I found .. Admittedly I am using vorbisfile
> but
> > > that’s because its so easy to use :) ..
> >
> > Vorbisfile shouldn't be a big overhead over straight libogg/libvorbis;
> > if it is, it's time we do something about it.
> 
> I don’t know the implementation (ive been just a 'user' and havent

Neither do I, really; I've always just worked on libvorbis.

> looked into how it works for a long time), but you could be right .. I
> could easily track where most of the big mallocs are coming from.

Please do; that would help us help you :-)

> 
> > > Per ogg file my memory manager is telling me it is using 885,201
> bytes
> > > per file (I thought I would maybe get a constant overhead and lower
> > > overhead per file but its 885k each time).
> >
> > That's way too much, I think... you might forget to free memory?  If
> > you've overridden alloca() as well, you need to free that memeory at
> > function exit of the function that did the alloca() (or per C block,
> > even).
> 
> I overrode _ogg_malloc , _ogg_calloc , _ogg_realloc, _ogg_free, no
> memory leaks (it reports 0 bytes allocated when I clean up)..  I havent
> touched alloca.

Okido.  So it's our problem, not yours.  Argh.  Well, more work...

> 
> > > Ov_read reads 17k at a time in my streamer, and reducing it (to say
> 7k)
> > > made no difference.
> > > Is there a way to get this right down?  Remove vorbisfile I assume
> is
> > > the first thing (though I am loathe to do right now due to the extra
> > > complexity involved).  Mp3 currently uses about 32k of different
> buffers
> >
> > That's not right; the tightest mp3 decoder I've seen needs about 48k
> > R/W memory, and there's not much opportunities left to reduce that.
> 
> Yes it is .. per mp3 file my decoder has these arrays and NO dynamic
> memory allocation internally.. they add up to something like 32k
> roughly.
> 
> #define SBLIMIT         32
> #define SSLIMIT         18
> #define MAXFRAMESIZE    1792
> unsigned char bsspace[2][MAXFRAMESIZE+512];
> float    hybrid_block[2][2][SBLIMIT*SSLIMIT];
> float    synth_buffs[2][2][0x120];
> float    block[2][2][SBLIMIT * SSLIMIT];
> float  hybridIn[2][SBLIMIT][SSLIMIT];
> float  hybridOut[2][SSLIMIT][SBLIMIT];

That's 40.5k (roughly), for these buffers only...  So my 48k stands.
But anyway, 48k or 880k doesn't compare very nice for us...

> 
> There is static table data as well of course (bandinfo/huff/sin/cos
> tables - only a few k) but I am talking about the dynamic per file hit.
> 
> > > per file which is extremely efficient memory usage for consoles, is
> > > there any way vorbis might be able to get this low?
> >
> > There's work being done to reduce Vorbis memory usage (as well as cpu
> usage, and cache pressure).
> >
> > > I havent had a look at static usage yet, id have to take a look at
> the
> > > map file for that.
> > > Any ideas or comments appreciated.
> >
> > You can mail me your malloc() etc. implememntation, and I'll have a
> > look at it; and/or the raw output of your mem usage experiments.
> 
> Static code+data Its about 76k after a very quick map file check (on an
> intel with msvc), which is fine for me..
> Ill look forward to hear any updates on memory reduction (ive heard
> about compressed codebooks etc) As for my memory code I don’t have to
> prove that that works .. its tracking the memory usage perfectly (it
> didn’t find any leaks which is cool) and has worked for years in
> commercial code :) its not just something I hacked up for this.

Can it output how much mem it alloced where?  I'd like to see that
output :-)

> 
> Cheers
> Brett

Segher

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.