[Vorbis-dev] How can I minimize the memory use in Tremor?

[xiphmont at xiph.org]

On Mon, Sep 1, 2008 at 11:38 PM, 隽 徐 <china_xujun at yahoo.com.cn> wrote:

> After looking into the code, I found the header parse, especially, the
> codebook unpack part takes a lot of memory. Tremor decodes the codebooks to
> a cache to store them for further decode. And this is the dominant factor in
> decoder memory usage, am I right?

correct.  Other things can be trimmed as well, but the fully unpacked
codebooks are far and away the largest piece of memory.

> There are two structures in this code to hold the codebooks, static_codebook
> and codebook. But in low mem code, there are only one structure -  codebook.
> It seems static_codebook is redundant, and I wonder if I could replace this
> part with the low mem code?

It is not redundant, exactly-- in the main branch of tremor the
codebooks are fully unpacked, and one structure contains metadata for
handling the codebooks rapidly, where the other contains the raw
unpacked data.  In the low-mem branch they are only partially unpacked
and then during decode pieces are unpacked as needed.  The low mem
branch requires considerably more CPU for this reason (but far far
less memory).  The codebook and packet decode code are very different
in the two, although most of that is contained inside the codebook
code.

It is possible to swap in the low-mem codebook code but not directly.
Although the changes are all contained in the codebook code, I believe
the internal API for calling the codebook code is slightly different.

> Since I'm not very clear about the codebook structure, so, I'm not sure if
> this is feasible to reduce the memory usage? Or can someone tell
> me other methods to minimize the memory use?

This is not an unreasonable strategy.  I've not looked at this
specific code in a little while so I'm not able to immediately judge
the relative time needed to 'drop in' the lowmem codebook handling
code, as opposed to moving to the lowmem branch and moving over
modified/optimized pieces of code from your current port.   As an
educated guess, trying to 'drop in' the lowmem branch's codebook
handling code probably is the right choice.

Monty