[vorbis-dev] some ogg memory issues with managed bitrates

[oddsock]

I noticed that when using managed bitrates from within my streaming encoder 
(dsp_oddcast), memory seemed to grow without bounds....I've narrowed it 
down to only managed bitrates (i.e. it doesn't happen if I use quality 
levels)...

I modified the encoder_example to use managed bitrates (it defaulted to 
quality) and ran valgrind on it.....4 hours later (ugh) I got the following 
report :

==21460== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
==21460== malloc/free: in use at exit: 728 bytes in 2 blocks.
==21460== malloc/free: 27071 allocs, 27069 frees, 66711094 bytes allocated.
==21460== For counts of detected errors, rerun with: -v
==21460== searching for pointers to 2 not-freed blocks.
==21460== checked 4843636 bytes.

(ignore the 2 not-freed blocks as those are probably due to my fopen()s 
that I also added to the encoder example (nothing else was added))

the part of concern is the ~66MB of bytes allocated....this is running 
encoder_example on a approx 3 minute WAV file.

in a separate test I recompiled vorbis/ogg with dmalloc and dmalloc pointed 
to the following chunk of code in ogg/framing.c

void oggpack_write(oggpack_buffer *b,unsigned long value,int bits){
   if(b->endbyte+4>=b->storage){
     b->buffer=_ogg_realloc(b->buffer,b->storage+BUFFER_INCREMENT);
     b->storage+=BUFFER_INCREMENT;
     b->ptr=b->buffer+b->endbyte;
}

the part that dmalloc pointed out was the ogg_realloc() call above...

Here is the dmalloc report :
1029039756: 31968: alloc calls: malloc 13302, calloc 238, realloc 13311, 
free 5117
1029039756: 31968: alloc calls: recalloc 0, memalign 0, valloc 0
1029039756: 31968:  total memory allocated: 47480854 bytes (26851 pnts)
1029039756: 31968:  max in use at one time: 5127307 bytes (8424 pnts)
1029039756: 31968: max alloced with 1 call: 297824 bytes
1029039756: 31968: max alloc rounding loss: 1911221 bytes (27%)
1029039756: 31968: max memory space wasted: 0 bytes (0%)
1029039756: 31968: final user memory space: basic 1534, divided 303, 
5614582 bytes
1029039756: 31968:  final admin overhead: basic 27, divided 66, 380928 
bytes (4%)
1029039756: 31968:  final external space: 0 bytes (0 blocks)
1029039756: 31968: top 10 allocations:
1029039756: 31968:  total-size  count in-use-size  count  source
1029039756: 31968:    42650624  13281     3544832    614  bitwise.c:107
1029039756: 31968:     1062144   4149      196608    768  bitwise.c:38
1029039756: 31968:      732168     39      244056     13  sharedbook.c:189
1029039756: 31968:      671112      4      297824      1  block.c:139
1029039756: 31968:      378624   1632      126208    544  psy.c:180
1029039756: 31968:      344920    173           0      0  block.c:116
1029039756: 31968:      248580      3       82860      1  bitrate.c:105
1029039756: 31968:      163840      1      163840      1  cbuffer.c:52
1029039756: 31968:      151552      4       45056      2  block.c:372
1029039756: 31968:    47480854  26851     5122454   8423  Total of 115

note that these are separate runs, but the dmalloc report also show a very 
large allocation (42MB) right at chunk of code identified above...

Again, this only seems to happen with Managed Bitrates, and the memory 
growth has been observed on both win32 and linux platforms...

oddsock

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.