[theora] segfault on amd64 with ffmpeg

[Guillaume Pellerin]

Hi,

and thank to you all for this great codec !

I have this bug on Debian Lenny with compiled packages of last svn versions of
ffmpeg and libtheora. This seems to append only on the amd64 arch. Here is a
valgrind log :

pre-barreau at augustins:~/video$ export LD_LIBRARY_PATH=/usr/local/lib
pre-barreau at augustins:~/video$ valgrind
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg -i ffmpeg_pb_jack_test_14.mpg
-vcodec libtheora -f ogg -y test.ogg
==12051== Memcheck, a memory error detector.

==12051== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.

==12051== Using LibVEX rev 1854, a library for dynamic binary translation.

==12051== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.

==12051== Using valgrind-3.3.1-Debian, a dynamic binary instrumentation
framework.
==12051== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.

==12051== For more details, rerun with: -v

==12051==

FFmpeg version SVN-r17468, Copyright (c) 2000-2009 Fabrice Bellard, et al.

  configuration: --enable-libmp3lame --enable-gpl --enable-libfaad
--enable-libvorbis --enable-libfaac --enable-libxvid --enable-postproc
--enable-libamr-nb --enable-libamr-wb --enable-x11grab --enable-libgsm
--enable-libx264 --enable-libtheora --enable-swscale --enable-libdc1394
--enable-nonfree --enable-avfilter --enable-libdirac --disable-decoder=libdirac
--enable-libspeex --enable-avfilter-lavf --enable-vdpau --disable-altivec
--disable-armv5te --disable-armv6 --disable-vis
  libavutil     49.14. 0 / 49.14. 0

  libavcodec    52.18. 0 / 52.18. 0

  libavformat   52.29. 2 / 52.29. 2

  libavdevice   52. 1. 0 / 52. 1. 0

  libavfilter    0. 3. 0 /  0. 3. 0

  libswscale     0. 7. 0 /  0. 7. 0

  libpostproc   51. 2. 0 / 51. 2. 0

  built on Feb 20 2009 14:32:43, gcc: 4.3.2

Input #0, mpeg, from 'ffmpeg_pb_jack_test_14.mpg':

  Duration: 00:02:18.76, start: 0.500000, bitrate: 510 kb/s

    Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 432x200 [PAR 1:1 DAR 54:25],
104857 kb/s, 25.00 tb(r)
    Stream #0.1[0x1c0]: Audio: mp3, 48000 Hz, stereo, s16, 96 kb/s

Output #0, ogg, to 'test.ogg':

    Stream #0.0: Video: libtheora, yuv420p, 432x200 [PAR 1:1 DAR 54:25], q=2-31,
200 kb/s, 25.00 tb(c)
    Stream #0.1: Audio: flac, 48000 Hz, stereo, s16, 64 kb/s

Stream mapping:

  Stream #0.0 -> #0.0

  Stream #0.1 -> #0.1

Press [q] to stop encoding

==12051== Invalid write of size 4

==12051==    at 0x64D1485: PickIntra (in /usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x64DA9B2: theora_encode_YUVin (in
/usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x762598: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x4E2DB8: avcodec_encode_video (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x4256D2: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x425F61: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x42A3A0: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==  Address 0x9dc2294 is 4 bytes before a block of size 8,100 alloc'd

==12051==    at 0x4C2260E: malloc (vg_replace_malloc.c:207)

==12051==    by 0x64E64C9: InitFragmentInfo (in
/usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x64E6722: InitFrameDetails (in
/usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x64D9F2E: theora_encode_init (in
/usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x762874: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x4E38FE: avcodec_open (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x4292DB: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)

==12051==

==12051== Invalid write of size 4

==12051==    at 0x64D14AC: PickIntra (in /usr/local/lib/libtheora.so.0.3.4)

==12051==    by 0x64DA9B2: theora_encode_YUVin (in
/usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x762598: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4E2DB8: avcodec_encode_video (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4256D2: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x425F61: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A3A0: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==  Address 0x9dc2294 is 4 bytes before a block of size 8,100 alloc'd
==12051==    at 0x4C2260E: malloc (vg_replace_malloc.c:207)
==12051==    by 0x64E64C9: InitFragmentInfo (in /usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x64E6722: InitFrameDetails (in /usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x64D9F2E: theora_encode_init (in /usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x762874: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4E38FE: avcodec_open (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4292DB: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==
==12051== Invalid write of size 4
==12051==    at 0x64D14F8: PickIntra (in /usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x64DA9B2: theora_encode_YUVin (in
/usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x762598: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4E2DB8: avcodec_encode_video (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4256D2: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x425F61: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A3A0: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==  Address 0x109dc378c is not stack'd, malloc'd or (recently) free'd
==12051==
==12051== Process terminating with default action of signal 11 (SIGSEGV)
==12051==  Access not within mapped region at address 0x109DC378C
==12051==    at 0x64D14F8: PickIntra (in /usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x64DA9B2: theora_encode_YUVin (in
/usr/local/lib/libtheora.so.0.3.4)
==12051==    by 0x762598: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4E2DB8: avcodec_encode_video (in
/home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x4256D2: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x425F61: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A3A0: (within /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==    by 0x42A97B: main (in /home/pub/apps/ffmpeg_dev/ffmpeg_svn/ffmpeg)
==12051==
==12051== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 8 from 1)
==12051== malloc/free: in use at exit: 10,930,852 bytes in 5,498 blocks.
==12051== malloc/free: 6,241 allocs, 743 frees, 12,668,639 bytes allocated.
==12051== For counts of detected errors, rerun with: -v
==12051== searching for pointers to 5,498 not-freed blocks.
==12051== checked 12,611,608 bytes.
==12051==
==12051== LEAK SUMMARY:
==12051==    definitely lost: 50 bytes in 1 blocks.
==12051==      possibly lost: 29 bytes in 1 blocks.
==12051==    still reachable: 10,930,773 bytes in 5,496 blocks.
==12051==         suppressed: 0 bytes in 0 blocks.
==12051== Rerun with --leak-check=full to see details of leaked memory.
Erreur de segmentation


Any idea ?

Cheers,
Guillaume