vlc crashes in avcodec_decode_audio2

[Ken Arnold]

Binary package hint: vlc

Jaunty, vlc 0.9.9a-2ubuntu1, libavcodec-unstripped-52: 3:0.svn20090303-1ubuntu2+unstripped1

I have a m4v file that plays fine in ffplay and used to play in vlc before I upgraded intrepid->jaunty. (It plays without sound in mplayer; the codec is 0x332D6361 / ffmpeg ac3 it seems. *shrug*) Now in Jaunty it consistently crashes. I've tried to debug the crash a little, and at least got it before it destroyed its stack:

#0 0xaf152c10 in avcodec_decode_audio2 () from /usr/lib/i686/cmov/libavcodec.so.52
#1 0xaf8dbba1 in DecodeAudio (p_dec=0xa0e9e9c, pp_block=0xae60c298) at audio.c:315
#2 0xb7eb9051 in DecoderDecodeAudio (p_dec=0xa0e9e9c, p_block=0xa16dae0) at input/decoder.c:629
#3 0xb7eba8f5 in DecoderDecode (p_dec=0xa0e9e9c, p_block=0xae60d008) at input/decoder.c:1006
#4 0xb7ebb2e1 in DecoderThread (p_this=0xa0e9e9c) at input/decoder.c:594
#5 0xb7f12e79 in thread_entry (data=0x9edb318) at misc/threads.c:465
#6 0xb7e714ff in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7 0xb7dec49e in clone () from /lib/tls/i686/cmov/libc.so.6

It SEGV's while storing parameters for the avctx->codec->decode call, at one of these instructions (intuition says it should be the `call`, but it crashes when I nexti in gdb when the eip is on the mov; maybe I just misunderstand gdb):
0xaf0baccc <avcodec_decode_audio2+188>: mov %ebx,(%esp)
0xaf0baccf <avcodec_decode_audio2+191>: call *0x1c(%edi)

Here's the avctx->codec structure, i.e., `p *p_sys->p_context->codec` in DecodeAudio:
{name = 0xaf4bbe9d "ac3", type = CODEC_TYPE_AUDIO, id = CODEC_ID_AC3, priv_data_size = 88656, init = 0xaf164fef, encode = 0, close = 0xaf164fb3, decode = 0xaf1618a0,
  capabilities = 0, next = 0xaf558360, flush = 0, supported_framerates = 0x0, pix_fmts = 0x0, long_name = 0xaf4bbe8a "ATSC A/52A (AC-3)", supported_samplerates = 0x0,
  sample_fmts = 0x0, channel_layouts = 0x0}

That looks pretty similar to (but is not the same memory location as) ac3_decoder in ac3dec.c. So it baffles me why it would be crashing there, but hopefully this basic tracing helps (I certainly learned a few things about gdb et al because of this; any good resources that would have gotten me further?).

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
NonfreeKernelModules: openafs nvidia
Package: vlc 0.9.9a-2ubuntu1
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: vlc
Uname: Linux 2.6.28-11-generic i686

[JB VideoLAN (jb-videolan)]

Can you sahre the file?

[Ken Arnold (kenneth-arnold)]

It's been way too long; I forgot which file this was and may not even still have it.