vivi memory leak = kernel panic

[Carl Karsten]

Overview:

sudo modprobe vivi
then as a user, build/run the sample code from the V4L2 spec (either the original from the spec or the updated one on linuxtv.org) It eats up some memory pool, the OOM killer whacks what it can till:
[ 290.760597] Kernel panic - not syncing: Out of memory and no killable processes...
vivi = kernel module that ships with ubuntu (which is the same as the one in Linus's kernel.)

Background (answers "Why?" but can be skipped)
I am trying to use vivi to help with the development of some V4L2 drivers and apps and python bindings. It does not require any hardware, is sposed to be an example of how to do things, so should be very handy for working with 5 people who don't all have the same hardware, and we don't want the results to depend on any one piece of hardware anyway. the apps should use the API correctly, and the drivers should implement the API correctly. vivi is like a hardware driver, but does not require any hardware - it just returns a test pattern. Also, there is an example program as part of the V4L2 spec: capture.c. It too seems a good candidate for evaluating if a driver is behaving correctly. So getting these 2 pieces working correctly will help the development of the real software.

Steps:
1. Install u-server (or -desktop, or -xubuntu, and I am sure any other flavor) on any hardware (well, so far I have only tested on 32 bit systems, but this issues first came out when someone running 64 os on x64 hardware. (I don't feel like asking someone to kernel panic their main box.)
2. sudo adduser foo video
3. sudo modprobe vivi
4. dmesg | grep /dev/video
[ 227.874764] vivi: V4L2 device registered as /dev/video0
4. wget http://www.linuxtv.org/downloads/video4linux/API/V4L2_API/v4l2spec/capture.c
5. gcc capture.c -o capture
6. watch grep Vmalloc /proc/meminfo

VmallocTotal: 897016 kB
VmallocUsed: 3084 kB
VmallocChunk: 893708 kB

7. in another shell: ./capture -d /dev/video0 --read

should see 100 dots: ....................................................................................................
and
VmallocUsed: 19392 kB
VmallocChunk: 877400 kB

repeat #7 and watch VmallocUsed grow
see:
VmallocUsed: 35700 52008 68316 84624 100,932 101,536

On my 128mb qemu u-server, the 6th time:

[ 6237.229155] Out of memory: kill process 4038 (sshd) score 2808 or a child
[ 6237.232929] Killed process 4039 (bash)
and klogd, dbus-daemon

[ 6237.483670] Killed process 9424 (capture)
[ 6237.699045] vmalloc (614400 pages) failed

[ 6237.722821] Killed process 9374 (watch)
and atd, syslogd, portmap, sshd*2 cron dhclient3 dd getty*5 acpid.

[ 6238.328635] Kernel panic - not syncing: Out of memory and no killable processes...
[ 6238.328685]

here is all of meminfo in a spreadsheet:
http://spreadsheets.google.com/ccc?key=pIfz0wOzPtW1-oZkXXbvcLA&hl=en

Linux dell29 2.6.24-19-generic #1 SMP Wed Aug 20 22:56:21 UTC 2008 i686 GNU/Linux

updated version of capture.c:
http://linuxtv.org/hg/v4l-dvb/file/b45ffc93fb82/v4l2-apps/test/capture_example.c

current vivi.c
http://linuxtv.org/hg/v4l-dvb/file/50e4cdc46320/linux/drivers/media/video/vivi.c

version in Intrepid:
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-intrepid.git;a=blob;f=drivers/media/video/vivi.c;h=8ba8daafd7ea2a3a2dff574b535a2908128b8d8b;hb=HEAD

[Carl Karsten (carlfk)]

any hardware, including qemu, which is how I captured the kernel messages:

$ sudo qemu -net tap -net nic,model=rtl8139 ibex-server-crashme.qcow2 -serial stdio
[ 0.000000] Kernel command line: root=UUID=1d4e76ad-ff3b-4736-8fb3-ce8e18944d61 ro console=ttyS0,9600n8

[Carl Karsten (carlfk)]

huh - on an x64 box:

carl@ibexx64:~/vga2usb/v4l.org/examples$ uname -a
Linux ibexx64 2.6.27-7-generic #1 SMP Fri Oct 24 06:40:41 UTC 2008 x86_64 GNU/Linux

carl@ibexx64:~/vga2usb/v4l.org/examples$ ./memleak.sh
malloc: ../bash/dispose_cmd.c:241: assertion botched
free: called with unallocated block argument
Aborting...Aborted

Not sure what that is about.

carl@ibexx64:~/vga2usb/v4l.org/examples$ ./capture_example --read -c 1
.
carl@ibexx64:~/vga2usb/v4l.org/examples$ grep Vmalloc /proc/meminfo
VmallocTotal: 34359738367 kB
VmallocUsed: 115508 kB
VmallocChunk: 34359622691 kB

Took it a few more times. looks like 16 total.

[Carl Karsten (carlfk)]

> carl@ibexx64:~/vga2usb/v4l.org/examples$ ./memleak.sh
> malloc: ../bash/dispose_cmd.c:241: assertion botched
> free: called with unallocated block argument
> Aborting...Aborted
> Not sure what that is about.

That is about the missing #!/bin/bash - works on 32, but whatever it is, has nothing to do with this bug.

[Charlie Kravetz (cjkgeek)]

Thanks for reporting this bug and any supporting documentation.

I did this on my PIII 1.2GHz emachines, using Xubuntu Jaunty 9.04 (Development Version). I can confirm it does happen as described.

Since this bug has enough information provided for a developer to begin work, I'm going to mark it as confirmed and let them handle it from here. Thanks for taking the time to make Ubuntu better!

[Drew Haven (drew-haven)]

I just want to confirm I'm having this problem as well in hardy with 2.6.24-16-server x86_64. I have users that run some large processing jobs on our cluster, and the Ubuntu machines I added will kernel panic with "Out of memory and no killable processes" when all the physical RAM is used up.

I do not have any swap on these systems, and in fact, they are diskless.

[penalvch (penalvch)]

Carl Karsten, this bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? If so, could you please test for this with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/ .

If it remains an issue, could you please run the following command in the development release from a Terminal (Applications->Accessories->Terminal), as it will automatically gather and attach updated debug information to this report:

apport-collect -p linux <replace-with-bug-number>

Also, could you please test the latest upstream kernel available following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue. Please do not test the kernel in the daily folder, but the one all the way at the bottom. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested and remove the tag:
needs-upstream-testing

This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the text:
needs-upstream-testing

If this bug is fixed in the mainline kernel, please add the following tags:
kernel-fixed-upstream
kernel-fixed-upstream-VERSION-NUMBER

where VERSION-NUMBER is the version number of the kernel you tested.

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-VERSION-NUMBER

If you are unable to test the mainline kernel, please comment as to why specifically you were unable to test it and add the following tags:
kernel-unable-to-test-upstream
kernel-unable-to-test-upstream-VERSION-NUMBER

Please let us know your results. Thank you for your understanding.

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]