depmod fills disk to 100% with invisible modules.ccwmap

[5zwvKGFr]

I ran update manager, it was installing a new kernel (3.2.0-54-generic). Then an error dialog/message popped up that the disk (root partition) was full, the dialog had two buttons (examine - starts baobab and skip/cancel).

At first I tried to find any big files (du didn't help, showed that only 8GB of 20GB was used). Long story short, googled a bit about why df and du might disagree and someone recommended lsof | grep deleted.

depmod (i think) had created a file and deleted it then kept writing to it filling up my root partition. In the end I had to kill it, because it just kept writing to it even when I zeroed out that deleted file.

Here's a log (shortened) of what I typed in the console:

root@localhost:/# lsof | grep deleted | grep depmod
depmod 8673 root 3w REG 8,1 14525296640 676388 /lib/modules/3.2.0-54-generic/modules.ccwmap (deleted)

root@localhost:/# cd /proc/8673/fd

root@localhost:/proc/8673/fd# ll -h | grep ccwmap
l-wx------ 1 root root 64 sep 28 16:52 3 -> /lib/modules/3.2.0-54-generic/modules.ccwmap (deleted)

root@localhost:/proc/8673/fd# > 3

root@localhost:/proc/8673/fd# df -h | grep sda1
/dev/sda1 22G 9,3G 12G 45% /

root@localhost:/proc/8673/fd# lsof | grep deleted | grep depmod
depmod 8673 root 3w REG 8,1 27424661504 676388 /lib/modules/3.2.0-54-generic/modules.ccwmap (deleted)

root@localhost:/proc/8673/fd# df -h | grep sda1
/dev/sda1 22G 22G 0 100% /

root@localhost:/proc/8673/fd# > 3

root@localhost:/proc/8673/fd# df -h | grep sda1
/dev/sda1 22G 8,8G 13G 42% /

root@localhost:/proc/8673/fd# df -h | grep sda1
/dev/sda1 22G 11G 11G 49% /

root@localhost:/proc/8673/fd# pkill depmod

Seems I'm not the only one who's seen this issue with kernel updates: http://askubuntu.com/questions/157740/updating-to-linux-image-3-2-0-26-generic-occupies-all-disk-space

---
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 2.0.1-0ubuntu17.4
Architecture: amd64
ArecordDevices:
 **** List of CAPTURE Hardware Devices ****
 card 0: PCH [HDA Intel PCH], device 0: ALC269VB Analog [ALC269VB Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: andris 2142 F.... pulseaudio
Card0.Amixer.info:
 Card hw:0 'PCH'/'HDA Intel PCH at 0xf1c00000 irq 50'
   Mixer name : 'Intel CougarPoint HDMI'
   Components : 'HDA:10ec0269,10280446,00100100 HDA:80862805,80860101,00100000'
   Controls : 25
   Simple ctrls : 12
DistroRelease: Ubuntu 12.04
EcryptfsInUse: Yes
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111012)
MachineType: Dell Inc. Dell System XPS 15Z
MarkForUpload: True
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.2.0-53-generic root=UUID=9543673d-3cae-4de0-ad02-31f9d0ae1614 ro quiet splash elevator=noop vt.handoff=7
ProcVersionSignature: Ubuntu 3.2.0-53.81-generic 3.2.50
RelatedPackageVersions:
 linux-restricted-modules-3.2.0-53-generic N/A
 linux-backports-modules-3.2.0-53-generic N/A
 linux-firmware 1.79.6
StagingDrivers: mei
Tags: precise running-unity staging
Uname: Linux 3.2.0-53-generic x86_64
UpgradeStatus: Upgraded to precise on 2012-06-02 (482 days ago)
UserGroups: adm admin cdrom dialout fuse lpadmin plugdev sambashare vboxusers wireshark
dmi.bios.date: 08/11/2011
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A05
dmi.board.name: 0MFNCV
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 8
dmi.chassis.vendor: Dell Inc.
dmi.chassis.version: 0.1
dmi.modalias: dmi:bvnDellInc.:bvrA05:bd08/11/2011:svnDellInc.:pnDellSystemXPS15Z:pvr:rvnDellInc.:rn0MFNCV:rvrA00:cvnDellInc.:ct8:cvr0.1:
dmi.product.name: Dell System XPS 15Z
dmi.sys.vendor: Dell Inc.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1232473

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[beetlebolt (beetlebolt)]

I'm running 12.04. When I ran update manager today it tried to install kernel 3.8.0-33 which very quickly filled the hdd to 100% and this particular file had become 35G:
/lib/modules/3.8.0-33-generic/modules.ccwmap

in htop I saw that depmod was using 100% of one of the cores.

I've since deleted that file and apt-get purged everything related to this kernel.

[Lauri Tirkkonen (lotheac)]

We also ran into this on one of our machines. depmod seems to be writing to the file in question in a loop:

    # strace -s 100 -p $(pgrep depmod)
    write(3, "0x00\nnvidia_304 0x2887 0xfbad 0x00 0x0000 0x00\nnvidia_304 0x2887 "..., 4096) = 4096

    Attaching to process 17565
    Reading symbols from /sbin/depmod...(no debugging symbols found)...done.
    Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.15.so...done.
    done.
    Loaded symbols for /lib/x86_64-linux-gnu/libc.so.6
    Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/lib/x86_64-linux-gnu/ld-2.15.so...done.
    done.
    Loaded symbols for /lib64/ld-linux-x86-64.so.2
    0x00007fed6ba89040 in __write_nocancel () at ../sysdeps/unix/syscall-template.S:82
    82 ../sysdeps/unix/syscall-template.S: No such file or directory.
    (gdb) bt
    #0 0x00007fed6ba89040 in __write_nocancel () at ../sysdeps/unix/syscall-template.S:82
    #1 0x00007fed6ba1b883 in _IO_new_file_write (f=0x18164b0, data=0x7fed621c2000, n=4096) at fileops.c:1289
    #2 0x00007fed6ba1b74a in new_do_write (fp=0x18164b0,
        data=0x7fed621c2000 "0x00\nnvidia_304", ' ' <repeats 11 times>, "0x2887 0xfbad 0x00 0x0000 0x00\nnvidia_304", ' ' <repeats 11 times>, "0x2887 0xfbad 0x00 0x0000 0x00\nnvidia_304", ' ' <repeats 11 times>, "0x2887 0xfbad 0x00 0x0000 0x00\nnvi"..., to_do=4096) at fileops.c:543
    #3 0x00007fed6ba1ceb5 in _IO_new_do_write (fp=<optimized out>, data=<optimized out>, to_do=4096) at fileops.c:516
    #4 0x00007fed6ba1c025 in _IO_new_file_xsputn (n=4, data=<optimized out>, f=0x18164b0) at fileops.c:1371
    #5 _IO_new_file_xsputn (f=0x18164b0, data=<optimized out>, n=4) at fileops.c:1305
    #6 0x00007fed6b9ec4a7 in _IO_vfprintf_internal (s=<optimized out>, format=<optimized out>, ap=<optimized out>) at vfprintf.c:1660
    #7 0x00007fed6baab80b in ___fprintf_chk (fp=0x18164b0, flag=1, format=<optimized out>) at fprintf_chk.c:37
    #8 0x0000000000405a24 in ?? ()
    #9 0x00000000004021a4 in ?? ()
    #10 0x00007fed6b9c376d in __libc_start_main (main=0x401770, argc=5, ubp_av=0x7fff69f88ea8, init=<optimized out>, fini=<optimized out>,
        rtld_fini=<optimized out>, stack_end=0x7fff69f88e98) at libc-start.c:226
    #11 0x0000000000402535 in ?? ()
    #12 0x00007fff69f88e98 in ?? ()
    #13 0x000000000000001c in ?? ()
    #14 0x0000000000000005 in ?? ()
    #15 0x00007fff69f8ae05 in ?? ()
    #16 0x00007fff69f8ae12 in ?? ()
    #17 0x00007fff69f8ae26 in ?? ()
    #18 0x00007fff69f8ae29 in ?? ()
    #19 0x00007fff69f8ae15 in ?? ()
    #20 0x0000000000000000 in ?? ()

I'll build module-init-tools with debug symbols and try to reproduce.

[Lauri Tirkkonen (lotheac)]

Unfortunately I could not reproduce this after building module-init-tools with debug symbols.

[penalvch (penalvch)]

Andris Zāģeris, as per your https://launchpadlibrarian.net/151741411/BootDmesg.txt :
[ 0.927247] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
...
[ 3.325597] [Firmware Bug]: ACPI(PEGP) defines _DOD but not _DOS

Hence, as per http://www.dell.com/support/drivers/us/en/19/Product/xps-15z an update is available for your BIOS (A12). If you update to this following https://help.ubuntu.com/community/BiosUpdate , does it change anything? If it doesn't, could you please both specify what happened, and just provide the output of the following terminal command:
sudo dmidecode -s bios-version && sudo dmidecode -s bios-release-date

Please note your current BIOS is already in the Bug Description, so posting this on the old BIOS would not be helpful.

For more on BIOS updates and linux, please see https://help.ubuntu.com/community/ReportingBugs#Bug_reporting_etiquette .

Thank you for your understanding.

[5zwvKGFr (5zwvkgfr-deactivatedaccount)]

@penalvch

I haven't encountered this issue since I made this report (there have been several kernel updates and the issue hasn't repeated). So I'd rather not update my BIOS at this time (some risk involved, also time investment doing backups, etc.).

I guess we can just let this bug expire (unless this issue still affects other users).

[penalvch (penalvch)]

Andris Zāģeris, thank you for your comments. If other people have a bug, they would need to file a new report. Hence, this bug report is being closed due to your last comment https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1232473/comments/28 regarding this being fixed with an update. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.

[Mario Rohkrämer (ligh-de)]

Occasionally, this happens to me too. Currently, updating Ubuntu "PP" 12.04 LTS with kernel 3.13.0-34-generic. The result is /lib/modules/

Unfortunately, because it usually gets fixed after a forced reboot, the best time to collect information about this bug would be when it happens. But what should I do to collect useful information for the developers?

I will probably archive the content of /var/log/apt/ for now.

[Mario Rohkrämer (ligh-de)]

Oops, incomplete:

The result is /lib/modules/3.13.0-34-generic/modules.ccwmap filling the root FS with the remaining 22GB (which were free space before updating).

[penalvch (penalvch)]

Mario Rohkrämer, thank you for your comment. Unfortunately, this bug report is not scoped to you, or your problem. So your hardware and problem may be tracked, could you please file a new report with Ubuntu by executing the following in a terminal while booted into the default Ubuntu kernel (not a mainline one) via:
ubuntu-bug linux

For more on this, please read the official Ubuntu documentation:
Ubuntu Bug Control and Ubuntu Bug Squad: https://wiki.ubuntu.com/Bugs/BestPractices#X.2BAC8-Reporting.Focus_on_One_Issue
Ubuntu Kernel Team: https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Filing_Kernel_Bug_reports
Ubuntu Community: https://help.ubuntu.com/community/ReportingBugs#Bug_reporting_etiquette

When opening up the new report, please feel free to subscribe me to it.

As well, please do not announce in this report you created a new bug report.

Thank you for your understanding.

Helpful bug reporting tips:
https://wiki.ubuntu.com/ReportingBugs