Comment 0 for bug 1813244

Revision history for this message
Po-Hsu Lin (cypressyew) wrote : systemd logrotate cause kernel trace "BUG: unable to handle kernel paging request at 6db23a14" on Cosmic i386

This issue was found after leaving a SUT to run overnight (was testing the ubuntu_cts_kernel test 13 hours ago before this happens). Can't tell if this is a regression, as I haven't find a way to reproduce it. But I do see similar reports on the Internet [1]

After checking the systemd service timers, it looks like this is caused by the logrotate.service:

$ sudo systemctl list-timers --all
NEXT LEFT LAST PASSED UNIT ACTIVATES
Fri 2019-01-25 06:18:58 UTC 1h 40min left Thu 2019-01-24 06:34:15 UTC 22h ago apt-daily-upgrade.timer apt-daily-upgrade.service
Fri 2019-01-25 09:15:54 UTC 4h 37min left Fri 2019-01-25 03:43:24 UTC 55min ago apt-daily.timer apt-daily.service
Fri 2019-01-25 17:02:47 UTC 12h left Fri 2019-01-25 04:20:17 UTC 18min ago motd-news.timer motd-news.service
Sat 2019-01-26 00:00:00 UTC 19h left Fri 2019-01-25 00:00:37 UTC 4h 38min ago logrotate.timer logrotate.service
Sat 2019-01-26 04:02:38 UTC 23h left Fri 2019-01-25 04:02:38 UTC 36min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.serv
Mon 2019-01-28 00:00:00 UTC 2 days left Wed 2019-01-23 10:44:18 UTC 1 day 17h ago fstrim.timer fstrim.service
n/a n/a n/a n/a snapd.snap-repair.timer snapd.snap-repair.service

7 timers listed.

After this happens, you won't be able to reboot it with the command.
$ sudo reboot
sudo: unable to resolve host onza: Resource temporarily unavailable
Killed

Here is the error message, please refer to the attachment for a complete syslog:

Jan 25 00:00:37 onza systemd[1]: Starting Rotate log files...
Jan 25 00:00:37 onza kernel: [45282.956634] BUG: unable to handle kernel paging request at 6db23a14
Jan 25 00:00:37 onza kernel: [45282.962902] *pdpt = 0000000033fa1001 *pde = 0000000000000000
Jan 25 00:00:37 onza kernel: [45282.968650] Oops: 0000 [#1] SMP
Jan 25 00:00:37 onza kernel: [45282.971796] CPU: 0 PID: 407 Comm: systemd-journal Not tainted 4.18.0-14-generic #15-Ubuntu
Jan 25 00:00:37 onza kernel: [45282.980330] Hardware name: Dell Inc. PowerEdge R310/05XKKK, BIOS 1.11.0 09/18/2012
Jan 25 00:00:37 onza kernel: [45282.988127] EIP: __kmalloc+0xc9/0x240
Jan 25 00:00:37 onza kernel: [45282.991785] Code: 4d e0 8d 4a 01 31 c6 8b 45 f0 89 75 dc 31 f3 8b 37 64 0f c7 0e 0f 94 c0 84 c0 74 b9 8b 75 e4 8b 45 dc 39 45 e0 74 0e 03 5f 14 <33> 1b 33 9f b4 00 00 00 0f 18 03 f7 45 ec 00 80 00 00 0f 85 3f 01
Jan 25 00:00:37 onza kernel: [45283.011254] EAX: 9e20b374 EBX: 6db23a14 ECX: 0000350d EDX: 0000350c
Jan 25 00:00:37 onza kernel: [45283.017769] ESI: eac03a00 EDI: eac03a00 EBP: f461fd9c ESP: f461fd74
Jan 25 00:00:37 onza kernel: [45283.024304] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010206
Jan 25 00:00:37 onza kernel: [45283.031245] CR0: 80050033 CR2: 6db23a14 CR3: 29d304a0 CR4: 000006f0
Jan 25 00:00:37 onza kernel: [45283.037770] Call Trace:
Jan 25 00:00:37 onza kernel: [45283.040395] ? ext4_htree_store_dirent+0x2e/0x120
Jan 25 00:00:37 onza kernel: [45283.045255] ext4_htree_store_dirent+0x2e/0x120
Jan 25 00:00:37 onza kernel: [45283.049951] htree_dirblock_to_tree+0xd2/0x230
Jan 25 00:00:37 onza kernel: [45283.054562] ext4_htree_fill_tree+0x7f/0x2c0
Jan 25 00:00:37 onza kernel: [45283.058991] ? kmem_cache_alloc_trace+0x167/0x1e0
Jan 25 00:00:37 onza kernel: [45283.063849] ? ext4_readdir+0x9d4/0xb10
Jan 25 00:00:37 onza kernel: [45283.067854] ext4_readdir+0x6aa/0xb10
Jan 25 00:00:37 onza kernel: [45283.071687] ? _copy_to_user+0x21/0x30
Jan 25 00:00:37 onza kernel: [45283.075597] ? do_statfs64+0x34/0x60
Jan 25 00:00:37 onza kernel: [45283.079334] ? security_file_permission+0x7c/0xb0
Jan 25 00:00:37 onza kernel: [45283.084306] iterate_dir+0x15c/0x1a0
Jan 25 00:00:37 onza kernel: [45283.087883] ksys_getdents64+0x6f/0x110
Jan 25 00:00:37 onza kernel: [45283.091853] ? iterate_dir+0x1a0/0x1a0
Jan 25 00:00:37 onza kernel: [45283.095771] sys_getdents64+0x16/0x20
Jan 25 00:00:37 onza kernel: [45283.099595] do_fast_syscall_32+0x7f/0x1e0
Jan 25 00:00:37 onza kernel: [45283.103858] entry_SYSENTER_32+0x4e/0x7c
Jan 25 00:00:37 onza kernel: [45283.107938] EIP: 0xb7ef4d41
Jan 25 00:00:37 onza kernel: [45283.110898] Code: f6 ff ff 55 89 e5 8b 55 08 8b 80 5c cd ff ff 85 d2 74 02 89 02 5d c3 8b 04 24 c3 8b 1c 24 c3 90 90 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d 76 00 58 b8 77 00 00 00 cd 80 90 8d 76
Jan 25 00:00:37 onza kernel: [45283.130298] EAX: ffffffda EBX: 00000021 ECX: 02219dec EDX: 00008000
Jan 25 00:00:37 onza kernel: [45283.136824] ESI: ffffffb4 EDI: 00000000 EBP: 02219dd0 ESP: bff8f5e8
Jan 25 00:00:37 onza kernel: [45283.143245] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000296
Jan 25 00:00:37 onza kernel: [45283.150299] Modules linked in: iptable_nat ip_gre gre ip_tunnel dummy iptable_filter xt_recent bpfilter openvswitch nsh nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_conncount nf_nat nf_conntrack intel_powerclamp coretemp ipmi_ssif kvm_intel kvm dcdbas ipmi_si joydev irqbypass intel_cstate input_leds ipmi_devintf i7core_edac ipmi_msghandler acpi_power_meter mac_hid sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 i2c_algo_bit ttm hid_generic drm_kms_helper gpio_ich syscopyarea sysfillrect sysimgblt mpt3sas fb_sys_fops drm
Jan 25 00:00:37 onza kernel: [45283.222723] usbhid raid_class pata_acpi lpc_ich hid scsi_transport_sas bnx2 wmi
Jan 25 00:00:37 onza kernel: [45283.230378] CR2: 000000006db23a14
Jan 25 00:00:37 onza kernel: [45283.233894] ---[ end trace a12425d45f6ac772 ]---

[1] https://forum.manjaro.org/t/systemd-timer-crashing-my-system-at-midnight/19609
[2]

ProblemType: Bug
DistroRelease: Ubuntu 18.10
Package: linux-image-4.18.0-14-generic 4.18.0-14.15
ProcVersionSignature: User Name 4.18.0-14.15-generic 4.18.20
Uname: Linux 4.18.0-14-generic i686
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Jan 25 03:48 seq
 crw-rw---- 1 root audio 116, 33 Jan 25 03:48 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.10-0ubuntu13.1
Architecture: i386
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CurrentDmesg:

Date: Fri Jan 25 04:18:08 2019
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
MachineType: Dell Inc. PowerEdge R310
PciMultimedia:

ProcFB: 0 mgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.18.0-14-generic root=UUID=6aaa11f6-d386-4c0c-b4b8-38e6c408980a ro console=ttyS0,115200n8
RelatedPackageVersions:
 linux-restricted-modules-4.18.0-14-generic N/A
 linux-backports-modules-4.18.0-14-generic N/A
 linux-firmware 1.175.1
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/18/2012
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 1.11.0
dmi.board.name: 05XKKK
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr1.11.0:bd09/18/2012:svnDellInc.:pnPowerEdgeR310:pvr:rvnDellInc.:rn05XKKK:rvrA05:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R310
dmi.sys.vendor: Dell Inc.