kernel crash with 0010:ovl_open_realfile+0x4a/0x150 [overlay] in Qemu with focal daily

Bug #1877575 reported by Jean-Baptiste Lallement
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Expired
Critical
Unassigned
qemu (Ubuntu)
Invalid
Critical
Unassigned

Bug Description

Focal daily 20200507 (started earlier)
Kernel is 5.4.0-30.34

Kernel crashes on boot in Qemu with:

[ 4.019922] RIP: 0010:ovl_open_realfile+0x4a/0x150 [overlay]
[ 4.022101] Code: 44 8b 6f 40 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 49 8b 7e 28 41 81 cd 00 00 04 04 e8 9d d4 ff ff 49 89 c7 48 8b 45 c8 <48> 8b 40 68 48 81 78 60 62 6a 65 6a 0f 84 a6 00 00 00 65 48 8b 04
[ 4.027487] RSP: 0018:ffffac44c0707c08 EFLAGS: 00010202
[ 4.029424] RAX: 0f230ca096f12900 RBX: ffff9e44e9c2eaa0 RCX: 0000000000000001
[ 4.031645] RDX: ffff9e44ea9ae540 RSI: ffff9e44e9c2eaa0 RDI: ffff9e44fb281f00
[ 4.033811] RBP: ffffac44c0707c50 R08: ffff9e44e9e3cc00 R09: 0000000000000000
[ 4.036428] R10: ffff9e44ea9ae540 R11: 0000000000000000 R12: ffff9e44f9ec9c00
[ 4.038737] R13: 0000000004048002 R14: ffff9e44eaa51b00 R15: ffff9e44e9e3cc00
[ 4.046087] FS: 00007f91962a6600(0000) GS:ffff9e44fbb00000(0000) knlGS:0000000000000000
[ 4.051277] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.055455] CR2: 000056295fbc4a59 CR3: 0000000169c80000 CR4: 00000000000006e0
[ 4.059614] general protection fault: 0000 [#4] SMP NOPTI
[ 4.061700] CPU: 1 PID: 360 Comm: cp Tainted: G D 5.4.0-30-generic #34-Ubuntu
[ 4.063016] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-1ubuntu1 04/01/2014
[ 4.063016] RIP: 0010:ovl_open_realfile+0x4a/0x150 [overlay]
[ 4.063016] Code: 44 8b 6f 40 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 49 8b 7e 28 41 81 cd 00 00 04 04 e8 9d d4 ff ff 49 89 c7 48 8b 45 c8 <48> 8b 40 68 48 81 78 60 62 6a 65 6a 0f 84 a6 00 00 00 65 48 8b 04
[ 4.063016] RSP: 0018:ffffac44c0373c08 EFLAGS: 00010206
[ 4.063016] RAX: 1579f0d048c0c300 RBX: ffff9e44eaa3f478 RCX: 0000000000000000
[ 4.063016] RDX: 0000000000000000 RSI: ffff9e44eaa3f478 RDI: ffff9e44fb281f00
[ 4.063016] RBP: ffffac44c0373c50 R08: ffff9e44fb3ac6c0 R09: 0000000000000000
[ 4.063016] R10: 0000000000000001 R11: 0000007461642e73 R12: ffff9e44f9b2d700
[ 4.063016] R13: 0000000004048000 R14: ffff9e44fa3d65c0 R15: ffff9e44fb3ac6c0
[ 4.063016] FS: 00007f91962a6600(0000) GS:ffff9e44fbb00000(0000) knlGS:0000000000000000
[ 4.063016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.063016] CR2: 000056295fbc4a59 CR3: 0000000169c80000 CR4: 00000000000006e0
[ 4.063016] Call Trace:
[ 4.063016] ? ovl_llseek+0x120/0x120 [overlay]
[ 4.063016] ovl_open+0x57/0x90 [overlay]
[ 4.063016] do_dentry_open+0x143/0x3a0
[ 4.063016] vfs_open+0x2d/0x30
[ 4.063016] do_last+0x194/0x900
[ 4.063016] path_openat+0x8d/0x290
[ 4.063016] do_filp_open+0x91/0x100
[ 4.063016] ? __alloc_fd+0x46/0x150
[ 4.063016] do_sys_open+0x17e/0x290
[ 4.063016] __x64_sys_openat+0x20/0x30
[ 4.124120] do_syscall_64+0x57/0x190
[ 4.124120] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 4.124120] RIP: 0033:0x7f91961c3d1b
[ 4.124120] Code: 25 00 00 41 00 3d 00 00 41 00 74 4b 64 8b 04 25 18 00 00 00 85 c0 75 67 44 89 e2 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 91 00 00 00 48 8b 4c 24 28 64 48 33 0c 25
[ 4.124120] RSP: 002b:00007ffebc71f1b0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
[ 4.124120] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f91961c3d1b
[ 4.124120] RDX: 0000000000000000 RSI: 0000562960241670 RDI: 00000000ffffff9c
[ 4.124120] RBP: 0000562960241670 R08: 0000000000000001 R09: 0000000000000013
[ 4.124120] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 4.124120] R13: 00000000ffffffff R14: 00007ffebc71f298 R15: 0000000000000000
[ 4.124120] Modules linked in: overlay nls_utf8 isofs dm_mirror dm_region_hash dm_log hid_generic usbhid hid qxl ttm drm_kms_helper ahci syscopyarea sysfillrect virtio_net net_failover i2c_i801 libahci psmouse lpc_ich sysimgblt fb_sys_fops failover virtio_blk drm

Reproduced on bionic and focal hosts.
Groovy daily boots fine.
boot message attached.

ProblemType: Bug
DistroRelease: Ubuntu 20.10
Package: qemu 1:4.2-3ubuntu6
ProcVersionSignature: Ubuntu 5.4.0-18.22-generic 5.4.24
Uname: Linux 5.4.0-18-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu32
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Fri May 8 15:36:38 2020
Dependencies:

InstallationDate: Installed on 2014-07-15 (2124 days ago)
InstallationMedia: Ubuntu 14.10 "Utopic Unicorn" - Alpha amd64 (20140520)
MachineType: Gigabyte Technology Co., Ltd. GA-890GPA-UD3H
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-18-generic root=UUID=0889d734-ea91-4bdb-9d16-2a0a923ad3d1 ro
SourcePackage: qemu
UpgradeStatus: Upgraded to groovy on 2018-03-24 (775 days ago)
dmi.bios.date: 07/23/2010
dmi.bios.vendor: Award Software International, Inc.
dmi.bios.version: FD
dmi.board.name: GA-890GPA-UD3H
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.modalias: dmi:bvnAwardSoftwareInternational,Inc.:bvrFD:bd07/23/2010:svnGigabyteTechnologyCo.,Ltd.:pnGA-890GPA-UD3H:pvr:rvnGigabyteTechnologyCo.,Ltd.:rnGA-890GPA-UD3H:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvr:
dmi.product.name: GA-890GPA-UD3H
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :
tags: added: focal rls-ff-incoming
Changed in linux (Ubuntu):
importance: Undecided → Critical
Changed in qemu (Ubuntu):
importance: Undecided → Critical
summary: - kernel crash in Qemu with focal daily
+ kernel crash with 0010:ovl_open_realfile+0x4a/0x150 [overlay] in Qemu
+ with focal daily
Revision history for this message
Paride Legovini (paride) wrote :

This is happening with the Focal live-server images too, starting from the image with serial 20200506. The previous daily (20200505) booted and installed fine.

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Status changed to Confirmed

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in qemu (Ubuntu):
status: New → Confirmed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

This is reported against 5.4.0-30.34 (in the guest I assume).
Per [1] that date window is the change from

2020-05-08 12:58:43 Published Focal proposed main devel 5.4.0-31.35
2020-05-08 13:00:19 Superseded Focal proposed main devel 5.4.0-30.34
2020-05-05 12:18:15 Published Focal security main devel 5.4.0-29.33
2020-05-04 19:13:24 Published Focal updates main devel 5.4.0-29.33

Given the crash it is either the new proposed kernel if it is in there or something in how the overlayfs is created (that is where the crashes are).

I checked myself the images from [2][3][4] and they all worked for me.
This boots for example:
$ sudo qemu-system-x86_64 -m 1024 -cpu host -smp 4 -enable-kvm -hda focal-server-cloudimg-amd64-20200507.img -boot d -curses

@Jobel/Paride - could you add which images and which commandline you used?

[1]: https://launchpad.net/ubuntu/+source/linux/+publishinghistory
[2]: https://cloud-images.ubuntu.com/focal/20200505/focal-server-cloudimg-amd64.img
[3]: https://cloud-images.ubuntu.com/focal/20200506/focal-server-cloudimg-amd64.img
[4]: https://cloud-images.ubuntu.com/focal/20200507/focal-server-cloudimg-amd64.img

Changed in qemu (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Paride Legovini (paride) wrote :

Speaking for server: the images are working now. I think the images were built with the proposed kernel and this caused trouble.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

As it seems the issue is neither triggered nor resolvable by qemu - marking the bug task invalid.

Changed in qemu (Ubuntu):
status: Incomplete → Invalid
Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

The bug automagically fixed itself. Setting to incomplete until it expires or can be reproduced.

Changed in linux (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

Changed in linux (Ubuntu):
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.