[Ubuntu24.04] virsh detach-interface is crashing the guest

[bugproxy]

== Comment: #0 - Kowshik Jois B S <email address hidden> - 2024-05-28 01:07:02 ==
---Problem Description---
While trying virsh attach-interface and virsh detach-interface, It is observed that, attaching an interface is successful. But trying to detach the same results in the guest crash with the below trace messages on the console.

root@ubuntulp3guest1:~# [ 5363.726428] Kernel attempted to read user page (10ec00000058) - exploit attempt? (uid: 0)
[ 5363.726570] BUG: Unable to handle kernel data access on read at 0x10ec00000058
[ 5363.726662] Faulting instruction address: 0xc0000000012d4828
[ 5363.726739] Oops: Kernel access of bad area, sig: 11 [#1]
[ 5363.726800] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries
[ 5363.726880] Modules linked in: 8139too 8139cp mii qrtr cfg80211 binfmt_misc uio_pdrv_genirq vmx_crypto uio dm_multipath nfnetlink ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 poly1305_p10_crypto chacha_p10_crypto libchacha crct10dif_vpmsum crc32c_vpmsum xhci_pci xhci_pci_renesas aes_gcm_p10_crypto
[ 5363.727302] CPU: 0 PID: 1614 Comm: drmgr Not tainted 6.8.0-31-generic #31-Ubuntu
[ 5363.727426] Hardware name: IBM pSeries (emulated by qemu) POWER10 (raw) 0x800200 0xf000006 of:SLOF,HEAD hv:linux,kvm pSeries
[ 5363.727563] NIP: c0000000012d4828 LR: c0000000012d68f0 CTR: 0000000000000000
[ 5363.727653] REGS: c0000000149cb440 TRAP: 0300 Not tainted (6.8.0-31-generic)
[ 5363.727742] MSR: 800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 44088282 XER: 20040000
[ 5363.727855] CFAR: c0000000012d68ec DAR: 000010ec00000058 DSISR: 40000000 IRQMASK: 0
[ 5363.727855] GPR00: c0000000012d68f0 c0000000149cb6e0 c000000002254800 000010ec00000048
[ 5363.727855] GPR04: c0000000149cb748 0000000000000000 0000000000000000 0000000000000000
[ 5363.727855] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 5363.727855] GPR12: 0000000000000000 c000000003e80000 0000000000000000 0000000000000000
[ 5363.727855] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 5363.727855] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 5363.727855] GPR24: 0000000000000000 0000000000000000 c0000000048585a0 c0000000149cb7d4
[ 5363.727855] GPR28: 0000000000000001 c000000014de9400 000010ec00000048 0000000000000000
[ 5363.728644] NIP [c0000000012d4828] __of_changeset_entry_invert+0x10/0x1ac
[ 5363.728732] LR [c0000000012d68f0] __of_changeset_revert_entries+0x98/0x180
[ 5363.728813] Call Trace:
[ 5363.728845] [c0000000149cb7b0] [c0000000012d6b60] of_changeset_revert+0x58/0xd8
[ 5363.728937] [c0000000149cb800] [c000000000d0d498] of_pci_remove_node+0x74/0xb0
[ 5363.729029] [c0000000149cb830] [c000000000cdbde0] pci_stop_bus_device+0xf4/0x138
[ 5363.729126] [c0000000149cb870] [c000000000cdbf40] pci_stop_and_remove_bus_device_locked+0x34/0x64
[ 5363.729232] [c0000000149cb8a0] [c000000000cf2950] remove_store+0xf0/0x108
[ 5363.729311] [c0000000149cb8f0] [c000000000e88384] dev_attr_store+0x34/0x78
[ 5363.729389] [c0000000149cb910] [c0000000007f8234] sysfs_kf_write+0x70/0xa4
[ 5363.729467] [c0000000149cb930] [c0000000007f66a8] kernfs_fop_write_iter+0x1d0/0x2e0
[ 5363.729558] [c0000000149cb980] [c0000000006c8fc8] vfs_write+0x27c/0x558
[ 5363.729639] [c0000000149cba30] [c0000000006c9628] ksys_write+0x90/0x170
[ 5363.729716] [c0000000149cba80] [c000000000033248] system_call_exception+0xf8/0x290
[ 5363.729811] [c0000000149cbe50] [c00000000000d05c] system_call_vectored_common+0x15c/0x2ec
[ 5363.729903] --- interrupt: 3000 at 0x74191e15c720
[ 5363.729964] NIP: 000074191e15c720 LR: 000074191e15c720 CTR: 0000000000000000
[ 5363.730053] REGS: c0000000149cbe80 TRAP: 3000 Not tainted (6.8.0-31-generic)
[ 5363.730143] MSR: 800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE> CR: 48088202 XER: 00000000
[ 5363.730257] IRQMASK: 0
[ 5363.730257] GPR00: 0000000000000004 00007ffffbdfb730 000074191e296d00 000000000000000b
[ 5363.730257] GPR04: 00000be4ed58d640 0000000000000001 00000000ffffffff 0000000000000031
[ 5363.730257] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 5363.730257] GPR12: 0000000000000000 000074191e3eb300 0000000000000000 0000000000000000
[ 5363.730257] GPR16: 0000000000000000 00000be4b90f2de0 00000be4b90f0298 00000be4b90f2da0
[ 5363.730257] GPR20: 00000be4b90f11b8 00000be4b90eff08 00007ffffbdfb910 00000be4b90f2220
[ 5363.730257] GPR24: 00000be4b90f2da8 00000be4ed54f440 000074191e28e0c0 0000000000000001
[ 5363.730257] GPR28: 00000be4ed4902a0 00000be4ed58d640 0000000000000001 00007ffffbdfb730
[ 5363.730991] NIP [000074191e15c720] 0x74191e15c720
[ 5363.731055] LR [000074191e15c720] 0x74191e15c720
[ 5363.731115] --- interrupt: 3000
[ 5363.731161] Code: 60000000 4bfffec8 7c0802a6 60000000 f8630000 f8630008 38600000 4e800020 3c4c00f8 3842ffe8 7c0802a6 60000000 <e9430010> e9230018 e8e30000 e9030008
[ 5363.731337] ---[ end trace 0000000000000000 ]---
[ 5363.734069] pstore: backend (nvram) writing error (-1)
[ 5363.734143]
[ 5364.734175] Kernel panic - not syncing: Fatal exception

root@ubuntulp3host:~#

---uname output---
Linux ubuntulp3host 6.8.0-31-generic #31-Ubuntu SMP Sat Apr 20 00:05:55 UTC 2024 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = P10 Denali

---Steps to Reproduce---
1. Create a Ubuntu 24.04 guest on Ubuntu 24.04 Host
2. attach an interface
    virsh attach-interface Ubuntu bridge --source virbr0
3. detach the attached interface
    virsh detach-interface Ubuntu bridge 52:54:00:3d:89:5a

Expected output:
interface should get detached by leaving the guest in running state

Actual output:
command output was seen as "Interface detached successfully" but the guest crashed with the above trace messages on the console.

Contact Information = <email address hidden>

Could you please help mirror this bug to Canonical?

Context: As per the ongoing discussion over mailing list, Ubuntu kernel team willll be disabling the config: CONFIG_PCI_DYNAMIC_OF_NODES in next updates of the kernel.

https://<email address hidden>/

Thanks,

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Libera.chat.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/2075721/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Frank Heimes (fheimes)]

I just noticed that the Canonical kernel team has already a Launchpad bug open on this and already started to work on it (it's meanwhile Fix Committed for noble/24.04 and oracular/24.10).
So I'm marking this bug as a duplicate of the kernel team's Launchpad bug: https://bugs.launchpad.net/bugs/2074376

I'll keep this bug status updated and aligned with LP#2074376,
so that the synched IBM BZ entry with be updated accordingly.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2024-08-09 11:36 EDT-------
I have tested this scenario with the noble-proposed kernel. I could attach and detach interfaces successfully. No crash/trace messages found.

Guest Env:
===========
Linux ubuntu 6.8.0-43-generic #43-Ubuntu SMP Fri Aug 2 19:46:18 UTC 2024 ppc64le ppc64le ppc64le GNU/Linux

root@ubuntu:~# cat /boot/config-6.8.0-43-generic | grep PCI_DYNAMIC
# CONFIG_PCI_DYNAMIC_OF_NODES is not set

Before Attaching the Interface:
=================================

root@ubuntu:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:24:e5:58 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.187/24 metric 100 brd 192.168.122.255 scope global dynamic enp0s1
valid_lft 2749sec preferred_lft 2749sec
inet6 fe80::5054:ff:fe24:e558/64 scope link
valid_lft forever preferred_lft forever

After Attaching the Interface:
=================================

# virsh attach-interface Ubuntu2404 bridge --source virbr0
Interface attached successfully

root@ubuntu:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:24:e5:58 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.187/24 metric 100 brd 192.168.122.255 scope global dynamic enp0s1
valid_lft 2738sec preferred_lft 2738sec
inet6 fe80::5054:ff:fe24:e558/64 scope link
valid_lft forever preferred_lft forever
3: enp0s7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 52:54:00:96:d9:83 brd ff:ff:ff:ff:ff:ff

After Detaching the Interface:
=================================

# virsh detach-interface Ubuntu2404 bridge 52:54:00:96:d9:83
Interface detached successfully

root@ubuntu:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:24:e5:58 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.187/24 metric 100 brd 192.168.122.255 scope global dynamic enp0s1
valid_lft 2720sec preferred_lft 2720sec
inet6 fe80::5054:ff:fe24:e558/64 scope link
valid_lft forever preferred_lft forever

[Frank Heimes (fheimes)]

Many thanks Kowshik Jois for the successful verification!