Activity log for bug #1415178

Date Who What changed Old value New value Message
2015-01-27 19:39:19 bugproxy bug added bug
2015-01-27 19:50:20 bugproxy tags architecture-ppc64le bugnameltc-115767 severity-medium targetmilestone-inin1504
2015-01-27 19:55:09 Luciano Chavez affects ubuntu linux (Ubuntu)
2015-01-28 17:42:03 Joseph Salisbury tags architecture-ppc64le bugnameltc-115767 severity-medium targetmilestone-inin1504 architecture-ppc64le bugnameltc-115767 kernel-da-key severity-medium targetmilestone-inin1504
2015-01-29 15:10:24 Chris J Arges nominated for series Ubuntu Utopic
2015-01-29 15:10:24 Chris J Arges bug task added linux (Ubuntu Utopic)
2015-01-29 15:10:24 Chris J Arges nominated for series Ubuntu Trusty
2015-01-29 15:10:24 Chris J Arges bug task added linux (Ubuntu Trusty)
2015-01-29 15:14:47 Chris J Arges bug task deleted linux (Ubuntu Trusty)
2015-02-02 21:24:26 Chris J Arges linux (Ubuntu): importance Undecided Medium
2015-04-02 13:16:35 Chris J Arges linux (Ubuntu): status New Confirmed
2015-06-10 18:35:53 Chris J Arges bug task deleted linux (Ubuntu Utopic)
2015-06-10 18:35:58 Chris J Arges linux (Ubuntu): assignee Chris J Arges (arges)
2015-06-10 18:36:01 Chris J Arges linux (Ubuntu): status Confirmed In Progress
2015-06-10 18:36:15 Chris J Arges nominated for series Ubuntu Vivid
2015-06-10 18:36:15 Chris J Arges bug task added linux (Ubuntu Vivid)
2015-06-10 18:38:22 Chris J Arges description pKVM version: [root@yangtze-lp1 ~]# cat /etc/issue IBM_PowerKVM release 2.1.1 build 10 alpha (pkvm2_1_1) Kernel \r on a \m (\l) Ubuntu version: root@ubuntushinner:~# uname -a Linux ubuntushinner 3.16.0-14-generic #20-Ubuntu SMP Sat Sep 6 23:45:12 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux steps to reproduce: 1. rmmod the ipr driver and kernel traces are seen on the syslog. root@ubuntushinner:~# lsmod Module Size Used by pseries_rng 2849 0 ses 9046 0 enclosure 11198 1 ses rtc_generic 2249 0 ipr 140038 0 ohci_pci 6794 0 root@ubuntushinner:~# uname -a Linux ubuntushinner 3.16.0-14-generic #20-Ubuntu SMP Sat Sep 6 23:45:12 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux root@ubuntushinner:~# rmmod ipr root@ubuntushinner:~# lsscsi [0:0:0:0] disk QEMU QEMU HARDDISK 2.0. /dev/sda [0:0:0:1] cd/dvd QEMU QEMU CD-ROM 2.0. /dev/sr0 syslog: Sep 9 02:57:57 ubuntushinner kernel: [ 576.546878] kernfs: can not remove 'device', no directory Sep 9 02:57:57 ubuntushinner kernel: [ 576.546903] ------------[ cut here ]------------ Sep 9 02:57:57 ubuntushinner kernel: [ 576.546906] WARNING: at /build/buildd/linux-3.16.0/fs/kernfs/dir.c:1220 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546908] Modules linked in: pseries_rng ses enclosure rtc_generic ipr(-) ohci_pci Sep 9 02:57:57 ubuntushinner kernel: [ 576.546922] CPU: 1 PID: 3537 Comm: rmmod Not tainted 3.16.0-14-generic #20-Ubuntu Sep 9 02:57:57 ubuntushinner kernel: [ 576.546926] task: c0000003ebad6830 ti: c0000003e5da8000 task.ti: c0000003e5da8000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546928] NIP: c000000000350c34 LR: c000000000350c30 CTR: c000000000517a60 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546931] REGS: c0000003e5dab380 TRAP: 0700 Not tainted (3.16.0-14-generic) Sep 9 02:57:57 ubuntushinner kernel: [ 576.546932] MSR: 8000000100029033 <SF,EE,ME,IR,DR,RI,LE> CR: 28088844 XER: 20000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] CFAR: c0000000009fd270 SOFTE: 1 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR00: c000000000350c30 c0000003e5dab600 c0000000013d49e0 000000000000002d Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR04: c000000001845db0 c000000001856618 0000000000000175 0000000000000175 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR08: c000000000e449e0 0000000000000000 0000000000000000 c000000000c41b80 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR12: 0000000000008800 c00000000fb80900 0000000000000000 00000100311e01f0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR16: 0000000000000000 000000004c7133a0 000000004c713358 0000000000000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR20: 000000004c713380 000000004c7133b8 0000000000000000 000000004c713398 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR24: 0000000000000000 0000000000000001 00003fffe6f71750 c0000003e4e01d28 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR28: 0000000000000000 d000000005fe18e0 c0000003e4e01d18 0000000000000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546980] NIP [c000000000350c34] kernfs_remove_by_name_ns+0xe4/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546984] LR [c000000000350c30] kernfs_remove_by_name_ns+0xe0/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546987] Call Trace: Sep 9 02:57:57 ubuntushinner kernel: [ 576.546990] [c0000003e5dab600] [c000000000350c30] kernfs_remove_by_name_ns+0xe0/0xf0 (unreliable) Sep 9 02:57:57 ubuntushinner kernel: [ 576.546995] [c0000003e5dab680] [c0000000003542c0] sysfs_remove_link+0x40/0x90 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547001] [c0000003e5dab6c0] [d000000005fe0de0] enclosure_remove_links.part.2+0x80/0xb0 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547005] [c0000003e5dab730] [d000000005fe0e54] enclosure_component_release+0x44/0x70 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547012] [c0000003e5dab760] [c000000000639360] device_release+0x60/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547018] [c0000003e5dab7e0] [c00000000050b73c] kobject_release+0xdc/0x250 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547022] [c0000003e5dab870] [c000000000639f3c] device_unregister+0x4c/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547025] [c0000003e5dab8e0] [d000000005fe0810] enclosure_unregister+0xb0/0x100 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547029] [c0000003e5dab920] [d0000000060a0178] ses_intf_remove+0xb8/0x160 [ses] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547032] [c0000003e5dab950] [c000000000639d84] device_del+0x104/0x270 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547036] [c0000003e5dab990] [c000000000639f2c] device_unregister+0x3c/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547041] [c0000003e5daba00] [c0000000006bb8b4] __scsi_remove_device+0x124/0x150 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547044] [c0000003e5daba30] [c0000000006b8834] scsi_forget_host+0xa4/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547048] [c0000003e5daba60] [c0000000006a88e4] scsi_remove_host+0xa4/0x1b0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547054] [c0000003e5dabaa0] [d000000003b74bd0] ipr_remove+0x80/0x100 [ipr] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547058] [c0000003e5dabb20] [c000000000567bb0] pci_device_remove+0x70/0x110 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547061] [c0000003e5dabb60] [c00000000063ff9c] __device_release_driver+0xac/0x130 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547064] [c0000003e5dabb90] [c000000000640e08] driver_detach+0x148/0x190 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547067] [c0000003e5dabbe0] [c00000000063fa28] bus_remove_driver+0x98/0x150 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547070] [c0000003e5dabc50] [c000000000641aac] driver_unregister+0x4c/0x80 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547074] [c0000003e5dabcc0] [c00000000056613c] pci_unregister_driver+0x4c/0x120 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547078] [c0000003e5dabd10] [d000000003b7573c] ipr_exit+0x34/0x1f18 [ipr] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547083] [c0000003e5dabd40] [c00000000015ddc8] SyS_delete_module+0x228/0x280 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547087] [c0000003e5dabe30] [c00000000000a17c] syscall_exit+0x0/0x7c Sep 9 02:57:57 ubuntushinner kernel: [ 576.547089] Instruction dump: Sep 9 02:57:57 ubuntushinner kernel: [ 576.547090] e8010010 eb81ffe0 eba1ffe8 ebc1fff0 7c0803a6 ebe1fff8 4e800020 3c62ff89 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547095] 7fa4eb78 38638f40 486ac5b9 60000000 <0fe00000> 3860fffe 4bffff94 3c4c0108 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547101] ---[ end trace 4a44cd6dfffd2bb9 ]--- Guest xml: [root@yangtze-lp1 ~]# virsh dumpxml GManu_Ubuntu_14_10_GTO <domain type='kvm' id='11'> <name>GManu_Ubuntu_14_10_GTO</name> <uuid>22a274f5-e9bc-4189-b4bb-5cf7f848889e</uuid> <memory unit='KiB'>17532928</memory> <currentMemory unit='KiB'>17532672</currentMemory> <vcpu placement='static'>16</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='ppc64' machine='pseries'>hvm</type> <boot dev='hd'/> <boot dev='cdrom'/> <boot dev='network'/> <bootmenu enable='yes'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/manu/Ubunut_14_10_shinner.qcow2'/> <backingStore/> <target dev='sda' bus='scsi'/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/manu/utopic-server-ppc64el.iso'/> <backingStore/> <target dev='sdb' bus='scsi'/> <readonly/> <alias name='scsi0-0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='scsi' index='0'> <alias name='scsi0'/> <address type='spapr-vio' reg='0x2000'/> </controller> <controller type='scsi' index='1'> <alias name='scsi1'/> <address type='spapr-vio' reg='0x3000'/> </controller> <interface type='bridge'> <mac address='52:54:00:d7:00:34'/> <source bridge='virbr0'/> <target dev='vnet1'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target port='0'/> <alias name='serial0'/> <address type='spapr-vio' reg='0x30000000'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> <address type='spapr-vio' reg='0x30000000'/> </console> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='selinux' relabel='yes'> <label>system_u:system_r:svirt_t:s0:c16,c940</label> <imagelabel>system_u:object_r:svirt_image_t:s0:c16,c940</imagelabel> </seclabel> </domain> We saw the similar issue when we tried to "rmmod ipr" with daul controllers configuration. Looks his guest only has one GTO adapter. Also I checked the latest kernel source code for drivers/misc/enclosure.c, it included the fix patch for dual controller case. static void enclosure_remove_links(struct enclosure_component *cdev) { char name[ENCLOSURE_NAME_SIZE]; /* * In odd circumstances, like multipath devices, something else may * already have removed the links, so check for this condition first. */ if (!cdev->dev->kobj.sd) -------------> fix patch for dual controllers return; enclosure_link_name(cdev, name); sysfs_remove_link(&cdev->dev->kobj, name); sysfs_remove_link(&cdev->cdev.kobj, "device"); } Thanks, Wendy I re-created the issue on another guest. Looks the following patch caused the issue. I need to re-build the kernel and do more debug. http://git.kernel.org/cgit/linux/kernel/git/jejb/scsi.git/commit/fs/sysfs/symlink.c?h=for-next&id=879f40d193bb3c6c13930e88e3e9d5d7baf84d19 Thanks, Wendy Here is the upstream link for the patch: http://marc.info/?l=linux-scsi&m=142145523316373&w=2 SRU Justification: [Impact] Removing the ipr driver causes kernel warnings. [Fix] commit 11e52a699afff576606ceb6cf697270459f1a4aa upstream $ git describe --contains 11e52a699afff576606ceb6cf697270459f1a4aa v4.0~8^2~2 [Test Case] See Steps to Reproduce below. -- pKVM version: [root@yangtze-lp1 ~]# cat /etc/issue IBM_PowerKVM release 2.1.1 build 10 alpha (pkvm2_1_1) Kernel \r on a \m (\l) Ubuntu version: root@ubuntushinner:~# uname -a Linux ubuntushinner 3.16.0-14-generic #20-Ubuntu SMP Sat Sep 6 23:45:12 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux steps to reproduce: 1. rmmod the ipr driver and kernel traces are seen on the syslog. root@ubuntushinner:~# lsmod Module Size Used by pseries_rng 2849 0 ses 9046 0 enclosure 11198 1 ses rtc_generic 2249 0 ipr 140038 0 ohci_pci 6794 0 root@ubuntushinner:~# uname -a Linux ubuntushinner 3.16.0-14-generic #20-Ubuntu SMP Sat Sep 6 23:45:12 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux root@ubuntushinner:~# rmmod ipr root@ubuntushinner:~# lsscsi [0:0:0:0] disk QEMU QEMU HARDDISK 2.0. /dev/sda [0:0:0:1] cd/dvd QEMU QEMU CD-ROM 2.0. /dev/sr0 syslog: Sep 9 02:57:57 ubuntushinner kernel: [ 576.546878] kernfs: can not remove 'device', no directory Sep 9 02:57:57 ubuntushinner kernel: [ 576.546903] ------------[ cut here ]------------ Sep 9 02:57:57 ubuntushinner kernel: [ 576.546906] WARNING: at /build/buildd/linux-3.16.0/fs/kernfs/dir.c:1220 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546908] Modules linked in: pseries_rng ses enclosure rtc_generic ipr(-) ohci_pci Sep 9 02:57:57 ubuntushinner kernel: [ 576.546922] CPU: 1 PID: 3537 Comm: rmmod Not tainted 3.16.0-14-generic #20-Ubuntu Sep 9 02:57:57 ubuntushinner kernel: [ 576.546926] task: c0000003ebad6830 ti: c0000003e5da8000 task.ti: c0000003e5da8000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546928] NIP: c000000000350c34 LR: c000000000350c30 CTR: c000000000517a60 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546931] REGS: c0000003e5dab380 TRAP: 0700 Not tainted (3.16.0-14-generic) Sep 9 02:57:57 ubuntushinner kernel: [ 576.546932] MSR: 8000000100029033 <SF,EE,ME,IR,DR,RI,LE> CR: 28088844 XER: 20000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] CFAR: c0000000009fd270 SOFTE: 1 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR00: c000000000350c30 c0000003e5dab600 c0000000013d49e0 000000000000002d Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR04: c000000001845db0 c000000001856618 0000000000000175 0000000000000175 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR08: c000000000e449e0 0000000000000000 0000000000000000 c000000000c41b80 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR12: 0000000000008800 c00000000fb80900 0000000000000000 00000100311e01f0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR16: 0000000000000000 000000004c7133a0 000000004c713358 0000000000000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR20: 000000004c713380 000000004c7133b8 0000000000000000 000000004c713398 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR24: 0000000000000000 0000000000000001 00003fffe6f71750 c0000003e4e01d28 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546942] GPR28: 0000000000000000 d000000005fe18e0 c0000003e4e01d18 0000000000000000 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546980] NIP [c000000000350c34] kernfs_remove_by_name_ns+0xe4/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546984] LR [c000000000350c30] kernfs_remove_by_name_ns+0xe0/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.546987] Call Trace: Sep 9 02:57:57 ubuntushinner kernel: [ 576.546990] [c0000003e5dab600] [c000000000350c30] kernfs_remove_by_name_ns+0xe0/0xf0 (unreliable) Sep 9 02:57:57 ubuntushinner kernel: [ 576.546995] [c0000003e5dab680] [c0000000003542c0] sysfs_remove_link+0x40/0x90 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547001] [c0000003e5dab6c0] [d000000005fe0de0] enclosure_remove_links.part.2+0x80/0xb0 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547005] [c0000003e5dab730] [d000000005fe0e54] enclosure_component_release+0x44/0x70 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547012] [c0000003e5dab760] [c000000000639360] device_release+0x60/0xf0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547018] [c0000003e5dab7e0] [c00000000050b73c] kobject_release+0xdc/0x250 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547022] [c0000003e5dab870] [c000000000639f3c] device_unregister+0x4c/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547025] [c0000003e5dab8e0] [d000000005fe0810] enclosure_unregister+0xb0/0x100 [enclosure] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547029] [c0000003e5dab920] [d0000000060a0178] ses_intf_remove+0xb8/0x160 [ses] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547032] [c0000003e5dab950] [c000000000639d84] device_del+0x104/0x270 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547036] [c0000003e5dab990] [c000000000639f2c] device_unregister+0x3c/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547041] [c0000003e5daba00] [c0000000006bb8b4] __scsi_remove_device+0x124/0x150 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547044] [c0000003e5daba30] [c0000000006b8834] scsi_forget_host+0xa4/0xb0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547048] [c0000003e5daba60] [c0000000006a88e4] scsi_remove_host+0xa4/0x1b0 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547054] [c0000003e5dabaa0] [d000000003b74bd0] ipr_remove+0x80/0x100 [ipr] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547058] [c0000003e5dabb20] [c000000000567bb0] pci_device_remove+0x70/0x110 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547061] [c0000003e5dabb60] [c00000000063ff9c] __device_release_driver+0xac/0x130 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547064] [c0000003e5dabb90] [c000000000640e08] driver_detach+0x148/0x190 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547067] [c0000003e5dabbe0] [c00000000063fa28] bus_remove_driver+0x98/0x150 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547070] [c0000003e5dabc50] [c000000000641aac] driver_unregister+0x4c/0x80 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547074] [c0000003e5dabcc0] [c00000000056613c] pci_unregister_driver+0x4c/0x120 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547078] [c0000003e5dabd10] [d000000003b7573c] ipr_exit+0x34/0x1f18 [ipr] Sep 9 02:57:57 ubuntushinner kernel: [ 576.547083] [c0000003e5dabd40] [c00000000015ddc8] SyS_delete_module+0x228/0x280 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547087] [c0000003e5dabe30] [c00000000000a17c] syscall_exit+0x0/0x7c Sep 9 02:57:57 ubuntushinner kernel: [ 576.547089] Instruction dump: Sep 9 02:57:57 ubuntushinner kernel: [ 576.547090] e8010010 eb81ffe0 eba1ffe8 ebc1fff0 7c0803a6 ebe1fff8 4e800020 3c62ff89 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547095] 7fa4eb78 38638f40 486ac5b9 60000000 <0fe00000> 3860fffe 4bffff94 3c4c0108 Sep 9 02:57:57 ubuntushinner kernel: [ 576.547101] ---[ end trace 4a44cd6dfffd2bb9 ]--- Guest xml: [root@yangtze-lp1 ~]# virsh dumpxml GManu_Ubuntu_14_10_GTO <domain type='kvm' id='11'>   <name>GManu_Ubuntu_14_10_GTO</name>   <uuid>22a274f5-e9bc-4189-b4bb-5cf7f848889e</uuid>   <memory unit='KiB'>17532928</memory>   <currentMemory unit='KiB'>17532672</currentMemory>   <vcpu placement='static'>16</vcpu>   <resource>     <partition>/machine</partition>   </resource>   <os>     <type arch='ppc64' machine='pseries'>hvm</type>     <boot dev='hd'/>     <boot dev='cdrom'/>     <boot dev='network'/>     <bootmenu enable='yes'/>   </os>   <features>     <acpi/>     <apic/>     <pae/>   </features>   <clock offset='utc'/>   <on_poweroff>destroy</on_poweroff>   <on_reboot>restart</on_reboot>   <on_crash>restart</on_crash>   <devices>     <emulator>/usr/bin/qemu-kvm</emulator>     <disk type='file' device='disk'>       <driver name='qemu' type='qcow2'/>       <source file='/var/lib/libvirt/images/manu/Ubunut_14_10_shinner.qcow2'/>       <backingStore/>       <target dev='sda' bus='scsi'/>       <alias name='scsi0-0-0-0'/>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>     </disk>     <disk type='file' device='cdrom'>       <driver name='qemu' type='raw'/>       <source file='/var/lib/libvirt/images/manu/utopic-server-ppc64el.iso'/>       <backingStore/>       <target dev='sdb' bus='scsi'/>       <readonly/>       <alias name='scsi0-0-0-1'/>       <address type='drive' controller='0' bus='0' target='0' unit='1'/>     </disk>     <controller type='usb' index='0'>       <alias name='usb0'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>     </controller>     <controller type='pci' index='0' model='pci-root'>       <alias name='pci.0'/>     </controller>     <controller type='scsi' index='0'>       <alias name='scsi0'/>       <address type='spapr-vio' reg='0x2000'/>     </controller>     <controller type='scsi' index='1'>       <alias name='scsi1'/>       <address type='spapr-vio' reg='0x3000'/>     </controller>     <interface type='bridge'>       <mac address='52:54:00:d7:00:34'/>       <source bridge='virbr0'/>       <target dev='vnet1'/>       <model type='virtio'/>       <alias name='net0'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>     </interface>     <serial type='pty'>       <source path='/dev/pts/0'/>       <target port='0'/>       <alias name='serial0'/>       <address type='spapr-vio' reg='0x30000000'/>     </serial>     <console type='pty' tty='/dev/pts/0'>       <source path='/dev/pts/0'/>       <target type='serial' port='0'/>       <alias name='serial0'/>       <address type='spapr-vio' reg='0x30000000'/>     </console>     <hostdev mode='subsystem' type='pci' managed='yes'>       <driver name='vfio'/>       <source>         <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>       </source>       <alias name='hostdev0'/>     </hostdev>     <memballoon model='virtio'>       <alias name='balloon0'/>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>     </memballoon>   </devices>   <seclabel type='dynamic' model='selinux' relabel='yes'>     <label>system_u:system_r:svirt_t:s0:c16,c940</label>     <imagelabel>system_u:object_r:svirt_image_t:s0:c16,c940</imagelabel>   </seclabel> </domain> We saw the similar issue when we tried to "rmmod ipr" with daul controllers configuration. Looks his guest only has one GTO adapter. Also I checked the latest kernel source code for drivers/misc/enclosure.c, it included the fix patch for dual controller case. static void enclosure_remove_links(struct enclosure_component *cdev) {         char name[ENCLOSURE_NAME_SIZE];         /*          * In odd circumstances, like multipath devices, something else may          * already have removed the links, so check for this condition first.          */         if (!cdev->dev->kobj.sd) -------------> fix patch for dual controllers                 return;         enclosure_link_name(cdev, name);         sysfs_remove_link(&cdev->dev->kobj, name);         sysfs_remove_link(&cdev->cdev.kobj, "device"); } Thanks, Wendy I re-created the issue on another guest. Looks the following patch caused the issue. I need to re-build the kernel and do more debug. http://git.kernel.org/cgit/linux/kernel/git/jejb/scsi.git/commit/fs/sysfs/symlink.c?h=for-next&id=879f40d193bb3c6c13930e88e3e9d5d7baf84d19 Thanks, Wendy Here is the upstream link for the patch: http://marc.info/?l=linux-scsi&m=142145523316373&w=2
2015-06-10 19:16:21 Chris J Arges linux (Ubuntu Vivid): assignee Chris J Arges (arges)
2015-06-10 19:16:22 Chris J Arges linux (Ubuntu Vivid): importance Undecided Medium
2015-06-10 19:16:24 Chris J Arges linux (Ubuntu Vivid): status New In Progress
2015-06-11 16:05:26 Brad Figg linux (Ubuntu Vivid): status In Progress Fix Committed
2015-06-18 21:19:13 Launchpad Janitor linux (Ubuntu): status In Progress Fix Released
2015-06-18 21:19:13 Launchpad Janitor cve linked 2015-1420
2015-06-18 21:19:13 Launchpad Janitor cve linked 2015-4001
2015-06-18 21:19:13 Launchpad Janitor cve linked 2015-4002
2015-06-18 21:19:13 Launchpad Janitor cve linked 2015-4003
2015-06-21 13:53:46 Launchpad Janitor branch linked lp:ubuntu/trusty-proposed/linux-lts-vivid
2015-06-24 09:35:07 Luis Henriques tags architecture-ppc64le bugnameltc-115767 kernel-da-key severity-medium targetmilestone-inin1504 architecture-ppc64le bugnameltc-115767 kernel-da-key severity-medium targetmilestone-inin1504 verification-needed-vivid
2015-06-26 14:16:51 Breno Leitão tags architecture-ppc64le bugnameltc-115767 kernel-da-key severity-medium targetmilestone-inin1504 verification-needed-vivid architecture-ppc64le bugnameltc-115767 kernel-da-key severity-medium targetmilestone-inin1504 verification-done-vivid
2015-07-06 15:44:34 Launchpad Janitor linux (Ubuntu Vivid): status Fix Committed Fix Released