ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: cpu hotplug on boslcp3g4 guest dumping call traces continuously.

Bug #1759722 reported by bugproxy on 2018-03-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Medium
Canonical Kernel Team
linux (Ubuntu)
Undecided
Ubuntu on IBM Power Systems Bug Triage

Bug Description

Problem Description:
===================
After performing HOTPLUG CPU on guest boslcp3g4 , saw dumping messages on the guest console continously

Steps to re-create:
==================

> boslcp3g4 is a guest of Boston LC system "boslcp3". Both of these systems are installed with ubuntu1804 kernel.

root@boslcp3:/home# uname -a
Linux boslcp3 4.15.0-12-generic #13+leo20180320 SMP Tue Mar 20 13:10:42 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3:/home# uname -r
4.15.0-12-generic

root@boslcp3g4:/kte/tools/setup.d# uname -a
Linux boslcp3g4 4.15.0-12-generic #13+leo20180320 SMP Tue Mar 20 13:10:42 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3g4:/kte/tools/setup.d# uname -r
4.15.0-12-generic

> When I perform hotplug of the cpu , it was successful but throwing these call traces in the Guest console continuously-

root@boslcp3:~# virsh setvcpus boslcp3g4 64 --live

root@boslcp3g4:~# [22273.371233] Using 'aes-generic' as fallback implementation.
[22273.371377] Using 'ctr(aes-generic)' as fallback implementation.
[22273.450867] Using 'aes-generic' as fallback implementation.
[22273.450967] Using 'ctr(aes-generic)' as fallback implementation.
[22300.779703] Using 'aes-generic' as fallback implementation.
[22300.779810] Using 'ctr(aes-generic)' as fallback implementation.
[22300.789192] Using 'aes-generic' as fallback implementation.
[22300.789327] Using 'ctr(aes-generic)' as fallback implementation.
[22327.714322] Using 'aes-generic' as fallback implementation.
[22327.714423] Using 'ctr(aes-generic)' as fallback implementation.
[22327.722548] Using 'aes-generic' as fallback implementation.
[22327.722654] Using 'ctr(aes-generic)' as fallback implementation.
[22354.401415] Using 'aes-generic' as fallback implementation.
[22354.401546] Using 'ctr(aes-generic)' as fallback implementation.
[22354.422227] Using 'aes-generic' as fallback implementation.
[22354.422354] Using 'ctr(aes-generic)' as fallback implementation.
[22368.150663] WARNING: CPU: 34 PID: 40114 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.153463] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.154166] CPU: 34 PID: 40114 Comm: kworker/34:0H Not tainted 4.15.0-12-generic #13+leo20180320
[22368.154241] Workqueue: kblockd blk_mq_run_work_fn
[22368.154283] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.154344] REGS: c000000145a2f9d0 TRAP: 0700 Not tainted (4.15.0-12-generic)
[22368.154402] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.154480] CFAR: c00000000068e070 SOFTE: 1
[22368.154480] GPR00: c000000000131538 c000000145a2fc50 c0000000016eaf00 c0000001346f7800
[22368.154480] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8
[22368.154480] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000145985d30
[22368.154480] GPR12: c00000000068e140 c00000000fa97600 c00000000013a368 c0000001760c35c0
[22368.154480] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.154480] GPR20: c00000017ee78180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.154480] GPR24: 0000000000000402 0000000000000000 c00000017ee78500 0000000000000000
[22368.154480] GPR28: c0000001346f7880 c00000017ee7c900 c00000017ee78180 c0000001346f7800
[22368.155001] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.155065] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.155115] Call Trace:
[22368.155139] [c000000145a2fc50] [c000000145a2fce0] 0xc000000145a2fce0 (unreliable)
[22368.155200] [c000000145a2fc90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.155259] [c000000145a2fd20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.155308] [c000000145a2fdc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.155358] [c000000145a2fe30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.155415] Instruction dump:
[22368.155446] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.155506] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.155566] ---[ end trace 70e3c0daa532c721 ]---
[22368.204754] WARNING: CPU: 34 PID: 40224 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.204856] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.205275] CPU: 34 PID: 40224 Comm: kworker/34:1H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.205352] Workqueue: kblockd blk_mq_run_work_fn
[22368.205392] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.205449] REGS: c00000017ac8b9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.205513] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 44002284 XER: 00000000
[22368.205588] CFAR: c00000000068e070 SOFTE: 1
[22368.205588] GPR00: c000000000131538 c00000017ac8bc50 c0000000016eaf00 c0000001346f7800
[22368.205588] GPR04: c0000001346f7880 0000000000000000 0000000044008282 c000000001721ed8
[22368.205588] GPR08: 0000000000000000 0000000000000001 0000000000000000 00000000ffffffff
[22368.205588] GPR12: c00000000068e140 c00000000fa97600 c00000000013a368 c000000179360000
[22368.205588] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.205588] GPR20: c00000017ee78180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.205588] GPR24: 0000000000000402 0000000000000000 c00000017ee78500 0000000000000000
[22368.205588] GPR28: c0000001346f7880 c00000017ee7c900 c00000017ee78180 c0000001346f7800
[22368.206068] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.206118] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.206165] Call Trace:
[22368.206263] [c00000017ac8bc90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.206322] [c00000017ac8bd20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.206373] [c00000017ac8bdc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.206424] [c00000017ac8be30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.206480] Instruction dump:
[22368.206511] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.206588] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.206662] ---[ end trace 70e3c0daa532c722 ]---
[22368.340886] WARNING: workqueue cpumask: online intersect > possible intersect
[22368.378115] WARNING: CPU: 36 PID: 40286 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.378224] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.378684] CPU: 36 PID: 40286 Comm: kworker/36:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.378762] Workqueue: kblockd blk_mq_run_work_fn
[22368.378801] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.378859] REGS: c00000017ace79d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.378942] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.379015] CFAR: c00000000068e070 SOFTE: 1
[22368.379015] GPR00: c000000000131538 c00000017ace7c50 c0000000016eaf00 c000000142f0fc00
[22368.379015] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8
[22368.379015] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001788e2930
[22368.379015] GPR12: c00000000068e140 c00000000fa98c00 c00000000013a368 c0000001760c2400
[22368.379015] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.379015] GPR20: c00000017eef8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.379015] GPR24: 0000000000000402 0000000000000000 c00000017eef8500 0000000000000000
[22368.379015] GPR28: c000000142f0fc80 c00000017eefc900 c00000017eef8180 c000000142f0fc00
[22368.379497] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.379547] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.379594] Call Trace:
[22368.379617] [c00000017ace7c50] [c00000017ace7ce0] 0xc00000017ace7ce0 (unreliable)
[22368.379678] [c00000017ace7c90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.379739] [c00000017ace7d20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.379790] [c00000017ace7dc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.379842] [c00000017ace7e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.379907] Instruction dump:
[22368.379938] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.380005] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.380070] ---[ end trace 70e3c0daa532c723 ]---
[22368.488613] WARNING: CPU: 39 PID: 40552 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.488724] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.489289] CPU: 39 PID: 40552 Comm: kworker/39:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.489367] Workqueue: kblockd blk_mq_run_work_fn
[22368.489409] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.489467] REGS: c00000017285b9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.489532] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.489605] CFAR: c00000000068e070 SOFTE: 1
[22368.489605] GPR00: c000000000131538 c00000017285bc50 c0000000016eaf00 c0000001346f7800
[22368.489605] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8
[22368.489605] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000142ca9c30
[22368.489605] GPR12: c00000000068e140 c00000000fa9ad00 c00000000013a368 c0000001760c9e40
[22368.489605] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.489605] GPR20: c00000017efb8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.489605] GPR24: 0000000000000402 0000000000000000 c00000017efb8500 0000000000000000
[22368.489605] GPR28: c0000001346f7880 c00000017efbc900 c00000017efb8180 c0000001346f7800
[22368.490093] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.490144] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.490191] Call Trace:
[22368.490213] [c00000017285bc50] [c00000017285bce0] 0xc00000017285bce0 (unreliable)
[22368.490272] [c00000017285bc90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.490330] [c00000017285bd20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.490379] [c00000017285bdc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.490431] [c00000017285be30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.490487] Instruction dump:
[22368.490517] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.490576] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.490636] ---[ end trace 70e3c0daa532c724 ]---
[22368.522251] WARNING: CPU: 39 PID: 41050 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.522353] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.522772] CPU: 39 PID: 41050 Comm: kworker/39:1H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.522851] Workqueue: kblockd blk_mq_run_work_fn
[22368.522890] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.522962] REGS: c0000000e95e39d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.523029] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 44002884 XER: 00000000
[22368.523103] CFAR: c00000000068e070 SOFTE: 1
[22368.523103] GPR00: c000000000131538 c0000000e95e3c50 c0000000016eaf00 c0000001346f7800
[22368.523103] GPR04: c0000001346f7880 0000000000000000 0000000044008882 c000000001721ed8
[22368.523103] GPR08: 0000000000000000 0000000000000001 0000000000000000 00000000ffffffff
[22368.523103] GPR12: c00000000068e140 c00000000fa9ad00 c00000000013a368 c000000261facac0
[22368.523103] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.523103] GPR20: c00000017efb8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.523103] GPR24: 0000000000000402 0000000000000000 c00000017efb8500 0000000000000000
[22368.523103] GPR28: c0000001346f7880 c00000017efbc900 c00000017efb8180 c0000001346f7800
[22368.523584] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.523635] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.523682] Call Trace:
[22368.523704] [c0000000e95e3c90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.523761] [c0000000e95e3d20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.523811] [c0000000e95e3dc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.523862] [c0000000e95e3e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.523919] Instruction dump:
[22368.523948] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.524009] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.524068] ---[ end trace 70e3c0daa532c725 ]---
[22368.525105] WARNING: CPU: 38 PID: 40474 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.525198] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.525604] CPU: 38 PID: 40474 Comm: kworker/38:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.525692] Workqueue: kblockd blk_mq_run_work_fn
[22368.525736] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.525800] REGS: c00000017282f9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.525869] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.525944] CFAR: c00000000068e070 SOFTE: 1
[22368.525944] GPR00: c000000000131538 c00000017282fc50 c0000000016eaf00 c000000142f0fc00
[22368.525944] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8
[22368.525944] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000142ca5730
[22368.525944] GPR12: c00000000068e140 c00000000fa9a200 c00000000013a368 c0000001760c4040
[22368.525944] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.525944] GPR20: c00000017ef78180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.525944] GPR24: 0000000000000402 0000000000000000 c00000017ef78500 0000000000000000
[22368.525944] GPR28: c000000142f0fc80 c00000017ef7c900 c00000017ef78180 c000000142f0fc00
[22368.526428] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.526477] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.526524] Call Trace:
[22368.526546] [c00000017282fc50] [c00000017282fce0] 0xc00000017282fce0 (unreliable)
[22368.526604] [c00000017282fc90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.526662] [c00000017282fd20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.526711] [c00000017282fdc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.526761] [c00000017282fe30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.526818] Instruction dump:
[22368.526849] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.526908] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.526984] ---[ end trace 70e3c0daa532c726 ]---
[22368.829929] WARNING: CPU: 32 PID: 39931 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.830040] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.830473] CPU: 32 PID: 39931 Comm: kworker/32:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.830588] Workqueue: kblockd blk_mq_run_work_fn
[22368.830629] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.830688] REGS: c0000001789779d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.830754] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.830830] CFAR: c00000000068e070 SOFTE: 1
[22368.830830] GPR00: c000000000131538 c000000178977c50 c0000000016eaf00 c0000001346f7800
[22368.830830] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8
[22368.830830] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001727a5730
[22368.830830] GPR12: c00000000068e140 c00000000fa96000 c00000000013a368 c0000001760ca940
[22368.830830] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.830830] GPR20: c00000017edf8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.830830] GPR24: 0000000000000402 0000000000000000 c00000017edf8500 0000000000000000
[22368.830830] GPR28: c0000001346f7880 c00000017edfc900 c00000017edf8180 c0000001346f7800
[22368.831330] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.831383] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.831472] Call Trace:
[22368.831495] [c000000178977c50] [c000000178977ce0] 0xc000000178977ce0 (unreliable)
[22368.831554] [c000000178977c90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.831612] [c000000178977d20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.831662] [c000000178977dc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.831714] [c000000178977e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.831773] Instruction dump:
[22368.831803] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.831863] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.831924] ---[ end trace 70e3c0daa532c727 ]---
[22368.831972] WARNING: CPU: 32 PID: 39931 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.832047] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.832450] CPU: 32 PID: 39931 Comm: kworker/32:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.832527] Workqueue: kblockd blk_mq_run_work_fn
[22368.832565] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.832624] REGS: c0000001789779d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.832690] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.832761] CFAR: c00000000068e070 SOFTE: 1
[22368.832761] GPR00: c000000000131538 c000000178977c50 c0000000016eaf00 c0000001346f7800
[22368.832761] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8
[22368.832761] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001727a5730
[22368.832761] GPR12: c00000000068e140 c00000000fa96000 c00000000013a368 c0000001760ca940
[22368.832761] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.832761] GPR20: c00000017edf8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.832761] GPR24: 0000000000000402 0000000000000000 c00000017edf8500 0000000000000000
[22368.832761] GPR28: c0000001346f7880 c00000017edfc900 c00000017edf8180 c0000001346f7800
[22368.833253] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.833302] LR [c000000000131538] process_one_work+0x298/0x5a0
[22368.833351] Call Trace:
[22368.833371] [c000000178977c50] [c000000178977c90] 0xc000000178977c90 (unreliable)
[22368.833429] [c000000178977c90] [c000000000131538] process_one_work+0x298/0x5a0
[22368.833488] [c000000178977d20] [c0000000001318d8] worker_thread+0x98/0x630
[22368.833537] [c000000178977dc0] [c00000000013a508] kthread+0x1a8/0x1b0
[22368.833587] [c000000178977e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4
[22368.833642] Instruction dump:
[22368.833673] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010
[22368.833732] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef
[22368.833792] ---[ end trace 70e3c0daa532c728 ]---
[22368.913232] WARNING: CPU: 35 PID: 40184 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140
[22368.913335] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net
[22368.913750] CPU: 35 PID: 40184 Comm: kworker/35:0H Tainted: G W 4.15.0-12-generic #13+leo20180320
[22368.913830] Workqueue: kblockd blk_mq_run_work_fn
[22368.913869] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140
[22368.913928] REGS: c00000017acc79d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic)
[22368.913996] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000
[22368.914070] CFAR: c00000000068e070 SOFTE: 1
[22368.914070] GPR00: c000000000131538 c00000017acc7c50 c0000000016eaf00 c000000142f0fc00
[22368.914070] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8
[22368.914070] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000172428530
[22368.914070] GPR12: c00000000068e140 c00000000fa98100 c00000000013a368 c0000001760c2ac0
[22368.914070] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[22368.914070] GPR20: c00000017eeb8180 0000000000000000 0000000000000000 fffffffffffffef7
[22368.914070] GPR24: 0000000000000402 0000000000000000 c00000017eeb8500 0000000000000000
[22368.914070] GPR28: c000000142f0fc80 c00000017eebc900 c00000017eeb8180 c000000142f0fc00
[22368.914583] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140
[22368.914635] LR [c000000000131538] process_one_work+0x298/0x5a0
[

> virsh dumpxml boslcp3g4

root@boslcp3:~# virsh dumpxml boslcp3g4 | grep vcpu
  <vcpu placement='static' current='64'>80</vcpu>
  <vcpus>
    <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='1' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='2' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='3' enabled='yes' hotpluggable='no' order='1'/>
    <vcpu id='4' enabled='yes' hotpluggable='no' order='2'/>
    <vcpu id='5' enabled='yes' hotpluggable='no' order='2'/>
    <vcpu id='6' enabled='yes' hotpluggable='no' order='2'/>
    <vcpu id='7' enabled='yes' hotpluggable='no' order='2'/>
    <vcpu id='8' enabled='yes' hotpluggable='no' order='3'/>
    <vcpu id='9' enabled='yes' hotpluggable='no' order='3'/>
    <vcpu id='10' enabled='yes' hotpluggable='no' order='3'/>
    <vcpu id='11' enabled='yes' hotpluggable='no' order='3'/>
    <vcpu id='12' enabled='yes' hotpluggable='no' order='4'/>
    <vcpu id='13' enabled='yes' hotpluggable='no' order='4'/>
    <vcpu id='14' enabled='yes' hotpluggable='no' order='4'/>
    <vcpu id='15' enabled='yes' hotpluggable='no' order='4'/>
    <vcpu id='16' enabled='yes' hotpluggable='no' order='5'/>
    <vcpu id='17' enabled='yes' hotpluggable='no' order='5'/>
    <vcpu id='18' enabled='yes' hotpluggable='no' order='5'/>
    <vcpu id='19' enabled='yes' hotpluggable='no' order='5'/>
    <vcpu id='20' enabled='yes' hotpluggable='no' order='6'/>
    <vcpu id='21' enabled='yes' hotpluggable='no' order='6'/>
    <vcpu id='22' enabled='yes' hotpluggable='no' order='6'/>
    <vcpu id='23' enabled='yes' hotpluggable='no' order='6'/>
    <vcpu id='24' enabled='yes' hotpluggable='no' order='7'/>
    <vcpu id='25' enabled='yes' hotpluggable='no' order='7'/>
    <vcpu id='26' enabled='yes' hotpluggable='no' order='7'/>
    <vcpu id='27' enabled='yes' hotpluggable='no' order='7'/>
    <vcpu id='28' enabled='yes' hotpluggable='no' order='8'/>
    <vcpu id='29' enabled='yes' hotpluggable='no' order='8'/>
    <vcpu id='30' enabled='yes' hotpluggable='no' order='8'/>
    <vcpu id='31' enabled='yes' hotpluggable='no' order='8'/>
    <vcpu id='32' enabled='yes' hotpluggable='yes' order='9'/>
    <vcpu id='33' enabled='yes' hotpluggable='yes' order='9'/>
    <vcpu id='34' enabled='yes' hotpluggable='yes' order='9'/>
    <vcpu id='35' enabled='yes' hotpluggable='yes' order='9'/>
    <vcpu id='36' enabled='yes' hotpluggable='yes' order='10'/>
    <vcpu id='37' enabled='yes' hotpluggable='yes' order='10'/>
    <vcpu id='38' enabled='yes' hotpluggable='yes' order='10'/>
    <vcpu id='39' enabled='yes' hotpluggable='yes' order='10'/>
    <vcpu id='40' enabled='yes' hotpluggable='yes' order='11'/>
    <vcpu id='41' enabled='yes' hotpluggable='yes' order='11'/>
    <vcpu id='42' enabled='yes' hotpluggable='yes' order='11'/>
    <vcpu id='43' enabled='yes' hotpluggable='yes' order='11'/>
    <vcpu id='44' enabled='yes' hotpluggable='yes' order='12'/>
    <vcpu id='45' enabled='yes' hotpluggable='yes' order='12'/>
    <vcpu id='46' enabled='yes' hotpluggable='yes' order='12'/>
    <vcpu id='47' enabled='yes' hotpluggable='yes' order='12'/>
    <vcpu id='48' enabled='yes' hotpluggable='yes' order='13'/>
    <vcpu id='49' enabled='yes' hotpluggable='yes' order='13'/>
    <vcpu id='50' enabled='yes' hotpluggable='yes' order='13'/>
    <vcpu id='51' enabled='yes' hotpluggable='yes' order='13'/>
    <vcpu id='52' enabled='yes' hotpluggable='yes' order='14'/>
    <vcpu id='53' enabled='yes' hotpluggable='yes' order='14'/>
    <vcpu id='54' enabled='yes' hotpluggable='yes' order='14'/>
    <vcpu id='55' enabled='yes' hotpluggable='yes' order='14'/>
    <vcpu id='56' enabled='yes' hotpluggable='yes' order='15'/>
    <vcpu id='57' enabled='yes' hotpluggable='yes' order='15'/>
    <vcpu id='58' enabled='yes' hotpluggable='yes' order='15'/>
    <vcpu id='59' enabled='yes' hotpluggable='yes' order='15'/>
    <vcpu id='60' enabled='yes' hotpluggable='yes' order='16'/>
    <vcpu id='61' enabled='yes' hotpluggable='yes' order='16'/>
    <vcpu id='62' enabled='yes' hotpluggable='yes' order='16'/>
    <vcpu id='63' enabled='yes' hotpluggable='yes' order='16'/>
    <vcpu id='64' enabled='no' hotpluggable='yes'/>
    <vcpu id='65' enabled='no' hotpluggable='yes'/>
    <vcpu id='66' enabled='no' hotpluggable='yes'/>
    <vcpu id='67' enabled='no' hotpluggable='yes'/>
    <vcpu id='68' enabled='no' hotpluggable='yes'/>
    <vcpu id='69' enabled='no' hotpluggable='yes'/>
    <vcpu id='70' enabled='no' hotpluggable='yes'/>
    <vcpu id='71' enabled='no' hotpluggable='yes'/>
    <vcpu id='72' enabled='no' hotpluggable='yes'/>
    <vcpu id='73' enabled='no' hotpluggable='yes'/>
    <vcpu id='74' enabled='no' hotpluggable='yes'/>
    <vcpu id='75' enabled='no' hotpluggable='yes'/>
    <vcpu id='76' enabled='no' hotpluggable='yes'/>
    <vcpu id='77' enabled='no' hotpluggable='yes'/>
    <vcpu id='78' enabled='no' hotpluggable='yes'/>
    <vcpu id='79' enabled='no' hotpluggable='yes'/>
  </vcpus>
root@boslcp3:~#

> lspcu output from guest

root@boslcp3g4:~# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0,4,8,12,16,20,24,28,32-63
Off-line CPU(s) list: 1-3,5-7,9-11,13-15,17-19,21-23,25-27,29-31
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0,4,8,12,16,20,24,28,32-39
NUMA node1 CPU(s): 40-63
root@boslcp3g4:~#

> Attched guest consoel logs

Hi, there is a patch to remove the WARNING message but it's harmless according to the patch author:

4058ba173eb411422482125e14e7bb12b70c8b38

 blk-mq: turn WARN_ON in __blk_mq_run_hw_queue into printk

 We know this WARN_ON is harmless and in reality it may be trigged,
 so convert it to printk() and dump_stack() to avoid to confusing
 people.

I'll prepare a backport to Canonical but I lowering this importance to medium, since it's not a problem.

Thank you

Default Comment by Bridge

tags: added: architecture-ppc64le bugnameltc-165971 severity-medium targetmilestone-inin1804

Default Comment by Bridge

Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → linux (Ubuntu)
Changed in ubuntu-power-systems:
importance: Undecided → Medium
status: New → Triaged
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
tags: added: triage-g
Seth Forshee (sforshee) on 2018-03-29
Changed in linux (Ubuntu):
status: New → Fix Committed
Changed in ubuntu-power-systems:
status: Triaged → Fix Committed
Launchpad Janitor (janitor) wrote :
Download full text (40.4 KiB)

This bug was fixed in the package linux - 4.15.0-15.16

---------------
linux (4.15.0-15.16) bionic; urgency=medium

  * linux: 4.15.0-15.16 -proposed tracker (LP: #1761177)

  * FFe: Enable configuring resume offset via sysfs (LP: #1760106)
    - PM / hibernate: Make passing hibernate offsets more friendly

  * /dev/bcache/by-uuid links not created after reboot (LP: #1729145)
    - SAUCE: (no-up) bcache: decouple emitting a cached_dev CHANGE uevent

  * Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default machine
    type(pseries-bionic) complaining "KVM implementation does not support
    Transactional Memory, try cap-htm=off" (kvm) (LP: #1752026)
    - powerpc: Use feature bit for RTC presence rather than timebase presence
    - powerpc: Book E: Remove unused CPU_FTR_L2CSR bit
    - powerpc: Free up CPU feature bits on 64-bit machines
    - powerpc: Add CPU feature bits for TM bug workarounds on POWER9 v2.2
    - powerpc/powernv: Provide a way to force a core into SMT4 mode
    - KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9
    - KVM: PPC: Book3S HV: Work around XER[SO] bug in fake suspend mode
    - KVM: PPC: Book3S HV: Work around TEXASR bug in fake suspend state

  * Important Kernel fixes to be backported for Power9 (kvm) (LP: #1758910)
    - powerpc/mm: Fixup tlbie vs store ordering issue on POWER9

  * Ubuntu 18.04 - IO Hang on some namespaces when running HTX with 16
    namespaces (Bolt / NVMe) (LP: #1757497)
    - powerpc/64s: Fix lost pending interrupt due to race causing lost update to
      irq_happened

  * fwts-efi-runtime-dkms 18.03.00-0ubuntu1: fwts-efi-runtime-dkms kernel module
    failed to build (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

linux (4.15.0-14.15) bionic; urgency=medium

  * linux: 4.15.0-14.15 -proposed tracker (LP: #1760678)

  * [Bionic] mlx4 ETH - mlnx_qos failed when set some TC to vendor
    (LP: #1758662)
    - net/mlx4_en: Change default QoS settings

  * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10
    (LP: #1759312)
    - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features

  * Bionic update to 4.15.15 stable release (LP: #1760585)
    - net: dsa: Fix dsa_is_user_port() test inversion
    - openvswitch: meter: fix the incorrect calculation of max delta_t
    - qed: Fix MPA unalign flow in case header is split across two packets.
    - tcp: purge write queue upon aborting the connection
    - qed: Fix non TCP packets should be dropped on iWARP ll2 connection
    - sysfs: symlink: export sysfs_create_link_nowarn()
    - net: phy: relax error checking when creating sysfs link netdev->phydev
    - devlink: Remove redundant free on error path
    - macvlan: filter out unsupported feature flags
    - net: ipv6: keep sk status consistent after datagram connect failure
    - ipv6: old_dport should be a __be16 in __ip6_datagram_connect()
    - ipv6: sr: fix NULL pointer dereference when setting encap source address
    - ipv6: sr: fix scheduling in RCU when creating seg6 lwtunnel state
    - mlxsw: spectrum_buffers: Set a minimum quota for CPU port traffic
    - net: phy: Tell caller result ...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ubuntu-power-systems:
status: Fix Committed → Fix Released
Brad Figg (brad-figg) on 2019-07-24
tags: added: cscc
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers