Checking

On Wed, Aug 24, 2022 at 7:50 PM Jeff Lane  <email address hidden>
wrote:

> So this is actually in 5.19, and has been pulled into 22.04 via our 5.15
> kernel.  Is there anything more to do here?
>
> commit eb2f403f098fedb4c58283e9532df2ff0d2a36e9
> Author: James Smart <email address hidden>
> Date:   Tue Apr 12 15:19:44 2022 -0700
>
>     scsi: lpfc: Move cfg_log_verbose check before calling lpfc_dmp_dbg()
>
>     BugLink: https://bugs.launchpad.net/bugs/1981864
>
>     [ Upstream commit e294647b1aed4247fe52851f3a3b2b19ae906228 ]
>
>     In an attempt to log message 0126 with LOG_TRACE_EVENT, the following
> hard
>     lockup call trace hangs the system.
>
>     Call Trace:
>      _raw_spin_lock_irqsave+0x32/0x40
>      lpfc_dmp_dbg.part.32+0x28/0x220 [lpfc]
>      lpfc_cmpl_els_fdisc+0x145/0x460 [lpfc]
>      lpfc_sli_cancel_jobs+0x92/0xd0 [lpfc]
>      lpfc_els_flush_cmd+0x43c/0x670 [lpfc]
>      lpfc_els_flush_all_cmd+0x37/0x60 [lpfc]
>      lpfc_sli4_async_event_proc+0x956/0x1720 [lpfc]
>      lpfc_do_work+0x1485/0x1d70 [lpfc]
>      kthread+0x112/0x130
>      ret_from_fork+0x1f/0x40
>     Kernel panic - not syncing: Hard LOCKUP
>
>     The same CPU tries to claim the phba->port_list_lock twice.
>
>     Move the cfg_log_verbose checks as part of the lpfc_printf_vlog() and
>     lpfc_printf_log() macros before calling lpfc_dmp_dbg().  There is no
> need
>     to take the phba->port_list_lock within lpfc_dmp_dbg().
>
>     Link:
> https://<email address hidden>
>     Co-developed-by: Justin Tee <email address hidden>
>     Signed-off-by: Justin Tee <email address hidden>
>     Signed-off-by: James Smart <email address hidden>
>     Signed-off-by: Martin K. Petersen <email address hidden>
>     Signed-off-by: Sasha Levin <email address hidden>
>     Signed-off-by: Kamal Mostafa <email address hidden>
>     Signed-off-by: Stefan Bader <email address hidden>
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1971193
>
> Title:
>   Server Crash while running IO and switch port bounce test with 2K
>   login session
>
> Status in linux package in Ubuntu:
>   Incomplete
>
> Bug description:
>   [Impact]
>   Server crash and Call trace reported on one of the servers running IO and
>   switch port bounce test from the 2K login session configuration.
>
>   Call Trace:
>   [56048.470488] Call Trace:
>   [56048.470489]  _raw_spin_lock_irqsave+0x32/0x40
>   [56048.470489]  lpfc_dmp_dbg.part.32+0x28/0x220 [lpfc]
>   [56048.470490]  lpfc_cmpl_els_fdisc+0x145/0x460 [lpfc]
>   [56048.470490]  lpfc_sli_cancel_jobs+0x92/0xd0 [lpfc]
>   [56048.470490]  lpfc_els_flush_cmd+0x43c/0x670 [lpfc]
>   [56048.470491]  lpfc_els_flush_all_cmd+0x37/0x60 [lpfc]
>   [56048.470491]  lpfc_sli4_async_event_proc+0x956/0x1720 [lpfc]
>   [56048.470492]  lpfc_do_work+0x1485/0x1d70 [lpfc]
>   [56048.470492]  ? __schedule+0x280/0x700
>   [56048.470492]  ? finish_wait+0x80/0x80
>   [56048.470493]  ? lpfc_unregister_unused_fcf+0x80/0x80 [lpfc]
>   [56048.470493]  kthread+0x112/0x130
>   [56048.470493]  ? kthread_flush_work_fn+0x10/0x10
>   [56048.470494]  ret_from_fork+0x1f/0x40
>   [56048.470494] Kernel panic - not syncing: Hard LOCKUP
>   [56048.470495] CPU: 0 PID: 682 Comm: lpfc_worker_0 Kdump: loaded
> Tainted: G
>        IOE    --------- -  - 4.18.0-240.el8.x86_64 #1
>   [56048.470496] Hardware name: Dell Inc. PowerEdge R740/0DY2X0, BIOS
> 2.11.2
>   004/21/2021
>   [56048.470496] Call Trace:
>   [56048.470496]  <NMI>
>   [56048.470496]  dump_stack+0x5c/0x80
>   [56048.470497]  panic+0xe7/0x2a9
>   [56048.470497]  ? __switch_to_asm+0x51/0x70
>   [56048.470497]  nmi_panic.cold.9+0xc/0xc
>   [56048.470498]  watchdog_overflow_callback.cold.7+0x5c/0x70
>   [56048.470498]  __perf_event_overflow+0x52/0xf0
>   [56048.470499]  handle_pmi_common+0x1db/0x270
>   [56048.470499]  ? __set_pte_vaddr+0x32/0x50
>   [56048.470499]  ? __native_set_fixmap+0x24/0x30
>   [56048.470500]  ? ghes_copy_tofrom_phys+0xd3/0x1c0
>   [56048.470500]  ? __ghes_peek_estatus.isra.12+0x49/0xa0
>   [56048.470500]  intel_pmu_handle_irq+0xbf/0x160
>   [56048.470501]  perf_event_nmi_handler+0x2d/0x50
>   [56048.470501]  nmi_handle+0x63/0x110
>   [56048.470501]  default_do_nmi+0x4e/0x100
>   [56048.470502]  do_nmi+0x128/0x190
>   [56048.470502]  end_repeat_nmi+0x16/0x6a
>   [56048.470503] RIP: 0010:native_queued_spin_lock_slowpath+0x5d/0x1d0
>   [56048.470504] Code: 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07
> 30 e4
>   09 d0 a9 00 01 ff ff 75 47 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 8b 07
> <84> c0 75
>   f8 b8 01 00 00 00 66 89 07 c3 8b 37 81 fe 00 01 00 00 75
>   [56048.470504] RSP: 0018:ffffacebc7877ca8 EFLAGS: 00000002
>   [56048.470505] RAX: 0000000000000101 RBX: 0000000000000246 RCX:
>   000000000000001f
>   [56048.470505] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
>   ffff94dcf5341dc0
>   [56048.470506] RBP: ffff94dcf5340000 R08: 0000000000000002 R09:
>   0000000000029600
>   [56048.470506] R10: 000060d29656a45c R11: ffff94dcf534fd12 R12:
>   ffff94dcf5341db0
>   [56048.470507] R13: ffff94dcf5341dc0 R14: ffff94dcc4ae8a00 R15:
>   0000000000000003
>   [56048.470507]  ? native_queued_spin_lock_slowpath+0x5d/0x1d0
>   [56048.470507]  ? native_queued_spin_lock_slowpath+0x5d/0x1d0
>   [56048.470508]  </NMI>
>   [56048.470508]  _raw_spin_lock_irqsave+0x32/0x40
>   [56048.470509]  lpfc_dmp_dbg.part.32+0x28/0x220 [lpfc]
>   [56048.470509]  lpfc_cmpl_els_fdisc+0x145/0x460 [lpfc]
>   [56048.470509]  lpfc_sli_cancel_jobs+0x92/0xd0 [lpfc]
>   [56048.470510]  lpfc_els_flush_cmd+0x43c/0x670 [lpfc]
>   [56048.470510]  lpfc_els_flush_all_cmd+0x37/0x60 [lpfc]
>   [56048.470510]  lpfc_sli4_async_event_proc+0x956/0x1720 [lpfc]
>   [56048.470511]  lpfc_do_work+0x1485/0x1d70 [lpfc]
>   [56048.470511]  ? __schedule+0x280/0x700
>   [56048.470511]  ? finish_wait+0x80/0x80
>   [56048.470512]  ? lpfc_unregister_unused_fcf+0x80/0x80 [lpfc]
>   [56048.470512]  kthread+0x112/0x130
>   [56048.470513]  ? kthread_flush_work_fn+0x10/0x10
>   [56048.470513]  ret_from_fork+0x1f/0x40
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]#
>
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]# cat
>   /etc/redhat-release
>   Red Hat Enterprise Linux release 8.3 (Ootpa)
>
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]# cat
>   /sys/module/lpfc/version
>   0:14.0.390.2
>
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]# cat
>   /sys/class/scsi_host/host*/modeldesc
>   Emulex LightPulse LPe32002-M2 2-Port 32Gb Fibre Channel Adapter
>   Emulex LightPulse LPe32002-M2 2-Port 32Gb Fibre Channel Adapter
>
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]# cat
>   /sys/class/scsi_host/host*/fwrev
>   14.0.390.1, sli-4:2:c
>   14.0.390.1, sli-4:2:c
>
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]# cat
>   /sys/class/fc_host/host*/port_name
>   0x10000090faf09459
>   0x10000090faf0945a
>   [root@ms-svr3-10-231-131-160 127.0.0.1-2021-11-20-05:14:30]#
>
>   HBA Attributes for 10:00:00:90:fa:f0:94:59
>
>   Host Name                     : ms-svr3-10-231-131-160
>   Manufacturer                  : Emulex Corporation
>   Serial Number                 : FC70793283
>   Model                         : LPe32002-M2
>   Model Desc                    : Emulex LightPulse LPe32002-M2 2-Port
> 32Gb Fibre
>   Channel Adapter
>   Node WWN                      : 20 00 00 90 fa f0 94 59
>   Node Symname                  :
>   HW Version                    : 0000000c 00000001 00000000
>   FW Version                    : 14.0.390.1
>   Vendor Spec ID                : 10DF
>   Number of Ports               : 1
>   Driver Name                   : lpfc
>   Driver Version                : 14.0.390.2; HBAAPI(I) v2.3.d, 07-12-10
>   Device ID                     : E300
>   HBA Type                      : LPe32002-M2
>   Operational FW                : 14.0.390.1
>   IEEE Address                  : 00 90 fa f0 94 59
>   Boot Code                     : Enabled
>   Boot Version                  : 14.0.390.1
>   Board Temperature             : Normal
>   Function Type                 : FC
>   Sub Device ID                 : E300
>   PCI Bus Number                : 94
>   PCI Func Number               : 0
>   Sub Vendor ID                 : 10DF
>   IPL Filename                  : H62LEX1
>   Service Processor FW Name     : 14.0.390.1
>   ULP FW Name                   : 14.0.390.1
>   FC Universal BIOS Version     : 14.0.390.1
>   FC x86 BIOS Version           : 14.0.390.1
>   FC EFI BIOS Version           : 14.0.388.0
>   FC FCODE Version              : 14.0.386.0
>   Flash Firmware Version        : 14.0.390.1
>   Secure Firmware               : Enabled
>
>   [root@ms-svr3-10-231-131-160 log]# hbacmd portattrib
>   10:00:00:90:fa:f0:94:59
>
>   Port Attributes for 10:00:00:90:fa:f0:94:59
>
>   Node WWN                  : 20 00 00 90 fa f0 94 59
>   Port WWN                  : 10 00 00 90 fa f0 94 59
>   Port Symname              :
>   Port FCID                 : 0000
>   Port Type                 : Unknown
>   Port State                : Link Down
>   Port Service Type         : 8
>   Port Supported FC4        : 00 00 01 00 00 00 00 01
>                               00 00 00 00 00 00 00 00
>                               00 00 00 00 00 00 00 00
>                               00 00 00 00 00 00 00 00
>   Port Active FC4           : 00 00 01 00 00 00 00 01
>                               00 00 00 00 00 00 00 00
>                               00 00 00 00 00 00 00 00
>                               00 00 00 00 00 00 00 00
>   Port Supported Speed      : 8 16 32 Gbit/sec
>   Configured Port Speed     : Auto Detect
>   Port Speed                : Not Available
>   Max Frame Size            : 2048
>   OS Device Name            : /sys/class/scsi_host/host15
>   Num Discovered Ports      : 0
>   Fabric Name               : 00 00 00 00 00 00 00 00
>   Function Type             : FC
>   FEC                       : Enabled
>
>   [Fixes]
>   The following patch will resolve the issue:
>   scsi: lpfc: Move cfg_log_verbose check before calling lpfc_dmp_dbg()
>   In an attempt to log message 0126 with LOG_TRACE_EVENT, the following
> hard
>   lockup call trace hangs the system.
>
>   [Testcase]
>
>
>   [root@ms-svr3-10-231-131-160 log]#
>   [reply] [-]Comment 3James Smart 2022-04-13 09:12:37 PDT
>   Patches pushed upstream 4/12/22:
>
>   https://lore.kernel.org/linux-
>   <email address hidden>/T/#t
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1971193/+subscriptions
>
> --

*Laurie Barry*
Software Program Manager |  Emulex Connectivity Division
office/soft phone: 978-719-1503 | mobile: 603-930-1771
200 Brickstone Square  |  Andover, MA 01810
<email address hidden>   |   broadcom.com

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.