[SRU][HPE 24.04] Intel FVL NIC FW flash fails with inbox driver, causing driver not detected

[Michael Reed]

[Impact]
Description of problem:

Within 24.04 an intel FVL Idea adapter flash fails through a FW smart component RPM install. The flash successfully completes on the firmware(NVM) and fails on the firmware(ROM). The process takes roughly thirty minutes and still fails.

Following this the NIC will not be recognized by the OS or RBSU until the next power cycle.

Version-Release number of selected component (if applicable):

Kernel 6.8.0-38-generic

How reproducible:

100% of the time

Steps to Reproduce:
1. Attempt to upgrade NIC (Intel Ethernet Converged Network Adapter X710-DA2 was used in my case) using upgrade located at https://support.hpe.com/connect/s/softwaredetails?language=en_US&collectionId=MTX-eeaaff922a754e17
2. Wait 30 minutes for upgrade to fail
3. Observe that NIC is no longer recognized
4. AC Power Cycle to bring the NIC back up

Actual results:

Upgrade fails and system requires a power cycle to use NIC again.

Expected results:

Upgrade succeeds

[Fix]
Additional info:

There is an upstream patch from Intel that remediates this issue, where the NIC successfully upgrades in a reasonable amount of time.

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=8b9b59e27aa88ba133fbac85def3f8be67f2d5a8

I have patched the kernel, 6.8.0-38-generic, with the patch above and observed success.

[Test Plan]
Steps to Reproduce:
1. Attempt to upgrade NIC (Intel Ethernet Converged Network Adapter X710-DA2 was used in my case) using upgrade located at https://support.hpe.com/connect/s/softwaredetails?language=en_US&collectionId=MTX-eeaaff922a754e17
2. Wait 30 minutes for upgrade to fail
3. Observe that NIC is no longer recognized
4. AC Power Cycle to bring the NIC back up

Expected results:

Upgrade succeeds

[Where problems could occur]

[Other Info]

[Michael Reed (mreed8855)]

Hi Ian,

Can you provide the regression risk for the [Where problems could occur] field?

[Michael Reed (mreed8855)]

Hi Ian,

Can you test this test kernel?

https://people.canonical.com/~mreed/hpe/lp_2073913_hpe_nic_fw_intel/noble/

[Ian Taylor (tayloian)]

There are currently some difficulties with the SUT infrastructure due to the typhoon in Taiwan so I am unable to at this moment, but will once services are back.

I am not entirely sure what your regression risk question is asking, are you asking how many releases back this issue spans?

[Michael Reed (mreed8855)]

Hi Ian,

When I refer to regression risk, I am asking what are the chances this breaks the kernel for any release.

[Ian Taylor (tayloian)]

Hello Michael,

I was able to install the kernel you have given me, however two of the packages has dependency issues that prevented their install, the linux-cloud-tools-6.8.0-40 & linux-tools-6.8.0-40. I have attached screenshots showing the errors after install of the other kernel packages. After a reboot, this kernel enabled me to successfully update the firmware in a timely manner. (Screenshot attached)

In my experience working with both the kernel I had patched and the kernel given to me, I had not run into any issues or anything that hints that the kernel may be broken.