Cannot provision VF's on multiple N3000 FEC devices

[Caio Bruchert]

Brief Description

After provisioning vf's driver and VF's on N3000 FEC on controller-1, not only sriov_vfs_pci_address remain empty but sriov_numvfs goes back to default value '0' after a few seconds

Failing Automation Test Case:
test_fpga.py::test_fpga_sriovdp_dpdk_connectivity

Severity

Critical: System/Feature is not usable after the defect

Steps to Reproduce
system host-device-modify controller-1 0000:be:00.0 --driver igb_uio --vf-driver igb_uio --num-vfs 8

After this command is executed at
test_fpga_sriovdp_dpdk_connectivity(), we can no longer unlock controller-1 due to following error msg:
Expecting sriov_vfs_pci_address length=8 for FEC device pciaddr=0000:b4:00.0. Please wait a few minutes for inventory update and retry host-unlock.

This failure leads testcase teardown to fail and several other TCs also.

During automation logs analysis we manually tried to add VFs to the expected number for the device trying to get the controller unlocked looking for an automation fix or workaround and we realized that the number we were setting via:

system host-device-modify controller-1 0000:be:00.0 --num-vfs 8

dissapeared in a few seconds from being configured

Expected Behavior

Write down what was expected after taking the steps written above

Actual Behavior

State what is the actual behavior

Reproducibility

Reproducible 100%

System Configuration

AIO-DX ipv4

Branch/Pull Time/Commit

SW_VERSION="6.0"

Last Pass

11/11

Timestamp/Logs

Alarms

No alarms were raised.

Test Activity

Regression Testing

Workaround

None available

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/839962

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/stx-puppet/+/840909

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/840909
Committed: https://opendev.org/starlingx/stx-puppet/commit/a4823c6d2c209c13e68d8aa424c725daf6302e61
Submitter: "Zuul (22348)"
Branch: master

commit a4823c6d2c209c13e68d8aa424c725daf6302e61
Author: Caio Bruchert <email address hidden>
Date: Fri May 6 09:54:14 2022 -0400

    Add support for multiple FPGA_5GNR devices

    When multiple devices are present, pf_bb_config can be called
    with the -a option to automatically configure the probed devices
    and thus prevent entering interactive mode.

    Test Plan:
        PASS: Verify that pf_bb_config command executed successfully
        PASS: Verify if everything worked after host lock/unlock

    Partial-Bug: 1970785

    Signed-off-by: Caio Bruchert <email address hidden>
    Change-Id: I3f6e8381f296f97ae9cd6f557bf7c9dd0c2b50b0

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/839962
Committed: https://opendev.org/starlingx/config/commit/ad27075671bbe8b0351dedad7ccf8cf7327a6371
Submitter: "Zuul (22348)"
Branch: master

commit ad27075671bbe8b0351dedad7ccf8cf7327a6371
Author: Caio Bruchert <email address hidden>
Date: Thu Apr 28 16:05:15 2022 -0400

    Fix yaml generation for fpga_fec_device_config

    Regardless of how many FPGA devices is present, the host
    YAML file always contains one entry.

    This is caused due to a list being overwritten with the
    last entry instead of being updated.

    To detect similar issues a new puppet unit test was added.

    Test Plan:
        PASS: Run new unit test
        PASS: Run all unit tests
        PASS: Verify live system

    Closes-Bug: 1970785
    Depends-On: https://review.opendev.org/c/starlingx/stx-puppet/+/840909

    Change-Id: I80dfaffc470083e09658c49982a2b5fe355962b8
    Signed-off-by: Caio Bruchert <email address hidden>