SRIOV vf's are not created using "system host-if-modify" command

[Sriram Dharwadkar]

Brief Description
-----------------
SRIOV vfs are not created using "system host-if-modify" command
system host-if-modify controller-0 -n sriov0 -c pci-sriov -N 8 --vf-driver=netdevice ae29d14b-5c2d-416d-985f-2ad4a9f4c570

Severity
--------
Provide the severity of the defect.
Critical

Steps to Reproduce
------------------
1) while installing distributed starlingX 4.0, as part of subcloud bring up, SRIOV vf's are to be created by using below commands.
system host-label-assign controller-0 sriovdp=enabled
system host-memory-modify controller-0 0 -1G 30
system host-memory-modify controller-0 1 -1G 30
DATA0IF=enp59s0f0
export NODE=controller-0
PHYSNET0='physnet0'
SPL=/tmp/tmp-system-port-list
SPIL=/tmp/tmp-system-host-if-list
system host-port-list ${NODE} --nowrap > ${SPL}
system host-if-list -a ${NODE} --nowrap > ${SPIL}
DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}')
DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}')
DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}')
DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}')
system datanetwork-add ${PHYSNET0} vlan
system host-if-modify $NODE -n sriov0 -c pci-sriov -N 8 --vf-driver=netdevice $DATA0IFUUID
system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0}
system host-unlock controller-0

Expected Behavior
------------------
After the running the list of commands as part of subcloud installation, it is expected that
8 vf's are created. "ip link show" should show those 8 vfs. Configmap(/etc/pcidp/config.json) has to be updated for sriov device plugin

Actual Behavior
----------------
vf's are not created. ip link show doesnt show any vfs.
But config.json is updated
controller-0:/home/sysadmin# cat /etc/pcidp/config.json
{
  "resourceList": [
    {
      "resourceName": "pci_sriov_net_physnet1",
      "selectors": {
        "vendors": [
          "14e4"
        ],
        "drivers": [
          "bnxt_en"
        ],
        "devices": [
          "16dc"
        ],
        "pfNames": [
          "enp59s0f0"
        ]
      }
    }
  ]
}

and controller-0 has gone to "failed" state
[sysadmin@controller-1 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | disabled | failed |

Since it is in failed state, controller-0 node is not able to run sriov-dp pod.

Reproducibility
---------------
Always reproducible

System Configuration
--------------------
All in one controller - duplex in edge OR
Standard controller with storage + 2 compute nodes in edge

Branch/Pull Time/Commit
-----------------------
rel-20.06 - in Starlingx 4.0 (Distributed StarlingX)

Last Pass
---------

Timestamp/Logs
--------------

Test Activity
-------------

Workaround
----------
Describe workaround if available

[Steven Webster (swebster-wr)]

Notes:

Check the logs in /var/log/puppet/*. The controller-0 is likely failing as the puppet manifest has errors when trying to enable / bind the SR-IOV devices.

- Ensure you have VT-d enabled in the BIOS.
- Ensure this NIC supports SR-IOV

It may be useful to try to enable SR-IOV manually on the NIC. You can check the puppet logs for the appropriate commands

[Ghada Khalil (gkhalil)]

Marking as low priority given the lack of response of the reporter