"Unable to find a system nic" if openstack lists down ports in json config

[Julian Andres Klode]

We see most ppc64el machines fail to acquire network IPs on scalingstack with errors like

ValueError: Unable to find a system nic for {'mtu': 1458, 'type': 'physical', 'subnets': [{'type': 'dhcp4'}], 'mac_address': 'fa:16:3e:6d:95:51'}

In those cases, openstack generated two networks, but only one device is actually present in the VM.

ubuntu@juju-4d1272-prod-proposed-migration-9:~$ openstack console log show jak-s390x-impish | grep fa:16
[ 7.113131] cloud-init[439]: ValueError: Unable to find a system nic for {'mtu': 1458, 'type': 'physical', 'subnets': [{'type': 'dhcp4'}], 'mac_address': 'fa:16:3e:a4:29:ce'}
[ 7.681157] cloud-init[464]: ci-info: | enp0s1 | False | . | . | . | fa:16:3e:c1:f9:61 |
[ 7.761093] cloud-init[464]: ValueError: Unable to find a system nic for {'mtu': 1458, 'type': 'physical', 'subnets': [{'type': 'dhcp4'}], 'mac_address': 'fa:16:3e:a4:29:ce'}
ubuntu@juju-4d1272-prod-proposed-migration-9:~$ openstack port list --server jak-s390x-impish
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| 7c68497d-348a-4438-99df-4a7bd31734df | | fa:16:3e:a4:29:ce | ip_address='10.44.124.6', subnet_id='694adc9c-c852-4654-937f-b2438a1bad23' | DOWN |
| ec5f644c-6151-4ad9-bd09-7908e8956faa | | fa:16:3e:c1:f9:61 | ip_address='10.44.124.73', subnet_id='694adc9c-c852-4654-937f-b2438a1bad23' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+

cloud-init must not abort here, but continue configuring the active network device.

[Chad Smith (chad.smith)]

Thanks for filing the bug Julian. Could you also please attach the tarfile from `cloud-init collect-logs` to aid in triage. I think what openstack is telling cloud init probably needs to take into account optional devices.

[Julian Andres Klode (juliank)]

I unfortunately cannot SSH into the machine without having the network setup first, chicken and egg problem. But it's probably similar to the above.

In any case, there should be two devices sent, and it should not hard early error out just because it can't find one of them.

[Julian Andres Klode (juliank)]

Presumably it should log the error instead of raising it, and then at the end, before writing the config delete any items without a name.

[Chad Smith (chad.smith)]

The problem is that cloud-init is being told there should be two devices associated with this instance, and it isn't being told that one of them is optional for boot. We might need to tease out why that is the case in this deployment

[Launchpad Janitor (janitor)]

[Expired for cloud-init because there has been no activity for 60 days.]

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3920