ironic incorrectly modified neutron port bidding for ports associated with nova instances

Bug #2076265 reported by sean mooney
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ironic
Triaged
High
Unassigned

Bug Description

nova was previosuly imacted by a cve where external modification of port binding on a neutron port could crash the compute agent https://bugs.launchpad.net/ossa/+bug/1981813

nova has never supported modification fo a neutron ports vnic type while it is attached to a nova instance.

virt drivers in nova are not allowed to modify the vnic_type of any port that is associated with an instance.

when reviewing the output of ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa

the following error is observed

Jul 25 20:34:42.508653 np0038062841 nova-compute[106292]: ERROR nova.network.neutron [req-a7169636-31a7-4d11-9017-a6865cd16b59 req-102a3eac-094a-42f8-a114-8ac7a841a33c service nova] [instance: 929bb573-a06a-4fa7-a77a-c643bd9655ce] The vnic_type of the bound port 88d554aa-8c7a-46ac-b54b-efeac135c643 has been changed in neutron from "normal" to "baremetal". Changing vnic_type of a bound port is not supported by Nova. To avoid breaking the connectivity of the instance please change the port vnic_type back to "normal".

https://zuul.opendev.org/t/openstack/build/0c7147a1aea541eb88ae387cb5487fe0/log/controller/logs/screen-n-cpu.txt

there are potentially 2 issues with this job.

1st nova only supports creating ports of vnic_type=normal and all other vnic types require the port to be precreated and passed in when creating the instance.

its possible the relevant test is not written with that in mind and is not valid to execute when using ironic

2nd the test may support specifying the vnic_type and the relevant config option may just not be set
https://github.com/openstack/tempest/blob/master/tempest/config.py#L784-L791

outside of the job issue, there is a large bug which is that there is existing technical debt where ironic is violating the contract for the virt driver nova-neutron interaction by modifying a neutron port associated with a VM.

this is entirely unsupported from a nova perspective and has previously been identified as technical debt

as with volume, it is not valid for ironic to ever modify a Netron port or volume attachments for a volume or port associated with an ironic instance.

we need to fix these codepaths in ironic eventually but that is a larger problem.

This bug is simply tracking the incorrect configuration fo the ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa job. this misconfiguration may also impact other ironic jobs.

Afonne-CID (cidelight)
Changed in ironic:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Afonne-CID (cidelight) wrote :

It's worth noting that I'm not certain if this is of a "high" importance.

Revision history for this message
Michael Sherman (msherman-uchicago) wrote :

Commenting as I wasn't sure if the concern was solely for CI, or also some production impact:

I observe that nova error message in production, running 2023.1 for nova,neutron,ironic, and the networking-generic-switch ML2 plugin, but haven't (so far) noticed any particular issues besides the log noise.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.