Regenerate mac address of a port

[Harald Jensås]

Openstack Ironic need to set the mac address of neutron port to the actual mac address of the hardware for DHCP/PXE deployment to work. When using pre-created neutron ports there is an issue where previously used ports cause a conflict when a new instance is scheduled onto the same baremetal hosts.

Example:
 openstack port create --network private test-port1
 openstack port create --network private test-port2

  openstack server create \
    --image cirros-0.3.5-x86_64-disk \
    --port test-port1\
    --flavor baremetal \
    test-server

  openstack server delete test-server

  openstack server create \
    --image cirros-0.3.5-x86_64-disk \
    --port test-port2 \
    --flavor baremetal \
    test-server

The second server create command fails with:
fault:
  code: 500
  created: '2018-04-29T12:21:37Z'
  < .. snip .. >
  message: 'Build of instance 5982f29a-3d48-4eb8-88ca-cf794c8f31dd aborted: Failure
    prepping block device.'

An in conductor log:
Apr 29 14:21:35 ironic-devstack.lab.example.com ironic-conductor[445]: ERROR ironic.common.neutron MacAddressInUseClient: Unable to complete operation for network 52497681-18f4-47de-b48e-1265799caea9. The mac address 52:54:00:53:b5:b9 is in use.

It should be possible to "return" the port by resetting a mac in the base MAC address Neutron uses for VIFs. e.g a port update request that triggers regeneration of the mac address. Ironic would add this to it's interface detach method, to ensure no previously used ports cause issues when deploying new baremetal instances.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/565931

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/565932

[Brian Haley (brian-haley)]

Question - you showed two "port create" calls - you're seeing them generate the same MAC address for both ports? I don't see that happening but maybe I'm mis-understanding.

[Brian Haley (brian-haley)]

I changed Status back to New so that it shows up on the https://wiki.openstack.org/wiki/Meetings/NeutronDrivers "new" list, that way we will talk about it at the meeting next week.

[Harald Jensås (harald-jensas)]

@Brian,

When the two ports are initially created, they do not get the same MAC address.

What happens is that Ironic changes[1] the MAC address of the port when it is attaching the port.

Ironic needs to do this to ensure that neutrons DHCP server responds to the DHCP request from the baremetal servers network interface.

The problem arises when the Ironic instance is later deleted, and then I try to create a new Ironic instance using a different port in the same network. If the new instance is scheduled onto the same baremetal server, Ironic attempts to change the MAC address of the latter port to match the servers network interface. Neutron refuses because the prior port is still there with the MAC address of the baremetal server.

[1] https://github.com/openstack/ironic/blob/master/ironic/common/neutron.py#L133

[Brian Haley (brian-haley)]

@Harald - thanks for the information, I understand now.

But just a follow-up since I'm sure it will be asked at the drivers meeting - why can't ironic set the mac address to some other value when the server is destroyed? That way there shouldn't be a conflict. Of course it would have to be a unique value to not conflict with other ports, and since we don't save the original value we're right back in the regenerate case most likely. Just want to make sure you thought of other options.

[Harald Jensås (harald-jensas)]

@Brian

Indeed, Ironic does not store the original MAC address. Ironic could generate a MAC address. But, the problem is that Ironic is not aware of neutrons ``base_mac`` option[1]. While not strictly necessary I think it would be good to do the regenerating in neutron so that the same base_mac is used.

[1] https://docs.openstack.org/neutron/pike/configuration/neutron.html#DEFAULT.base_mac

[Miguel Lavalle (minsel)]

RFE was approved during today's Drivers meeting

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565931
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=56033ba643812a30577f6ab17648806c2ee494ba
Submitter: Zuul
Branch: master

commit 56033ba643812a30577f6ab17648806c2ee494ba
Author: Harald Jensås <email address hidden>
Date: Thu May 3 03:11:19 2018 +0200

    Add support to regenerate port mac_address

    - Adds converter to generate a valid mac address if the
      provided data is 'null' (None).
    - Adds api extension definition for port using the new converter.

    Related-Bug: #1768690
    Change-Id: Ibfcf179c2051d2bf47d4d1e62e8146cbee29fd64

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565932
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8f3a066b20b7ffafec95a618d60e40727504f37c
Submitter: Zuul
Branch: master

commit 8f3a066b20b7ffafec95a618d60e40727504f37c
Author: Harald Jensås <email address hidden>
Date: Thu May 3 03:15:07 2018 +0200

    Add support to regenerate port mac_address

    Passing 'null' (None) as the mac address in a port update
    request causes the port's mac address to be re-generated
    using the base MAC address Neutron uses for VIFs.

    This change implementes a temporary lib api definition
    with a new converter that will generate valid mac if the
    data provided is None.

    APIImpact: Port mac_addr regenerated if None passed on update.
    Closes-Bug: #1768690
    Change-Id: I7d04beea4810718c3b745de8ea97897b1323267e

[Harald Jensås (harald-jensas)]

I removed [RFE] from subject because this patch is part of fixing a bug in the ironic nova neutron interaction triangle. Where a port in a network can be left with the mac address of a baremetal server if it is not deleted after a deployment or deployment failure.

I want to use this in Ironic to fix the bug, and it needs to be backported.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/580599

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/580600

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/580599

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/580600