Deferred IP allocation, port update with binding_host_id + set new mac address fails

[Harald Jensås]

On a routed provider network IP allocation will be deffered if insufficent binding information is available.

When a port is updated with binding_host_id only this works as expected:
------------------------------------------------------------------------

$ openstack port create --network ctlplane testport -f yaml
admin_state_up: UP
allowed_address_pairs: ''
binding_host_id: ''
binding_profile: ''
binding_vif_details: ''
binding_vif_type: unbound
binding_vnic_type: normal
created_at: '2019-01-16T00:13:19Z'
data_plane_status: null
description: ''
device_id: ''
device_owner: ''
dns_assignment: null
dns_domain: null
dns_name: null
extra_dhcp_opts: ''
fixed_ips: ''
id: 9e05e8fe-c11a-4682-ba69-2a402094758d
mac_address: fa:16:3e:31:de:01
name: testport
network_id: 1f731773-6dcf-45ce-aa11-aaf205f2faf6
port_security_enabled: true
project_id: 9d8aea1921f04207b35a9e1fc4923ae1
qos_policy_id: null
revision_number: 1
security_group_ids: 2354cbbe-5726-42ef-8b45-e642cb9bf2ea
status: DOWN
tags: ''
trunk_details: null
updated_at: '2019-01-16T00:13:19Z'

$ openstack port set --host 05a94a66-27ca-4642-ad62-8f53827055c7 testport

$ openstack port show testport -f yaml
admin_state_up: UP
allowed_address_pairs: ''
binding_host_id: 05a94a66-27ca-4642-ad62-8f53827055c7
binding_profile: ''
binding_vif_details: ''
binding_vif_type: binding_failed
binding_vnic_type: normal
created_at: '2019-01-16T00:13:19Z'
data_plane_status: null
description: ''
device_id: ''
device_owner: ''
dns_assignment: null
dns_domain: null
dns_name: null
extra_dhcp_opts: ''
fixed_ips: ip_address='172.20.0.11', subnet_id='61745187-973c-4515-a025-c719776bbf44'
id: 9e05e8fe-c11a-4682-ba69-2a402094758d
mac_address: fa:16:3e:31:de:01
name: testport
network_id: 1f731773-6dcf-45ce-aa11-aaf205f2faf6
port_security_enabled: true
project_id: 9d8aea1921f04207b35a9e1fc4923ae1
qos_policy_id: null
revision_number: 3
security_group_ids: 2354cbbe-5726-42ef-8b45-e642cb9bf2ea
status: DOWN
tags: ''
trunk_details: null
updated_at: '2019-01-16T00:13:58Z'

2019-01-16 01:13:58.064 38 DEBUG neutron.api.v2.base [req-3baa886c-f409-4a2f-82b4-a46055e6653b 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] Request body: {u'port': {u'binding:host_id':
 u'05a94a66-27ca-4642-ad62-8f53827055c7'}} prepare_request_body /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:715

2019-01-16 01:13:58.310 38 DEBUG neutron.db.db_base_plugin_common [req-3baa886c-f409-4a2f-82b4-a46055e6653b 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] Allocated IP 172.20.0.11 (1f731
773-6dcf-45ce-aa11-aaf205f2faf6/61745187-973c-4515-a025-c719776bbf44/9e05e8fe-c11a-4682-ba69-2a402094758d) _store_ip_allocation /usr/lib/python2.7/site-packages/neutron/db/db_base_plugin_common.py:124

When a port update request contain both binding_host_id and a new mac address,
IP allocation does not happen:
------------------------------

$ openstack port create --network ctlplane testport -f yaml
admin_state_up: UP
allowed_address_pairs: ''
binding_host_id: ''
binding_profile: ''
binding_vif_details: ''
binding_vif_type: unbound
binding_vnic_type: normal
created_at: '2019-01-16T00:15:35Z'
data_plane_status: null
description: ''
device_id: ''
device_owner: ''
dns_assignment: null
dns_domain: null
dns_name: null
extra_dhcp_opts: ''
fixed_ips: ''
id: 3c6ecca2-3f77-4528-86c2-66971c89ef2e
mac_address: fa:16:3e:aa:f4:46
name: testport
network_id: 1f731773-6dcf-45ce-aa11-aaf205f2faf6
port_security_enabled: true
project_id: 9d8aea1921f04207b35a9e1fc4923ae1
qos_policy_id: null
revision_number: 1
security_group_ids: 2354cbbe-5726-42ef-8b45-e642cb9bf2ea
status: DOWN
tags: ''
trunk_details: null
updated_at: '2019-01-16T00:15:35Z'

$ openstack port set --host 05a94a66-27ca-4642-ad62-8f53827055c7 --mac-address 52:54:00:76:4f:56 testport

$ openstack port show testport -f yaml
admin_state_up: UP
allowed_address_pairs: ''
binding_host_id: 05a94a66-27ca-4642-ad62-8f53827055c7
binding_profile: ''
binding_vif_details: ''
binding_vif_type: binding_failed
binding_vnic_type: normal
created_at: '2019-01-16T00:15:35Z'
data_plane_status: null
description: ''
device_id: ''
device_owner: ''
dns_assignment: null
dns_domain: null
dns_name: null
extra_dhcp_opts: ''
fixed_ips: ''
id: 3c6ecca2-3f77-4528-86c2-66971c89ef2e
mac_address: 52:54:00:76:4f:56
name: testport
network_id: 1f731773-6dcf-45ce-aa11-aaf205f2faf6
port_security_enabled: true
project_id: 9d8aea1921f04207b35a9e1fc4923ae1
qos_policy_id: null
revision_number: 3
security_group_ids: 2354cbbe-5726-42ef-8b45-e642cb9bf2ea
status: DOWN
tags: ''
trunk_details: null
updated_at: '2019-01-16T00:15:46Z'

2019-01-16 01:15:45.684 39 DEBUG neutron.api.v2.base [req-8373dd06-d3fe-4fdc-841e-3850ec6eb374 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] Request body: {u'port': {u'binding:host_id':
 u'05a94a66-27ca-4642-ad62-8f53827055c7', u'mac_address': u'52:54:00:76:4f:56'}} prepare_request_body /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:715

[Harald Jensås (harald-jensas)]

Some additional debug logging:
------------------------------

--- Port update with binding_host_id only:

2019-01-16 02:04:36.755 39 DEBUG neutron.db.ipam_backend_mixin [req-7c09a78b-e6b7-4592-9bb1-9d0cb513e367 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] HARALD - fixed_ips_requested False update_port /usr/lib/python2.7/site-packages/neutron/db/ipam_backend_mixin.py:658
2019-01-16 02:04:36.755 39 DEBUG neutron.db.ipam_backend_mixin [req-7c09a78b-e6b7-4592-9bb1-9d0cb513e367 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] HARALD - new_port {u'binding:host_id': u'05a94a66-27ca-4642-ad62-8f53827055c7'} update_port /usr/lib/python2.7/site-packages/neutron/db/ipam_backend_mixin.py:659

--- Port update with binding_host_id and mac_address:

2019-01-16 02:06:23.887 39 DEBUG neutron.db.ipam_backend_mixin [req-ab23fb97-2f15-4836-ba3d-e31f4357d1e5 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] HARALD - fixed_ips_requested True update_port /usr/lib/python2.7/site-packages/neutron/db/ipam_backend_mixin.py:658
2019-01-16 02:06:23.888 39 DEBUG neutron.db.ipam_backend_mixin [req-ab23fb97-2f15-4836-ba3d-e31f4357d1e5 944d2af27c0e4e41bc8203acf9b4e6fb 9d8aea1921f04207b35a9e1fc4923ae1 - default default] HARALD - new_port {'fixed_ips': [], u'binding:host_id': u'05a94a66-27ca-4642-ad62-8f53827055c7', u'mac_address': u'52:54:00:76:4f:56'} update_port /usr/lib/python2.7/site-packages/neutron/db/ipam_backend_mixin.py:659

For some reason 'fixed_ips': [] is added to the new_port data when updating be MAC address.

[Harald Jensås (harald-jensas)]

Here if new_mac is in the request fixed_ips defaults to [] if original port does not have fixed_ips.

neutron/db/ipam_pluggable_backend.py:387

        if new_mac:
            original = self._make_port_dict(db_port, process_extensions=False)
            if original.get('mac_address') != new_mac:
                original_ips = original.get('fixed_ips', [])
                new_ips = new_port.setdefault('fixed_ips', original_ips)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/631116

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/631116
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b0d758e1b4af9aed8c2da6e0bac4723d8e4f5443
Submitter: Zuul
Branch: master

commit b0d758e1b4af9aed8c2da6e0bac4723d8e4f5443
Author: Harald Jensås <email address hidden>
Date: Wed Jan 16 02:44:32 2019 +0100

    Fix port update deferred IP allocation with host_id + new MAC

    IP allocation was initially deffered due to lack of binding
    information. On port update the with both `mac_address` and
    `binding_host_id`` in the request 'fixed_ips: []' was
    appended to the new_port data. This caused the check for
    fixed_ips_requested to return True, which in turn cause
    deferred_ip_allocation to evaluates False.

    Only set the new_port default fixed_ips to original_ips if
    the original port had fixed_ips.

    Closes-Bug: #1811905
    Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/634111

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/634112

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/634111
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b3f32e190090dc7fa922ef4d7c88e3510c6b3c7b
Submitter: Zuul
Branch: stable/rocky

commit b3f32e190090dc7fa922ef4d7c88e3510c6b3c7b
Author: Harald Jensås <email address hidden>
Date: Wed Jan 16 02:44:32 2019 +0100

    Fix port update deferred IP allocation with host_id + new MAC

    IP allocation was initially deffered due to lack of binding
    information. On port update the with both `mac_address` and
    `binding_host_id`` in the request 'fixed_ips: []' was
    appended to the new_port data. This caused the check for
    fixed_ips_requested to return True, which in turn cause
    deferred_ip_allocation to evaluates False.

    Only set the new_port default fixed_ips to original_ips if
    the original port had fixed_ips.

    Closes-Bug: #1811905
    Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
    (cherry picked from commit b0d758e1b4af9aed8c2da6e0bac4723d8e4f5443)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/634112
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1e76ddf711e392267c3da093e41c7935bf4493d6
Submitter: Zuul
Branch: stable/queens

commit 1e76ddf711e392267c3da093e41c7935bf4493d6
Author: Harald Jensås <email address hidden>
Date: Wed Jan 16 02:44:32 2019 +0100

    Fix port update deferred IP allocation with host_id + new MAC

    IP allocation was initially deffered due to lack of binding
    information. On port update with both `mac_address` and
    `binding_host_id`` in the request, 'fixed_ips: []' was
    appended to the new_port data. This caused the check for
    fixed_ips_requested to return True, which in turn cause
    deferred_ip_allocation to evaluates False.

    Only set the new_port default fixed_ips to original_ips if
    the original port had fixed_ips.

    Conflicts:
        test_segment.py

    Closes-Bug: #1811905
    Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
    (cherry picked from commit b0d758e1b4af9aed8c2da6e0bac4723d8e4f5443)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/650254

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/650254
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e6ec35f6796806ec5c3b6a5d39c25f906056a93d
Submitter: Zuul
Branch: stable/pike

commit e6ec35f6796806ec5c3b6a5d39c25f906056a93d
Author: Harald Jensås <email address hidden>
Date: Wed Jan 16 02:44:32 2019 +0100

    Fix port update deferred IP allocation with host_id + new MAC

    IP allocation was initially deffered due to lack of binding
    information. On port update with both `mac_address` and
    `binding_host_id`` in the request, 'fixed_ips: []' was
    appended to the new_port data. This caused the check for
    fixed_ips_requested to return True, which in turn cause
    deferred_ip_allocation to evaluates False.

    Only set the new_port default fixed_ips to original_ips if
    the original port had fixed_ips.

    Conflicts:
        test_segment.py

    Closes-Bug: #1811905
    Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
    (cherry picked from commit b0d758e1b4af9aed8c2da6e0bac4723d8e4f5443)
    (cherry picked from commit 1e76ddf711e392267c3da093e41c7935bf4493d6)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.7 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.6 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.3 release.