Failed to set DHCP BOOT options for any port

[Ladislav Smola]

Ironic condutor fails to update neutron because of:
neutron-server[5252]: 2014-09-19 08:01:33.846 5252 WARNING keystonemiddleware.auth_token [-] Authorization failed for token

That leads to:
 ironic-conductor[4728]: FailedToUpdateDHCPOptOnPort: Failed to set DHCP BOOT options for any port on node df50441c-c7a7-46d0-b774-a0a07158c88a.

Leading to stack-create fail with:
No valid host was found. Exceeded max scheduling

This happens after > 24h of running Ironic. heat stack-create works again if I restart ironic-conductor. Seems like Ironic is holding some old auth_token there.

full log around that event:

http://paste.openstack.org/show/113198/

[Yuriy Zveryanskyy (yzveryanskyy)]

Ladislav, you are right about old token. Neutron DHCP provider uses first token always.
1) https://github.com/openstack/ironic/blob/master/ironic/common/dhcp_factory.py#L41-L43
2) https://github.com/openstack/ironic/blob/master/ironic/dhcp/neutron.py#L61
3) https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/pxe.py#L325

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/122728

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/122728
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=efd63cbf53846ae34ad8ba319ff221c7e26e5e65
Submitter: Jenkins
Branch: master

commit efd63cbf53846ae34ad8ba319ff221c7e26e5e65
Author: Dmitry Tantsur <email address hidden>
Date: Fri Sep 19 15:08:02 2014 +0200

    Do not cache auth token in Neutron DHCP provider

    Currently token expire after some time and deploy breaks.
    Also refactored all tests for Neutron DHCP provider, as
    many of them were just broken (e.g. using non-existing asserts).

    Change-Id: I38a8e95a09fd4737682e37b1162eb445fbc8012f
    Closes-Bug: #1371503

[Ladislav Smola (lsmola)]

solved in ironic