Spurious ComputeHostNotFound warnings in nova-compute logs during ironic node re-balance

Bug #1841476 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Triaged
Low
Unassigned

Bug Description

Seen here:

https://d01b2e57f0a56cb7edf0-b6bc206936c08bb07a5f77cfa916a2d4.ssl.cf5.rackcdn.com/678298/4/check/ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode/92c65ac/compute1/logs/screen-n-cpu.txt.gz

We see a warning that a compute node could not be found by host and node but then later is found just by nodename and is moving to the current host:

Aug 26 18:41:38.800657 ubuntu-bionic-rax-ord-0010443319 nova-compute[747]: WARNING nova.compute.resource_tracker [None req-a894abee-a2f1-4423-8ede-2a1b9eef28a4 None None] No compute node record for ubuntu-bionic-rax-ord-0010443319:61dbc9c7-828b-4c42-b19c-a3716037965f: ComputeHostNotFound_Remote: Compute host ubuntu-bionic-rax-ord-0010443319 could not be found.

Aug 26 18:41:38.818412 ubuntu-bionic-rax-ord-0010443319 nova-compute[747]: INFO nova.compute.resource_tracker [None req-a894abee-a2f1-4423-8ede-2a1b9eef28a4 None None] ComputeNode 61dbc9c7-828b-4c42-b19c-a3716037965f moving from ubuntu-bionic-rax-ord-0010443317 to ubuntu-bionic-rax-ord-0010443319

The warning comes from this call:

https://github.com/openstack/nova/blob/71478c3eedd95e2eeb219f47460603221ee249b9/nova/compute/resource_tracker.py#L554

And the re-balance is found here:

https://github.com/openstack/nova/blob/71478c3eedd95e2eeb219f47460603221ee249b9/nova/compute/resource_tracker.py#L561

The warning is then a red herring. We could:

1. add something to the warning message saying this could be due to a re-balance but that might be confusing for non-ironic computes

and/or

2. check if self.driver.rebalances_nodes and if True, change the warning to an info level message (and potentially modify the message with the re-balance wording in #1 above).

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.