Comment 1 for bug 1876772

Hi,

Adding missing Traceback you have shared with me:

2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager [req-34c9d727-3683-45bd-824c-569a9dd8124d - - - - -] Error updating resources for node b1b.blr.example.cloud.: ResourceProviderCreationFailed: Failed to create resource provider b1b.blr.example.cloud
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager Traceback (most recent call last):
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/compute/manager.py", line 7778, in _update_available_resource_for_node
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager rt.update_available_resource(context, nodename)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 721, in update_available_resource
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager self._update_available_resource(context, resources)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 274, in inner
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager return f(*args, **kwargs)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 798, in _update_available_resource
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager self._update(context, cn)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/retrying.py", line 49, in wrapped_f
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager return Retrying(*dargs, **dkw).call(f, *args, **kw)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/retrying.py", line 206, in call
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager return attempt.get(self._wrap_exception)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/retrying.py", line 247, in get
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager six.reraise(self.value[0], self.value[1], self.value[2])
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/retrying.py", line 200, in call
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 960, in _update
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager self._update_to_placement(context, compute_node)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 912, in _update_to_placement
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager context, compute_node.uuid, name=compute_node.hypervisor_hostname)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 35, in __run_method
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager return getattr(self.instance, __name)(*args, **kwargs)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 1006, in get_provider_tree_and_ensure_root
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager parent_provider_uuid=parent_provider_uuid)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 672, in _ensure_resource_provider
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager parent_provider_uuid=parent_provider_uuid)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 74, in wrapper
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager return f(self, *a, **k)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager File "/openstack/venvs/nova-18.1.2/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 606, in _create_resource_provider
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager raise exception.ResourceProviderCreationFailed(name=name)
2020-05-05 00:56:53.160 29284 ERROR nova.compute.manager ResourceProviderCreationFailed: Failed to create resource provider b1b.blr.example.cloud

It really depends what happened during the upgrade, the error is trying to create resource provider that already exists.

Is it possible the upgrade has removed resource provider or were supposed to and now it's failing to be re-created.

It would be great to get more insighst about the resource provider and if it has any allocations:

openstack resource provider list --name example.cloud
openstack resource provider show --allocations <resource-provider-uuid>

I guess the way would be to remove the resource provider, but if you already have allocations to it, you will have to save and restore them. I am not sure if the nova would re-create those.

How to delete the resource provider and save allocations please check https://bugzilla.redhat.com/show_bug.cgi?id=1774243#c2

Also note that the CLI can be installed if its not available for you. I would try something like:

pipsi install osc-placement

it will create osc-placement under $HOME/venvs/osc-placement virtual environment so you should be able to use the tool without messing up the system.

Or you can edit directly the DB, I wouldn't recommend that much but if there is no other option.

Please note it all may wary, and depends on which intaller you are using. I believe this is a valid bug, that makes sense to look into. But you may need to provide more information of what exactly happened.