Hi, 

we have a similar problem on instance resizes. 
We are currently running on OpenStack Victoria.

Steps to reproduce
==================
* create a server group
* start some instances in this group
* delete the server group
* resize an instance of the former server group

This also results in the error message 'Instance group ad661086-ca64-47ee-ad16-3226734de605 could not be found.' and vm_state 'error'. 

I traced the error down and it seems to originate from nova-compute on the destination host candidates.
Here are the error logs from nova-compute on one of the destinations candidates when trying to do the resize: 


2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [req-64de0df0-e144-4dba-ada4-1cb048e82706 4e9bb6ded0ed4a248be15edf71ea8944 f7c7480026ff49fcb5d439f66ea05203 - default default] [instance: b9a023da-26ae-4cef-af6e-20b549c0236f] Error trying to reschedule: nova.exception_Remote.NoValidHost_Remote: No valid host
was found. No valid host found for resize
Traceback (most recent call last):

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/manager.py", line 353, in _cold_migrate
    task.execute()

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/tasks/base.py", line 26, in wrap
    self.rollback(ex)

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/tasks/base.py", line 23, in wrap
    return original(self)

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/tasks/base.py", line 40, in execute
    return self._execute()

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/tasks/migrate.py", line 300, in _execute
    selection = self._reschedule()

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/tasks/migrate.py", line 382, in _reschedule
    raise exception.MaxRetriesExceeded(reason=reason)

nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b9a023da-26ae-4cef-af6e-20b549c0236f.


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 241, in inner
    return func(*args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/manager.py", line 98, in wrapper
    return fn(self, context, *args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/utils.py", line 1434, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/manager.py", line 299, in migrate_server
    self._cold_migrate(context, instance, flavor,

  File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/manager.py", line 368, in _cold_migrate
    raise exception.NoValidHost(reason=msg)

nova.exception.NoValidHost: No valid host was found. No valid host found for resize
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f] Traceback (most recent call last):
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 5233, in prep_resize
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     self._validate_instance_group_policy(context, instance,
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 1640, in _validate_instance_group_policy
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     group = objects.InstanceGroup.get_by_hint(context, group_hint)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/objects/instance_group.py", line 384, in get_by_hint
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return cls.get_by_uuid(context, hint)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_versionedobjects/base.py", line 175, in wrapper
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     result = cls.indirection_api.object_class_action_versions(
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/rpcapi.py", line 240, in object_class_action_versions
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return cctxt.call(context, 'object_class_action_versions',
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/rpc/client.py", line 175, in call
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     self.transport._send(self.target, msg_ctxt, msg,
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/transport.py", line 123, in _send
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return self._driver.send(target, ctxt, message,
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 680, in send
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return self._send(target, ctxt, message, wait_for_reply, timeout,
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 672, in _send
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     raise result
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f] nova.exception_Remote.InstanceGroupNotFound_Remote: Instance group ad661086-ca64-47ee-ad16-3226734de605 could not be found.
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f] Traceback (most recent call last):
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/conductor/manager.py", line 138, in _object_dispatch
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return getattr(target, method)(*args, **kwargs)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     result = fn(cls, context, *args, **kwargs)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/objects/instance_group.py", line 368, in get_by_uuid
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     db_group = cls._get_from_db_by_uuid(context, uuid)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1014, in wrapper
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     return fn(*args, **kwargs)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]   File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/objects/instance_group.py", line 223, in _get_from_db_by_uuid
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]     raise exception.InstanceGroupNotFound(group_uuid=uuid)
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f]
2021-11-02 14:24:09.560 7 ERROR nova.compute.manager [instance: b9a023da-26ae-4cef-af6e-20b549c0236f] nova.exception.InstanceGroupNotFound: Instance group ad661086-ca64-47ee-ad16-3226734de605 could not be found.


Server group show before the delete operation: 

~ $ openstack server group show ad661086-ca64-47ee-ad16-3226734de605
+----------+--------------------------------------+
| Field    | Value                                |
+----------+--------------------------------------+
| id       | ad661086-ca64-47ee-ad16-3226734de605 |
| members  | b9a023da-26ae-4cef-af6e-20b549c0236f |
| name     | affinity-testgroup                   |
| policies | affinity                             |
+----------+--------------------------------------+


Checking the nova_api DB I found that the requests_specs table entry for my instance (b9a023da-26ae-4cef-af6e-20b549c0236f) still contains a scheduler_hints reference to the deleted server group: 

SELECT spec FROM request_specs WHERE instance_uuid = 'b9a023da-26ae-4cef-af6e-20b549c0236f';
..."scheduler_hints": {"group": ["ad661086-ca64-47ee-ad16-3226734de605"]},...

Removing the group ID from the array and setting the instance state back to 'active' I was able to successfully resize afterwards. 

UPDATE request_specs SET spec = '{"nova_object.name": "RequestSpec", ..., "scheduler_hints": {"group": []}, ...}' WHERE instance_uuid = 'b9a023da-26ae-4cef-af6e-20b549c0236f';

openstack server set --state active b9a023da-26ae-4cef-af6e-20b549c0236f