changing openstack_domain does not change in nova DB

Bug #2033209 reported by Aref
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
Undecided
Unassigned

Bug Description

In inventory/group_vars/all.yml file and in openstack_hosts/defaults/main.yml role openstack_domain variable defined as openstack.local

openstack_domain: openstack.local

However, during OSA deployment for some reason, we used the FQDN name. such as example.com

after updating the openstack_domain.
all existing instances has error while live-migration that they can not find the new compute.

Unable to find record for source node compute-02.openstack.local on compute-02:
nova.exception.ComputeHostNotFound: Compute host compute-02 could not be found.

it searched for compute-02.openstack.local but it does not exist anymore. After I checked the Nova DB
I found that the compute_nodes and instances table is not updated with the new openstack_domain name.
now I want a way other than manually changing the DB fields

Aref (rfak)
description: updated
Revision history for this message
Aref (rfak) wrote :

all vms become orphened with old name resource allocation. there has to be a way to migrate all instances attached to old resource allocation to new node name ,
its odd that when openstack_domain is change it create new compute_node in DB and new resource allocation.

is there any way to correct that?

Revision history for this message
Dmitriy Rabotyagov (noonedeadpunk) wrote :

Hey,

Sorry, I am inclined to say that the problem you are describing is specifically nova issue. I will change affected project to Nova, as I believe this bug report fits better there.

As far as I know regarding the topic - this is pretty much intended behavior and a known dropback. The only workaround I'm aware of is to update the database. So in case a hostname for compute needs to change, we always ensure that no VM is running there before doing so.

affects: openstack-ansible → nova
Revision history for this message
Aref (rfak) wrote :

thanks for your response.
the problem is with nova and placement.

each instances is bond to resource provider in placement. when the openstack_domain changes, it remove thos placement resource provider and add new ones. that cause the all running instances is bond to old resource provider name which is deleted. so they become orohaned instances and no longer to do some tasks like live-migration and resizeing. changing in DB wont effect unless with the help of this doc on
https://docs.openstack.org/nova/xena/admin/troubleshooting/orphaned-allocations.html
we have to change the resource group for each instances.
we need thats to be done either automatically or whenever changing the openstack_domain it will not delete old resource provider and just update the name of existings one.
I know changing openstack_domain is not a repetitive task but when enabling ssl certs for all parts of openstack. we use wildcard ssl certs that need the changes of openstack_domain.

another approaches that i havent tested yet is that to manully change the resource provider full name to new domain and then run all playbooks. but i dont have any idea whether it will work or not.

Revision history for this message
Aref (rfak) wrote :

Its really a pain in the neck. anyone has any idea?

Revision history for this message
Artom Lifshitz (notartom) wrote :

If I understand correctly, it looks like the hostnames of your compute hosts has changed because of a deployment error. I understand that this is not your fault, but renaming compute host names is essentially forbidden, as it causes the kind of breakage that you're reporting in this bug.

We have implemented protections from this in [1], but that work will only be released in Caracal. I'm afraid for now hostnames are immutable, and manual repair and recovery is needed if one of more compute hosts get renamed.

[1] https://review.opendev.org/c/openstack/nova/+/872432

Changed in nova:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.