Hypervisors are present in the cloud after corresponding units have been removed

[Vladimir Grevtsev]

latest charms, bionic-train
steps to reproduce: bootstrap a cloud, remove nova-compute units, do a "openstack hypervisor list"

juju status: https://pastebin.canonical.com/p/2RwktsDQ9r/

$ openstack hypervisor list
+----+---------------------+-----------------+---------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+---------------+-------+
| 1 | node02.maas | QEMU | 172.27.84.238 | down |
| 4 | node04.maas | QEMU | 172.27.84.240 | down |
| 7 | node01.maas | QEMU | 172.27.84.237 | down |
| 10 | node03.maas | QEMU | 172.27.84.239 | down |
| 13 | node05.maas | QEMU | 172.27.84.242 | down |
+----+---------------------+-----------------+---------------+-------+

$ openstack network agent list
+--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
| 0a00645c-2711-4457-8729-6a9d689d3e5c | DHCP agent | node04.maas | nova | :-) | UP | neutron-dhcp-agent |
| 0f1e61f8-bee3-4134-b56a-28f122b5bda3 | Open vSwitch agent | node04.maas | None | :-) | UP | neutron-openvswitch-agent |
| 1966a277-d634-4255-b14d-9409ec2429cc | L3 agent | node03.maas | nova | :-) | UP | neutron-l3-agent |
| 19b02fba-9f16-4c55-879c-6f633b5b3bcb | L3 agent | node02.maas | nova | :-) | UP | neutron-l3-agent |
| 1a7ee5e8-d2bd-4581-9d4c-e8990cb8d14e | DHCP agent | node02.maas | nova | :-) | UP | neutron-dhcp-agent |
| 2f7ffae1-ca6e-4cd0-91ea-9d5ff56c0174 | Metadata agent | node01.maas | None | :-) | UP | neutron-metadata-agent |
| 314880a7-3224-4ec7-a9b9-23d239567566 | Metadata agent | node03.maas | None | :-) | UP | neutron-metadata-agent |
| 3fe5e46a-fe87-4be7-b5c2-35f1746ed378 | Open vSwitch agent | node03.maas | None | :-) | UP | neutron-openvswitch-agent |
| 5220c51b-e42e-4542-89ec-e1944922efd9 | DHCP agent | node03.maas | nova | :-) | UP | neutron-dhcp-agent |
| 58d76307-91f1-4d11-9791-3987998502e3 | L3 agent | node04.maas | nova | :-) | UP | neutron-l3-agent |
| 62528a41-6ffb-4cb0-8046-9e1877193d7d | Metadata agent | node02.maas | None | :-) | UP | neutron-metadata-agent |
| 6de39cce-4ee0-499e-827b-8096e6ef5607 | L3 agent | node01.maas | nova | :-) | UP | neutron-l3-agent |
| 7a515b87-db15-458c-b32e-4cab5585d34d | Metadata agent | node04.maas | None | :-) | UP | neutron-metadata-agent |
| 84162297-d7e8-413b-b17e-16ec9dd4483f | Metadata agent | node05.maas | None | :-) | UP | neutron-metadata-agent |
| 853d87c2-e8d6-4d8e-ae5c-6ae56779a0b4 | Open vSwitch agent | node02.maas | None | :-) | UP | neutron-openvswitch-agent |
| aa2c5522-b4ee-4e6d-80de-da57caa78e5e | L3 agent | node05.maas | nova | :-) | UP | neutron-l3-agent |
| ac84d7bd-ef79-4de9-9556-3666af65753b | Open vSwitch agent | node01.maas | None | :-) | UP | neutron-openvswitch-agent |
| d2945aa7-87d1-4a19-b86b-368db2a78235 | Open vSwitch agent | node05.maas | None | :-) | UP | neutron-openvswitch-agent |
| d5cf1059-70b9-49b6-8112-bea32d9f37cf | DHCP agent | node05.maas | nova | :-) | UP | neutron-dhcp-agent |
| f2a79451-a09b-4ff0-8601-66223fadbe90 | DHCP agent | node01.maas | nova | :-) | UP | neutron-dhcp-agent |
+--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+

[Aurelien Lourot (aurelien-lourot)]

I can easily reproduce it. Hypervisors are marked as 'down' instead of fully going away from the list. It doesn't sound very severe though, so setting this to 'Low' for now.

Steps to reproduce:

git clone https://github.com/openstack-charmers/openstack-bundles
cd openstack-bundles/stable/openstack-base/
juju deploy ./bundle.yaml --overlay ../overlays/openstack-base-virt-overlay.yaml
watch -c "juju status --color" # wait for active/idle on all units
. openrc
openstack hypervisor list # -> 3 hypervisors up
juju remove-unit nova-compute/2
watch -c "juju status --color" # wait for active/idle on all units

openstack hypervisor list
# expected: 2 hypervisors up, 0 hypervisors down
# actual: 2 hypervisors up, 1 hypervisor down