Activity log for bug #1867041

Date Who What changed Old value New value Message
2020-03-11 18:40:14 Vladimir Grevtsev bug added bug
2020-03-11 18:40:58 Vladimir Grevtsev description latest charms, bionic-train steps to reproduce: bootstrap a cloud, remove nova-compute units, do a "openstack hypervisor list" $ juju status Model Controller Cloud/Region Version SLA Timestamp openstack foundations-maas maas_cloud/default 2.7.3 unsupported 18:36:41Z App Version Status Scale Charm Store Rev OS Notes ceph-mon 14.2.4 active 3 ceph-mon jujucharms 45 ubuntu ceph-osd 14.2.4 active 4 ceph-osd jujucharms 298 ubuntu ceph-radosgw 14.2.4 active 1 ceph-radosgw jujucharms 285 ubuntu cinder 15.0.1 active 1 cinder jujucharms 300 ubuntu cinder-ceph 15.0.1 active 1 cinder-ceph jujucharms 253 ubuntu glance 19.0.0 active 1 glance jujucharms 294 ubuntu keystone 16.0.0 active 1 keystone jujucharms 310 ubuntu memcached active 2 memcached jujucharms 28 ubuntu mysql 5.7.20 active 3 percona-cluster jujucharms 284 ubuntu neutron-api 15.0.1 active 1 neutron-api jujucharms 283 ubuntu neutron-openvswitch maintenance 1 neutron-openvswitch jujucharms 273 ubuntu nova-cloud-controller 20.0.1 active 1 nova-cloud-controller jujucharms 341 ubuntu nova-compute-kvm 20.0.1 waiting 1 nova-compute jujucharms 312 ubuntu ntp waiting 0/1 ntp jujucharms 39 ubuntu openstack-dashboard 16.0.0 active 1 openstack-dashboard jujucharms 299 ubuntu placement 2.0.0 active 1 placement jujucharms 4 ubuntu rabbitmq-server 3.6.10 active 3 rabbitmq-server jujucharms 99 ubuntu Unit Workload Agent Machine Public address Ports Message ceph-mon/0* active idle 8/lxd/0 172.27.84.246 Unit is ready and clustered ceph-mon/1 active idle 1/lxd/0 172.27.85.2 Unit is ready and clustered ceph-mon/2 active idle 4/lxd/0 172.27.84.255 Unit is ready and clustered ceph-osd/0* active idle 0 172.27.84.236 Unit is ready (1 OSD) ceph-osd/1 active idle 1 172.27.84.237 Unit is ready (1 OSD) ceph-osd/2 active idle 2 172.27.84.238 Unit is ready (1 OSD) ceph-osd/3 active idle 3 172.27.84.239 Unit is ready (1 OSD) ceph-radosgw/0* active idle 4/lxd/1 172.27.85.1 80/tcp Unit is ready cinder/0* active idle 7/lxd/0 172.27.84.251 8776/tcp Unit is ready cinder-ceph/0* active idle 172.27.84.251 Unit is ready glance/0* active idle 7/lxd/1 172.27.84.247 9292/tcp Unit is ready keystone/0* active idle 1/lxd/1 172.27.85.3 5000/tcp Unit is ready memcached/0 active idle 7/lxd/2 172.27.84.249 11211/tcp Unit is ready and clustered memcached/1* active idle 8/lxd/1 172.27.84.250 11211/tcp Unit is ready and clustered mysql/0 active idle 7/lxd/3 172.27.84.253 3306/tcp Unit is ready mysql/1 active idle 1/lxd/2 172.27.85.5 3306/tcp Unit is ready mysql/2* active idle 2/lxd/0 172.27.84.254 3306/tcp Unit is ready neutron-api/0* active idle 8/lxd/2 172.27.84.252 9696/tcp Unit is ready nova-cloud-controller/0* active executing 8/lxd/3 172.27.84.248 8774/tcp,8775/tcp Unit is ready nova-compute-kvm/5* waiting executing 0 172.27.84.236 Incomplete relations: image, messaging neutron-openvswitch/48* maintenance executing 172.27.84.236 (install) Installing packages ntp/103* waiting allocating 172.27.84.236 agent initializing openstack-dashboard/0* active idle 3/lxd/0 172.27.85.4 80/tcp,443/tcp Unit is ready placement/0* active idle 6/lxd/1 172.27.85.7 8778/tcp Unit is ready rabbitmq-server/0 active idle 3/lxd/1 172.27.85.6 5672/tcp Unit is ready and clustered rabbitmq-server/1 active idle 5/lxd/0 172.27.85.0 5672/tcp Unit is ready and clustered rabbitmq-server/2* active idle 6/lxd/0 172.27.84.245 5672/tcp Unit is ready and clustered $ openstack hypervisor list +----+---------------------+-----------------+---------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+---------------+-------+ | 1 | node02.maas | QEMU | 172.27.84.238 | down | | 4 | node04.maas | QEMU | 172.27.84.240 | down | | 7 | node01.maas | QEMU | 172.27.84.237 | down | | 10 | node03.maas | QEMU | 172.27.84.239 | down | | 13 | node05.maas | QEMU | 172.27.84.242 | down | +----+---------------------+-----------------+---------------+-------+ $ openstack network agent list +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ | 0a00645c-2711-4457-8729-6a9d689d3e5c | DHCP agent | node04.maas | nova | :-) | UP | neutron-dhcp-agent | | 0f1e61f8-bee3-4134-b56a-28f122b5bda3 | Open vSwitch agent | node04.maas | None | :-) | UP | neutron-openvswitch-agent | | 1966a277-d634-4255-b14d-9409ec2429cc | L3 agent | node03.maas | nova | :-) | UP | neutron-l3-agent | | 19b02fba-9f16-4c55-879c-6f633b5b3bcb | L3 agent | node02.maas | nova | :-) | UP | neutron-l3-agent | | 1a7ee5e8-d2bd-4581-9d4c-e8990cb8d14e | DHCP agent | node02.maas | nova | :-) | UP | neutron-dhcp-agent | | 2f7ffae1-ca6e-4cd0-91ea-9d5ff56c0174 | Metadata agent | node01.maas | None | :-) | UP | neutron-metadata-agent | | 314880a7-3224-4ec7-a9b9-23d239567566 | Metadata agent | node03.maas | None | :-) | UP | neutron-metadata-agent | | 3fe5e46a-fe87-4be7-b5c2-35f1746ed378 | Open vSwitch agent | node03.maas | None | :-) | UP | neutron-openvswitch-agent | | 5220c51b-e42e-4542-89ec-e1944922efd9 | DHCP agent | node03.maas | nova | :-) | UP | neutron-dhcp-agent | | 58d76307-91f1-4d11-9791-3987998502e3 | L3 agent | node04.maas | nova | :-) | UP | neutron-l3-agent | | 62528a41-6ffb-4cb0-8046-9e1877193d7d | Metadata agent | node02.maas | None | :-) | UP | neutron-metadata-agent | | 6de39cce-4ee0-499e-827b-8096e6ef5607 | L3 agent | node01.maas | nova | :-) | UP | neutron-l3-agent | | 7a515b87-db15-458c-b32e-4cab5585d34d | Metadata agent | node04.maas | None | :-) | UP | neutron-metadata-agent | | 84162297-d7e8-413b-b17e-16ec9dd4483f | Metadata agent | node05.maas | None | :-) | UP | neutron-metadata-agent | | 853d87c2-e8d6-4d8e-ae5c-6ae56779a0b4 | Open vSwitch agent | node02.maas | None | :-) | UP | neutron-openvswitch-agent | | aa2c5522-b4ee-4e6d-80de-da57caa78e5e | L3 agent | node05.maas | nova | :-) | UP | neutron-l3-agent | | ac84d7bd-ef79-4de9-9556-3666af65753b | Open vSwitch agent | node01.maas | None | :-) | UP | neutron-openvswitch-agent | | d2945aa7-87d1-4a19-b86b-368db2a78235 | Open vSwitch agent | node05.maas | None | :-) | UP | neutron-openvswitch-agent | | d5cf1059-70b9-49b6-8112-bea32d9f37cf | DHCP agent | node05.maas | nova | :-) | UP | neutron-dhcp-agent | | f2a79451-a09b-4ff0-8601-66223fadbe90 | DHCP agent | node01.maas | nova | :-) | UP | neutron-dhcp-agent | +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ latest charms, bionic-train steps to reproduce: bootstrap a cloud, remove nova-compute units, do a "openstack hypervisor list" juju status: https://pastebin.canonical.com/p/2RwktsDQ9r/ $ openstack hypervisor list +----+---------------------+-----------------+---------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+---------------+-------+ | 1 | node02.maas | QEMU | 172.27.84.238 | down | | 4 | node04.maas | QEMU | 172.27.84.240 | down | | 7 | node01.maas | QEMU | 172.27.84.237 | down | | 10 | node03.maas | QEMU | 172.27.84.239 | down | | 13 | node05.maas | QEMU | 172.27.84.242 | down | +----+---------------------+-----------------+---------------+-------+ $ openstack network agent list +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ | 0a00645c-2711-4457-8729-6a9d689d3e5c | DHCP agent | node04.maas | nova | :-) | UP | neutron-dhcp-agent | | 0f1e61f8-bee3-4134-b56a-28f122b5bda3 | Open vSwitch agent | node04.maas | None | :-) | UP | neutron-openvswitch-agent | | 1966a277-d634-4255-b14d-9409ec2429cc | L3 agent | node03.maas | nova | :-) | UP | neutron-l3-agent | | 19b02fba-9f16-4c55-879c-6f633b5b3bcb | L3 agent | node02.maas | nova | :-) | UP | neutron-l3-agent | | 1a7ee5e8-d2bd-4581-9d4c-e8990cb8d14e | DHCP agent | node02.maas | nova | :-) | UP | neutron-dhcp-agent | | 2f7ffae1-ca6e-4cd0-91ea-9d5ff56c0174 | Metadata agent | node01.maas | None | :-) | UP | neutron-metadata-agent | | 314880a7-3224-4ec7-a9b9-23d239567566 | Metadata agent | node03.maas | None | :-) | UP | neutron-metadata-agent | | 3fe5e46a-fe87-4be7-b5c2-35f1746ed378 | Open vSwitch agent | node03.maas | None | :-) | UP | neutron-openvswitch-agent | | 5220c51b-e42e-4542-89ec-e1944922efd9 | DHCP agent | node03.maas | nova | :-) | UP | neutron-dhcp-agent | | 58d76307-91f1-4d11-9791-3987998502e3 | L3 agent | node04.maas | nova | :-) | UP | neutron-l3-agent | | 62528a41-6ffb-4cb0-8046-9e1877193d7d | Metadata agent | node02.maas | None | :-) | UP | neutron-metadata-agent | | 6de39cce-4ee0-499e-827b-8096e6ef5607 | L3 agent | node01.maas | nova | :-) | UP | neutron-l3-agent | | 7a515b87-db15-458c-b32e-4cab5585d34d | Metadata agent | node04.maas | None | :-) | UP | neutron-metadata-agent | | 84162297-d7e8-413b-b17e-16ec9dd4483f | Metadata agent | node05.maas | None | :-) | UP | neutron-metadata-agent | | 853d87c2-e8d6-4d8e-ae5c-6ae56779a0b4 | Open vSwitch agent | node02.maas | None | :-) | UP | neutron-openvswitch-agent | | aa2c5522-b4ee-4e6d-80de-da57caa78e5e | L3 agent | node05.maas | nova | :-) | UP | neutron-l3-agent | | ac84d7bd-ef79-4de9-9556-3666af65753b | Open vSwitch agent | node01.maas | None | :-) | UP | neutron-openvswitch-agent | | d2945aa7-87d1-4a19-b86b-368db2a78235 | Open vSwitch agent | node05.maas | None | :-) | UP | neutron-openvswitch-agent | | d5cf1059-70b9-49b6-8112-bea32d9f37cf | DHCP agent | node05.maas | nova | :-) | UP | neutron-dhcp-agent | | f2a79451-a09b-4ff0-8601-66223fadbe90 | DHCP agent | node01.maas | nova | :-) | UP | neutron-dhcp-agent | +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
2020-03-11 18:44:05 Vladimir Grevtsev summary Hypervisors are present in the cloud after corresponding unit has been removed Hypervisors are present in the cloud after corresponding units have been removed
2020-03-12 14:01:34 Aurelien Lourot charm-nova-compute: assignee Aurelien Lourot (aurelien-lourot)
2020-03-12 15:17:07 Aurelien Lourot tags scaleback
2020-03-12 15:26:21 Aurelien Lourot charm-nova-compute: importance Undecided Low
2020-03-12 15:26:30 Aurelien Lourot charm-nova-compute: status New Confirmed
2020-03-12 15:26:35 Aurelien Lourot charm-nova-compute: assignee Aurelien Lourot (aurelien-lourot)