Steps to reproduce
==================
1) remove all instances from the hypervisor:
(env) vance@zs95k5:~$ nova hypervisor-servers zs93k23
+----+------+---------------+---------------------+
| ID | Name | Hypervisor ID | Hypervisor Hostname |
+----+------+---------------+---------------------+
+----+------+---------------+---------------------+
2) disable the hypervisor:
(env) vance@zs95k5:~$ nova service-list
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
| 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:34.000000 | - |
| 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:27.000000 | - |
| 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:30.000000 | - |
| 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:13:28.000000 | - |
| 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - |
| 17 | nova-compute | zs93k23 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - |
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
(env) vance@zs95k5:~$ nova service-disable zs93k23 nova-compute
+---------+--------------+----------+
| Host | Binary | Status |
+---------+--------------+----------+
| zs93k23 | nova-compute | disabled |
+---------+--------------+----------+
3) delete the compute service
(env) vance@zs95k5:~$ nova service-delete 17
(env) vance@zs95k5:~$ nova service-list
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
| 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:54.000000 | - |
| 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:47.000000 | - |
| 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:56.000000 | - |
| 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:14:48.000000 | - |
| 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:14:53.000000 | - |
+----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+
4) delete the neutron agent
(env) vance@zs95k5:~$ openstack network agent list
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
| 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent |
| 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent |
| 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent |
| 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent |
| a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent |
| aaee3bf0-f8bd-41b7-94ed-f9213f120016 | Open vSwitch agent | zs93k23 | None | True | UP | neutron-openvswitch-agent |
| c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent |
| cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent |
| f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
(env) vance@zs95k5:~$ openstack network agent delete aaee3bf0-f8bd-41b7-94ed-f9213f120016
(env) vance@zs95k5:~$ openstack network agent list
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
| 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent |
| 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent |
| 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent |
| 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent |
| a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent |
| c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent |
| cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent |
| f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+
5) check hypervisor list
(env) vance@zs95k5:~$ nova hypervisor-list
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.ComputeHostNotFound'> (HTTP 500) (Request-ID: req-2b773323-be4f-4c8c-be05-a41144ebed76)
trying to remove compute node properly
Steps to reproduce -----+- ------- ------- +------ ------- ------- -+ -----+- ------- ------- +------ ------- ------- -+ -----+- ------- ------- +------ ------- ------- -+
==================
1) remove all instances from the hypervisor:
(env) vance@zs95k5:~$ nova hypervisor-servers zs93k23
+----+-
| ID | Name | Hypervisor ID | Hypervisor Hostname |
+----+-
+----+-
2) disable the hypervisor: ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+ ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+ 30T21:13: 34.000000 | - | 30T21:13: 27.000000 | - | 30T21:13: 30.000000 | - | 30T21:13: 28.000000 | - | 30T21:13: 33.000000 | - | 30T21:13: 33.000000 | - | ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+ ---+--- ------- ----+-- ------- -+ ---+--- ------- ----+-- ------- -+ ---+--- ------- ----+-- ------- -+
(env) vance@zs95k5:~$ nova service-list
+----+-
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+-
| 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-
| 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-
| 17 | nova-compute | zs93k23 | nova | enabled | up | 2016-11-
+----+-
(env) vance@zs95k5:~$ nova service-disable zs93k23 nova-compute
+------
| Host | Binary | Status |
+------
| zs93k23 | nova-compute | disabled |
+------
3) delete the compute service ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+ ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+ 30T21:14: 54.000000 | - | 30T21:14: 47.000000 | - | 30T21:14: 56.000000 | - | 30T21:14: 48.000000 | - | 30T21:14: 53.000000 | - | ------- ------- -+----- ------- ------- --+---- ------+ ------- ---+--- ----+-- ------- ------- ------- -----+- ------- ------- --+
(env) vance@zs95k5:~$ nova service-delete 17
(env) vance@zs95k5:~$ nova service-list
+----+-
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+-
| 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-
| 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-
| 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-
+----+-
4) delete the neutron agent ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+ ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+ 3dbe-4e87- a9a5-d4b569d311 3d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron- openvswitch- agent | 0e62-4198- 96d9-dfe732d787 4d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | 69cb-4a7f- 8ae4-541d3b2bdd 3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | e459-46c8- 945c-eea4197c01 ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron- openvswitch- agent | 1fb4-436a- becd-fccc737518 fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron- metering- agent | f8bd-41b7- 94ed-f9213f1200 16 | Open vSwitch agent | zs93k23 | None | True | UP | neutron- openvswitch- agent | ad20-4258- a926-bc4a90dccc 96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | 2888-4c3d- 8241-1d3fcd018a 1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron- metadata- agent | f030-43ae- a598-0d01825298 04 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron- openvswitch- agent | ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+ f8bd-41b7- 94ed-f9213f1200 16 ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+ ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+ 3dbe-4e87- a9a5-d4b569d311 3d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron- openvswitch- agent | 0e62-4198- 96d9-dfe732d787 4d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | 69cb-4a7f- 8ae4-541d3b2bdd 3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | e459-46c8- 945c-eea4197c01 ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron- openvswitch- agent | 1fb4-436a- becd-fccc737518 fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron- metering- agent | ad20-4258- a926-bc4a90dccc 96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | 2888-4c3d- 8241-1d3fcd018a 1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron- metadata- agent | f030-43ae- a598-0d01825298 04 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron- openvswitch- agent | ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ----+-- ------- ------- ---+--- ----+-- -----+- ------- ------- ------- -----+
(env) vance@zs95k5:~$ openstack network agent list
+------
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+------
| 039e4b7a-
| 2aa13570-
| 2ab2320a-
| 48d6d83b-
| a36eecd5-
| aaee3bf0-
| c6039c81-
| cfecc66c-
| f60cbf28-
+------
(env) vance@zs95k5:~$ openstack network agent delete aaee3bf0-
(env) vance@zs95k5:~$ openstack network agent list
+------
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+------
| 039e4b7a-
| 2aa13570-
| 2ab2320a-
| 48d6d83b-
| a36eecd5-
| c6039c81-
| cfecc66c-
| f60cbf28-
+------
5) check hypervisor list bugs.launchpad. net/nova/ and attach the Nova API log if possible. .ComputeHostNot Found'> (HTTP 500) (Request-ID: req-2b773323- be4f-4c8c- be05-a41144ebed 76)
(env) vance@zs95k5:~$ nova hypervisor-list
ERROR (ClientException): Unexpected API Error. Please report this at http://
<class 'nova.exception
Okay, here is the error from the log: paste.ubuntu. com/23559892/
http://
Expected result
===============
Nova compute node is properly removed from the OS database.
Environment
===========
1. OpenStack nova version 2:13.1.2-0ubuntu2
2. Hypervisor:
Libvirt+KVM (KVM for IBM z Systems 1.1.3-beta4.3)
3. Networking type: OVS+GRE