Activity log for bug #1781880

Date Who What changed Old value New value Message
2018-07-16 09:06:42 s10 bug added bug
2018-07-16 09:09:29 s10 description Affected version: Pike+ Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 How to reproduce: 1. Set servicegroup_driver=mc in and enable_new_services=False in nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list: Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> Description =========== nova service-list for new compute service causes TypeError. Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 Steps to reproduce ================== 1. Set servicegroup_driver=mc in and enable_new_services=False in /etc/nova/nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list. Expected result =============== List of the nova services Actual result ============= ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-cbd9f4ca-7634-4378-8dcc-5a03d9d4193f) Environment =========== OpenStack Pike Logs ============== Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'>
2018-07-16 09:09:48 s10 description Description =========== nova service-list for new compute service causes TypeError. Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 Steps to reproduce ================== 1. Set servicegroup_driver=mc in and enable_new_services=False in /etc/nova/nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list. Expected result =============== List of the nova services Actual result ============= ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-cbd9f4ca-7634-4378-8dcc-5a03d9d4193f) Environment =========== OpenStack Pike Logs ============== Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> Description =========== nova service-list for new compute service causes TypeError. Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 Steps to reproduce ================== 1. Set servicegroup_driver=mc in and enable_new_services=False in /etc/nova/nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list. Expected result =============== List of the nova services Actual result ============= ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-cbd9f4ca-7634-4378-8dcc-5a03d9d4193f) Environment =========== OpenStack Pike Logs ============== Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'>
2018-07-16 09:19:22 Pavel Abalikhin nova: status New Confirmed
2018-07-17 16:44:06 s10 description Description =========== nova service-list for new compute service causes TypeError. Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 Steps to reproduce ================== 1. Set servicegroup_driver=mc in and enable_new_services=False in /etc/nova/nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list. Expected result =============== List of the nova services Actual result ============= ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-cbd9f4ca-7634-4378-8dcc-5a03d9d4193f) Environment =========== OpenStack Pike Logs ============== Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> Description =========== nova service-list for new compute service causes TypeError. Related commit: https://git.openstack.org/cgit/openstack/nova/commit/?id=0df91a7f799060cd2e9b8a0adac1efacb974bcb3 Steps to reproduce ================== 1. Set servicegroup_driver=mc and enable_new_services=False in /etc/nova/nova.conf 2. Add new compute host, start nova-compute service. Service becomes disabled, and field updated_at is NULL in nova.services table in DB. 3. Execute nova service-list. Expected result =============== List of the nova services Actual result ============= ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-cbd9f4ca-7634-4378-8dcc-5a03d9d4193f) Environment =========== OpenStack Pike Logs ============== Trace in the nova-api-os-compute logs: ERROR nova.api.openstack.extensions [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] Unexpected exception in API method: TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions Traceback (most recent call last): TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 336, in wrapped TRACE nova.api.openstack.extensions return f(*args, **kwargs) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 261, in index TRACE nova.api.openstack.extensions _services = self._get_services_list(req, ['forced_down']) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 115, in _get_services_list TRACE nova.api.openstack.extensions for svc in _services] TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/services.py", line 83, in _get_service_detail TRACE nova.api.openstack.extensions updated_time = self.servicegroup_api.get_updated_time(svc) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/api.py", line 84, in get_updated_time TRACE nova.api.openstack.extensions return self._driver.updated_time(member) TRACE nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/mc.py", line 81, in updated_time TRACE nova.api.openstack.extensions if updated_time_in_db <= updated_time_in_mc: TRACE nova.api.openstack.extensions TypeError: can't compare datetime.datetime to NoneType TRACE nova.api.openstack.extensions 2018-07-16 11:40:16,437.437 2096 INFO nova.api.openstack.wsgi [req-6034517c-bc29-4ebe-931e-8726fd934bee e05fb82b34cd4265a839f2482debb973 b5d263d6d7c24b84b335f4f00ae6d7c9] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'>
2018-07-17 16:48:46 Matt Riedemann nova: importance Undecided Medium
2018-07-17 16:48:49 Matt Riedemann nova: status Confirmed Triaged
2018-07-17 16:48:58 Matt Riedemann tags memcache servicegroup
2018-07-17 16:51:57 Matt Riedemann nominated for series nova/queens
2018-07-17 16:51:57 Matt Riedemann bug task added nova/queens
2018-07-17 16:51:57 Matt Riedemann nominated for series nova/pike
2018-07-17 16:51:57 Matt Riedemann bug task added nova/pike
2018-07-17 16:52:06 Matt Riedemann nova/pike: status New Confirmed
2018-07-17 16:52:08 Matt Riedemann nova/queens: status New Confirmed
2018-07-17 16:55:18 Matt Riedemann nova/pike: importance Undecided Medium
2018-07-17 16:55:20 Matt Riedemann nova/queens: importance Undecided Medium
2018-07-19 01:16:48 Matt Riedemann tags memcache servicegroup low-hanging-fruit memcache servicegroup
2018-07-19 01:17:07 Zhenyu Zheng nova: assignee Zhenyu Zheng (zhengzhenyu)
2018-07-19 11:50:51 OpenStack Infra nova: status Triaged In Progress
2018-08-16 00:51:21 OpenStack Infra nova: status In Progress Fix Released
2018-08-16 00:54:03 Matt Riedemann nominated for series nova/rocky
2018-08-16 00:54:03 Matt Riedemann bug task added nova/rocky
2018-08-16 00:54:11 Matt Riedemann nova/rocky: importance Undecided Medium
2018-08-16 00:54:13 Matt Riedemann nova/rocky: status New Confirmed
2018-08-16 08:01:41 OpenStack Infra nova/rocky: status Confirmed In Progress
2018-08-16 08:01:41 OpenStack Infra nova/rocky: assignee Zhenyu Zheng (zhengzhenyu)
2018-08-16 08:01:59 OpenStack Infra nova/queens: status Confirmed In Progress
2018-08-16 08:01:59 OpenStack Infra nova/queens: assignee Zhenyu Zheng (zhengzhenyu)
2018-08-16 10:14:40 s10 nova/pike: assignee s10 (vlad-esten)
2018-08-16 10:14:48 OpenStack Infra nova/pike: status Confirmed In Progress
2018-09-16 19:13:12 OpenStack Infra nova/queens: assignee Zhenyu Zheng (zhengzhenyu) s10 (vlad-esten)
2018-09-22 19:57:10 OpenStack Infra nova/rocky: status In Progress Fix Committed
2018-09-25 08:55:43 OpenStack Infra nova/queens: status In Progress Fix Committed
2018-10-04 22:05:51 OpenStack Infra nova/pike: status In Progress Fix Committed