OpenStack Compute (nova)

  1. Bug #1921306
  2. Activity log

Activity log for bug #1921306

Date Who What changed Old value New value Message
2021-03-25 02:26:47 Xinxin Shen bug added bug
2021-03-25 14:44:40 Balazs Gibizer nova: status New Incomplete
2021-03-26 01:36:38 Xinxin Shen description When the virtual machine is in the paused state and continuously live migrated twice, the Instance state changes to ERROR. 2021-03-22 19:34:38.388 6 DEBUG nova.virt.libvirt.guest [- req-None - - - - -] Failed to get job stats: Unable to read from monitor: Connection reset by peer get_job_info /var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py:767 2021-03-22 19:34:38.389 6 WARNING nova.virt.libvirt.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Error monitoring migration: Unable to read from monitor: Connection reset by peer: libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Traceback (most recent call last): 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7793, in _live_migration 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] finish_event, disk_paths) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7593, in _live_migration_monitor 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] info = guest.get_job_info() 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 751, in get_job_info 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] stats = self._domain.jobStats() 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] result = proxy_call(self._autowrap, f, *args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = execute(f, *args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] six.reraise(c, e, tb) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = meth(*args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1433, in jobStats 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] if ret is None: raise libvirtError ('virDomainGetJobStats() failed', dom=self) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] 2021-03-22 19:34:38.394 6 DEBUG nova.virt.libvirt.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live migration monitoring is all done _live_migration /var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:7800 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live migration failed.: libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Traceback (most recent call last): 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 6510, in _do_live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] block_migration, migrate_data) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7279, in live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] migrate_data) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7793, in _live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] finish_event, disk_paths) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7593, in _live_migration_monitor 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] info = guest.get_job_info() 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 751, in get_job_info 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] stats = self._domain.jobStats() 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] result = proxy_call(self._autowrap, f, *args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = execute(f, *args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] six.reraise(c, e, tb) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = meth(*args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1433, in jobStats 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] if ret is None: raise libvirtError ('virDomainGetJobStats() failed', dom=self) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] 2021-03-22 19:34:38.637 6 ERROR nova.inspur.virt.inspur.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live Migration failure: operation failed: domain is not running: libvirtError: operation failed: domain is not running When the virtual machine is in the paused state and continuously live migrated twice, the Instance state changes to ERROR. The version of nova is Rocky。 2021-03-22 19:34:38.388 6 DEBUG nova.virt.libvirt.guest [- req-None - - - - -] Failed to get job stats: Unable to read from monitor: Connection reset by peer get_job_info /var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py:767 2021-03-22 19:34:38.389 6 WARNING nova.virt.libvirt.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Error monitoring migration: Unable to read from monitor: Connection reset by peer: libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Traceback (most recent call last): 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7793, in _live_migration 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] finish_event, disk_paths) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7593, in _live_migration_monitor 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] info = guest.get_job_info() 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 751, in get_job_info 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] stats = self._domain.jobStats() 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] result = proxy_call(self._autowrap, f, *args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = execute(f, *args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] six.reraise(c, e, tb) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = meth(*args, **kwargs) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1433, in jobStats 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] if ret is None: raise libvirtError ('virDomainGetJobStats() failed', dom=self) 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.389 6 ERROR nova.virt.libvirt.driver [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] 2021-03-22 19:34:38.394 6 DEBUG nova.virt.libvirt.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live migration monitoring is all done _live_migration /var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:7800 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live migration failed.: libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Traceback (most recent call last): 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 6510, in _do_live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] block_migration, migrate_data) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7279, in live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] migrate_data) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7793, in _live_migration 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] finish_event, disk_paths) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7593, in _live_migration_monitor 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] info = guest.get_job_info() 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 751, in get_job_info 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] stats = self._domain.jobStats() 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] result = proxy_call(self._autowrap, f, *args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = execute(f, *args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] six.reraise(c, e, tb) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] rv = meth(*args, **kwargs) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1433, in jobStats 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] if ret is None: raise libvirtError ('virDomainGetJobStats() failed', dom=self) 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] libvirtError: Unable to read from monitor: Connection reset by peer 2021-03-22 19:34:38.394 6 ERROR nova.compute.manager [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] 2021-03-22 19:34:38.637 6 ERROR nova.virt.libvirt.driver [- req-None - - - - -] [instance: d1d2af1f-e973-438c-a6b6-b628091d3596] Live Migration failure: operation failed: domain is not running: libvirtError: operation failed: domain is not running
2021-05-25 04:17:23 Launchpad Janitor nova: status Incomplete Expired
2021-06-21 12:38:50 Pierre Thévenet nova: status Expired New
2021-06-22 06:57:26 Pierre Thévenet bug added subscriber Pierre Thévenet
2021-06-22 07:24:07 Sébastien Gremion bug added subscriber Sébastien Gremion
2021-06-22 16:28:57 Artom Lifshitz nova: status New Incomplete
2021-06-23 09:25:50 Pierre Thévenet nova: status Incomplete New
2021-07-09 10:00:46 Lee Yarwood bug watch added https://bugzilla.redhat.com/show_bug.cgi?id=1713009
2021-07-09 10:00:55 Lee Yarwood nova: status New Invalid