Comment 2 for bug 1670055

Revision history for this message
MarginHu (margin2017) wrote :

I find a clue from compute node "kode5".

Line 1786: 2017-03-04 00:19:07.075 6 DEBUG oslo_concurrency.lockutils [req-cc95fa42-5db9-43d8-856f-6854ac45c9fd - - - - -] Lock "/var/lib/nova/instances/50837923-6cc8-4aa7-918e-6ce0c121a390/disk.info" released by "nova.virt.libvirt.imagebackend.write_to_disk_info_file" :: held 0.001s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
 Line 1790: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [req-cc95fa42-5db9-43d8-856f-6854ac45c9fd - - - - -] [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] Instance failed to spawn
 Line 1791: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] Traceback (most recent call last):
 Line 1792: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources
 Line 1793: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] yield resources
 Line 1794: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1930, in _build_and_run_instance
 Line 1795: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] block_device_info=block_device_info)
 Line 1796: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2676, in spawn
 Line 1797: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] block_device_info=block_device_info)
 Line 1798: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3081, in _create_image
 Line 1799: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] fallback_from_host)
 Line 1800: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3197, in _create_and_inject_local_root
 Line 1801: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] instance, size, fallback_from_host)
 Line 1802: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6758, in _try_fetch_image_cache
 Line 1803: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] size=size)
 Line 1804: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 227, in cache
 Line 1805: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] *args, **kwargs)
 Line 1806: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 569, in create_image
 Line 1807: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] prepare_template(target=base, *args, **kwargs)
 Line 1808: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
 Line 1809: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] return f(*args, **kwargs)
 Line 1810: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 223, in fetch_func_sync
 Line 1811: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] fetch_func(target=target, *args, **kwargs)
 Line 1812: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 458, in fetch_image
 Line 1813: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] images.fetch_to_raw(context, image_id, target)
 Line 1814: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 127, in fetch_to_raw
 Line 1815: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] fetch(context, image_href, path_tmp)
 Line 1816: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 118, in fetch
 Line 1817: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] IMAGE_API.download(context, image_href, dest_path=path)
 Line 1818: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/image/api.py", line 184, in download
 Line 1819: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] dst_path=dest_path)
 Line 1820: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 602, in download
 Line 1821: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] {'path': dst_path, 'exception': ex})
 Line 1822: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
 Line 1823: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] self.force_reraise()
 Line 1824: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
 Line 1825: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] six.reraise(self.type_, self.value, self.tb)
 Line 1826: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 586, in download
 Line 1827: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] for chunk in image_chunks:
 Line 1828: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390] TypeError: 'NoneType' object is not iterable
 Line 1829: 2017-03-04 00:19:07.321 6 ERROR nova.compute.manager [instance: 50837923-6cc8-4aa7-918e-6ce0c121a390]