Comment 2 for bug 1956475

Revision history for this message
chandan kumar (chkumar246) wrote :

https://logserver.rdoproject.org/38/37738/8/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/53cd474/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz

```
{3} tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_cold_migration [73.700506s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/usr/lib/python3.9/site-packages/tempest/common/utils/__init__.py", line 70, in wrapper
    return f(*func_args, **func_kwargs)

      File "/usr/lib/python3.9/site-packages/tempest/scenario/test_network_advanced_server_ops.py", line 228, in test_server_connectivity_cold_migration
    waiters.wait_for_server_status(self.servers_client, server['id'],

      File "/usr/lib/python3.9/site-packages/tempest/common/waiters.py", line 78, in wait_for_server_status
    raise exceptions.BuildErrorException(server_id=server_id)

    tempest.exceptions.BuildErrorException: Server 608b8c1b-ca90-4654-935f-08ec69ffecb6 failed to build and is in ERROR status

Captured traceback-1:
~~~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/usr/lib/python3.9/site-packages/tempest/common/waiters.py", line 124, in wait_for_server_termination
    raise lib_exc.DeleteErrorException(

    tempest.lib.exceptions.DeleteErrorException: Resource %(resource_id)s failed to delete and is in ERROR status
Details: Server 608b8c1b-ca90-4654-935f-08ec69ffecb6 failed to delete and is in ERROR status

```

By taking a look at compute logs
https://logserver.rdoproject.org/38/37738/8/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/53cd474/logs/overcloud-novacompute-0/var/log/extra/errors.txt.gz

```
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] Traceback (most recent call last):
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 10259, in _error_out_instance_on_exception
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] yield
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 5627, in _resize_instance
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] disk_info = self.driver.migrate_disk_and_power_off(
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 11005, in migrate_disk_and_power_off
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] self._cleanup_remote_migration(dest, inst_base,
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] self.force_reraise()
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] raise self.value
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 10983, in migrate_disk_and_power_off
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] libvirt_utils.copy_image(from_path, img_path, host=dest,
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/utils.py", line 249, in copy_image
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] remote_filesystem_driver.copy_file(src, dest,
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/remotefs.py", line 104, in copy_file
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] self.driver.copy_file(src, dst, on_execute=on_execute,
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/volume/remotefs.py", line 196, in copy_file
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] processutils.execute(
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] File "/usr/lib/python3.9/site-packages/oslo_concurrency/processutils.py", line 438, in execute
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] raise ProcessExecutionError(exit_code=_returncode,
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] Command: scp -r /var/lib/nova/instances/608b8c1b-ca90-4654-935f-08ec69ffecb6_resize/disk 172.17.0.113:/var/lib/nova/instances/608b8c1b-ca90-4654-935f-08ec69ffecb6/disk
2022-01-13 02:46:58.604 ERROR /var/log/containers/nova/nova-compute.log: 2 ERROR nova.compute.manager [instance: 608b8c1b-ca90-4654-935f-08ec69ffecb6] Exit code: 255
```

It might be linked with this bug.