tempest.api.compute.servers.test_device_tagging fails with libvirt-xen

[Anthony PERARD]

On the Xen Project CI, two new tests always fails:

tempest.api.compute.servers.test_device_tagging.DeviceTaggingTest.test_device_tagging
tempest.api.compute.servers.test_device_tagging.DeviceTaggingTestV2_42.test_device_tagging

It appear that the test is trying to ssh to the instance while the instance itself is powering off.

from tempest, Captured pythonlogging:
    2017-05-31 07:41:22,451 28296 INFO [tempest.api.compute.servers.test_device_tagging] Attempting to verify tagged devices in server d7004fb9-c241-4976-befb-85af3771778f via the metadata service: http://169.254.169.254/openstack/latest/meta_data.json
    2017-05-31 07:41:22,452 28296 DEBUG [tempest.lib.common.utils.linux.remote_client] Remote command: set -eu -o pipefail; PATH=$PATH:/sbin; curl -V
    2017-05-31 07:41:22,453 28296 INFO [tempest.lib.common.ssh] Creating ssh connection to '172.24.5.4:22' as 'cirros' with public key authentication
    2017-05-31 07:41:25,451 28296 WARNING [tempest.lib.common.ssh] Failed to establish authenticated ssh connection to cirros@172.24.5.4 ([Errno None] Unable to connect to port 22 on 172.24.5.4). Number attempts: 1. Retry after 2 seconds.

From screen-n-cpu.txt.gz:
May 31 07:41:30.385202 ds-xen-rax-iad-10748 nova-compute[21012]: DEBUG nova.virt.driver [-] Emitting event <LifecycleEvent: 1496216475.24, d7004fb9-c241-4976-befb-85af3771778f => Stopped> {{(pid=21012) emit_event /opt/stack/new/nova/nova/virt/driver.py:1446}}
May 31 07:41:30.385567 ds-xen-rax-iad-10748 nova-compute[21012]: INFO nova.compute.manager [-] [instance: d7004fb9-c241-4976-befb-85af3771778f] VM Stopped (Lifecycle Event)

More logs can be found at:
http://logs.openstack.xenproject.org/80/394480/35/check/dsvm-tempest-xen/ef343be/

[Matt Riedemann (mriedem)]

These are only run when ssh validation is enabled, which was made the default in devstack last week:

https://github.com/openstack-dev/devstack/commit/6d302a83242e2e83ca911da9c2c3cd7cb93b789c

That's likely why it's on in the xenproject CI now. You could disable it explicitly until the bug is fixed. I'm not sure why this would cause issues with libvirt+xen but not libvirt+qemu though.

[Anthony PERARD (anthony-perard)]

tempest.api.compute.servers.test_device_tagging is the only test that fails whith ssh validation enabled, the other tests works fine. We may need to find out what the guest is doing, it maybe could not start properly.

As suggested, I've disable ssh validation on the CI for now.

Thanks.