'check_instance_connectivity' is called too early after creating an instance
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Released
|
High
|
Dmitry Tyzhnenko |
Bug Description
Reproduced on CI: https:/
In the system test 'neutron_
...
- Create an instance with a key pair
- Manually reschedule router from primary controller to another one
- Check network connectivity from instance via dhcp namespace
...
As the result, the test starts to check for connectivity from instance to 8.8.8.8 before the instance started it's network services.
In check_instance_
wait(lambda: remote.
### From https:/
http://
When ssh started on the instance, neutron l3 router had been already migrated, and the first ping from instance to 8.8.8.8 failed:
AssertionError: Instance has no connectivity, exit code 1,stdout ['PING 8.8.8.8 (8.8.8.8): 56 data bytes\n', '\n', '--- 8.8.8.8 ping statistics ---\n', '1 packets transmitted, 0 packets received, 100% packet loss\n'], stderr []
But the next pings started manually are working fine:
root@node-2:~# ip netns exec qrouter-
root@node-2:~# ssh cirros@
The authenticity of host '192.168.111.4 (192.168.111.4)' can't be established.
RSA key fingerprint is e7:6d:36:
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.111.4' (RSA) to the list of known hosts.
cirros@
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=55 time=4.754 ms
64 bytes from 8.8.8.8: seq=1 ttl=55 time=4.520 ms
64 bytes from 8.8.8.8: seq=2 ttl=55 time=5.508 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 4.520/4.927/5.508 ms
=======
In the system test, we should perform more than just a single ping.
The test can be extended to not just ping something outside the cluster, but something like the following:
- establish a TCP/IP connection to some external resource;
- check that the connection alive (perform some request, do not close the connection)
- migrate the neutron l3 router;
- check that the connection alive (perform some request using the established connection)
TCP/IP connections should be alive after migrating, even if some re-tries will be performed by TCP stack in cases like described above.
Also this approach could be used instead of 'downloading a file' in appropriate tests to make sure that the connection remains alive.
Changed in fuel: | |
status: | New → Confirmed |
tags: | added: area-qa |
Changed in fuel: | |
status: | Fix Committed → Fix Released |
Fix proposed to branch: master /review. openstack. org/232561
Review: https:/