Detailed bug description:
A customer has added a new compute node to the environment. They noticed, that live-migration works FROM that compute node, but not TO it. The issue appeared to be connected to the new node not being in /etc/hosts of other nodes:
root@mosp-10133:~# grep mosp-10133 /etc/hosts
10.64.253.17 mosp-10133.srv.tgos.de mosp-10133
10.64.253.17 messaging-mosp-10133.srv.tgos.de messaging-mosp-10133
root@mosp-10133:~# cat /etc/hosts
# HEADER: This file was autogenerated at 2017-02-10 10:28:45 +0000
root@mosp-10152:~# grep mosp-10133 /etc/hosts
root@mosp-10152:~#
root@mosp-10152:/etc/libvirt# cat /etc/hosts
# HEADER: This file was autogenerated at 2017-01-30 09:52:06 +0000
However, according to the logs, the hostname was used in attempts to connect for migration:
<179>Feb 10 11:26:56 mosp-10152 nova-compute: 2017-02-10 11:26:56.674 15781 ERROR nova.virt.libvirt.driver [req-2bf88e9f-391d-4428-9661-e298c53f2207 186f6d384e58469794c8bd398638da77 23e1af30aba74019a7c044bf140da3e2 - - -] [instance: ec781752-a63e-4422-9a6f-2d48c87c8b91] Live Migration failure: operation failed: Failed to connect to remote libvirt URI qemu+tcp://mosp-10133.srv.tgos.de/system: unable to connect to server at 'mosp-10133.srv.tgos.de:16509': Connection timed out
Steps to reproduce:
Deploy an environment, add a new compute node, check /etc/hosts on nodes.
Expected results:
/etc/hosts should be updated and have the new hostname listed on all nodes as needed.
Actual result:
/etc/hosts is not updated.
Reproducibility:
-
Workaround:
Manually put the hostname to /etc/hosts on nodes.
Impact:
Live migration to the new compute node does not work.
Description of the environment:
Operation system: Ubuntu 14.04
Versions of components: MOS 9.1
Reference architecture: HA
Network model: Neutron with Provider Network
Related projects installed: -
Additional information:
-
We need astute log from master and puppet log from previously deployed node to debug this.