[fuel 10.0 newton] Deployment fails because vrouter ports were not created

[Alexander]

We are trying to deploy openstack on 3 nodes: 1 controller+cinder, 2 compute. Network verification is successful, but the deployment fails with error:
Failed tasks: Task[ntp-check/4] Stopping the deployment process!

After investigating we found out that the reason ntp server is not available is that vrouter ports has not been created:
$ip netns exec vrouter ifconfig
conntrd Link encap:Ethernet HWaddr f6:7e:f2:95:34:44
          inet addr:240.1.0.4 Bcast:0.0.0.0 Mask:255.255.255.0
          inet6 addr: fe80::f47e:f2ff:fe95:3444/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:1638 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:71707 (71.7 KB) TX bytes:2064 (2.0 KB)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

vr-host-ns Link encap:Ethernet HWaddr aa:3e:10:ce:20:10
          inet addr:240.0.0.6 Bcast:0.0.0.0 Mask:255.255.255.252
          inet6 addr: fe80::a83e:10ff:fece:2010/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:648 (648.0 B) TX bytes:648 (648.0 B)

Attaching diagnostic snapshot for controller

fuel version: 10.0

[Oleksiy Molchanov (omolchanov)]

It seems you do not have access to internet or they just were down

2017-01-18 10:54:51 +0000 Puppet (err): ERROR: Unable to communicate with at least one of NTP server, checked the following host(s): ["0.fuel.pool.ntp.org", "1.fuel.pool.ntp.org", "2.fuel.pool.ntp.org"] on node node-7.domain.tld

[Alexander (alexanderermilov)]

there is no internet access because nameserver in /etc/resolv.conf is set to 192.168.0.1 which is the IP address of the nonexistant vrouter's port. We noticed that this port exists during deploy and internet access is working fine, but at some point before ntp_check.pp the port disappears from vrouter (we couldn't find why) and the check fails. As you can see from the output of the ip netns exec vrouter ifconfig command there is no port with IP 192.168.0.1 in vrouter

[Oleksiy Molchanov (omolchanov)]

Please attach full diagnostic snapshot then

[Alexander (alexanderermilov)]

Full diagnostic snapshot in attachment.the vrouter port b_vrouter disappears after puppet manifest virtual_ips/conntrackd.pp. the same way command crm_mon

Stack: corosync
Current DC: my-node (version 1.1.14-70404b0) - partition with quorum
1 node and 16 resources configured

Online: [my-node ]

[Alexander (alexanderermilov)]

Oleksiy good day. what did you learn about this error?

[Oleksiy Molchanov (omolchanov)]

Alexander, sorry I missed this issue, because it is marked as Incomplete.

I have checked logs and the problem is related to pacemaker, seems it just died.

[Oleksiy Molchanov (omolchanov)]

Alexander, can you just try once more?

[Alexander (alexanderermilov)]

I tried a few times with the same result. I can try again if the fuel image has been updated after I filed this bug.
I downloaded the image from fuel-infra.org. Is there a newer version of newton fuel?

[Oleksiy Molchanov (omolchanov)]

Please download latest one

https://www.fuel-infra.org/#fuelget