Deployment failed for environment with nodes already provisioned

[Julia Aranovich]

Can be reproduced on Fuel UI on custom #567 ISO.

Steps to reproduce:
1. create default env, add a controller node.
2. Start provisioning only for the environment (provisioning button can be found in Deploy Changes button dropdown) (PUT /api/clusters/x/provision request).
3. Wait for provisioning successfully finished.
4. Start deployment for the environment (PUT /api/clusters/x/changes request)

Expected result: deployment completed successfully.

Actual result: deployment failed with an error "Deployment has failed. Method granular_deploy. Deployment failed on nodes 1.
Inspect Astute logs for the details"

[Kate Pimenova (kpimenova)]

Snapshot is attached

[Alexander Kislitsky (akislitsky)]

Deployment was failed due to failed connectivity check:

2015-11-12T09:48:06.195238+00:00 err: ERROR: Unable to fetch url 'http://archive.ubuntu.com/ubuntu/', error 'execution expired'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid. on node node-1.domain.tld
2015-11-12T09:48:06.195754+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:95:in `rescue in fetch'
2015-11-12T09:48:06.196862+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:83:in `fetch'
2015-11-12T09:48:06.196862+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:106:in `block (4 levels) in <top (required)>'
2015-11-12T09:48:06.197397+00:00 err: ERROR: Unable to fetch url 'http://archive.ubuntu.com/ubuntu/', error 'execution expired'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid. on node node-1.domain.tld
2015-11-12T09:48:06.198369+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:95:in `rescue in fetch'
2015-11-12T09:48:06.198369+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:83:in `fetch'
2015-11-12T09:48:06.198369+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:106:in `block (4 levels) in <top (required)>'
2015-11-12T09:48:06.199288+00:00 err: Could not run: ERROR: Unable to fetch url 'http://archive.ubuntu.com/ubuntu/', error 'execution expired'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid.
2015-11-12T09:48:06.199798+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:95:in `rescue in fetch'
2015-11-12T09:48:06.200900+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:83:in `fetch'
2015-11-12T09:48:06.200900+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:106:in `block (4 levels) in <top (required)>'

root@node-1:~# ping archive.ubuntu.com
PING archive.ubuntu.com (91.189.91.15) 56(84) bytes of data.

root@node-1:~# ifconfig [58/617]
br-ex Link encap:Ethernet HWaddr 08:00:27:10:89:c6
          inet addr:172.16.0.4 Bcast:172.16.0.255 Mask:255.255.255.0
          inet6 addr: fe80::5834:d7ff:fe85:352d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:451 errors:0 dropped:0 overruns:0 frame:0
          TX packets:230 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:109004 (109.0 KB) TX bytes:18700 (18.7 KB)

br-floating Link encap:Ethernet HWaddr e2:1a:74:b5:42:4c
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:454 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueue...

[Ihor Kalnytskyi (ikalnytskyi)]

Julia, are you sure you have properly configures networks? It looks like you simply didn't have access to Internet through br-ex (though it's configures, which is fine). Did you use VBox scripts?

[Julia Aranovich (jkirnosova)]

Igor, seems yes, I just run launch.sh script with minimal configuration (4GB) that uses vBox

[Matthew Mosesohn (raytrac3r)]

This is related to a local env configuration. The gateway on the public interface isn't forwarding traffic correctly.