Fuel for OpenStack

Fuel Web does not start, if change subnet

Bug #1285573 reported by Vadim Rovachev on 2014-02-27

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Fuel for OpenStack	Invalid	Medium	Matthew Mosesohn	Fuel for OpenStack 4.1.1

Bug Description

{"build_id": "2014-02-26_17-35-21", "mirantis": "yes", "build_number": "214", "nailgun_sha": "ea08cef3e06a72f47cfaa8cd8fe6d034e2cf722e", "ostf_sha": "8e6681b6d06c7cb20a84c1cc740d5f2492fb9d85", "fuelmain_sha": "6e6f6b31b8880b7600a1af5cc94d48ac01d60a07", "astute_sha": "10cccc87f2ee35510e43c8fa19d2bf916ca1fced", "release": "4.1", "fuellib_sha": "0a2e5bdc01c1e3bb285acb7b39125101e950ac72"}

Steps to reproduce:
1. Start KVM with fuel ISO.
2. Await Fuel Setup.
http://picpaste.com/01-wZSNZlzD.png
Change IP address -> 10.20.1.2 and GW -> 10.20.1.1
click < Apply >
Await 20 sec
http://picpaste.com/02-7VNCh3B6.png
3. Go to "Quit Setup"
Click "Sava and Quit"
4. Await end of install
http://picpaste.com/04-yfgeFLSV.png

5.
Open WebUI(10.20.1.2:8000) in browser
Expected result: openstak WebUI
Actual result: 502 Bad Gateway

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

curl http://10.20.1.2:8000/api/version

Expected:
{"build_id": "2014-02-26_17-35-21", "mirantis": "yes", "build_number": "214", "nailgun_sha": "ea08cef3e06a72f47cfaa8cd8fe6d034e2cf722e", "ostf_sha": "8e6681b6d06c7cb20a84c1cc740d5f2492fb9d85", "fuelmain_sha": "6e6f6b31b8880b7600a1af5cc94d48ac01d60a07", "astute_sha": "10cccc87f2ee35510e43c8fa19d2bf916ca1fced", "release": "4.1", "fuellib_sha": "0a2e5bdc01c1e3bb285acb7b39125101e950ac72"}

Actual:
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.0.15</center>
</body>
</html>

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@fuel ~]# grep -r 'error' /var/log/
/var/log/puppet/bootstrap_admin_node.log:notice: /Stage[main]/Nailgun::Cobbler/File[/var/www/cobbler/aux/20-puppet_error_syslog.conf]/ensure: defined content as '{md5}e2436ac978b00d6f8b4b278e7bb53771'
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.enable-disable-network: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.sleep-wake: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.enable-disable-wifi: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.enable-disable-wwan: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.use-user-connections: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.NetworkManager.network-control: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.network-manager-settings.system.wifi.share.open: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.syslog:08:24:08,667 WARNING NetworkManager: <warn> error requesting auth for org.freedesktop.network-manager-settings.system.wifi.share.protected: (26) Remote Exception invoking org.freedesktop.PolicyKit1.Authority.CheckAuthorization() on /org/freedesktop/PolicyKit1/Authority at name org.freedesktop.PolicyKit1: org.freedesktop.DBus.Error.Spawn.ExecFailed: Cannot launch daemon, file not found or permissions invalid
/var/log/anaconda.yum.log:[2014-02-27 08:26:30,787] DEBUG : TSINFO: Marking libgpg-error-1.7-4.el6.x86_64 as install for gpgme-1.1.8-3.el6.x86_64
/var/log/fuelmenu.log:You may see errors during provisioning and in system logs. NTP errors are not fatal.
/var/log/fuelmenu.log:2014-02-27 08:50:20,711 INFO No errors found
/var/log/mcollective.log:E, [2014-02-27T09:04:49.259475 #2239] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@localhost:61613: es_recv: connection.receive returning EOF as nil - resetting connection.
/var/log/nginx/error_repo.log:2014/02/27 09:14:25 [error] 7597#0: *12 open() "/var/www/nailgun/favicon.ico" failed (2: No such file or directory), client: 10.20.1.200, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.20.1.2:8080"
/var/log/nginx/error_nailgun.log:2014/02/27 09:10:01 [error] 7597#0: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:11:04 [error] 7597#0: *4 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:12:35 [error] 7597#0: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:13:22 [error] 7597#0: *8 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:14:09 [error] 7597#0: *10 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:14:29 [error] 7597#0: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.200, server: localhost, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8001/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:15:10 [error] 7597#0: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:16:20 [error] 7597#0: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:17:31 [error] 7597#0: *25 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:18:28 [error] 7597#0: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:19:22 [error] 7597#0: *29 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:20:04 [error] 7597#0: *31 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:21:09 [error] 7597#0: *33 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:22:24 [error] 7597#0: *35 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:23:04 [error] 7597#0: *37 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:24:22 [error] 7597#0: *39 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:25:25 [error] 7597#0: *41 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:26:31 [error] 7597#0: *43 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:27:15 [error] 7597#0: *45 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:28:12 [error] 7597#0: *47 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:29:16 [error] 7597#0: *49 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:30:15 [error] 7597#0: *51 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:31:03 [error] 7597#0: *53 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:32:29 [error] 7597#0: *55 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:33:20 [error] 7597#0: *57 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:34:04 [error] 7597#0: *59 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:35:06 [error] 7597#0: *61 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:36:28 [error] 7597#0: *63 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:37:29 [error] 7597#0: *67 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:38:16 [error] 7597#0: *69 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:38:55 [error] 7597#0: *71 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.200, server: localhost, request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:8001/api/version", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:39:08 [error] 7597#0: *73 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:40:12 [error] 7597#0: *75 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:41:23 [error] 7597#0: *77 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/nginx/error_nailgun.log:2014/02/27 09:42:15 [error] 7597#0: *79 connect() failed (111: Connection refused) while connecting to upstream, client: 10.20.1.5, server: localhost, request: "POST /api/nodes/ HTTP/1.1", upstream: "http://127.0.0.1:8001/api/nodes/", host: "10.20.1.2:8000"
/var/log/rabbitmq/startup_log:starting error log relay ...done
/<email address hidden>:STOMP error frame sent:

See original description

Vadim Rovachev (vrovachev) on 2014-02-27

description:

updated

Matthew Mosesohn (raytrac3r) on 2014-02-27

Changed in fuel:
assignee:	nobody → Matthew Mosesohn (raytrac3r)

Revision history for this message

Matthew Mosesohn (raytrac3r) wrote on 2014-02-27:

Vadim, can you post your /etc/astute.yaml on Fuel master for me? The only thing I would suspect is if you change the master IP address, but skip viewing the PXE Setup pane, it might write the wrong info in /etc/astute.yaml.

Matthew Mosesohn (raytrac3r) on 2014-02-27

Changed in fuel:
milestone:	none → 5.0
milestone:	5.0 → 4.1.1

Revision history for this message

Vadim Rovachev (vrovachev) wrote on 2014-02-27:

/etc/astute.yaml

HOSTNAME: fuel
DNS_DOMAIN: domain.tld
DNS_SEARCH: domain.tld
DNS_UPSTREAM: 8.8.8.8
NTP1: 0.pool.ntp.org
NTP2: 1.pool.ntp.org
NTP3: 2.pool.ntp.org
ADMIN_NETWORK:
  interface: eth0
  ipaddress: 10.20.1.2
  netmask: 255.255.255.0
  cidr: 10.20.0.0/24
  size: '256'
  static_pool_start: 10.20.1.4
  static_pool_end: 10.20.1.128
  dhcp_pool_start: 10.20.1.129
  dhcp_pool_end: 10.20.1.254
TEST_DNS: www.google.com

.
In
{"build_id": "2014-02-27_19-37-46", "mirantis": "yes", "build_number": "224", "nailgun_sha": "12a7e7a99557f2bc302f0806ad3beef02e94b974", "ostf_sha": "8e6681b6d06c7cb20a84c1cc740d5f2492fb9d85", "fuelmain_sha": "ba019bf15a9597a154e7c1d6ecc840614d21414c", "astute_sha": "10cccc87f2ee35510e43c8fa19d2bf916ca1fced", "release": "4.1", "fuellib_sha": "61d3a150402da3ce1160836c8d659f6d9d1f9640"}
bug is not reproduced

Evgeniy L (rustyrobot) on 2014-02-27

Changed in fuel:
status:	New → Incomplete
importance:	Undecided → Medium

Revision history for this message

Matthew Mosesohn (raytrac3r) wrote on 2014-03-06:

I couldn't reproduce this on 4.1 RC3 ISO using the same steps as reporter. When changing IP and gateway to a different network and then finishing deployment, I can still access Fuel UI with no issues.

Vadim Rovachev (vrovachev) on 2014-04-03

Changed in fuel:
status:	Incomplete → Invalid

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.