[2.1.3] A node enlistment fails to contact metadata service
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Invalid
|
High
|
Mike Pontillo | ||
2.1 |
Fix Released
|
High
|
Mike Pontillo |
Bug Description
This one probably related to https:/
A node enlistment fails to contact metadata service
MAAS node is receiving DHCP PXE calls on MAAS subnet 172.16.1.x, fails to connect to metadata service and finishes boot process, but never shows up in MAAS Device discovery list.
The server:
Ubuntu 16.04.2 LTS
Linux cs-srv-233 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Network:
1: lo: <LOOPBACK,
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp3s0f0: <BROADCAST,
link/ether d4:85:64:51:16:c8 brd ff:ff:ff:ff:ff:ff
inet 120.263.220.233/24 brd 120.263.220.255 scope global enp3s0f0
valid_lft forever preferred_lft forever
inet6 fe80::d685:
valid_lft forever preferred_lft forever
3: enp3s0f1: <BROADCAST,
link/ether d4:85:64:51:16:ca brd ff:ff:ff:ff:ff:ff
inet 172.16.1.100/24 brd 172.16.1.255 scope global enp3s0f1
valid_lft forever preferred_lft forever
inet6 fe80::d685:
valid_lft forever preferred_lft forever
Network configuration:
auto lo
iface lo inet loopback
# The primary network interface
auto enp3s0f0
iface enp3s0f0 inet static
address 120.263.220.233
netmask 255.255.255.0
network 120.263.220.0
broadcast 120.263.220.255
gateway 120.263.220.254
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 172.16.1.100 120.263.55.2 120.263.5.3 8.8.8.8
dns-search cs.du.edu
auto enp3s0f1
iface enp3s0f1 inet static
address 172.16.1.100
netmask 255.255.255.0
network 172.16.1.0
broadcast 172.16.1.255
Region and Rack controllers:
dpkg -s maas-region-
Package: maas-region-
Status: install ok installed
Priority: optional
Section: net
Installed-Size: 45
Maintainer: Ubuntu Developers <email address hidden>
Architecture: all
Source: maas
Version: 2.1.3+bzr5573-
Depends: avahi-utils, dbconfig-pgsql, iputils-ping, maas-dns (= 2.1.3+bzr5573-
Recommends: openssh-server
Suggests: nmap
Description: Region Controller for MAAS
dpkg -s maas-rack-
Package: maas-rack-
Status: install ok installed
Priority: optional
Section: net
Installed-Size: 96
Maintainer: Ubuntu Developers <email address hidden>
Architecture: all
Source: maas
Version: 2.1.3+bzr5573-
Replaces: maas-cluster-
Depends: authbind, avahi-utils, bind9utils, distro-info, freeipmi-tools, grub-common, iputils-ping, maas-cli (= 2.1.3+bzr5573-
Suggests: amtterm, ipmitool, libvirt-bin, nmap, wsmancli
Breaks: maas-cluster-
Conflicts: tftpd-hpa
Conffiles:
/etc/logrotate
/etc/sudoers.
Description: Rack Controller for MAAS
MAAS packages installed:
maas-cli 2.1.3+bzr5573-
maas-common 2.1.3+bzr5573-
maas-dhcp 2.1.3+bzr5573-
maas-dns 2.1.3+bzr5573-
maas-proxy 2.1.3+bzr5573-
maas-rack-
maas-region-api 2.1.3+bzr5573-
maas-region-
python3-django-maas 2.1.3+bzr5573-
python3-maas-client 2.1.3+bzr5573-
python3-
cat /etc/maas/
cluster_uuid: c2278486-
maas_url: http://
http://
#cloud-config
datasource:
MAAS:
timeout : 50
max_wait : 120
# there are no default values for metadata_url or oauth credentials
# If no credentials are present, non-authed attempts will be made.
metadata_url: http://
output: {all: '| tee -a /var/log/
Patch http://
MAAS region info:
maas-region shell
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveCon
>>> from maasserver.preseed import get_preseed_context
>>> from pprint import pprint
>>> from maasserver.models import RackController
>>> pprint(
[{'metadata_
'osystem': '',
'release': '',
'server_host': '172.16.1.100',
'server_url': 'http://
'syslog_
The error I see on the node PXE boot:
285.792403} cloud-init[1190]: 2-17-02-21 21:07:02, 027 - url_helper.py
Calling ‘http://
request error [HTTPConnection
Max retries exceeded with url: /2009-04-
(‘<requests.
Please, let me know if more information is needed.
Thanks,
Changed in maas: | |
status: | Incomplete → Triaged |
importance: | Undecided → High |
assignee: | nobody → Mike Pontillo (mpontillo) |
milestone: | none → 2.2.0 |
Changed in maas: | |
status: | Triaged → Incomplete |
Changed in maas: | |
status: | Incomplete → Invalid |
Could you please post the kernel parameters sent on the PXE process ? That should tell us what IP address is being used to download the metadata (based from your bug, however, seems to be the correct)
The error here:
285.792403} cloud-init[1190]: 2-17-02-21 21:07:02, 027 - url_helper.py 169.254. 169.254/ 2009-04- 04/meta- data/instance- id' Pool(host= ‘169.254. 169.254’ , port=80)
Calling ‘http://
request error [HTTPConnection
Is because cloud-init was unable to contact MAAS' metadata and it fallsback to the 169 address.