cloud-init

fails to fetch meta-data from cloudstack

Bug #1311107 reported by Atilla Mas
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
New
Undecided
Unassigned

Bug Description

Downloaded the latest ubuntu 14.04 cloud-image from cloud-images.ubuntu.com with cloud-init 0.7.5.

cloud-init tries to get the meta-data from CloudStack via http://10.1.209.130/latest/meta-data
this url doesn't exist, but the "http://10.1.209.130/latest/meta-data/" does, so changing the code in this file fixes the error:

/usr/lib/python2.7/dist-packages/cloudinit/ec2_utils.py

165 def get_instance_metadata(api_version='latest',
166 metadata_address='http://169.254.169.254',
167 ssl_details=None, timeout=5, retries=5):
168 md_url = url_helper.combine_url(metadata_address, api_version)
169 md_url = url_helper.combine_url(md_url, 'meta-data/')
170 caller = functools.partial(util.read_file_or_url,
171 ssl_details=ssl_details, timeout=timeout,
172 retries=retries)

on line 169, the trailing slash in the meta-data string makes this work.

entries from logfile before changing ec2_utils.py:

2014-04-22 08:24:24,781 - importer.py[DEBUG]: Looking for modules ['ubuntu', 'cloudinit.distros.ubuntu'] that have attributes ['Distro']
2014-04-22 08:24:24,781 - importer.py[DEBUG]: Failed at attempted import of 'ubuntu' due to: No module named ubuntu
2014-04-22 08:24:24,782 - importer.py[DEBUG]: Found ubuntu with attributes ['Distro'] in ['cloudinit.distros.ubuntu']
2014-04-22 08:24:24,782 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
2014-04-22 08:24:24,782 - __init__.py[DEBUG]: Looking for for data source in: ['CloudStack', 'NoCloud'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK']
2014-04-22 08:24:24,782 - importer.py[DEBUG]: Looking for modules ['DataSourceCloudStack', 'cloudinit.sources.DataSourceCloudStack'] that have attributes ['get_datasource_list']
2014-04-22 08:24:24,783 - importer.py[DEBUG]: Failed at attempted import of 'DataSourceCloudStack' due to: No module named DataSourceCloudStack
2014-04-22 08:24:24,783 - importer.py[DEBUG]: Found DataSourceCloudStack with attributes ['get_datasource_list'] in ['cloudinit.sources.DataSourceCloudStack']
2014-04-22 08:24:24,784 - importer.py[DEBUG]: Looking for modules ['DataSourceNoCloud', 'cloudinit.sources.DataSourceNoCloud'] that have attributes ['get_datasource_list']
2014-04-22 08:24:24,784 - importer.py[DEBUG]: Failed at attempted import of 'DataSourceNoCloud' due to: No module named DataSourceNoCloud
2014-04-22 08:24:24,784 - importer.py[DEBUG]: Found DataSourceNoCloud with attributes ['get_datasource_list'] in ['cloudinit.sources.DataSourceNoCloud']
2014-04-22 08:24:24,784 - __init__.py[DEBUG]: Searching for data source in: ['DataSourceCloudStack', 'DataSourceNoCloudNet']
2014-04-22 08:24:24,784 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceCloudStack.DataSourceCloudStack'>
2014-04-22 08:24:24,785 - DataSourceCloudStack.py[DEBUG]: Using /var/lib/dhcp lease directory
2014-04-22 08:24:24,785 - DataSourceCloudStack.py[DEBUG]: Found DHCP identifier 10.1.209.130
2014-04-22 08:24:24,785 - DataSourceCloudStack.py[DEBUG]: Found DHCP identifier 10.1.209.130
2014-04-22 08:24:24,785 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/cs/meta-data (quiet=False)
2014-04-22 08:24:24,785 - url_helper.py[DEBUG]: [0/1] open 'http://10.1.209.130//latest/meta-data/instance-id' with {'url': 'http://10.1.209.130//latest/meta-data/instance-id', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2014-04-22 08:24:24,849 - url_helper.py[DEBUG]: Read from http://10.1.209.130//latest/meta-data/instance-id (200, 36b) after 1 attempts
2014-04-22 08:24:24,849 - DataSourceCloudStack.py[DEBUG]: Using metadata source: 'http://10.1.209.130//latest/meta-data/instance-id'
2014-04-22 08:24:24,850 - url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/user-data' with {'url': 'http://10.1.209.130/latest/user-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:24,853 - url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/user-data (200, 0b) after 1 attempts
2014-04-22 08:24:24,854 - url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:24,857 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2014-04-22 08:24:25,858 - url_helper.py[DEBUG]: [1/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:25,861 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2014-04-22 08:24:26,863 - url_helper.py[DEBUG]: [2/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:26,866 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2014-04-22 08:24:27,868 - url_helper.py[DEBUG]: [3/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:27,871 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2014-04-22 08:24:28,873 - url_helper.py[DEBUG]: [4/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:28,876 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2014-04-22 08:24:29,877 - url_helper.py[DEBUG]: [5/6] open 'http://10.1.209.130/latest/meta-data' with {'url': 'http://10.1.209.130/latest/meta-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
2014-04-22 08:24:29,885 - util.py[WARNING]: Failed fetching metadata from url http://10.1.209.130/latest/meta-data
2014-04-22 08:24:29,888 - util.py[DEBUG]: Failed fetching metadata from url http://10.1.209.130/latest/meta-data
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cloudinit/ec2_utils.py", line 175, in get_instance_metadata
    response = caller(md_url)
  File "/usr/lib/python2.7/dist-packages/cloudinit/util.py", line 704, in read_file_or_url
    exception_cb=exception_cb)
  File "/usr/lib/python2.7/dist-packages/cloudinit/url_helper.py", line 257, in readurl
    raise excps[-1]
UrlError: 404 Client Error: Not Found

after the fix:
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Looking for modules ['DataSourceCloudStack', 'cloudinit.sources.DataSourceCloudStack'] that have attributes ['get_datasource_list']
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Failed at attempted import of 'DataSourceCloudStack' due to: No module named DataSourceCloudStack
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Found DataSourceCloudStack with attributes ['get_datasource_list'] in ['cloudinit.sources.DataSourceCloudStack']
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Looking for modules ['DataSourceNoCloud', 'cloudinit.sources.DataSourceNoCloud'] that have attributes ['get_datasource_list']
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Failed at attempted import of 'DataSourceNoCloud' due to: No module named DataSourceNoCloud
Apr 22 11:21:55 ubuntu [CLOUDINIT] importer.py[DEBUG]: Found DataSourceNoCloud with attributes ['get_datasource_list'] in ['cloudinit.sources.DataSourceNoCloud']
Apr 22 11:21:55 ubuntu [CLOUDINIT] __init__.py[DEBUG]: Searching for data source in: ['DataSourceCloudStack', 'DataSourceNoCloudNet']
Apr 22 11:21:55 ubuntu [CLOUDINIT] __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceCloudStack.DataSourceCloudStack'>
Apr 22 11:21:55 ubuntu [CLOUDINIT] DataSourceCloudStack.py[DEBUG]: Using /var/lib/dhcp lease directory
Apr 22 11:21:55 ubuntu [CLOUDINIT] DataSourceCloudStack.py[DEBUG]: Found DHCP identifier 10.1.209.130
Apr 22 11:21:55 ubuntu [CLOUDINIT] DataSourceCloudStack.py[DEBUG]: Found DHCP identifier 10.1.209.130
Apr 22 11:21:55 ubuntu [CLOUDINIT] util.py[DEBUG]: Reading from /var/lib/cloud/seed/cs/meta-data (quiet=False)
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/1] open 'http://10.1.209.130//latest/meta-data/instance-id' with {'url': 'http://10.1.209.130//latest/meta-data/instance-id', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130//latest/meta-data/instance-id (200, 36b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] DataSourceCloudStack.py[DEBUG]: Using metadata source: 'http://10.1.209.130//latest/meta-data/instance-id'
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/user-data' with {'url': 'http://10.1.209.130/latest/user-data', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/user-data (200, 0b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/' with {'url': 'http://10.1.209.130/latest/meta-data/', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/ (200, 136b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/local-ipv4' with {'url': 'http://10.1.209.130/latest/meta-data/local-ipv4', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/local-ipv4 (200, 11b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/local-hostname' with {'url': 'http://10.1.209.130/latest/meta-data/local-hostname', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/local-hostname (200, 16b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/public-hostname' with {'url': 'http://10.1.209.130/latest/meta-data/public-hostname', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/public-hostname (200, 12b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/public-keys' with {'url': 'http://10.1.209.130/latest/meta-data/public-keys', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/public-keys (200, 0b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/cloud-identifier' with {'url': 'http://10.1.209.130/latest/meta-data/cloud-identifier', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/cloud-identifier (200, 49b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/service-offering' with {'url': 'http://10.1.209.130/latest/meta-data/service-offering', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/service-offering (200, 21b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/public-ipv4' with {'url': 'http://10.1.209.130/latest/meta-data/public-ipv4', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/public-ipv4 (200, 12b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/vm-id' with {'url': 'http://10.1.209.130/latest/meta-data/vm-id', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/vm-id (200, 36b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/availability-zone' with {'url': 'http://10.1.209.130/latest/meta-data/availability-zone', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/availability-zone (200, 9b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: [0/6] open 'http://10.1.209.130/latest/meta-data/instance-id' with {'url': 'http://10.1.209.130/latest/meta-data/instance-id', 'headers': {'User-Agent': 'Cloud-Init/0.7.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0} configuration
Apr 22 11:21:55 ubuntu [CLOUDINIT] url_helper.py[DEBUG]: Read from http://10.1.209.130/latest/meta-data/instance-id (200, 36b) after 1 attempts
Apr 22 11:21:55 ubuntu [CLOUDINIT] DataSourceCloudStack.py[DEBUG]: Crawl of metadata service took 0 seconds
Apr 22 11:21:55 ubuntu [CLOUDINIT] stages.py[INFO]: Loaded datasource DataSourceCloudStack - DataSourceCloudStack
Apr 22 11:21:55 ubuntu [CLOUDINIT] util.py[DEBUG]: Reading from /proc/cmdline (quiet=False)
Apr 22 11:21:55 ubuntu [CLOUDINIT] util.py[DEBUG]: Read 147 bytes from /proc/cmdline
Apr 22 11:21:55 ubuntu [CLOUDINIT] util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg (quiet=False)

Revision history for this message
Scott Moser (smoser) wrote :

Atilla, Thanks for the good bug report.
sorry i didn't see this earlier (and sorry for the bug at all)

Revision history for this message
Carlos (creategui) wrote :

BTW a trailing / is also required on AWS EC2. The difference is that EC2 returns a 301 instead of the 404 that Cloudstack returns.

on AWS EC2:
$ curl -i http://169.254.169.254/latest/meta-data
HTTP/1.0 301 Moved Permanently
Location: http://169.254.169.254/latest/meta-data/
Content-Length: 0
Connection: close
Date: Mon, 25 Aug 2014 15:58:00 GMT
Server: EC2ws

So this is not entirely a CS bug.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.