metadata proxy not support Https Metadata-api

[Liping Mao]

In neutron/agent/metadata/agent.py, we have :

        url = urlparse.urlunsplit((
            'http',
            '%s:%s' % (self.conf.nova_metadata_ip,
                       self.conf.nova_metadata_port),
            req.path_info,
            req.query_string,
            ''))

This means we can only use http to access metadata api server. and if we have loadbalance for metadata api server and has enabled the SSL on Loadbalance VIP, it can't work.

I think we need something like metadata_protocal to config http or https.

[Xiaolin Zhang (zhangxiaolins)]

We've encountered the same issue that metadata agent failed with supporting https Keystone authentication.
When we configured Keystone to use https, and modified metadata_agent.ini to
auth_url = https://192.168.8.101:5000/v2.0
then restart metadata-agent.

The newly started VM thereafter would not get its metadata:
in console we saw:
2014-01-08 13:50:24,592 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: http error [500]

in metadata-agent.log we saw:
2014-01-08 21:50:25.453 1774 DEBUG neutronclient.client [-]
REQ: curl -i https://192.168.8.101:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-neutronclient" -d '{"auth": {"tenantName": "ser
vice", "passwordCredentials": {"username": "neutron", "password": "REDACTED"}}}'
 http_log_req /usr/lib/python2.7/dist-packages/neutronclient/common/utils.py:173
2014-01-08 21:50:25.492 1774 ERROR neutron.agent.metadata.agent [-] Unexpected error.
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent Traceback (most recent call last):
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 94, in __call__
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent instance_id = self._get_instance_id(req)
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 118, in _get_instance_id
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent device_owner=DEVICE_OWNER_ROUTER_INTF)['ports']
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 108, in with_params
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent ret = self.function(instance, *args, **kwargs)
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 298, in list_ports
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent **_params)
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1197, in list
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent for r in self._pagination(collection, path, **params):
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1210, in _pagination
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent res = self.get(path, params=params)
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1183, in get
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent headers=headers, params=params)
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1168, in retry_request
2014-01-08 21:50:25.492 1774 TRACE neutron.agent.metadata.agen...

[Xiaolin Zhang (zhangxiaolins)]

It's a need to add https protocol support for metadata proxy processing and add new config options, e.g. `auth_protocol`, in metadata_agent.ini

[Xiaolin Zhang (zhangxiaolins)]

Propose to add https options -insecure and ca_cert- in _get_neutron_client

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/67181

[Ashok kumaran B (ashokkumaran-b)]

Hi Xiaolin, dont you think your booted instance need to run cloud-init 0.7.5 as well? what do you think

http://<email address hidden>/msg14857.html

[Ashok kumaran B (ashokkumaran-b)]

I mean when you run https enabled metadata-api , i suppose cloud-init 0.7.5 is needed in images

[Xiaolin Zhang (zhangxiaolins)]

Hi Ashok, I agree with you, if the VM client fetch metadata from https://169.254.169.254/, the cloud-init need 0.7.5.
Also, neutron needs supporting metadata agent to proxy request via https client, which resolved by above proposed fix.
Thanks for you kind advice.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/67181
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a06c2f6cbdff64bdc9750b57cb8782cc337774ff
Submitter: Jenkins
Branch: master

commit a06c2f6cbdff64bdc9750b57cb8782cc337774ff
Author: Xiaolin Zhang <email address hidden>
Date: Thu Jan 16 22:28:28 2014 +0800

    Adds https support for metadata agent

    Adds two configure options to support https client for metadata proxy
    * auth_insecure: turn off verification of the certificate for ssl, or
    * auth_ca_cert: CA cert to check against with for ssl.

    Change-Id: I3ffb86ae9ce31931436a4e13957aae30eebf9d92
    Closes-Bug: #1263872

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/79658

[Jakub Libosvar (libosvar)]

If I understand correctly this bug is about communication neutron-metadata-proxy<->keystone only. According comment 7 should we implement https support in namespace_proxy too and edit l3 nat rules?

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sascha Peilicke (<email address hidden>) on branch: stable/havana
Review: https://review.openstack.org/79658
Reason: @Ihar: thanks for your reply but this one was started ages ago. Meanwhile, it's affected by the stable branch policy and I don't care much for it anymore anyway.