full ha: Neutron metadata agent can not talk to keystone

[Sagi (Sergey) Shnaidman]

neutron metada agent configured as:
/etc/neutron/metadata_agent.ini:auth_url = http://localhost:5000/v2.0

but keystone listens on eth0 interface:
tcp 0 0 192.168.40.4:5000 0.0.0.0:* LISTEN 6306/python

should be configured as VIP address:
auth_url = http://192.168.40.253:5000/v2.0

2014-07-30 10:05:08.994 6632 TRACE neutron.notifiers.nova ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 111] ECONNREFUSED)

[Sagi (Sergey) Shnaidman (sshnaidm)]

Perhaps it's not metadata agent, because reconfiguration does not help:
It still tries to connect to localhost:

2014-07-30 11:53:28.460 14327 ERROR neutron.notifiers.nova [req-9e511e04-6a3a-45f1-afe0-ee1b3d365fca None] Failed to notify nova on events: [{'status': 'completed', 'tag': u'358b4880-be54-44a2-a3e0-227b6ea8fadd', 'name': 'network-vif-unplugged', 'server_uuid': u'f861a43f-1df1-435b-a71f-05b65d627c4d'}]
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova Traceback (most recent call last):
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 221, in send_events
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova batched_events)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/contrib/server_external_events.py", line 39, in create
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova return_raw=True)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 152, in _create
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova _resp, body = self.api.client.post(url, body=body)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 286, in post
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova return self._cs_request(url, 'POST', **kwargs)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 269, in _cs_request
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova self.authenticate()
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 382, in authenticate
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova auth_url = self._v2_auth(auth_url)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 469, in _v2_auth
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova return self._authenticate(url, body)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 482, in _authenticate
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova **kwargs)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 242, in _time_request
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova resp, body = self.request(url, method, **kwargs)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 213, in request
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova **kwargs)
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
2014-07-30 11:53:28.460 14327 TRACE neutron.notifiers.nova resp = self.send(prep, **send_kwargs)
2014-07-30 11:53:28.460 14327...

[Sagi (Sergey) Shnaidman (sshnaidm)]

finally found it:
cfg.StrOpt('nova_admin_auth_url',
               default='http://localhost:5000/v2.0',
               help=_('Authorization URL for connecting to nova in admin '
                      'context')),

should be explicitly in neutron.conf:

nova_admin_auth_url=http://192.168.40.x:5000/v2.0
Moving it to another bug, this one is about metadata agent.

[Mark T. Voelker (mvoelker)]

The auth_url directive in the metadata.ini file typically gets set by this data mapping I believe:

https://github.com/CiscoSystems/puppet_openstack_builder/blob/icehouse/data/data_mappings/common.yaml#L211-L217

Do you have anything in your yaml files that would override that?