mitaka b2: neutron->nova external event notification "Service catalog is empty"

Bug #1549227 reported by James Page on 2016-02-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Unassigned

Bug Description

With the same configuration that is confirmed to work with mitaka b1, I'm seeing the following error with mitaka b2 - neutron is unable to send external event notifications back to nova:

2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'8cbc4861-97cb-484e-bd10-fd6a226c332a', 'name': 'network-vif-plugged', 'server_uuid': u'2831a5ed-26f9-42ed-bb9d-363b1117307f'}]
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova Traceback (most recent call last):
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 209, in send_events
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova batched_events)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py", line 39, in create
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return_raw=True)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova _resp, body = self.api.client.post(url, body=body)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return self.request(url, 'POST', **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 91, in request
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in request
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return self.session.request(url, method, **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/_utils.py", line 180, in inner
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return func(*args, **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 345, in request
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova base_url = self.get_endpoint(auth, **endpoint_filter)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 615, in get_endpoint
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return auth.get_endpoint(self, **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 213, in get_endpoint
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova service_name=service_name)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/_utils.py", line 180, in inner
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova return func(*args, **kwargs)
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova File "/usr/lib/python2.7/dist-packages/keystoneauth1/access/service_catalog.py", line 190, in url_for
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova raise exceptions.EmptyCatalog('The service catalog is empty.')
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova EmptyCatalog: The service catalog is empty.
2016-02-24 10:36:45.844 26152 ERROR neutron.notifiers.nova

neutron.conf:

[DEFAULT]
verbose = False
debug = False
use_syslog = False
state_path = /var/lib/neutron
bind_host = 0.0.0.0
auth_strategy = keystone
notification_driver = neutron.openstack.common.notifier.rpc_notifier
api_workers = 2
rpc_workers = 2

router_distributed = False

l3_ha = False
bind_port = 9686
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
service_plugins = router,firewall,lbaas,vpnaas,metering
allow_overlapping_ips = True
neutron_firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True

[nova]
auth_plugin = password
region_name = RegionOne
auth_url = http://10.5.3.86:35357/v2.0
username = neutron
password = XXX
tenant_id = 8679f8352c4b4be0a7edda3b41099731

[quotas]
quota_driver = neutron.db.quota_db.DbQuotaDriver
quota_items = network,subnet,port,security_group,security_group_rule
quota_security_group = 10
quota_security_group_rule = 100
quota_network = 10
quota_subnet = 10
quota_port = 50
quota_vip = 10
quota_pool = 10
quota_member = -1
quota_health_monitors = -1
quota_router = 10
quota_floatingip = 50

[agent]
root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf

[keystone_authtoken]
identity_uri = http://10.5.3.86:35357/
auth_uri = http://10.5.3.86:5000/
admin_tenant_name = services
admin_user = neutron
admin_password = XXXX
signing_dir = /var/cache/neutron

[database]
connection = mysql://neutron:XXXX@10.5.3.88/neutron
max_pool_size = 8

[oslo_messaging_rabbit]
rabbit_userid = neutron
rabbit_virtual_host = openstack
rabbit_password = XXX
rabbit_host = 10.5.3.96

[oslo_concurrency]
lock_path = $state_path/lock

James Page (james-page) wrote :

switching nova.tenant_id to nova.project_id resolved this problem, but I'm not sure that it should have done.

Gary Kotton (garyk) wrote :

I think that this something that we need to document

Changed in neutron:
status: New → Confirmed
importance: Undecided → Medium
Sreekumar S (sreesiv) on 2016-06-16
Changed in neutron:
assignee: nobody → Sreekumar S (sreesiv)
Sreekumar S (sreesiv) on 2016-11-07
Changed in neutron:
assignee: Sreekumar S (sreesiv) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers