Description
===========
When trying to live-migrate an instance, the instance start on the second node but then nova throw a "service catalog is empty".
Steps to reproduce
==================
* openstack server migrate 2a58b49e-08df-471b-9560-a9281621b2d1 --live ruby.makz.me
Expected result
===============
The instance is migrated on the ruby.makz.me node
Actual result
=============
Nova start the instance on the second node for the live migration, nova controller throw a "Migration error: The service catalog is empty"
Environment
===========
Controller
openstack-nova-scheduler-18.0.0-1.el7.noarch
python-nova-18.0.0-1.el7.noarch
openstack-nova-console-18.0.0-1.el7.noarch
openstack-nova-common-18.0.0-1.el7.noarch
openstack-nova-conductor-18.0.0-1.el7.noarch
openstack-nova-spicehtml5proxy-18.0.0-1.el7.noarch
python2-novaclient-11.0.0-1.el7.noarch
openstack-nova-api-18.0.0-1.el7.noarch
openstack-nova-placement-api-18.0.0-1.el7.noarch
openstack-nova-novncproxy-18.0.0-1.el7.noarch
Compute node
openstack-nova-compute-18.0.0-1.el7.noarch
openstack-nova-common-18.0.0-1.el7.noarch
python-nova-18.0.0-1.el7.noarch
python2-novaclient-11.0.0-1.el7.noarch
2. Which hypervisor did you use?
Libvirt + KVM
libvirt-daemon-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-secret-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-mpath-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-network-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-logical-3.9.0-14.el7_5.7.x86_64
libvirt-libs-3.9.0-14.el7_5.7.x86_64
libvirt-client-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-core-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-iscsi-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-nwfilter-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-qemu-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-rbd-3.9.0-14.el7_5.7.x86_64
libvirt-python-3.9.0-1.el7.x86_64
libvirt-daemon-driver-storage-disk-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-interface-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-scsi-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-kvm-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-nodedev-3.9.0-14.el7_5.7.x86_64
libvirt-daemon-driver-storage-gluster-3.9.0-14.el7_5.7.x86_64
qemu-kvm-ev-2.10.0-21.el7_5.4.1.x86_64
qemu-img-ev-2.10.0-21.el7_5.4.1.x86_64
libvirt-daemon-driver-qemu-3.9.0-14.el7_5.7.x86_64
qemu-kvm-common-ev-2.10.0-21.el7_5.4.1.x86_64
ipxe-roms-qemu-20170123-1.git4e85b27.el7_4.1.noarch
centos-release-qemu-ev-1.0-3.el7.centos.noarch
2. Which storage type did you use?
Cinder + LVM
python-cinder-13.0.0-1.el7.noarch
openstack-cinder-13.0.0-1.el7.noarch
python2-cinderclient-4.0.1-1.el7.noarch
3. Which networking type did you use?
openstack-neutron-13.0.1-1.el7.noarch
openstack-neutron-common-13.0.1-1.el7.noarch
openstack-neutron-ml2-13.0.1-1.el7.noarch
python2-neutron-lib-1.18.0-1.el7.noarch
python2-neutronclient-6.9.0-1.el7.noarch
openstack-neutron-linuxbridge-13.0.1-1.el7.noarch
python-neutron-13.0.1-1.el7.noarch
python2-openvswitch-2.9.0-3.el7.noarch
Logs & Configs
==============
Controller
==========
==> /var/log/neutron/server.log <==
2018-09-25 15:48:40.734 6677 INFO neutron.wsgi [req-b5ffce01-5f45-4099-9f4e-dca8d66bd04f 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] 192.168.58.1 "GET /v2.0/ports?device_id=2a58b49e-08df-471b-9560-a9281621b2d1 HTTP/1.1" status: 200 len: 1189 time: 0.5954452
2018-09-25 15:48:40.830 6677 INFO neutron.wsgi [req-d524e8ea-7201-4f31-a774-dd9b88e3cd1f 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] 192.168.58.1 "GET /v2.0/security-groups?id=2b2618f8-8c86-4ff0-8beb-52a9e8e5edad&id=1f47e4f6-62d5-4e91-bb91-f27c525bd0be&id=4e0d94e2-e915-41ea-8103-636af0d5c82f&id=9cfc779e-dd64-44f1-85b2-a9c6c852cfba HTTP/1.1" status: 200 len: 11665 time: 0.0915859
==> /var/log/nova/nova-api.log <==
2018-09-25 15:48:40.837 24668 INFO nova.osapi_compute.wsgi.server [req-c2c432cc-4911-4f8e-a8fc-9b2a732ed93c 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] 192.168.58.1 "GET /v2.1/servers/2a58b49e-08df-471b-9560-a9281621b2d1 HTTP/1.1" status: 200 len: 2108 time: 1.3573151
==> /var/log/nova/nova-placement-api.log <==
2018-09-25 15:48:42.552 8830 INFO nova.api.openstack.placement.requestlog [req-a2384cab-5e52-4c99-a4db-56c96475f4ee 0412e7701b674098b81afc2424781afb a03be5acfab64945a9259b798130d1eb - default default] 192.168.58.1 "GET /allocations/2a58b49e-08df-471b-9560-a9281621b2d1" status: 200 len: 245 microversion: 1.28\x1b[00m
2018-09-25 15:48:42.746 8831 INFO nova.api.openstack.placement.requestlog [req-95abd683-8e2f-4955-8c2d-25b6c88db49b 0412e7701b674098b81afc2424781afb a03be5acfab64945a9259b798130d1eb - default default] 192.168.58.1 "POST /allocations" status: 204 len: 0 microversion: 1.13\x1b[00m
==> /var/log/nova/nova-conductor.log <==
2018-09-25 15:48:43.132 24594 WARNING oslo_config.cfg [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Deprecated: Option "url" from group "neutron" is deprecated for removal (Endpoint lookup uses the service catalog via common keystoneauth1 Adapter configuration options. In the current release, "url" will override this behavior, but will be ignored and/or removed in a future release. To achieve the same result, use the endpoint_override option instead.). Its value may be silently ignored in the future.
==> /var/log/neutron/server.log <==
2018-09-25 15:48:43.151 6674 INFO neutron.wsgi [req-0751549f-ce04-4fd5-a590-e0fcbf4fc39d 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] 192.168.58.1 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 9702 time: 0.0112290
==> /var/log/nova/nova-placement-api.log <==
2018-09-25 15:48:45.234 8829 INFO nova.api.openstack.placement.requestlog [req-b61f462c-1f34-4855-aa4c-83042f503897 0412e7701b674098b81afc2424781afb a03be5acfab64945a9259b798130d1eb - default default] 192.168.58.1 "POST /allocations" status: 204 len: 0 microversion: 1.13\x1b[00m
==> /var/log/nova/nova-conductor.log <==
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Migration of instance 2a58b49e-08df-471b-9560-a9281621b2d1 to host ruby.makz.me unexpectedly failed.: EmptyCatalog: The service catalog is empty.
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager Traceback (most recent call last):
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 424, in _live_migrate
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager task.execute()
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/base.py", line 27, in wrap
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager self.rollback()
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager self.force_reraise()
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager six.reraise(self.type_, self.value, self.tb)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/base.py", line 24, in wrap
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager return original(self)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/base.py", line 42, in execute
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager return self._execute()
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line 97, in _execute
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager source_node, dest_node = self._check_requested_destination()
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line 171, in _check_requested_destination
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager self._call_livem_checks_on_host(self.destination)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line 252, in _call_livem_checks_on_host
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager self._bind_ports_on_destination(destination))
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/tasks/live_migrate.py", line 263, in _bind_ports_on_destination
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager self.context, self.instance, destination)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1277, in bind_ports_to_host
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager json=data, raise_exc=False)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 334, in post
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager return self.request(url, 'POST', **kwargs)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 213, in request
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager return self.session.request(url, method, **kwargs)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 706, in request
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager **endpoint_filter)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 1113, in get_endpoint
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager return auth.get_endpoint(self, **kwargs)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager allow_version_hack=allow_version_hack, **kwargs)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 279, in get_endpoint_data
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager service_name=service_name)
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/access/service_catalog.py", line 425, in endpoint_data_for
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager raise exceptions.EmptyCatalog('The service catalog is empty.')
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager EmptyCatalog: The service catalog is empty.
2018-09-25 15:48:45.237 24594 ERROR nova.conductor.manager
2018-09-25 15:48:45.269 24594 WARNING nova.scheduler.utils [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Failed to compute_task_migrate_server: The service catalog is empty.: EmptyCatalog: The service catalog is empty.
2018-09-25 15:48:45.270 24594 WARNING nova.scheduler.utils [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] [instance: 2a58b49e-08df-471b-9560-a9281621b2d1] Setting instance to ERROR state.: EmptyCatalog: The service catalog is empty.
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Exception during message handling: MigrationError: Migration error: The service catalog is empty.
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server return func(*args, **kwargs)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 86, in wrapper
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server return fn(self, context, *args, **kwargs)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1141, in decorated_function
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 281, in migrate_server
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server block_migration, disk_over_commit, request_spec)
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 454, in _live_migrate
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server raise exception.MigrationError(reason=six.text_type(ex))
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server MigrationError: Migration error: The service catalog is empty.
2018-09-25 15:48:45.424 24594 ERROR oslo_messaging.rpc.server
==> /var/log/nova/nova-api.log <==
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Unexpected exception in API method: MigrationError_Remote: Migration error: The service catalog is empty.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
res = self.dispatcher.dispatch(message)
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
return self._do_dispatch(endpoint, method, ctxt, args)
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
result = func(ctxt, **new_args)
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 86, in wrapper
return fn(self, context, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1141, in decorated_function
return function(self, context, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 281, in migrate_server
block_migration, disk_over_commit, request_spec)
File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 454, in _live_migrate
raise exception.MigrationError(reason=six.text_type(ex))
MigrationError: Migration error: The service catalog is empty.
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 801, in wrapped
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/migrate_server.py", line 107, in _migrate_live
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi async_)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 205, in inner
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 213, in _wrapped
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return fn(self, context, instance, *args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 153, in inner
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 4396, in live_migrate
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi request_spec=request_spec, async_=async_)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 112, in live_migrate_instance
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi request_spec=request_spec)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 338, in migrate_server
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return cctxt.call(context, 'migrate_server', **kw)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179, in call
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi retry=self.retry)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 133, in _send
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi retry=retry)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi call_monitor_timeout, retry=retry)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi raise result
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi MigrationError_Remote: Migration error: The service catalog is empty.
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi res = self.dispatcher.dispatch(message)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return self._do_dispatch(endpoint, method, ctxt, args)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi result = func(ctxt, **new_args)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 226, in inner
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 86, in wrapper
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return fn(self, context, *args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1141, in decorated_function
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi return function(self, context, *args, **kwargs)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 281, in migrate_server
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi block_migration, disk_over_commit, request_spec)
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 454, in _live_migrate
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi raise exception.MigrationError(reason=six.text_type(ex))
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi MigrationError: Migration error: The service catalog is empty.
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.446 24668 ERROR nova.api.openstack.wsgi
2018-09-25 15:48:45.481 24668 INFO nova.api.openstack.wsgi [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'MigrationError_Remote'>
2018-09-25 15:48:45.483 24668 INFO nova.osapi_compute.wsgi.server [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] 192.168.58.1 "POST /v2.1/servers/2a58b49e-08df-471b-9560-a9281621b2d1/action HTTP/1.1" status: 500 len: 628 time: 4.6382320
Compute node
============
2018-09-25 15:48:42.961 1741 INFO nova.virt.libvirt.driver [req-d26db4fb-79f2-47c3-b779-4c49e76791e7 3829056496a04d929af44e29efc7568b f2b81bf3d8ec40c191ae06bc2682e606 - default default] Instance launched has CPU info: {"vendor": "Intel", "model": "Westmere-IBRS", "arch": "x86_64", "features": ["pge", "clflush", "sep", "syscall", "vme", "dtes64", "stibp", "msr", "vmx", "xtpr", "cmov", "ssse3", "est", "pat", "monitor", "smx", "pbe", "lm", "tsc", "nx", "fxsr", "tm", "sse4.1", "pae", "sse4.2", "pclmuldq", "cx16", "pcid", "mmx", "arat", "cx8", "mce", "de", "rdtscp", "ht", "dca", "lahf_lm", "pdcm", "mca", "pdpe1gb", "apic", "sse", "pse", "ds", "invtsc", "pni", "tm2", "aes", "sse2", "ss", "ds_cpl", "acpi", "spec-ctrl", "fpu", "ssbd", "pse36", "mtrr", "popcnt"], "topology": {"cores": 6, "cells": 2, "threads": 2, "sockets": 1}}
Check the [neutron] section of the nova configuration file used for your nova-conductor service, because that is what is failing. Is that configured? And if so, what are the values? Are those consistent with what is in the service catalog for the networking service (neutron)?