neutron metadata agent erroring after Queens to Rocky upgrade

Bug #1802365 reported by Liam Young on 2018-11-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack neutron-gateway charm
Critical
Liam Young

Bug Description

After upgrading the neutron-gateway from Queens to Rocky new guests are not receiving metadata and the neutron-gateway contains the following error:

2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent [-] Unexpected error.: ImportError: No module named pyamqp
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent Traceback (most recent call last):
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 88, in __call__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 161, in _get_instance_and_tenant_id
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 149, in _get_ports
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/cache_utils.py", line 116, in __call__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 123, in _get_router_networks
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 105, in _get_ports_from_server
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 71, in get_ports
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 150, in call
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 526, in _send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 507, in _get_reply_q
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 498, in _get_connection
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 578, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 179, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/transport/__init__.py", line 83, in get_transport_cls
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/transport/__init__.py", line 68, in resolve_transport
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/utils/imports.py", line 56, in symbol_by_name
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent __import__(name)
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent ImportError: No module named pyamqp
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent
2018-11-08 18:14:19.406 18619 INFO eventlet.wsgi.server [-] 192.168.21.13,<local> "GET /2009-04-04/meta-data/instance-id HTTP/1.1" status: 500 len: 362 time: 0.0027149

Liam Young (gnuoy) wrote :

This was fixed by restarting neutron-metadata-agent.service after the upgrade.

Changed in charm-neutron-gateway:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Liam Young (gnuoy)
milestone: none → 18.11

Fix proposed to branch: master
Review: https://review.openstack.org/616866

Changed in charm-neutron-gateway:
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/616866
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-gateway/commit/?id=d3a6eb410e833b2a57f1f577e8480c4acf481632
Submitter: Zuul
Branch: master

commit d3a6eb410e833b2a57f1f577e8480c4acf481632
Author: Liam Young <email address hidden>
Date: Fri Nov 9 10:03:40 2018 +0000

    Restart nova-metadata-agent after upgrade to Rocky

    After an upgrade to Rocky the nova-metadata-agent is broken due to
    a problem importing the amqp lib. This patch simply restarts the
    service after an upgrade.

    Change-Id: I2c4df35b714a8af607937cb3ad4261db58ed3bbc
    Closes-Bug: #1802365

Changed in charm-neutron-gateway:
status: In Progress → Fix Committed
David Ames (thedac) on 2018-11-20
Changed in charm-neutron-gateway:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers