neutron metadata agent erroring after Queens to Rocky upgrade

Bug #1802365 reported by Liam Young
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Neutron Gateway Charm
Fix Released
Critical
Liam Young

Bug Description

After upgrading the neutron-gateway from Queens to Rocky new guests are not receiving metadata and the neutron-gateway contains the following error:

2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent [-] Unexpected error.: ImportError: No module named pyamqp
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent Traceback (most recent call last):
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 88, in __call__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 161, in _get_instance_and_tenant_id
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 149, in _get_ports
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/cache_utils.py", line 116, in __call__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 123, in _get_router_networks
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 105, in _get_ports_from_server
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 71, in get_ports
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 150, in call
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 526, in _send
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 507, in _get_reply_q
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 498, in _get_connection
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 578, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 179, in __init__
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/transport/__init__.py", line 83, in get_transport_cls
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/transport/__init__.py", line 68, in resolve_transport
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/kombu/utils/imports.py", line 56, in symbol_by_name
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent __import__(name)
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent ImportError: No module named pyamqp
2018-11-08 18:14:19.405 18619 ERROR neutron.agent.metadata.agent
2018-11-08 18:14:19.406 18619 INFO eventlet.wsgi.server [-] 192.168.21.13,<local> "GET /2009-04-04/meta-data/instance-id HTTP/1.1" status: 500 len: 362 time: 0.0027149

Revision history for this message
Liam Young (gnuoy) wrote :

This was fixed by restarting neutron-metadata-agent.service after the upgrade.

Changed in charm-neutron-gateway:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Liam Young (gnuoy)
milestone: none → 18.11
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-neutron-gateway (master)

Fix proposed to branch: master
Review: https://review.openstack.org/616866

Changed in charm-neutron-gateway:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-neutron-gateway (master)

Reviewed: https://review.openstack.org/616866
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-gateway/commit/?id=d3a6eb410e833b2a57f1f577e8480c4acf481632
Submitter: Zuul
Branch: master

commit d3a6eb410e833b2a57f1f577e8480c4acf481632
Author: Liam Young <email address hidden>
Date: Fri Nov 9 10:03:40 2018 +0000

    Restart nova-metadata-agent after upgrade to Rocky

    After an upgrade to Rocky the nova-metadata-agent is broken due to
    a problem importing the amqp lib. This patch simply restarts the
    service after an upgrade.

    Change-Id: I2c4df35b714a8af607937cb3ad4261db58ed3bbc
    Closes-Bug: #1802365

Changed in charm-neutron-gateway:
status: In Progress → Fix Committed
David Ames (thedac)
Changed in charm-neutron-gateway:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.