Comment 5 for bug 1965140

Revision history for this message
Nicolas Melot (nicmel) wrote (last edit ):

I have the same symptoms with l3-agent. I set /etc/neutron/l3_agent.ini\[DEFAULT]\default_log_levels

default_log_levels = amqp=DEBUG,amqplib=DEBUG,boto=DEBUG,qpid=DEBUG,sqlalchemy=DEBUG,suds=DEBUG,oslo.messaging=DEBUG,oslo_messaging=DEBUG,iso8601=DEBUG,requests.packages.urllib3.connectionpool=DEBUG,urllib3.connectionpool=DEBUG,websocket=DEBUG,requests.packages.urllib3.util.retry=DEBUG,urllib3.util.retry=DEBUG,keystonemiddleware=DEBUG,routes.middleware=DEBUG,stevedore=DEBUG,taskflow=DEBUG,keystoneauth=DEBUG,oslo.cache=DEBUG,oslo_policy=DEBUG,dogpile.core.dogpile=DEBUG

and I joined the content of /var/log/neutron/l3-agent.log to this post. I did not find anything helpful the l3-agent.log, except perhaps "'cluster_name': '<email address hidden>'" in line 765 where, assuming this is how to connect to rabbitmq, the username is wrong and the config refers to the correct host's A dns record (w3.int.lunarc) instead of its CNAME (osrabbitmq.int.lunarc) I usually use; I don't know where I can set the correct values. Finally, see below for the output of journalctl -fu neutron-l3-agent:
[root@w3 ~]# journalctl -fu neutron-l3-agent
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopping OpenStack Neutron Layer 3 Agent...
Mar 18 08:05:13 w3.int.lunarc systemd[1]: neutron-l3-agent.service: Succeeded.
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopped OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Started OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: Traceback (most recent call last):
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 476, in fire_timers
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: timer()
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/hubs/timer.py", line 59, in __call__
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: cb(*args, **kw)
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/semaphore.py", line 152, in _do_acquire
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: waiter.switch()
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: greenlet.error: cannot switch to a different thread

and the status of network agents
[root@w3 ~]# openstack network agent list
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+
| 330269b7-b73c-4207-abc7-21f1a2972b7b | Linux bridge agent | w6.int.lunarc | None | :-) | UP | neutron-linuxbridge-agent |
| 83d16241-8a3a-42b0-beda-87246d945dc1 | L3 agent | w3.int.lunarc | nova | XXX | UP | neutron-l3-agent |
| a52ab60f-d893-491d-a43e-823a0d482810 | Linux bridge agent | w3.int.lunarc | None | :-) | UP | neutron-linuxbridge-agent |
| abd75644-d895-41ae-94fa-6c4351cbc4bf | Metadata agent | w3.int.lunarc | None | XXX | UP | neutron-metadata-agent |
| c05c65bc-779e-4fe5-a19e-350c44900be4 | DHCP agent | w3.int.lunarc | nova | XXX | UP | neutron-dhcp-agent |
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+