and I joined the content of /var/log/neutron/l3-agent.log to this post. I did not find anything helpful the l3-agent.log, except perhaps "'cluster_name': '<email address hidden>'" in line 765 where, assuming this is how to connect to rabbitmq, the username is wrong and the config refers to the correct host's A dns record (w3.int.lunarc) instead of its CNAME (osrabbitmq.int.lunarc) I usually use; I don't know where I can set the correct values. Finally, see below for the output of journalctl -fu neutron-l3-agent:
[root@w3 ~]# journalctl -fu neutron-l3-agent
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopping OpenStack Neutron Layer 3 Agent...
Mar 18 08:05:13 w3.int.lunarc systemd[1]: neutron-l3-agent.service: Succeeded.
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopped OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Started OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: Traceback (most recent call last):
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 476, in fire_timers
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: timer()
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/hubs/timer.py", line 59, in __call__
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: cb(*args, **kw)
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: File "/usr/lib/python3.6/site-packages/eventlet/semaphore.py", line 152, in _do_acquire
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: waiter.switch()
Mar 18 08:05:18 w3.int.lunarc neutron-l3-agent[259430]: greenlet.error: cannot switch to a different thread
and the status of network agents
[root@w3 ~]# openstack network agent list
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+
| 330269b7-b73c-4207-abc7-21f1a2972b7b | Linux bridge agent | w6.int.lunarc | None | :-) | UP | neutron-linuxbridge-agent |
| 83d16241-8a3a-42b0-beda-87246d945dc1 | L3 agent | w3.int.lunarc | nova | XXX | UP | neutron-l3-agent |
| a52ab60f-d893-491d-a43e-823a0d482810 | Linux bridge agent | w3.int.lunarc | None | :-) | UP | neutron-linuxbridge-agent |
| abd75644-d895-41ae-94fa-6c4351cbc4bf | Metadata agent | w3.int.lunarc | None | XXX | UP | neutron-metadata-agent |
| c05c65bc-779e-4fe5-a19e-350c44900be4 | DHCP agent | w3.int.lunarc | nova | XXX | UP | neutron-dhcp-agent |
+--------------------------------------+--------------------+---------------+-------------------+-------+-------+---------------------------+
I have the same symptoms with l3-agent. I set /etc/neutron/ l3_agent. ini\[DEFAULT] \default_ log_levels
default_log_levels = amqp=DEBUG, amqplib= DEBUG,boto= DEBUG,qpid= DEBUG,sqlalchem y=DEBUG, suds=DEBUG, oslo.messaging= DEBUG,oslo_ messaging= DEBUG,iso8601= DEBUG,requests. packages. urllib3. connectionpool= DEBUG,urllib3. connectionpool= DEBUG,websocket =DEBUG, requests. packages. urllib3. util.retry= DEBUG,urllib3. util.retry= DEBUG,keystonem iddleware= DEBUG,routes. middleware= DEBUG,stevedore =DEBUG, taskflow= DEBUG,keystonea uth=DEBUG, oslo.cache= DEBUG,oslo_ policy= DEBUG,dogpile. core.dogpile= DEBUG
and I joined the content of /var/log/ neutron/ l3-agent. log to this post. I did not find anything helpful the l3-agent.log, except perhaps "'cluster_name': '<email address hidden>'" in line 765 where, assuming this is how to connect to rabbitmq, the username is wrong and the config refers to the correct host's A dns record (w3.int.lunarc) instead of its CNAME (osrabbitmq. int.lunarc) I usually use; I don't know where I can set the correct values. Finally, see below for the output of journalctl -fu neutron-l3-agent: l3-agent. service: Succeeded. l3-agent[ 259430] : Traceback (most recent call last): l3-agent[ 259430] : File "/usr/lib/ python3. 6/site- packages/ eventlet/ hubs/hub. py", line 476, in fire_timers l3-agent[ 259430] : timer() l3-agent[ 259430] : File "/usr/lib/ python3. 6/site- packages/ eventlet/ hubs/timer. py", line 59, in __call__ l3-agent[ 259430] : cb(*args, **kw) l3-agent[ 259430] : File "/usr/lib/ python3. 6/site- packages/ eventlet/ semaphore. py", line 152, in _do_acquire l3-agent[ 259430] : waiter.switch() l3-agent[ 259430] : greenlet.error: cannot switch to a different thread
[root@w3 ~]# journalctl -fu neutron-l3-agent
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopping OpenStack Neutron Layer 3 Agent...
Mar 18 08:05:13 w3.int.lunarc systemd[1]: neutron-
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Stopped OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:13 w3.int.lunarc systemd[1]: Started OpenStack Neutron Layer 3 Agent.
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
Mar 18 08:05:18 w3.int.lunarc neutron-
and the status of network agents ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------+ ------- ------- -----+- ------+ ------- +------ ------- ------- ------- + ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------+ ------- ------- -----+- ------+ ------- +------ ------- ------- ------- + b73c-4207- abc7-21f1a2972b 7b | Linux bridge agent | w6.int.lunarc | None | :-) | UP | neutron- linuxbridge- agent | 8a3a-42b0- beda-87246d945d c1 | L3 agent | w3.int.lunarc | nova | XXX | UP | neutron-l3-agent | d893-491d- a43e-823a0d4828 10 | Linux bridge agent | w3.int.lunarc | None | :-) | UP | neutron- linuxbridge- agent | d895-41ae- 94fa-6c4351cbc4 bf | Metadata agent | w3.int.lunarc | None | XXX | UP | neutron- metadata- agent | 779e-4fe5- a19e-350c44900b e4 | DHCP agent | w3.int.lunarc | nova | XXX | UP | neutron-dhcp-agent | ------- ------- ------- ------- ----+-- ------- ------- ----+-- ------- ------+ ------- ------- -----+- ------+ ------- +------ ------- ------- ------- +
[root@w3 ~]# openstack network agent list
+------
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+------
| 330269b7-
| 83d16241-
| a52ab60f-
| abd75644-
| c05c65bc-
+------