Comment 5 for bug 1950916

Tempest Test found router:
1-11-15 01:57:27,120 158054 INFO [tempest.lib.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 201 POST http://192.168.24.3:9696/v2.0/routers 3.188s
2021-11-15 01:57:27,121 158054 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"router": {"name": "tempest-TestNetworkBasicOps-router-1232673168", "admin_state_up": true, "project_id": "08157c21409d440dba084ccabed97461", "external_gateway_info": {"network_id": "aa047c87-117f-404e-98ff-5ccb675ff93a"}}}
    Response - Headers: {'content-type': 'application/json', 'content-length': '690', 'x-openstack-request-id': 'req-bf5c0242-4cea-4480-9dc7-f75cefbb98e2', 'date': 'Mon, 15 Nov 2021 01:57:27 GMT', 'connection': 'close', 'status': '201', 'content-location': 'http://192.168.24.3:9696/v2.0/routers'}
        Body: b'{"router": {"id": "93922895-a67b-4e2a-abee-ddff06583470", "name": "tempest-TestNetworkBasicOps-router-1232673168", "tenant_id": "08157c21409d440dba084ccabed97461", "admin_state_up": true, "status": "ACTIVE", "external_gateway_info": {"network_id": "aa047c87-117f-404e-98ff-5ccb675ff93a", "external_fixed_ips": [{"subnet_id": "34588ca7-5b76-4dee-9909-3926138ce219", "ip_address": "192.168.24.181"}], "enable_snat": true}, "description": "", "availability_zones": [], "availability_zone_hints": [], "routes": [], "flavor_id": null, "tags": [], "created_at": "2021-11-15T01:57:24Z", "updated_at": "2021-11-15T01:57:25Z", "revision_number": 3, "project_id": "08157c21409d440dba084ccabed97461"}}'
2021-11-15 01:57:27,480 158054 INFO [tempest.lib.common.rest_client] Request (TestNetworkBasicOps:test_network_basic_ops): 200 GET http://192.168.24.3:9696/v2.0/subnets?project_id=08157c21409d440dba084ccabed97461&cidr=10.100.0.0%2F28 0.358s
2021-11-15 01:57:27,481 158054 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}

- https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario007-standalone-master/8664fc6/logs/undercloud/var/log/tempest/stestr_results.html.gz

At the same time on neutron side, router is missing:
2021-11-15 01:57:26.855 129917 DEBUG neutron.agent.l3.agent [req-bf5c0242-4cea-4480-9dc7-f75cefbb98e2 c7a8f9be62a04c3d8a1407a13c46b26a 08157c21409d440dba084ccabed97461 - - -] Got routers updated notification :['93922895-a67b-4e2a-abee-ddff06583470'] routers_updated /usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py:582
2021-11-15 01:57:26.857 129917 INFO neutron.agent.l3.agent [-] Starting processing update 93922895-a67b-4e2a-abee-ddff06583470, action 3, priority 1, update_id bd1a5495-227d-45a6-91d8-303df87f84fb. Wait time elapsed: 0.001
2021-11-15 01:57:26.858 129917 INFO neutron.agent.l3.agent [-] Starting router update for 93922895-a67b-4e2a-abee-ddff06583470, action 3, priority 1, update_id bd1a5495-227d-45a6-91d8-303df87f84fb. Wait time elapsed: 0.001
2021-11-15 01:57:26.858 129917 DEBUG neutron.common.utils [-] Time-cost: call ae4c8533-8ad5-4e1c-9e11-975a01eb1a69 function get_routers start wrapper /usr/lib/python3.6/site-packages/neutron/common/utils.py:927
2021-11-15 01:57:27.096 129917 DEBUG neutron.agent.l3.agent [req-bf5c0242-4cea-4480-9dc7-f75cefbb98e2 c7a8f9be62a04c3d8a1407a13c46b26a 08157c21409d440dba084ccabed97461 - - -] Got routers updated notification :['93922895-a67b-4e2a-abee-ddff06583470'] routers_updated /usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py:582
2021-11-15 01:57:28.731 129917 DEBUG neutron.common.utils [-] Time-cost: call ae4c8533-8ad5-4e1c-9e11-975a01eb1a69 function get_routers took 1.874s seconds to run wrapper /usr/lib/python3.6/site-packages/oslo_utils/timeutils.py:388
2021-11-15 01:57:28.732 129917 DEBUG neutron.agent.l3.agent [-] Router 93922895-a67b-4e2a-abee-ddff06583470 info not in cache, will do the router add action. _process_router_if_compatible /usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py:637
2021-11-15 01:57:28.734 129917 DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.agent.linux.pd.add_router-8791547810197'] for router, before_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:176
2021-11-15 01:57:28.734 129917 DEBUG oslo_concurrency.lockutils [-] Lock "l3-agent-pd" acquired by "neutron.agent.linux.pd.add_router" :: waited 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:390
2021-11-15 01:57:28.735 129917 DEBUG oslo_concurrency.lockutils [-] Lock "l3-agent-pd" "released" by "neutron.agent.linux.pd.add_router" :: held 0.000s inner /usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:405
2021-11-15 01:57:28.738 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[0754215c-2f44-4e21-a953-836a76119faf]: (4, False) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.768 131158 DEBUG oslo.privsep.daemon [-] privsep: reply[83291ba2-3422-43fc-9754-27c24e4afbeb]: (4, None) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.807 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[6a3b47c1-0bb0-4aea-bbb3-a1b72ca0efc1]: (4, ('net.ipv4.conf.all.promote_secondaries = 1\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.851 152270 DEBUG oslo.privsep.daemon [-] privsep: reply[a6a6db33-803e-469c-9ad2-5bc23ca630af]: (4, ({'header': {'length': 36, 'type': 2, 'flags': 256, 'sequence_number': 255, 'pid': 159461, 'error': None, 'target': 'qrouter-93922895-a67b-4e2a-abee-ddff06583470', 'stats': Stats(qsize=0, delta=0, delay=0)}, 'event': 'NLMSG_ERROR'},)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.900 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[e251aed6-a9f8-4cb8-9867-225dc0c26bd0]: (4, ('net.ipv4.ip_forward = 1\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.931 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[082b3c8a-0a07-4d80-b340-630fc3d3f198]: (4, ('net.ipv4.conf.all.arp_ignore = 1\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:28.979 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[0cf05d48-6343-4606-a18d-f2820d9c4248]: (4, ('net.ipv4.conf.all.arp_announce = 2\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:29.008 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[d404593d-ccdf-456b-abab-c6f4c23dca30]: (4, ('net.ipv6.conf.all.forwarding = 0\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:29.047 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[6faab3ea-5cd3-4acd-bf24-8fed9b30b669]: (4, ('net.ipv4.ip_nonlocal_bind = 0\n', '', 0)) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503
2021-11-15 01:57:29.051 129917 DEBUG neutron.agent.linux.utils [-] Unable to access /var/lib/neutron/ha_confs/93922895-a67b-4e2a-abee-ddff06583470.pid.keepalived; Error: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/93922895-a67b-4e2a-abee-ddff06583470.pid.keepalived' get_value_from_file /usr/lib/python3.6/site-packages/neutron/agent/linux/utils.py:253
2021-11-15 01:57:29.070 151911 DEBUG neutron.privileged.agent.linux.ip_lib [-] Interface ha-345b8eb7-7f not found in namespace qrouter-93922895-a67b-4e2a-abee-ddff06583470 get_link_id /usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py:203
2021-11-15 01:57:29.071 151911 DEBUG oslo.privsep.daemon [-] privsep: reply[bfad6561-a712-4ffa-9659-9b4c9981cbb2]: (4, False) _call_back /usr/lib/python3.6/site-packages/oslo_privsep/daemon.py:503

- https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario007-standalone-master/8664fc6/logs/undercloud/var/log/containers/neutron/l3-agent.log.txt.gz