Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding static route with those interface failing as "Nexthop has invalid gateway"
The better way would be check the interface status before add the route entry at https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_fip_ns.py#L458
Stack trace: ============ 2024-03-29 08:00:19.027 5996 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway.
2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. : neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/common/utils.py", line 159, in call 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1207, in process 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.process_external() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 724, in process_external 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.connect_rtr_2_fip() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 739, in connect_rtr_2_fip 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.fip_ns.create_rtr_2_fip_link(self) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 456, in create_rtr_2_fip_link 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info rtr_2_fip_dev.route.add_gateway(str(fip_2_rtr.ip), table=FIP_RT_TBL) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 609, in add_gateway 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self._as_root([ip_version], tuple(args)) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 410, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 124, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info namespace=namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 132, in _execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 160, in execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info returncode=returncode) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info
Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding static route with those interface failing as "Nexthop has invalid gateway"
The better way would be check the interface status before add the route entry at https:/ /github. com/openstack/ neutron/ blob/master/ neutron/ agent/l3/ dvr_fip_ ns.py#L458
Stack trace: agent.linux. utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway.
============
2024-03-29 08:00:19.027 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron. agent.l3. router_ info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. lib.exceptions. ProcessExecutio nError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. agent.l3. router_ info Traceback (most recent call last): agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ common/ utils.py" , line 159, in call agent.l3. router_ info return func(*args, **kwargs) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/l3/ router_ info.py" , line 1207, in process agent.l3. router_ info self.process_ external( ) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/l3/ dvr_local_ router. py", line 724, in process_external agent.l3. router_ info self.connect_ rtr_2_fip( ) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/l3/ dvr_local_ router. py", line 739, in connect_rtr_2_fip agent.l3. router_ info self.fip_ ns.create_ rtr_2_fip_ link(self) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/l3/ dvr_fip_ ns.py", line 456, in create_ rtr_2_fip_ link agent.l3. router_ info rtr_2_fip_ dev.route. add_gateway( str(fip_ 2_rtr.ip) , table=FIP_RT_TBL) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/linux/ ip_lib. py", line 609, in add_gateway agent.l3. router_ info self._as_ root([ip_ version] , tuple(args)) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/linux/ ip_lib. py", line 410, in _as_root agent.l3. router_ info use_root_ namespace= use_root_ namespace) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/linux/ ip_lib. py", line 124, in _as_root agent.l3. router_ info namespace= namespace) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/linux/ ip_lib. py", line 132, in _execute agent.l3. router_ info log_fail_ as_error= self.log_ fail_as_ error) agent.l3. router_ info File "/opt/openstack /neutron/ lib/python3. 6/site- packages/ neutron/ agent/linux/ utils.py" , line 160, in execute agent.l3. router_ info returncode= returncode) agent.l3. router_ info neutron_ lib.exceptions. ProcessExecutio nError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. agent.l3. router_ info agent.l3. router_ info
: neutron_
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.
2024-03-29 08:00:19.028 5996 ERROR neutron.