neutron

  1. Bug #2069409
  2. Activity log

Activity log for bug #2069409

Date Who What changed Old value New value Message
2024-06-14 09:31:41 Mohankumar bug added bug
2024-06-14 09:31:57 Mohankumar neutron: assignee Mohankumar (mohankumar-n)
2024-06-14 09:33:28 Mohankumar description Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding static route with those interface failing as "Nexthop has invalid gateway" The better way would be check the interface status before add the route entry at https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_fip_ns.py#L458 Stack trace: ============ 2024-03-29 08:00:19.027 5996 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. : neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/common/utils.py", line 159, in call 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1207, in process 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.process_external() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 724, in process_external 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.connect_rtr_2_fip() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 739, in connect_rtr_2_fip 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.fip_ns.create_rtr_2_fip_link(self) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 456, in create_rtr_2_fip_link 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info rtr_2_fip_dev.route.add_gateway(str(fip_2_rtr.ip), table=FIP_RT_TBL) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 609, in add_gateway 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self._as_root([ip_version], tuple(args)) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 410, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 124, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info namespace=namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 132, in _execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 160, in execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info returncode=returncode) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding static route command with those interface in l3 agent code at dvr_fip_ns.py failing as "Nexthop has invalid gateway" The better way would be check the interface status before add the route entry at https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_fip_ns.py#L458 Stack trace: ============ 2024-03-29 08:00:19.027 5996 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. : neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/common/utils.py", line 159, in call 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1207, in process 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.process_external() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 724, in process_external 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.connect_rtr_2_fip() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 739, in connect_rtr_2_fip 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.fip_ns.create_rtr_2_fip_link(self) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 456, in create_rtr_2_fip_link 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info rtr_2_fip_dev.route.add_gateway(str(fip_2_rtr.ip), table=FIP_RT_TBL) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 609, in add_gateway 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self._as_root([ip_version], tuple(args)) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 410, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 124, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info namespace=namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 132, in _execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 160, in execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info returncode=returncode) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info
2024-06-14 09:33:39 Mohankumar description Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding static route command with those interface in l3 agent code at dvr_fip_ns.py failing as "Nexthop has invalid gateway" The better way would be check the interface status before add the route entry at https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_fip_ns.py#L458 Stack trace: ============ 2024-03-29 08:00:19.027 5996 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. : neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/common/utils.py", line 159, in call 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1207, in process 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.process_external() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 724, in process_external 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.connect_rtr_2_fip() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 739, in connect_rtr_2_fip 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.fip_ns.create_rtr_2_fip_link(self) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 456, in create_rtr_2_fip_link 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info rtr_2_fip_dev.route.add_gateway(str(fip_2_rtr.ip), table=FIP_RT_TBL) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 609, in add_gateway 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self._as_root([ip_version], tuple(args)) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 410, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 124, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info namespace=namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 132, in _execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 160, in execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info returncode=returncode) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Sometimes we see a case that rfp in qrouter namespace and fpr in fip namespace is down . which results in adding route with those interface in l3 agent code at dvr_fip_ns.py failing as "Nexthop has invalid gateway" The better way would be check the interface status before add the route entry at https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_fip_ns.py#L458 Stack trace: ============ 2024-03-29 08:00:19.027 5996 ERROR neutron.agent.linux.utils [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info [-] Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. : neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/common/utils.py", line 159, in call 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1207, in process 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.process_external() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 724, in process_external 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.connect_rtr_2_fip() 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_local_router.py", line 739, in connect_rtr_2_fip 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self.fip_ns.create_rtr_2_fip_link(self) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/l3/dvr_fip_ns.py", line 456, in create_rtr_2_fip_link 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info rtr_2_fip_dev.route.add_gateway(str(fip_2_rtr.ip), table=FIP_RT_TBL) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 609, in add_gateway 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info self._as_root([ip_version], tuple(args)) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 410, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 124, in _as_root 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info namespace=namespace) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 132, in _execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info File "/opt/openstack/neutron/lib/python3.6/site-packages/neutron/agent/linux/utils.py", line 160, in execute 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info returncode=returncode) 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Stdin: ; Stdout: ; Stderr: Error: Nexthop has invalid gateway. 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info 2024-03-29 08:00:19.028 5996 ERROR neutron.agent.l3.router_info
2024-06-14 13:32:00 Oleg Bondarev tags l3-dvr-backlog
2024-06-14 13:34:42 Oleg Bondarev neutron: status New Incomplete
2024-06-18 13:49:05 Mohankumar neutron: status Incomplete New
2024-06-18 14:52:14 Brian Haley bug added subscriber Brian Haley