For neutron-l3-agent, after the execution of the linux command fails, it is not displayed which command failed to execute

Bug #1893627 reported by boluan ace
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Wishlist
boluan ace

Bug Description

For neutron-l3-agent, there are many scenarios where linux commands must be executed. When a linux command execution fails and an exception occurs, the exception information only shows the reason for the error, but it does not show which command was executed incorrectly, so it is difficult to locate the cause of the error.

For example, neutron-l3-agent fails to set floating ip, the error message is as follows:

2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info [req-23d047f9-97bb-4d37-8509-ee5658ef241f - ddcde2e097774530a2180da53c1afa23 - - -] L3 agent failure to setup floating IPs: ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Device "qg-c9ff34e2-20" does not exist.
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 432, in configure_fip_addresses
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info return self.process_floating_ip_addresses(interface_name)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 372, in process_floating_ip_addresses
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info existing_cidrs = self.get_router_cidrs(device)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 353, in get_router_cidrs
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info return set([addr['cidr'] for addr in device.addr.list()])
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 679, in list
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info self.name, scope, to, filters, ip_version)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 644, in get_devices_with_ip
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info for line in self._run(options, tuple(args)).split('\n'):
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 379, in _run
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info return self._parent._run(options, self.COMMAND, args)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 89, in _run
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info return self._as_root(options, command, args)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 101, in _as_root
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info namespace=namespace)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 109, in _execute
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 149, in execute
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info returncode=returncode)
2020-08-28 11:06:50.090 13 ERROR neutron.agent.l3.router_info ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Device "qg-c9ff34e2-20" does not exist.

But it is impossible to know which command execution error is, so it is difficult to troubleshoot the problem

Our expectation is similar to ProcessExecutionError: Exit code: 1; Cmd: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-648a4b83-b900-4b34-a01d-f9c7fd3f443d', 'ip', 'addr', 'show', 'qg-c9ff34e2-20']; Stdin: ; Stdout: ; Stderr: Device "qg-c9ff34e2-20" does not exist.

boluan ace (boluanace)
summary: - Command detail is lost when command execution fails
+ For neutron-l3-agent, after the execution of the linux command fails, it
+ is not displayed which command failed to execute
Changed in neutron:
assignee: nobody → boluan ace (boluanace)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/749076

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → Wishlist
tags: added: low-hanging-fruit
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/749076
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7f494fa3ae5f456a66bd4ec2a9d9f67c60df2d86
Submitter: Zuul
Branch: master

commit 7f494fa3ae5f456a66bd4ec2a9d9f67c60df2d86
Author: boluanace <email address hidden>
Date: Mon Aug 31 07:56:17 2020 -0700

    Show cmd detail when execute linux command failed

    For neutron-l3-agent, after the execution of the linux command fails,
    display which command failed to execute

    Change-Id: Ia237e1b18e02c9da3ae9ee94e4244828e5c207b2
    Closes-Bug: #1893627

Changed in neutron:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.