Missing Binary Commands don't log in a sane way

[Matthew Treinish]

When running neutron in an environment with a missing binary command run by a linux agent the failure mode is far from clear. For example:

http://paste.openstack.org/show/604877/

Is what was logged on my system when I didn't have ipset installed. It's not clear at all from this error message or stack trace that was the cause of failure. The only way I was able to figure this out was to switch the default rootwrap-daemon config file to DEBUG level and use syslog to actually see the missing executable error message.

You shouldn't need to modify the default log levels to see the cause of a fatal error like a missing binary command.

[Brian Haley (brian-haley)]

Seems execute_rootwrap_daemon() should catch the error and print something useful at a higher log level, seems doable.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/453278

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/453278
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0476ad641dffb4cdcb6fb72e9dd6579ffc21b754
Submitter: Jenkins
Branch: master

commit 0476ad641dffb4cdcb6fb72e9dd6579ffc21b754
Author: Brian Haley <email address hidden>
Date: Tue Apr 4 13:54:42 2017 -0400

    Print useful error on rootwrap daemon failure

    If the rootwrap daemon fails to execute a command, it
    generates a cryptic message:

    Unserializable message: ('#ERROR', ValueError('I/O operation on closed file',))

    We should at least log the command that we were trying
    to run, which will help users figure out why it failed.

    Change-Id: I2c94e5a226630432028351f8287868f4fe5d2fa1
    Closes-bug: #1677742

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/455323

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/455324

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/455324
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4143a394cb71612ed49e5db409815540257e8f13
Submitter: Jenkins
Branch: stable/newton

commit 4143a394cb71612ed49e5db409815540257e8f13
Author: Brian Haley <email address hidden>
Date: Tue Apr 4 13:54:42 2017 -0400

    Print useful error on rootwrap daemon failure

    If the rootwrap daemon fails to execute a command, it
    generates a cryptic message:

    Unserializable message: ('#ERROR', ValueError('I/O operation on closed file',))

    We should at least log the command that we were trying
    to run, which will help users figure out why it failed.

    Change-Id: I2c94e5a226630432028351f8287868f4fe5d2fa1
    Closes-bug: #1677742
    (cherry picked from commit 0476ad641dffb4cdcb6fb72e9dd6579ffc21b754)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/455323
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=bfc428edb6ba7d94b7b087a52ddb28b354c7f9af
Submitter: Jenkins
Branch: stable/ocata

commit bfc428edb6ba7d94b7b087a52ddb28b354c7f9af
Author: Brian Haley <email address hidden>
Date: Tue Apr 4 13:54:42 2017 -0400

    Print useful error on rootwrap daemon failure

    If the rootwrap daemon fails to execute a command, it
    generates a cryptic message:

    Unserializable message: ('#ERROR', ValueError('I/O operation on closed file',))

    We should at least log the command that we were trying
    to run, which will help users figure out why it failed.

    Change-Id: I2c94e5a226630432028351f8287868f4fe5d2fa1
    Closes-bug: #1677742
    (cherry picked from commit 0476ad641dffb4cdcb6fb72e9dd6579ffc21b754)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 9.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.2 release.