STX-Openstack: privsep helper command exiting with non-zero code

[Lucas de Ataides Barreto]

Brief Description
-----------------
The neutron-netns-cleanup-cron-default pod is filled with log messages related to privsep: "Cleaning network namespaces caught an exception privsep helper command exited non zero (1)"

Severity
--------
Minor: System/Feature is usable with minor issue

Steps to Reproduce
------------------
1. Bootstrap a system with STX Master
2. Apply STX-Openstack Master
3. Observe neutron-netns-cleanup-cron-default logs with: kubectl -n openstack logs pod/neutron-netns-cleanup-cron-default-<id>

Expected Behavior
------------------
neutron-netns-cleanup-cron-default runs without errors.

Actual Behavior
----------------
neutron-netns-cleanup-cron-default runs with privsep errors.

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
Bare metal AIO-DX

Branch/Pull Time/Commit
-----------------------
starlingx/master/debian/monolithic/20230508T060000Z/

Timestamp/Logs
--------------
neutron@controller-0:~$ python -c 'from neutron.cmd.netns_cleanup import main; main()' --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/dhcp_agent.ini --config-file /etc/neutron/l3_agent.ini
/var/lib/openstack/lib/python3.9/site-packages/Crypto/Random/Fortuna/FortunaGenerator.py:28: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if sys.version_info[0] is 2 and sys.version_info[1] is 1:
/var/lib/openstack/lib/python3.9/site-packages/Crypto/Random/Fortuna/FortunaGenerator.py:28: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if sys.version_info[0] is 2 and sys.version_info[1] is 1:
/var/lib/openstack/lib/python3.9/site-packages/os_ken/lib/packet/cfm.py:271: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  assert interval is not 0
2023-05-16 22:31:58.315 251812 INFO neutron.common.config [-] Logging enabled!
2023-05-16 22:31:58.315 251812 INFO neutron.common.config [-] -c version 16.4.3.dev216
2023-05-16 22:31:58.332 251812 CRITICAL neutron [-] Unhandled error: oslo_privsep.daemon.FailedToDropPrivileges: privsep helper command exited non-zero (1)
2023-05-16 22:31:58.332 251812 ERROR neutron Traceback (most recent call last):
2023-05-16 22:31:58.332 251812 ERROR neutron File "<string>", line 1, in <module>
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/neutron/cmd/netns_cleanup.py", line 272, in main
2023-05-16 22:31:58.332 251812 ERROR neutron cleanup_network_namespaces(conf)
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/neutron/cmd/netns_cleanup.py", line 241, in cleanup_network_namespaces
2023-05-16 22:31:58.332 251812 ERROR neutron ip_lib.list_network_namespaces()
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/neutron/agent/linux/ip_lib.py", line 942, in list_network_namespaces
2023-05-16 22:31:58.332 251812 ERROR neutron return privileged.list_netns(**kwargs)
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/oslo_privsep/priv_context.py", line 246, in _wrap
2023-05-16 22:31:58.332 251812 ERROR neutron self.start()
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/oslo_privsep/priv_context.py", line 258, in start
2023-05-16 22:31:58.332 251812 ERROR neutron channel = daemon.RootwrapClientChannel(context=self)
2023-05-16 22:31:58.332 251812 ERROR neutron File "/var/lib/openstack/lib/python3.9/site-packages/oslo_privsep/daemon.py", line 367, in __init__
2023-05-16 22:31:58.332 251812 ERROR neutron raise FailedToDropPrivileges(msg)
2023-05-16 22:31:58.332 251812 ERROR neutron oslo_privsep.daemon.FailedToDropPrivileges: privsep helper command exited non-zero (1)
2023-05-16 22:31:58.332 251812 ERROR neutron

Test Activity
-------------
Feature Testing

Workaround
----------
N/A

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/openstack-armada-app/+/883679

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/openstack-armada-app/+/883679
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/abea418af5ec4592afec4b272715dbcbc0f66cb6
Submitter: "Zuul (22348)"
Branch: master

commit abea418af5ec4592afec4b272715dbcbc0f66cb6
Author: Lucas de Ataides <email address hidden>
Date: Fri May 19 12:11:10 2023 -0300

    Add runAsUser for neutron_netns_cleanup_cron pod

    The current user for the neutron_netns_cleanup_cron pod does not have
    permissions to execute necessary commands, and is causing several error
    logs. This is also true for other neutron pods as well, but by the time
    that [1] was included in the starlingx/openstack-armada-app, this
    neutron_netns_cleanup_cron pod didn't exist yet [2], and was not
    included in the change.

    This change adds the runAsUser as root (0) to the
    neutron_netns_cleanup_cron pod, so it runs without errors.

    [1] https://opendev.org/starlingx/openstack-armada-app/commit/982deda6e7732a070c824b26c3817206db8605d0
    [2] https://opendev.org/starlingx/openstack-armada-app/src/commit/982deda6e7732a070c824b26c3817206db8605d0/stx-openstack-helm/stx-openstack-helm/manifests/manifest.yaml#L1273

    Test plan:
    PASS - Build openstack-helm package
    PASS - Build stx-openstack-helm-fluxcd package
    PASS - Build stx-openstack helm charts
    PASS - Update/apply stx-openstack
    PASS - kubectl logs in the neutron-netns-cleanup pod and verify
           that no privsep errors are present
    PASS - Remove/delete stx-openstack

    Closes-Bug: 2020205

    Change-Id: Ibd93dacea87d1294b3e2704be2e0f452a5cfc275
    Signed-off-by: Lucas de Ataides <email address hidden>