nova and cinder containers throw sudo errors

[Michele Baldessari]

Deploy from today:
Aug 16 09:53:57 controller-0.alejandro.ftw sudo[148918]: cinder : command not allowed ; TTY=unknown ; PWD=/ ; USER=cinder ; COMMAND=/usr/bin/find /proc/8/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149072]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/8/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149075]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/32/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149078]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/33/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149205]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/6/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149209]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/30/fd/ -ilname socket* -printf %l\n
Aug 16 09:53:59 controller-0.alejandro.ftw sudo[149212]: nova : command not allowed ; TTY=unknown ; PWD=/ ; USER=nova ; COMMAND=/usr/bin/find /proc/31/fd/ -ilname socket* -printf %l\n

[Bogdan Dobrelya (bogdando)]

that spoils the healtchchecks work in it

[Bogdan Dobrelya (bogdando)]

A regression after https://review.opendev.org/c/openstack/tripleo-common/+/785004

[Bogdan Dobrelya (bogdando)]

We could approach fixing that via rootwrap filters based healthcheck command, like I prototyped earlier in https://review.opendev.org/c/openstack/tripleo-heat-templates/+/693369

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/puppet-tripleo/+/804702

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804706

[Cédric Jeanneret (cjeanner)]

Quick history:
- first, podman healthchecks were launched from a dedicated systemd unit, and were launched as "root" (podman exec --user root)
- now, it's using the native thing, and I suspect it's using the actual USER set in the Dockerfile (hence, "nova", "cinder", and so on)

The current way sudo is called is in order to allow the root user to run the find as the actual container user.

Soooo... I think the easiest way to correct this issue is just to drop the "sudo -u $puser"... Would that make any sense, Bogdan?

[Cédric Jeanneret (cjeanner)]

I think it's a win:

[root@oc0-controller-0 ~]# podman healthcheck run heat_engine
unhealthy
[root@oc0-controller-0 ~]# podman exec -u root -ti heat_engine bash
[root@oc0-controller-0 /]# vi /usr/share/openstack-tripleo-common/healthcheck/common.sh
## Here, I edit the file to remove "sudo -u $puser"
[root@oc0-controller-0 /]# exit
[root@oc0-controller-0 ~]# podman healthcheck run heat_engine
healthy

Running a "journalctl -f | grep sudo" in parallel shows the error for the first run:
Aug 17 09:17:04 oc0-controller-0 sudo[365895]: heat : command not allowed ; TTY=unknown ; PWD=/ ; USER=heat ; COMMAND=/usr/bin/find /proc/8/fd/ -ilname socket* -printf %l\n

Once I remove that sudo call, it's clean. And healthy.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804838

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Bogdan Dobrelya <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804706
Reason: sudo can be removed instead

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Bogdan Dobrelya <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/puppet-tripleo/+/804702
Reason: sudo can be removed instead

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/804838
Committed: https://opendev.org/openstack/tripleo-common/commit/36a7a4d093470ffe5d970cc288da292714b48f2a
Submitter: "Zuul (22348)"
Branch: master

commit 36a7a4d093470ffe5d970cc288da292714b48f2a
Author: Cédric Jeanneret <email address hidden>
Date: Tue Aug 17 11:51:20 2021 +0200

    Remove now useless sudo call for port check

    Back then, the "sudo" was needed because healthchecks were running as
    root, using a paunch created systemd unit/timer calling "podman exec
    --user root".

    Now that we're using native healthchecks, we can get rid of the sudo
    call, since it's already using as the intended user.

    Closes-Bug: #1940068

    Change-Id: I7eaabc91005952aa956456944c46224f08f4c7f8

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804863

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804864

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804865

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804866

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Cedric Jeanneret <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804866
Reason: no need for train

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Cedric Jeanneret <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804865
Reason: no need for ussuri

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Cedric Jeanneret <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-common/+/804864
Reason: no need for victoria

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/804863
Committed: https://opendev.org/openstack/tripleo-common/commit/69e5827ec6adfb0e436933bc8e03885f6254b098
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 69e5827ec6adfb0e436933bc8e03885f6254b098
Author: Cédric Jeanneret <email address hidden>
Date: Tue Aug 17 11:51:20 2021 +0200

    Remove now useless sudo call for port check

    Back then, the "sudo" was needed because healthchecks were running as
    root, using a paunch created systemd unit/timer calling "podman exec
    --user root".

    Now that we're using native healthchecks, we can get rid of the sudo
    call, since it's already using as the intended user.

    Closes-Bug: #1940068

    Change-Id: I7eaabc91005952aa956456944c46224f08f4c7f8
    (cherry picked from commit 36a7a4d093470ffe5d970cc288da292714b48f2a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 16.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 15.4.0 release.