sshd running inside nova_migration_target overrides /var/run/sshd.pid on host with pid 1

Bug #1830982 reported by Takashi Kajinami
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Takashi Kajinami

Bug Description

Now we have 2 sshd instances running on compute nodes.
 1. sshd running on host, to use conventional operation over remote login
 2. sshd running inside nova_migration_target, used for migration in nova

These two instances generally use separated resources, like ports, or conf files, but they are sharing pid, /var/run/sshd.pid on the host.

This causes that we have "1" in /var/run/sshd.pid, as pid file is overwritten by the sshd running inside nova_migration_target container with pid 1 inside the container.

Note that currently we do not see any specific problem caused by this, except for the below error log shown when we restart sshd running on host.
~~~
[heat-admin@compute-0 ~]$ sudo systemctl restart sshd
[heat-admin@compute-0 ~]$ sudo systemctl status sshd
● sshd.service - OpenSSH server daemon
...
May 28 23:31:45 compute-0 sshd[422824]: error: Couldn't create pid file "/var/run/sshd.pid": Permission denied
~~~

description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/662109

Changed in tripleo:
assignee: nobody → Takashi Kajinami (kajinamit)
status: New → In Progress
Changed in tripleo:
importance: Undecided → High
milestone: none → train-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/662109
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b4223ead2d1ed7e07cdfd5ceac52f43eb29e0f71
Submitter: Zuul
Branch: master

commit b4223ead2d1ed7e07cdfd5ceac52f43eb29e0f71
Author: Takashi Kajinami <email address hidden>
Date: Thu May 30 08:09:27 2019 +0900

    Do not bind /run on host to nova_migration_target

    Now we have 2 sshd instances running on compute nodes, (1)sshd running
    on host, and (2)sshd running inside nova_migration_target.
    Because we bind /run on host to nova_mgiration_target, these two prcesses
    share the same pid file, /var/run/sshd.pid , which causes that we have
    pid 1 in that pid file.

    As we do not require sshd running on the container to have access to
    host pid files, this patch removes bind about /run about the container,
    to avoid the overwriting problem.

    Change-Id: I71cb64997991a31b1b87bf73aa4109c355a90708
    Closes-bug: #1830982

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/663147

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/663148

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/663149

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/stein)

Reviewed: https://review.opendev.org/663147
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=5b1ba9846b68545751b930c7c4a65e97845d2e3d
Submitter: Zuul
Branch: stable/stein

commit 5b1ba9846b68545751b930c7c4a65e97845d2e3d
Author: Takashi Kajinami <email address hidden>
Date: Thu May 30 08:09:27 2019 +0900

    Do not bind /run on host to nova_migration_target

    Now we have 2 sshd instances running on compute nodes, (1)sshd running
    on host, and (2)sshd running inside nova_migration_target.
    Because we bind /run on host to nova_mgiration_target, these two prcesses
    share the same pid file, /var/run/sshd.pid , which causes that we have
    pid 1 in that pid file.

    As we do not require sshd running on the container to have access to
    host pid files, this patch removes bind about /run about the container,
    to avoid the overwriting problem.

    Change-Id: I71cb64997991a31b1b87bf73aa4109c355a90708
    Closes-bug: #1830982
    (cherry picked from commit b4223ead2d1ed7e07cdfd5ceac52f43eb29e0f71)

tags: added: in-stable-stein
tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/rocky)

Reviewed: https://review.opendev.org/663148
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=7329ca67a9f771e84458873c2ef2dc5adff2b674
Submitter: Zuul
Branch: stable/rocky

commit 7329ca67a9f771e84458873c2ef2dc5adff2b674
Author: Takashi Kajinami <email address hidden>
Date: Thu May 30 08:09:27 2019 +0900

    Do not bind /run on host to nova_migration_target

    Now we have 2 sshd instances running on compute nodes, (1)sshd running
    on host, and (2)sshd running inside nova_migration_target.
    Because we bind /run on host to nova_mgiration_target, these two prcesses
    share the same pid file, /var/run/sshd.pid , which causes that we have
    pid 1 in that pid file.

    As we do not require sshd running on the container to have access to
    host pid files, this patch removes bind about /run about the container,
    to avoid the overwriting problem.

    Change-Id: I71cb64997991a31b1b87bf73aa4109c355a90708
    Closes-bug: #1830982
    (cherry picked from commit b4223ead2d1ed7e07cdfd5ceac52f43eb29e0f71)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/queens)

Reviewed: https://review.opendev.org/663149
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=43b074c10955c424aa5918b305758f55d6eb384a
Submitter: Zuul
Branch: stable/queens

commit 43b074c10955c424aa5918b305758f55d6eb384a
Author: Takashi Kajinami <email address hidden>
Date: Thu May 30 08:09:27 2019 +0900

    Do not bind /run on host to nova_migration_target

    Now we have 2 sshd instances running on compute nodes, (1)sshd running
    on host, and (2)sshd running inside nova_migration_target.
    Because we bind /run on host to nova_mgiration_target, these two prcesses
    share the same pid file, /var/run/sshd.pid , which causes that we have
    pid 1 in that pid file.

    As we do not require sshd running on the container to have access to
    host pid files, this patch removes bind about /run about the container,
    to avoid the overwriting problem.

    Change-Id: I71cb64997991a31b1b87bf73aa4109c355a90708
    Closes-bug: #1830982
    (cherry picked from commit b4223ead2d1ed7e07cdfd5ceac52f43eb29e0f71)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.0.0

This issue was fixed in the openstack/tripleo-heat-templates 11.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 9.4.0

This issue was fixed in the openstack/tripleo-heat-templates 9.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 8.4.0

This issue was fixed in the openstack/tripleo-heat-templates 8.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 10.6.0

This issue was fixed in the openstack/tripleo-heat-templates 10.6.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.