TripleO deploys ceph client keyring with 644 permissions

[John Fulton]

When following steps to deploy OpenStack with Ceph, either an external ceph cluster or one deployed by TripleO:

 https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ceph_config.html
 https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ceph_external.html

The ceph.client.openstack.keyring permissions is world readable in all overcloud nodes.

  [root@overcloud-compute-1 ~]# ls -ltZ /etc/ceph/
  -rw-r--r--. root root system_u:object_r:etc_t:s0 ceph.conf
  -rw-------. root root system_u:object_r:etc_t:s0 ceph.client.admin.keyring
  -rw-r--r--. root root system_u:object_r:etc_t:s0 ceph.client.openstack.keyring <-- All user can read this keyring.
  -rwxr-xr-x. root root unconfined_u:object_r:etc_t:s0 rbdmap

This means that any local user can access to ceph cluster as 'client.openstack'. Though there is no remote exploit and if there is local access to overcloud nodes there is a bigger problem, the permissions on this key could be more restrictive; e.g. 600 with ACLs set to allow users like Nova, Cinder, etc to read they key.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/508975

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/509020

[John Fulton (jfulton-org)]

This bug requires different fixes depending on if the deployment is containerized or not.

Containerized deployment:
1. have kolla copy in the key files so that they are owned by the user in each container, e.g. nova, cinder, etc and chmod 600
- https://review.openstack.org/#/c/508975/

Non-containerized deployemtn:
2. update THT to indicate that the keys should be chmod 600 but also set ACLs for each user; e.g. nova, cinder, etc
- https://review.openstack.org/#/c/509020/

3. update puppet-ceph to that it supports ACLs
- https://review.openstack.org/#/c/509021/

The above are WIPs under revision

[John Fulton (jfulton-org)]

Here's an alternative to the puppet-ceph change that Keith came up with https://review.openstack.org/#/c/517347

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/517347
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=a4d12e02a7a44116aa8c7a327e14a09522a54b57
Submitter: Zuul
Branch: master

commit a4d12e02a7a44116aa8c7a327e14a09522a54b57
Author: Keith Schincke <email address hidden>
Date: Thu Nov 2 11:19:31 2017 -0400

    Set ACLs on ceph client keyrings

    This patch will set file system ACLs on the ceph client keyring.
    This will help resolve (1) for OSP Ocata and before

    Change-Id: I353b19a5a3f9a9af110587bd0996f08700335a44
    Partial-Bug: #1720787
    1: https://bugzilla.redhat.com/show_bug.cgi?id=1462657

[Alex Schultz (alex-schultz)]

The previous patch had to be reverted due to a typo. https://review.openstack.org/519041

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/519531

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/508975
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=ce7b65f443d38a6627631f53cb22336338e97d30
Submitter: Zuul
Branch: master

commit ce7b65f443d38a6627631f53cb22336338e97d30
Author: John Fulton <email address hidden>
Date: Mon Oct 2 15:22:08 2017 -0400

    Set file mode permission for Ceph keyrings in containers

    Pass mode parameter to ceph-ansible for Ceph keyrings on container
    host. Pass mode and ownership parameter to each Ceph client container
    using kolla_config. ACLs are set for Cinder if it is not running in
    containers.

    Change-Id: I11618b3fd696739ad9b86618a1f3f96570c61a30
    Partial-Bug: #1720787

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/522024

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/522024
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a18fd59077d97de83496c85c017b9d256a3eddd4
Submitter: Zuul
Branch: stable/pike

commit a18fd59077d97de83496c85c017b9d256a3eddd4
Author: John Fulton <email address hidden>
Date: Mon Oct 2 15:22:08 2017 -0400

    Set file mode permission for Ceph keyrings in containers

    Pass mode parameter to ceph-ansible for Ceph keyrings on container
    host. Pass mode and ownership parameter to each Ceph client container
    using kolla_config. ACLs are set for Cinder if it is not running in
    containers.

    Unclean cherry pick from ce7b65f443d38a6627631f53cb22336338e97d30

    Change-Id: I11618b3fd696739ad9b86618a1f3f96570c61a30
    Partial-Bug: #1720787

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519531
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=48c417519f88472d035c3ad6a92edcc2e6039d9b
Submitter: Zuul
Branch: master

commit 48c417519f88472d035c3ad6a92edcc2e6039d9b
Author: Keith Schincke <email address hidden>
Date: Mon Nov 13 23:15:21 2017 -0500

    Update cephx keys with ACLs for openstack services.

    This patch will set file system ACLs on the ceph client keyring.
    This will help resolve (1) for OSP Ocata and before

    Change-Id: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787
    1: https://bugzilla.redhat.com/show_bug.cgi?id=1462657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/509020
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=bdf1ade1b9de994c3098fb17cca6dd64b3d14cd5
Submitter: Zuul
Branch: master

commit bdf1ade1b9de994c3098fb17cca6dd64b3d14cd5
Author: John Fulton <email address hidden>
Date: Tue Oct 3 00:21:57 2017 +0000

    Set restrictive file permissions on Ceph keyrings for non-containerized deployment

    Pass mode parameter 0600 and user and group ownership to puppet-ceph for
    Ceph openstack client keyrings during non-containerized deployment.

    Author: Keith Schincke <email address hidden>
    Co-Author: John Fulton <email address hidden>
    Change-Id: Iccb24f5c2ee639ad2bc0869a37cec305f32b9fd1
    Depends-On: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/527969

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/527970

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/528014

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/527969
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=3357c38248f1894a9388cb6d8962ded9e216ec02
Submitter: Zuul
Branch: stable/pike

commit 3357c38248f1894a9388cb6d8962ded9e216ec02
Author: Keith Schincke <email address hidden>
Date: Mon Nov 13 23:15:21 2017 -0500

    Update cephx keys with ACLs for openstack services.

    This patch will set file system ACLs on the ceph client keyring.
    This will help resolve (1) for OSP Ocata and before

    Change-Id: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787
    1: https://bugzilla.redhat.com/show_bug.cgi?id=1462657
    (cherry picked from commit 48c417519f88472d035c3ad6a92edcc2e6039d9b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/527970
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=1e52acae86cfccea181eace94677507158b7eab3
Submitter: Zuul
Branch: stable/pike

commit 1e52acae86cfccea181eace94677507158b7eab3
Author: John Fulton <email address hidden>
Date: Tue Oct 3 00:21:57 2017 +0000

    Set restrictive file permissions on Ceph keyrings for non-containerized deployment

    Pass mode parameter 0600 and user and group ownership to puppet-ceph for
    Ceph openstack client keyrings during non-containerized deployment.

    Author: Keith Schincke <email address hidden>
    Co-Author: John Fulton <email address hidden>
    Change-Id: Iccb24f5c2ee639ad2bc0869a37cec305f32b9fd1
    Depends-On: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787
    (cherry picked from commit bdf1ade1b9de994c3098fb17cca6dd64b3d14cd5)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/528595

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/528599

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/532397

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/532398

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528595
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=c3bc69228084d39cfb11c4545100d8add4cddb09
Submitter: Zuul
Branch: stable/newton

commit c3bc69228084d39cfb11c4545100d8add4cddb09
Author: Keith Schincke <email address hidden>
Date: Mon Nov 13 23:15:21 2017 -0500

    Update cephx keys with ACLs for openstack services.

    This patch will set file system ACLs on the ceph client keyring.
    This will help resolve (1) for OSP Ocata and before

    Change-Id: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787
    1: https://bugzilla.redhat.com/show_bug.cgi?id=1462657
    (cherry picked from commit 48c417519f88472d035c3ad6a92edcc2e6039d9b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528014
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=5082232e0b5cc72440780898482ac475710bd3a5
Submitter: Zuul
Branch: stable/ocata

commit 5082232e0b5cc72440780898482ac475710bd3a5
Author: Keith Schincke <email address hidden>
Date: Mon Nov 13 23:15:21 2017 -0500

    Update cephx keys with ACLs for openstack services.

    This patch will set file system ACLs on the ceph client keyring.
    This will help resolve (1) for OSP Ocata and before

    Change-Id: I0c1bc3d2362c6500b1a515d99f641f8c1468754a
    Partial-Bug: #1720787
    1: https://bugzilla.redhat.com/show_bug.cgi?id=1462657
    (cherry picked from commit 48c417519f88472d035c3ad6a92edcc2e6039d9b)

[OpenStack Infra (hudson-openstack)]

Change abandoned by Giulio Fidente (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/532397
Reason: duplicate of https://review.openstack.org/#/c/528015/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Giulio Fidente (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/532398
Reason: duplicate of https://review.openstack.org/#/c/528599/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/537874

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/537889

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/538969

[OpenStack Infra (hudson-openstack)]

Change abandoned by Giulio Fidente (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/538969
Reason: this was submitted already in https://review.openstack.org/#/c/528015/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/537874
Reason: Gate resets is causing issue to land a critical patch (538346) - I need to clear the gate until 538346 is merged. Please do not restore this patch, I'll take care of it.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/537889
Reason: Gate resets is causing issue to land a critical patch (538346) - I need to clear the gate until 538346 is merged. Please do not restore this patch, I'll take care of it.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/537889
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=74760363a0995bbdd55162361519a1c5de20b4ca
Submitter: Zuul
Branch: stable/ocata

commit 74760363a0995bbdd55162361519a1c5de20b4ca
Author: Keith Schincke <email address hidden>
Date: Thu Jan 25 07:42:24 2018 -0500

    Update nova::compute::libvirt for bug 1720787

    Nova compute was missed in the backporting of the related patch.
    This resulted in the correct acls for the nova user not being applied.
    This patch adds code missed by patch
    https://review.openstack.org/#/c/517347/

    Change-Id: Ie968e6abc6969c37be0a62ac45999093120673d4
    Related-Bug: 1720787

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528015
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=8bf46a66e67c3245f72b8eea6dacf247163106bb
Submitter: Zuul
Branch: stable/ocata

commit 8bf46a66e67c3245f72b8eea6dacf247163106bb
Author: John Fulton <email address hidden>
Date: Tue Oct 3 00:21:57 2017 +0000

    Set restrictive file permissions on Ceph keyrings for non-containerized deployment

    Pass mode parameter 0640 and user and group ownership to puppet-ceph for
    Ceph openstack client keyrings during non-containerized deployment.

    Author: Keith Schincke <email address hidden>
    Co-Author: John Fulton <email address hidden>
    Change-Id: Iccb24f5c2ee639ad2bc0869a37cec305f32b9fd1
    Depends-On: Ie968e6abc6969c37be0a62ac45999093120673d4
    Partial-Bug: #1720787
    (cherry picked from commit bdf1ade1b9de994c3098fb17cca6dd64b3d14cd5)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/537874
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=cbbf5bfa8c7108f4f4b1c2db8715d1884f1a6c93
Submitter: Zuul
Branch: stable/newton

commit cbbf5bfa8c7108f4f4b1c2db8715d1884f1a6c93
Author: Keith Schincke <email address hidden>
Date: Thu Jan 25 07:06:30 2018 -0500

    Update nova::compute::libvirt for bug 1720787

    Nova compute was missed in the backporting of the related patch.
    This resulted in the correct acls for the nova user not being applied.
    This patch adds code missed in the backport of patch
    https://review.openstack.org/#/c/517347/

    Change-Id: I6f1b8f9fb410ac655c96cc9aab7f06e203cdc31b
    Related-Bug: 1720787

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528599
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=565c81a78dce5334d7105963ee3995a7b007fb14
Submitter: Zuul
Branch: stable/newton

commit 565c81a78dce5334d7105963ee3995a7b007fb14
Author: John Fulton <email address hidden>
Date: Tue Oct 3 00:21:57 2017 +0000

    Set restrictive file permissions on Ceph keyrings for non-containerized deployment

    Pass mode parameter 0640 and user and group ownership to puppet-ceph for
    Ceph openstack client keyrings during non-containerized deployment.

    Author: Keith Schincke <email address hidden>
    Co-Author: John Fulton <email address hidden>
    Change-Id: Iccb24f5c2ee639ad2bc0869a37cec305f32b9fd1
    Depends-On: I6f1b8f9fb410ac655c96cc9aab7f06e203cdc31b
    Partial-Bug: #1720787
    (cherry picked from commit bdf1ade1b9de994c3098fb17cca6dd64b3d14cd5)

[Alex Schultz (alex-schultz)]

I think all the patches have been accounted for. Feel free to reopen if any where missed.