Neutron DHCP agent unable to read TLS key/certificate due to neutron UID mismatch

[Tim Rozet]

When deploying with TLS and OpenDaylight, neutron dhcp agent is configured with TLS certificate/key in order to be able to communicate with OVSDB (listening in passive ssl). However, neutron dhcp agent fails to add the dhcp tap port to OVSDB because it cannot read the key/certificate. The reason for this bug is because the key and certificate are generated on the host with the uid of neutron on the host. They are then mounted into the container. However, the UID of neutron in the container is not the same as the UID of the host. The neutron packaging distgit spec does not specify a unique UID.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/556673

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/557469

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/557469
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=fe09335418b45f6963645c018232530d517b2659
Submitter: Zuul
Branch: master

commit fe09335418b45f6963645c018232530d517b2659
Author: Tim Rozet <email address hidden>
Date: Wed Mar 28 11:27:02 2018 -0700

    Removes neutron ownership of certs

    Since neutron UID is not static, setting the owners on the certificates
    in the host to be 'neutron' will not match the UID for neutron in the
    deployed container. Therefore this patch removes the host neutron
    ownership and leaves it as root, so that it can be later modified in the
    container to be chowned to neutron.

    Partial-Bug: 1759049

    Change-Id: I83b14b91d1ee600bd9d5863acba34303921368ce
    Signed-off-by: Tim Rozet <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/556673
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=16731819c5bb92d9bb33c6fb8086a6a776bdef8c
Submitter: Zuul
Branch: master

commit 16731819c5bb92d9bb33c6fb8086a6a776bdef8c
Author: Tim Rozet <email address hidden>
Date: Mon Mar 26 15:15:56 2018 -0700

    Fixes Neutron certificate and key permissions

    The Neutron UID is not static and may be different between the host and
    neutron container. Since we generate certificates and keys on the host
    for neutron and then mount them in a container, it is highly likely the
    container Neutron UID will not match the one used on the host to
    generate the files and reading these files will fail in the container.

    This patch modifies the permissions after the files are mounted in the
    container to be owned by the correct Neutron UID.

    Closes-Bug: 1759049

    Depends-On: I83b14b91d1ee600bd9d5863acba34303921368ce

    Change-Id: Ibad3f1af4b44459e96a6dc9937e5fcef3e6335f4
    Signed-off-by: Tim Rozet <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/558664

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/558667

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/558664
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=af2c34c202a9e272828d372009b95705dbd25296
Submitter: Zuul
Branch: stable/queens

commit af2c34c202a9e272828d372009b95705dbd25296
Author: Tim Rozet <email address hidden>
Date: Wed Mar 28 11:27:02 2018 -0700

    Removes neutron ownership of certs

    Since neutron UID is not static, setting the owners on the certificates
    in the host to be 'neutron' will not match the UID for neutron in the
    deployed container. Therefore this patch removes the host neutron
    ownership and leaves it as root, so that it can be later modified in the
    container to be chowned to neutron.

    Partial-Bug: 1759049

    Change-Id: I83b14b91d1ee600bd9d5863acba34303921368ce
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit fe09335418b45f6963645c018232530d517b2659)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/558667
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=7299ad4875963a97d128aa180b46503ebd42e247
Submitter: Zuul
Branch: stable/queens

commit 7299ad4875963a97d128aa180b46503ebd42e247
Author: Tim Rozet <email address hidden>
Date: Mon Mar 26 15:15:56 2018 -0700

    Fixes Neutron certificate and key permissions

    The Neutron UID is not static and may be different between the host and
    neutron container. Since we generate certificates and keys on the host
    for neutron and then mount them in a container, it is highly likely the
    container Neutron UID will not match the one used on the host to
    generate the files and reading these files will fail in the container.

    This patch modifies the permissions after the files are mounted in the
    container to be owned by the correct Neutron UID.

    Closes-Bug: 1759049

    Depends-On: I83b14b91d1ee600bd9d5863acba34303921368ce

    Change-Id: Ibad3f1af4b44459e96a6dc9937e5fcef3e6335f4
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit 16731819c5bb92d9bb33c6fb8086a6a776bdef8c)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.2 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/564614

[Tim Rozet (trozet)]

Previous fix didn't work because the files were set to RO so they could not be chowned:
ERROR:__main__:Failed to change ownership of /etc/pki/tls/certs/neutron.crt to 42435:42435
Traceback (most recent call last):
  File "/usr/local/bin/kolla_set_configs", line 345, in set_perms
    os.chown(path, uid, gid)
OSError: [Errno 30] Read-only file system: '/etc/pki/tls/certs/neutron.crt'
INFO:__main__:Setting permission for /etc/pki/tls/private/neutron.key
ERROR:__main__:Failed to change ownership of /etc/pki/tls/private/neutron.key to 42435:42435
Traceback (most recent call last):
  File "/usr/local/bin/kolla_set_configs", line 345, in set_perms
    os.chown(path, uid, gid)
OSError: [Errno 30] Read-only file system: '/etc/pki/tls/private/neutron.key'

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/564614
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=542ec3590fb8e5dac44c3efb05f40706cedd0079
Submitter: Zuul
Branch: master

commit 542ec3590fb8e5dac44c3efb05f40706cedd0079
Author: Tim Rozet <email address hidden>
Date: Thu Apr 26 16:03:16 2018 -0400

    Fixes chowning neutron cert/key perms

    Modifyign the certificate and key permissions for neutron was failing
    during kolla start because the files were mounted as read only in the
    container.

    Related-Bug: 1759049

    Change-Id: I99ccea35edb39ed98b537eb7f7947f1c957d79f9
    Signed-off-by: Tim Rozet <email address hidden>

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/565011

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/565185

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565011
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=ebc2a67f7a2f26ae20f20933b6a7e8cd2d826b9f
Submitter: Zuul
Branch: stable/queens

commit ebc2a67f7a2f26ae20f20933b6a7e8cd2d826b9f
Author: Tim Rozet <email address hidden>
Date: Thu Apr 26 16:03:16 2018 -0400

    Fixes chowning neutron cert/key perms

    Modifyign the certificate and key permissions for neutron was failing
    during kolla start because the files were mounted as read only in the
    container.

    Related-Bug: 1759049

    Change-Id: I99ccea35edb39ed98b537eb7f7947f1c957d79f9
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit 542ec3590fb8e5dac44c3efb05f40706cedd0079)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565185
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=bce345210475a49aa49ac881982afbfa517868dd
Submitter: Zuul
Branch: master

commit bce345210475a49aa49ac881982afbfa517868dd
Author: Bogdan Dobrelya <email address hidden>
Date: Mon Apr 30 12:46:42 2018 +0200

    Copy-in neutron cert via kolla extended/start

    Instead of bind-mounting in RW mode, follow the established
    approach for ditributing certificates in containers.

    Related-Bug: #1759049
    Partial-Bug: #1767998

    Change-Id: I6bcb72b8b600b6b1d916b64c161bca22c802cf07
    Signed-off-by: Bogdan Dobrelya <email address hidden>