Magnum fails to contact services if internal TLS is enabled

[Kyle Dean]

**Environment**:
* OS Ubuntu
* Kernel 5.4.0-66
* Kolla-Ansible version: stable/victoria
* Docker image Install type: source
* Docker image distribution: ubuntu
* Are you using official images from Docker Hub or self built: self built
* Kolla version and environment used to build: stable/victoria

magnum could not contact various services when kolla_enable_tls_backend: "yes"

I had to make the following changes in the magnum.conf to fix the issue.

[magnum_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[heat_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[octavia_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[cinder_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[barbican_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[glance_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[neutron_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[nova_client]
ca_file = /etc/ssl/certs/ca-certificates.crt

[keystone_authtoken]
cafile = /etc/ssl/certs/ca-certificates.crt

[docker]
ca_file = /etc/ssl/certs/ca-certificates.crt

[drivers]

openstack_ca_file = /etc/ssl/certs/ca-certificates.crt

Kind regards,

K

[Mark Goddard (mgoddard)]

Proposed a fix. Note the comment about which options are not included. https://review.opendev.org/c/openstack/kolla-ansible/+/781062

[Kyle Dean (k.s-dean)]

In regards to the custom configuration for docker and drivers.

If the CA is not specified for drivers, magnum is not able to connect to the fedora-coreos container to provision k8s. I believe that was the issue and is why i provided those configuration options.

Either way it should be documented.

[Mark Goddard (mgoddard)]

It's documented here for drivers: https://docs.openstack.org/kolla-ansible/latest/reference/containers/magnum-guide.html#optional-private-ca

[Kyle Dean (k.s-dean)]

So it is, I must have missed that page. Cheers.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/781062
Committed: https://opendev.org/openstack/kolla-ansible/commit/48f0957a1c9c3ef0cac77642f058fa9cb47fbbc8
Submitter: "Zuul (22348)"
Branch: master

commit 48f0957a1c9c3ef0cac77642f058fa9cb47fbbc8
Author: Mark Goddard <email address hidden>
Date: Wed Mar 17 09:32:33 2021 +0000

    magnum: Add CA certificate configuration for internal TLS

    Magnum has various sections in its configuration file for OpenStack
    clients. When internal TLS is enabled, these may need a CA certificate
    to be specified.

    This change adds a CA certificate configuration, based on
    openstack_cacert, for all clients using internal endpoints.

    Note: we are explicitly not adding the configuration for the
    [magnum_client] ca_file and [drivers] openstack_ca_file options, since
    these use the public endpoint by default. These options may be
    provided via custom configuration if necessary.

    Change-Id: Ie59b3777c0a2c142b580addd67e279bc4b2f2c90
    Co-Authored-By: Kyle Dean
    Closes-Bug: #1919389

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/798893

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/798894

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/798895

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/798893
Committed: https://opendev.org/openstack/kolla-ansible/commit/270b237b379ee23ce90fb1fd3a2533a4e5b6b9b9
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 270b237b379ee23ce90fb1fd3a2533a4e5b6b9b9
Author: Mark Goddard <email address hidden>
Date: Wed Mar 17 09:32:33 2021 +0000

    magnum: Add CA certificate configuration for internal TLS

    Magnum has various sections in its configuration file for OpenStack
    clients. When internal TLS is enabled, these may need a CA certificate
    to be specified.

    This change adds a CA certificate configuration, based on
    openstack_cacert, for all clients using internal endpoints.

    Note: we are explicitly not adding the configuration for the
    [magnum_client] ca_file and [drivers] openstack_ca_file options, since
    these use the public endpoint by default. These options may be
    provided via custom configuration if necessary.

    Change-Id: Ie59b3777c0a2c142b580addd67e279bc4b2f2c90
    Co-Authored-By: Kyle Dean
    Closes-Bug: #1919389
    (cherry picked from commit 48f0957a1c9c3ef0cac77642f058fa9cb47fbbc8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/798894
Committed: https://opendev.org/openstack/kolla-ansible/commit/338d977317e30aabe36ab32388542ae0381798b5
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 338d977317e30aabe36ab32388542ae0381798b5
Author: Mark Goddard <email address hidden>
Date: Wed Mar 17 09:32:33 2021 +0000

    magnum: Add CA certificate configuration for internal TLS

    Magnum has various sections in its configuration file for OpenStack
    clients. When internal TLS is enabled, these may need a CA certificate
    to be specified.

    This change adds a CA certificate configuration, based on
    openstack_cacert, for all clients using internal endpoints.

    Note: we are explicitly not adding the configuration for the
    [magnum_client] ca_file and [drivers] openstack_ca_file options, since
    these use the public endpoint by default. These options may be
    provided via custom configuration if necessary.

    Change-Id: Ie59b3777c0a2c142b580addd67e279bc4b2f2c90
    Co-Authored-By: Kyle Dean
    Closes-Bug: #1919389
    (cherry picked from commit 48f0957a1c9c3ef0cac77642f058fa9cb47fbbc8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/798895
Committed: https://opendev.org/openstack/kolla-ansible/commit/e3ef0dc3d1db70ce0c8bb8d19801079c53559aaf
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit e3ef0dc3d1db70ce0c8bb8d19801079c53559aaf
Author: Mark Goddard <email address hidden>
Date: Wed Mar 17 09:32:33 2021 +0000

    magnum: Add CA certificate configuration for internal TLS

    Magnum has various sections in its configuration file for OpenStack
    clients. When internal TLS is enabled, these may need a CA certificate
    to be specified.

    This change adds a CA certificate configuration, based on
    openstack_cacert, for all clients using internal endpoints.

    Note: we are explicitly not adding the configuration for the
    [magnum_client] ca_file and [drivers] openstack_ca_file options, since
    these use the public endpoint by default. These options may be
    provided via custom configuration if necessary.

    Change-Id: Ie59b3777c0a2c142b580addd67e279bc4b2f2c90
    Co-Authored-By: Kyle Dean
    Closes-Bug: #1919389
    (cherry picked from commit 48f0957a1c9c3ef0cac77642f058fa9cb47fbbc8)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 11.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 12.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 10.3.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 13.0.0.0rc1 release candidate.