TLS everywhere: the apache service wrongly requests certificates for all networks

Bug #1811207 reported by Juan Antonio Osorio Robles
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Harald Jensås
Queens
Fix Released
Critical
Harald Jensås
Rocky
Fix Released
Critical
Harald Jensås

Bug Description

we currently do a loop to go through all the networks as follows:

  ApacheNetworks:
    type: OS::Heat::Value
    properties:
      value:
        # NOTE(jaosorior) Get unique network names to create
        # certificates for those. We skip the tenant network since
        # we don't need a certificate for that.
        - ctlplane
{%- for network in networks %}
  {%- if network.name_lower != 'tenant' %}
        - {{network.name_lower}}
  {%- endif %}
{%- endfor %}

In puppet/services/apache.j2.yaml we don't/cannot filter based on the role.networks in the jinja loop[1] creating the ApacheNetworks value since this file is rendered for per stack, not per-role.
The ApacheNetworks is used later to pass the data to certmonger for certificate generation[2]:

              generate_service_certificates: true
              apache::mod::ssl::ssl_ca: {get_param: InternalTLSCAFile}
              apache::mod::ssl::ssl_protocol: ['all', '-SSLv2', '-SSLv3', '-TLSv1']
              tripleo::certmonger::apache_dirs::certificate_dir: '/etc/pki/tls/certs/httpd'
              tripleo::certmonger::apache_dirs::key_dir: '/etc/pki/tls/private/httpd'
              apache_certificates_specs:
                map_merge:
                  repeat:
                    template:
                      httpd-NETWORK:
                        service_certificate: '/etc/pki/tls/certs/httpd/httpd-NETWORK.crt'
                        service_key: '/etc/pki/tls/private/httpd/httpd-NETWORK.key'
                        hostname: "%{hiera('fqdn_NETWORK')}"
                        principal: "HTTP/%{hiera('fqdn_NETWORK')}"
                    for_each:
                        NETWORK: {get_attr: [ApacheNetworks, value]}

This results, in deployments with the default network setup, with certmonger trying to request a certificate for the management interface, and failing:

Request ID 'httpd-management':
        status: CA_REJECTED
        ca-error: Server at https://ipa.ooo.test/ipa/xml denied our request, giving up: 3007 (RPC failed at server. 'fqdn' is required).
        stuck: yes
        key pair storage: type=FILE,location='/etc/pki/tls/private/httpd/httpd-management.key'
        certificate: type=FILE,location='/etc/pki/tls/certs/httpd/httpd-management.crt'
        CA: IPA
        issuer:
        subject:
        expires: unknown
        pre-save command:
        post-save command: "systemctl reload httpd"
        track: yes
        auto-renew: yes

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/apache.j2.yaml#L65-L69
[2] https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/apache.j2.yaml#L106-L122

Changed in tripleo:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → stein-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (master)

Fix proposed to branch: master
Review: https://review.openstack.org/629835

Changed in tripleo:
assignee: nobody → Harald Jensås (harald-jensas)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/629835
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=a3aeb0b7faa6a817d7336d7fef86dd96bcf12731
Submitter: Zuul
Branch: master

commit a3aeb0b7faa6a817d7336d7fef86dd96bcf12731
Author: Harald Jensås <email address hidden>
Date: Thu Jan 10 12:22:46 2019 +0100

    Filter apache_certificates_specs if hostname is empty

    Not all roles have all networks, when creating certificates
    we need to filter out certificate specs where the hiera
    lookup returns 'nil'.

    Previously we created noop resources with fallback to the
    ctlplane network for all networks defined in network data.
    So the hiera lookup returned a hostname for all networks
    no matter if the roles data included that network. This is
    no longer the case, so we need to filter.

    Bonus, we no longer create certificates that are'nt actually
    used.

    Change-Id: I651919488cb68b0b9878b4e21ab376bfc6e3f0fe
    Closes-Bug: #1811207

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/629961

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/629962

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (stable/rocky)

Reviewed: https://review.openstack.org/629961
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=60a0f973fdfc9253fcdb8a335668d89f203a5d27
Submitter: Zuul
Branch: stable/rocky

commit 60a0f973fdfc9253fcdb8a335668d89f203a5d27
Author: Harald Jensås <email address hidden>
Date: Thu Jan 10 12:22:46 2019 +0100

    Filter apache_certificates_specs if hostname is empty

    Not all roles have all networks, when creating certificates
    we need to filter out certificate specs where the hiera
    lookup returns 'nil'.

    Previously we created noop resources with fallback to the
    ctlplane network for all networks defined in network data.
    So the hiera lookup returned a hostname for all networks
    no matter if the roles data included that network. This is
    no longer the case, so we need to filter.

    Bonus, we no longer create certificates that are'nt actually
    used.

    Change-Id: I651919488cb68b0b9878b4e21ab376bfc6e3f0fe
    Closes-Bug: #1811207
    (cherry picked from commit a3aeb0b7faa6a817d7336d7fef86dd96bcf12731)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (stable/queens)

Reviewed: https://review.openstack.org/629962
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=adfca27bd0d3ccdea8b57541a34c3cd13a7e1bb1
Submitter: Zuul
Branch: stable/queens

commit adfca27bd0d3ccdea8b57541a34c3cd13a7e1bb1
Author: Harald Jensås <email address hidden>
Date: Thu Jan 10 12:22:46 2019 +0100

    Filter apache_certificates_specs if hostname is empty

    Not all roles have all networks, when creating certificates
    we need to filter out certificate specs where the hiera
    lookup returns 'nil'.

    Previously we created noop resources with fallback to the
    ctlplane network for all networks defined in network data.
    So the hiera lookup returned a hostname for all networks
    no matter if the roles data included that network. This is
    no longer the case, so we need to filter.

    Bonus, we no longer create certificates that are'nt actually
    used.

    Change-Id: I651919488cb68b0b9878b4e21ab376bfc6e3f0fe
    Closes-Bug: #1811207
    (cherry picked from commit a3aeb0b7faa6a817d7336d7fef86dd96bcf12731)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 8.4.0

This issue was fixed in the openstack/puppet-tripleo 8.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 10.3.0

This issue was fixed in the openstack/puppet-tripleo 10.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 9.4.0

This issue was fixed in the openstack/puppet-tripleo 9.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.