container manila_api_db_sync fails with TLS everywhere

[Goutham Pacha Ravi]

When deploying TripleO > queens, overcloud deployment fails at step 3, manila_api_db_sync not being able to connect to the database and start its setup.

This container is missing a bind-mount to let the manila-manage tool access /etc/my.cnf.d/tripleo.cnf. Consequently, the SSL configuration is not being used and the setup fails.

Similar bugs on other services: [1][2]

[1] https://bugs.launchpad.net/tripleo/+bug/1746491 (Cinder)
[2] https://bugs.launchpad.net/tripleo/+bug/1782392 (Octavia, Barbican)

Sample Error Log:

Deployment error:

overcloud.AllNodesDeploySteps.ControllerDeployment_Step3.0:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: c1773ea6-9eff-499d-936f-8894b948d8a2
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |

[...]

    TASK [Debug output for task which failed: Run puppet host configuration for step 3] ***
    ok: [localhost] => {
        "failed_when_result": false,

[...]

            "Error running ['docker', 'run', '--name', 'manila_api_db_sync', '--label', 'config_id=tripleo_step3', '--label', 'container_name=manila_api_db_sync', '--label', 'managed_by=paunc
h', '--label', 'config_data={\"command\": \"/usr/bin/bootstrap_host_exec manila_api su manila -s /bin/bash -c \\'/usr/bin/manila-manage db sync\\'\", \"user\": \"root\", \"volumes\": [\"/etc/
hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.cr
t:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/ipa/ca.crt:/etc
/ipa/ca.crt:ro\", \"/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/manila/etc/manila/:/etc/manila/:ro\", \"/var/log/containers/
manila:/var/log/manila\", \"/var/log/containers/httpd/manila-api:/var/log/httpd\"], \"image\": \"hostname:5000/osp13_containers-manila-api:13.0-47\", \"detach\": false,
 \"net\": \"host\"}', '--net=host', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/ex
tracted:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--v
olume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/ipa/ca.crt:/etc/ipa/ca.crt:ro', '--volume=/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts
:ro', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/lib/config-data/manila/etc/manila/:/etc/manila/:ro', '--volume=/var/log/containers/manila:/var/log/manila', '--volume=/var/log/cont
ainers/httpd/manila-api:/var/log/httpd', 'hostname:5000osp13_containers-manila-api:13.0-47', '/usr/bin/bootstrap_host_exec', 'manila_api', 'su', 'manila', '-s', '/bin/
bash', '-c', \"'/usr/bin/manila-manage\", 'db', \"sync'\"]. [1]",

Error from /var/log/containers/manila/manila-manage.log on controller 0:

2018-08-16 23:32:43.351 11 ERROR manila OperationalError: (pymysql.err.OperationalError) (1045, u"Access denied for user 'manila'@'X.X.X.X' (using password: YES)") (Background on this error at: http://sqlalche.me/e/e3q8)

[Goutham Pacha Ravi (gouthamr)]

Fix proposed to branch: master
Review: https://review.openstack.org/#/c/594801

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/594801
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a4bb5ab1a6e1b981a93c31140d347473ab3483bf
Submitter: Zuul
Branch: master

commit a4bb5ab1a6e1b981a93c31140d347473ab3483bf
Author: Goutham Pacha Ravi <email address hidden>
Date: Tue Aug 21 21:43:40 2018 -0700

    Fix bind-mount to manila's bootstrap container

    When deploying with tls-everywhere, there are
    more connection options necessary for the Overcloud
    manila database bootstrap container to connect
    to mysql. These connection options are present in
    the configuration folder
    /var/lib/config-data/manila/etc/my.cnf.d/tripleo.cnf.

    Fix the bind-mounts on the manila_api_db_sync
    container so it doesn't fail to find this
    configuration.

    Closes-Bug: #1788337
    Change-Id: I44133b0b0c4367214649777680c94dcfa7bddc76

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/595014

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Juan Antonio Osorio Robles (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/595014
Reason: Purging the gate to free up resources and address the timeout issues

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/595014
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=3b1c2f135d95798eafbc81fc5699a2fa7c2ae728
Submitter: Zuul
Branch: stable/queens

commit 3b1c2f135d95798eafbc81fc5699a2fa7c2ae728
Author: Goutham Pacha Ravi <email address hidden>
Date: Tue Aug 21 21:43:40 2018 -0700

    Fix bind-mount to manila's bootstrap container

    When deploying with tls-everywhere, there are
    more connection options necessary for the Overcloud
    manila database bootstrap container to connect
    to mysql. These connection options are present in
    the configuration folder
    /var/lib/config-data/manila/etc/my.cnf.d/tripleo.cnf.

    Fix the bind-mounts on the manila_api_db_sync
    container so it doesn't fail to find this
    configuration.

    Closes-Bug: #1788337
    Change-Id: I44133b0b0c4367214649777680c94dcfa7bddc76
    (cherry picked from commit a4bb5ab1a6e1b981a93c31140d347473ab3483bf)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.1.0 release.