mysql_init_tasks throws ERROR 1045 on containerized undercloud, when doing consequent re-deployments

[Bogdan Dobrelya]

Containerized undercloud deployed by quickstart throws "ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)" from a mysql-init-tasks container, when doing consequent re-deployments.

Consequent re-deployment should be idempotent or deploy updated configuration in-place. That can't be done though and requires a full /var/lib/mysql wiped undercloud.

The fix is probably to not update mysql password if the stack for it is deployed and mysql service container is alive undercloud.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/474945

[Bogdan Dobrelya (bogdando)]

The .my.cnf file is in place, the issue is that the command /usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT 1; throws "ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)" from a container

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: master
Review: https://review.openstack.org/474945

[Bogdan Dobrelya (bogdando)]

Just have caught the same ERROR 1045 with queens' master undercloud containers. That's weird a bug...

[Bogdan Dobrelya (bogdando)]

For that case, it seems related to the missing /etc/hosts entry. I need to inspect my CloudDomain / CloudName / overcloud_domain_name setup perhaps.

[Bogdan Dobrelya (bogdando)]

related https://bugs.launchpad.net/tripleo/+bug/1733873

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/522266

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/522298

[Bogdan Dobrelya (bogdando)]

So the issue is that for a consequent redeployments of containerized underlcloud, the existing /var/lib/mysql host's dir prevents mysql_bootstrap init container to update root passwords (generated each time we do undercloud deploy). So this basically is the root cause. Removing the directory helped to fix error for consequent re-deployments.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bogdan Dobrelya (<email address hidden>) on branch: master
Review: https://review.openstack.org/522266
Reason: at first, I was thinking to allow custom undercloud endpoints starting from t-h-t, then quickstart, and the client as well... But that's too much of the things to start with. So I reworked a baby step as an autonomous client option, that just heals the hardcode in the client. Hopefully, we'll end up having those Cloud* for uncercloud in t-h-t and quickstart params as well, eventually!
   4:58:28 PM GMT+1 - bogdando: at least, this little change unblocks me with RDO cloud domain names != defaults :)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/522298
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=909a9f753007f4cb10fc36a45934cccd6ce753cf
Submitter: Zuul
Branch: master

commit 909a9f753007f4cb10fc36a45934cccd6ce753cf
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Nov 22 17:41:08 2017 +0100

    Allow to override t-h-t 'Cloud*' params undercloud

    Allow to parametrize the hardcoded 'undercloud' domain name.
    F.e. RDOcloud uses an 'rdocloud' domain, therefore some DNS things might be
    failing with the hardcoded value.
    The default domain name remains unchanged. It can be
    overriden with a new '--local-domain' option as well.

    Related-bug: #1698349
    Partial-bug: #1733873

    Change-Id: Iad63142b241f9f7f0bbef68473f18b69f36daf76
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[Bogdan Dobrelya (bogdando)]

Basically that is a valid bug, reopening. Consequent re-deployment should be idempotent or deploy updated configuration in-place. That can't be done though and requires a full /var/lib/mysql wiped undercloud.

[Bogdan Dobrelya (bogdando)]

Addressed by https://review.openstack.org/#/c/526156

[Bogdan Dobrelya (bogdando)]

Addressed by https://review.openstack.org/#/c/526151