mysql pacemaker upgrade fails when running docker

Bug #1898610 reported by Alex Schultz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned

Bug Description

In stein and train, we only supported running pacemaker with Docker when using centos7. Unfortunately https://review.opendev.org/#/c/693710/ and https://review.opendev.org/#/c/693773/ introduced a breaking task because it assumes podman with k8s-file. k8s-file is not available when running under docker.

2020-10-05 18:33:50.518 636624 WARNING tripleoclient.v1.tripleo_upgrade.Upgrade [-] TASK [Upgrade Mysql database from a temporary container] ******************************************************************************************************
2020-10-05 18:33:50.521 636624 WARNING tripleoclient.v1.tripleo_upgrade.Upgrade [-] fatal: [standalone-0]: FAILED! => {"changed": true, "cmd": "docker run --rm --log-driver=k8s-file --log-opt path=/var/log/containers/mysql/db-upgrade.log -u root --net=host -e \"KOLLA_CONFIG_STRATEGY=COPY_ALWAYS\" -v /etc/hosts:/etc/hosts:ro -v /etc/localtime:/etc/localtime:ro -v /etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro -v /etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro -v /etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro -v /etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro -v /etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro -v /dev/log:/dev/log -v /etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro -v /etc/puppet:/etc/puppet:ro -v /var/lib/kolla/config_files/mysql.json:/var/lib/kolla/config_files/config.json:rw,z -v /var/lib/config-data/puppet-generated/mysql/:/var/lib/kolla/config_files/src:ro,z -v /var/lib/mysql:/var/lib/mysql:rw,z -v /tmp/mariadb-upgrade:/var/log/mariadb:rw,z \"docker.io/tripleostein/centos-binary-mariadb:pcmklatest\" /bin/bash -ecx \"kolla_set_configs; if mysqladmin ping --silent; then exit 0; fi; chown -R mysql:mysql /var/lib/mysql; chown -R mysql:mysql /var/log/mariadb; mysqld_safe --user=mysql --wsrep-provider=none --skip-networking --wsrep-on=off & timeout 60 sh -c 'while ! mysqladmin ping --silent; do sleep 1; done'; mysql_upgrade; mysqladmin shutdown\"", "delta": "0:00:00.066754", "end": "2020-10-05 18:33:50.491301", "msg": "non-zero return code", "rc": 125, "start": "2020-10-05 18:33:50.424547", "stderr": "/usr/bin/docker-current: Error response from daemon: logger: no log driver named 'k8s-file' is registered.\nSee '/usr/bin/docker-current run --help'.", "stderr_lines": ["/usr/bin/docker-current: Error response from daemon: logger: no log driver named 'k8s-file' is registered.", "See '/usr/bin/docker-current run --help'."], "stdout": "", "stdout_lines": []}

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/756149

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/756149
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f35c3455097a91cf6e07456b339829d82c18ad65
Submitter: Zuul
Branch: stable/train

commit f35c3455097a91cf6e07456b339829d82c18ad65
Author: Emilien Macchi <email address hidden>
Date: Mon Oct 5 14:59:11 2020 -0400

    [train] mysql: adjust CLI args based on container cli

    k8s-file log driver only exists for podman, such as log-opt path option.
    For docker, we'll use syslogd log driver.

    Note: this patch is only done for train and stein, where we still
    support Docker.

    Change-Id: Ie47e1cd81b25914d28ac63668817ce6237e2e8a9
    Closes-Bug: #1898610

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/756564

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/stein)

Reviewed: https://review.opendev.org/756564
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9f9173da38ff81eb5e194590ac5cada212f4433a
Submitter: Zuul
Branch: stable/stein

commit 9f9173da38ff81eb5e194590ac5cada212f4433a
Author: Emilien Macchi <email address hidden>
Date: Mon Oct 5 14:59:11 2020 -0400

    [train] mysql: adjust CLI args based on container cli

    k8s-file log driver only exists for podman, such as log-opt path option.
    For docker, we'll use syslogd log driver.

    Note: this patch is only done for train and stein, where we still
    support Docker.

    Change-Id: Ie47e1cd81b25914d28ac63668817ce6237e2e8a9
    Closes-Bug: #1898610
    (cherry picked from commit f35c3455097a91cf6e07456b339829d82c18ad65)

tags: added: in-stable-stein
Changed in tripleo:
milestone: victoria-rc1 → wallaby-1
Changed in tripleo:
milestone: wallaby-1 → wallaby-2
Changed in tripleo:
milestone: wallaby-2 → wallaby-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.4.0

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

Revision history for this message
Marios Andreou (marios-b) wrote :

Bug status has been set to 'Fix-Released' based on the discussion and/or patches above. If you disagree please re-set 'Triaged' and reach out to us on freenode #tripleo thank you!

Changed in tripleo:
status: Triaged → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates stein-eol

This issue was fixed in the openstack/tripleo-heat-templates stein-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.