DB sync can be skipped during deploy if DB setup fails

[Mark Goddard]

A common class of problems goes like this:

* kolla-ansible deploy
* Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
* Re-run kolla-ansible deploy
* Service fails to start

This happens because the DB is created during the first run, but for some reason we fail before performing the DB sync. This means that on the second run we don't include ansible/roles/*/tasks/bootstrap_services.yml because the DB already exists, and therefore still don't perform the DB sync. However this time, the command may complete without apparent error.

We should be less careful about when we perform the DB sync, and do it whenever it is necessary.

There is an argument for not doing the sync during a 'reconfigure' command.

More generally, we should better document the difference between these commands. In general there is no longer any difference between deploy and reconfigure, although this is not always the case.

[Mark Goddard (mgoddard)]

https://review.openstack.org/650962

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/650962
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=d5e5e885d11e338806425839a361d868c1f4ff10
Submitter: Zuul
Branch: master

commit d5e5e885d11e338806425839a361d868c1f4ff10
Author: Mark Goddard <email address hidden>
Date: Mon Apr 8 17:51:07 2019 +0100

    During deploy, always sync DB

    A common class of problems goes like this:

    * kolla-ansible deploy
    * Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
    * Re-run kolla-ansible deploy
    * Service fails to start

    This happens because the DB is created during the first run, but for some
    reason we fail before performing the DB sync. This means that on the second run
    we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
    already exists, and therefore still don't perform the DB sync. However this
    time, the command may complete without apparent error.

    We should be less careful about when we perform the DB sync, and do it whenever
    it is necessary. There is an argument for not doing the sync during a
    'reconfigure' command, although we will not change that here.

    This change only always performs the DB sync during 'deploy' and
    'reconfigure' commands.

    Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
    Closes-Bug: #1823766
    Closes-Bug: #1797814

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/670546

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/670548

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/670549

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/670546
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=d62e927dacfa42822296e7a0720cb85019c08a9b
Submitter: Zuul
Branch: stable/stein

commit d62e927dacfa42822296e7a0720cb85019c08a9b
Author: Mark Goddard <email address hidden>
Date: Mon Apr 8 17:51:07 2019 +0100

    During deploy, always sync DB

    A common class of problems goes like this:

    * kolla-ansible deploy
    * Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
    * Re-run kolla-ansible deploy
    * Service fails to start

    This happens because the DB is created during the first run, but for some
    reason we fail before performing the DB sync. This means that on the second run
    we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
    already exists, and therefore still don't perform the DB sync. However this
    time, the command may complete without apparent error.

    We should be less careful about when we perform the DB sync, and do it whenever
    it is necessary. There is an argument for not doing the sync during a
    'reconfigure' command, although we will not change that here.

    This change only always performs the DB sync during 'deploy' and
    'reconfigure' commands.

    Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
    Closes-Bug: #1823766
    Closes-Bug: #1797814
    (cherry picked from commit d5e5e885d11e338806425839a361d868c1f4ff10)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 8.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/670549
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=8e219a91738048ff16f2caed4c24f85832211aad
Submitter: Zuul
Branch: stable/queens

commit 8e219a91738048ff16f2caed4c24f85832211aad
Author: Mark Goddard <email address hidden>
Date: Mon Apr 8 17:51:07 2019 +0100

    During deploy, always sync DB

    A common class of problems goes like this:

    * kolla-ansible deploy
    * Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
    * Re-run kolla-ansible deploy
    * Service fails to start

    This happens because the DB is created during the first run, but for some
    reason we fail before performing the DB sync. This means that on the second run
    we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
    already exists, and therefore still don't perform the DB sync. However this
    time, the command may complete without apparent error.

    We should be less careful about when we perform the DB sync, and do it whenever
    it is necessary. There is an argument for not doing the sync during a
    'reconfigure' command, although we will not change that here.

    This change only always performs the DB sync during 'deploy' and
    'reconfigure' commands.

    Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
    Closes-Bug: #1823766
    Closes-Bug: #1797814
    (cherry picked from commit d5e5e885d11e338806425839a361d868c1f4ff10)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/670548
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=441c5a03f7e788336f3a7c1cb15dcec207073b07
Submitter: Zuul
Branch: stable/rocky

commit 441c5a03f7e788336f3a7c1cb15dcec207073b07
Author: Mark Goddard <email address hidden>
Date: Mon Apr 8 17:51:07 2019 +0100

    During deploy, always sync DB

    A common class of problems goes like this:

    * kolla-ansible deploy
    * Hit a problem, often in ansible/roles/*/tasks/bootstrap.yml
    * Re-run kolla-ansible deploy
    * Service fails to start

    This happens because the DB is created during the first run, but for some
    reason we fail before performing the DB sync. This means that on the second run
    we don't include ansible/roles/*/tasks/bootstrap_service.yml because the DB
    already exists, and therefore still don't perform the DB sync. However this
    time, the command may complete without apparent error.

    We should be less careful about when we perform the DB sync, and do it whenever
    it is necessary. There is an argument for not doing the sync during a
    'reconfigure' command, although we will not change that here.

    This change only always performs the DB sync during 'deploy' and
    'reconfigure' commands.

    Change-Id: I82d30f3fcf325a3fdff3c59f19a1f88055b566cc
    Closes-Bug: #1823766
    Closes-Bug: #1797814
    (cherry picked from commit d5e5e885d11e338806425839a361d868c1f4ff10)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 6.2.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 7.1.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc1 release candidate.