kolla-ansible

During deploy, if image pull fails, container is left removed

Bug #1852572 reported by Mark Goddard
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
Medium
Mark Goddard
kolla-ansible 10.0.0 "ussuri"
Rocky
Fix Released
Medium
Radosław Piliszek
kolla-ansible 7.2.0 "Rocky"
Stein
Fix Released
Medium
Radosław Piliszek
kolla-ansible 8.1.0 "Stein"
Train
Fix Released
Medium
Radosław Piliszek
kolla-ansible 9.0.0 "Train"
Ussuri
Fix Released
Medium
Mark Goddard
kolla-ansible 10.0.0 "ussuri"

Bug Description

# Steps to reproduce

* Deploy services using kolla-ansible deploy
* Reconfigure the image for one or more services to use an invalid config
* Deploy/reconfigure services using kolla-ansible reconfigure

The invalid config could be a wrong docker registry, wrong image name, wrong tag, etc.

# Expected results

The restart handler for the service fails, and the old container is left running.

# Actual results

The restart handler for the service fails, and the old container is stopped and removed. This leaves the service in a broken state.

# Environment

OS: CentOS 7
Kolla ansible: rocky

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/694278

Changed in kolla-ansible:
assignee: nobody → Mark Goddard (mgoddard)
status: New → In Progress
Mark Goddard (mgoddard)
Changed in kolla-ansible:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/694278
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=64d07c0b7fedf96c489772f7e6619fe112e4189a
Submitter: Zuul
Branch: master

commit 64d07c0b7fedf96c489772f7e6619fe112e4189a
Author: Mark Goddard <email address hidden>
Date: Thu Nov 14 12:12:07 2019 +0000

    Attempt to pull image before stopping and removing container

    * Deploy services using kolla-ansible deploy
    * Reconfigure the image for one or more services to use an invalid
    * config
    * Deploy/reconfigure services using kolla-ansible reconfigure

    The invalid config could be a wrong docker registry, wrong image name,
    wrong tag, etc.

    The restart handler for the service fails, and the old container is
    left running.

    The restart handler for the service fails, and the old container is
    stopped and removed. This leaves the service in a broken state.

    This change fixes the issue by pulling the image if necessary prior to
    stopping and removing the container.

    Change-Id: I85b2a1b224d4c4d85c32c4922a2cd2c41171a1dc
    Closes-Bug: #1852572

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/694812

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/694813

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/694814

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/train)

Reviewed: https://review.opendev.org/694812
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=de0e516e0459dcb95a42b942e88114c791eed49e
Submitter: Zuul
Branch: stable/train

commit de0e516e0459dcb95a42b942e88114c791eed49e
Author: Mark Goddard <email address hidden>
Date: Thu Nov 14 12:12:07 2019 +0000

    Attempt to pull image before stopping and removing container

    * Deploy services using kolla-ansible deploy
    * Reconfigure the image for one or more services to use an invalid
    * config
    * Deploy/reconfigure services using kolla-ansible reconfigure

    The invalid config could be a wrong docker registry, wrong image name,
    wrong tag, etc.

    The restart handler for the service fails, and the old container is
    left running.

    The restart handler for the service fails, and the old container is
    stopped and removed. This leaves the service in a broken state.

    This change fixes the issue by pulling the image if necessary prior to
    stopping and removing the container.

    Change-Id: I85b2a1b224d4c4d85c32c4922a2cd2c41171a1dc
    Closes-Bug: #1852572
    (cherry picked from commit 64d07c0b7fedf96c489772f7e6619fe112e4189a)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/stein)

Reviewed: https://review.opendev.org/694813
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=48b106f40915c3d642499ca44efae1d78617c427
Submitter: Zuul
Branch: stable/stein

commit 48b106f40915c3d642499ca44efae1d78617c427
Author: Mark Goddard <email address hidden>
Date: Thu Nov 14 12:12:07 2019 +0000

    Attempt to pull image before stopping and removing container

    * Deploy services using kolla-ansible deploy
    * Reconfigure the image for one or more services to use an invalid
    * config
    * Deploy/reconfigure services using kolla-ansible reconfigure

    The invalid config could be a wrong docker registry, wrong image name,
    wrong tag, etc.

    The restart handler for the service fails, and the old container is
    left running.

    The restart handler for the service fails, and the old container is
    stopped and removed. This leaves the service in a broken state.

    This change fixes the issue by pulling the image if necessary prior to
    stopping and removing the container.

    Change-Id: I85b2a1b224d4c4d85c32c4922a2cd2c41171a1dc
    Closes-Bug: #1852572
    (cherry picked from commit 64d07c0b7fedf96c489772f7e6619fe112e4189a)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/rocky)

Reviewed: https://review.opendev.org/694814
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=da0a19abc747621d16e36ed98d2b1db97d57bf39
Submitter: Zuul
Branch: stable/rocky

commit da0a19abc747621d16e36ed98d2b1db97d57bf39
Author: Mark Goddard <email address hidden>
Date: Thu Nov 14 12:12:07 2019 +0000

    Attempt to pull image before stopping and removing container

    * Deploy services using kolla-ansible deploy
    * Reconfigure the image for one or more services to use an invalid
    * config
    * Deploy/reconfigure services using kolla-ansible reconfigure

    The invalid config could be a wrong docker registry, wrong image name,
    wrong tag, etc.

    The restart handler for the service fails, and the old container is
    left running.

    The restart handler for the service fails, and the old container is
    stopped and removed. This leaves the service in a broken state.

    This change fixes the issue by pulling the image if necessary prior to
    stopping and removing the container.

    Change-Id: I85b2a1b224d4c4d85c32c4922a2cd2c41171a1dc
    Closes-Bug: #1852572
    (cherry picked from commit 64d07c0b7fedf96c489772f7e6619fe112e4189a)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 9.0.0.0rc2

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 7.2.0

This issue was fixed in the openstack/kolla-ansible 7.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 8.1.0

This issue was fixed in the openstack/kolla-ansible 8.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.