Deploy steps executed twice during `update run`

Bug #1776206 reported by Jiří Stránský
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Jiří Stránský

Bug Description

It seems that we run deployment tasks twice during minor update. From observation, the likely order is:

1) update_tasks
2) deploy tasks (puppet, containers)
3) host_prep_tasks
4) deploy tasks again

Points 1 and 2 get executed in `serial: 1` manner (as they should). After that normal deployment playbook runs in parallel on all nodes -- that part seems superfluous, as `update converge` would do a very similar thing.

summary: - Deploy steps executed twice during minor version update
+ Deploy steps executed twice during `update run`
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/574230

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/574236

tags: added: queens-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (master)

Reviewed: https://review.openstack.org/574230
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=560e467016db6121b421e059e4751e912a566400
Submitter: Zuul
Branch: master

commit 560e467016db6121b421e059e4751e912a566400
Author: Jiri Stransky <email address hidden>
Date: Mon Jun 11 13:59:37 2018 +0200

    Remove deploy playbook from `update run`

    We need the deploy steps as part of update playbook, to execute them
    with `serial: 1` (we need to start services on node A before
    proceeding to update node B). This is already happening correctly
    within the update playbook.

    After the update playbook we have been additionaly running deploy
    playbook on all nodes in parallel. That seems like a superfluous
    operation, in some environments considerably prolonging the time for
    minor update. The full deployment will be once again executed on all
    nodes in parallel during `update converge`, we don't need to do it in
    `update run` too.

    Change-Id: I513f655707c8a31f3621b96a756be9c468e3bd8b
    Closes-Bug: #1776206

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/575149

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (master)

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/574236
Reason: The gate is having serious troubles with docker.io, we need to abandon this patch so it leaves the gate and when it's stable again I will restore this patch. Please do not restore or do anything, I'll take care of it as soon as things work again.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (master)

Reviewed: https://review.openstack.org/574236
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=416b35f4c30f4a65d1c09e8b0c540f28e1ca4f86
Submitter: Zuul
Branch: master

commit 416b35f4c30f4a65d1c09e8b0c540f28e1ca4f86
Author: Jiri Stransky <email address hidden>
Date: Mon Jun 11 14:06:40 2018 +0200

    Updates: run host_prep_tasks between update tasks and deployment tasks

    We should re-run host_prep_tasks as part of the minor update, to make
    sure the host is ready for starting the updated containers. The right
    place for them is between update tasks and deployment tasks.

    This is important in case we deliver changes to host_prep_tasks during
    minor update, or if update_tasks do something that would partially
    undo the host preparation, e.g. clear/delete some directories on the
    host to get rid of previous state.

    Change-Id: Ic0a905a8c4691cbe75113131bd84e8a39dea046d
    Related-Bug: #1776206

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (stable/queens)

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/575680

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (stable/queens)

Reviewed: https://review.openstack.org/575149
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=1f45ee746acea80d4a994df2e9a1db43f0738a7c
Submitter: Zuul
Branch: stable/queens

commit 1f45ee746acea80d4a994df2e9a1db43f0738a7c
Author: Jiri Stransky <email address hidden>
Date: Mon Jun 11 13:59:37 2018 +0200

    Remove deploy playbook from `update run`

    We need the deploy steps as part of update playbook, to execute them
    with `serial: 1` (we need to start services on node A before
    proceeding to update node B). This is already happening correctly
    within the update playbook.

    After the update playbook we have been additionaly running deploy
    playbook on all nodes in parallel. That seems like a superfluous
    operation, in some environments considerably prolonging the time for
    minor update. The full deployment will be once again executed on all
    nodes in parallel during `update converge`, we don't need to do it in
    `update run` too.

    Change-Id: I513f655707c8a31f3621b96a756be9c468e3bd8b
    Closes-Bug: #1776206
    (cherry picked from commit 560e467016db6121b421e059e4751e912a566400)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (stable/queens)

Reviewed: https://review.openstack.org/575680
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=cc3ff7ae2777e759bc9426a527c7ea4ae65c1eb4
Submitter: Zuul
Branch: stable/queens

commit cc3ff7ae2777e759bc9426a527c7ea4ae65c1eb4
Author: Jiri Stransky <email address hidden>
Date: Mon Jun 11 14:06:40 2018 +0200

    Updates: run host_prep_tasks between update tasks and deployment tasks

    We should re-run host_prep_tasks as part of the minor update, to make
    sure the host is ready for starting the updated containers. The right
    place for them is between update tasks and deployment tasks.

    This is important in case we deliver changes to host_prep_tasks during
    minor update, or if update_tasks do something that would partially
    undo the host preparation, e.g. clear/delete some directories on the
    host to get rid of previous state.

    Change-Id: Ic0a905a8c4691cbe75113131bd84e8a39dea046d
    Related-Bug: #1776206
    (cherry picked from commit 416b35f4c30f4a65d1c09e8b0c540f28e1ca4f86)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 9.2.3

This issue was fixed in the openstack/python-tripleoclient 9.2.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 10.3.0

This issue was fixed in the openstack/python-tripleoclient 10.3.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.