[upgrades] deployment of two additional controllers fails

[Ilya Kharin]

Octane uses deployment of the graph with a set of skipped tasks to deploy two additional controllers. Therefore, only specified nodes are involved in the deployment excluding the primary controller node, that leads to improper configuration of cluster software on OpeStack Controler Plane nodes.

Steps to reproduce:
 1. The 7.0 environment with 3xController nodes
                             3xCompute nodes
                             3xCeph-OSD nodes
 2. Upgrade the Fuel Master node to 8.0 and then to 9.0
 3. Perform upgrade of the Fuel Master node to 9.1
 4. Execute the next steps:
      fuel2 release clone <env_id> <release_id>
      octane upgrade-env <env_id> <release_id>
      octane upgrade-node --isolated <new_env_id> <primary_controller_node_id>
      octane upgrade-db <env_id> <new_env_id>
      octane upgrade-ceph <env_id> <new_env_id>
      octane upgrade-control <env_id> <new_env_id>
      octane preupgrade-compute <env_id> <compute_node_id> ...
      octane upgrade-node <new_env_id> <controller_node_id> ...

Expected results:
 Two additional controllers re-provisioned and deployed successfully.

Actual results:
 Deployment of two additional controllers failed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/366748

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/368263

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/366748
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=27531efb9567ba900cf5ba4c66704f9df0388d3d
Submitter: Jenkins
Branch: master

commit 27531efb9567ba900cf5ba4c66704f9df0388d3d
Author: Ilya Kharin <email address hidden>
Date: Wed Sep 7 16:16:41 2016 +0300

    Use deploy_changes for sets of nodes

    The deploy_changes method involves additional nodes into deployment to
    perform necessary tasks to keep clustered software in a consistent
    state. A direct execution of tasks on a specific set of nodes does not
    involve other necessary nodes and it leads to some failures during
    deployments.

    Change-Id: Ibbdeb40e4287a3170b57c11d88f5aea0afa2b5c7
    Closes-Bug: #1619821

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/368263
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=44c93bae19d1988ca760aae3f866602e4c0efbe1
Submitter: Jenkins
Branch: stable/mitaka

commit 44c93bae19d1988ca760aae3f866602e4c0efbe1
Author: Ilya Kharin <email address hidden>
Date: Wed Sep 7 16:16:41 2016 +0300

    Use deploy_changes for sets of nodes

    The deploy_changes method involves additional nodes into deployment to
    perform necessary tasks to keep clustered software in a consistent
    state. A direct execution of tasks on a specific set of nodes does not
    involve other necessary nodes and it leads to some failures during
    deployments.

    Change-Id: Ibbdeb40e4287a3170b57c11d88f5aea0afa2b5c7
    Closes-Bug: #1619821

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/368266

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/368266
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=e7e2e84e15b7c9559fc51e690a78f367fd5278e9
Submitter: Jenkins
Branch: stable/mitaka

commit 27531efb9567ba900cf5ba4c66704f9df0388d3d
Author: Ilya Kharin <email address hidden>
Date: Wed Sep 7 16:16:41 2016 +0300

    Use deploy_changes for sets of nodes

    The deploy_changes method involves additional nodes into deployment to
    perform necessary tasks to keep clustered software in a consistent
    state. A direct execution of tasks on a specific set of nodes does not
    involve other necessary nodes and it leads to some failures during
    deployments.

    Change-Id: Ibbdeb40e4287a3170b57c11d88f5aea0afa2b5c7
    Closes-Bug: #1619821

commit a9e7c1020f2426eb3805a2f0e487ac052751d68f
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 16:51:25 2016 +0300

    Don't set up duplicate repos

    This is required if seed and orig env repos has the same repos.
    Set up only that repos that not found in orig env.

    Change-Id: I6a953634e1482af1b89ab1f5da878f52152e7ada
    Closes-bug: 1621491

commit 12160e167694b81ea301700219e7f54a68faf827
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 13:08:18 2016 +0300

    Refactoring osd cmd

    * Use util/apt function for construct sources
    * Use util/apt function for contrstruct preferences

    Change-Id: I3bbdeb3a0bc31154be4cce7d8d354e05b0700084
    Partial-Bug: 1620277

commit 19ae8ddcdb0d96b77f941871cece9fe5afafe28c
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 12:24:58 2016 +0300

    Add packages in preference generation

    Change-Id: I465dd959b50b855f6ddc9e81fe7605c80415b5b9
    Partial-Bug: 1620277

commit 1aca7e4a2b8ff7d57729d1421debd8be6553b282
Author: Nguyen Hung Phuong <email address hidden>
Date: Wed Sep 7 10:30:30 2016 +0700

    TrivialFix: Remove logging import unused

    This patch removes logging import unused in
    octane/commands/cleanup.py
    octane/commands/enable_release.py
    octane/handlers/backup_restore/astute.py

    Change-Id: I53b6f83d72be729215d840ebe0be18794440008e

[Aleksei Stepanov (penguinolog)]

Test pass on separate CI

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-octane ocata-eol release.