octane upgrade-osd don't fail in upgrade not completed

Bug #1620277 reported by Sergey Abramov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Critical
Sergey Abramov
Mitaka
Fix Released
Critical
Ilya Kharin

Bug Description

Detailed bug description:

if upgrade osd not completed (they have the same version as before upgrade) than octane doesn't fail. Needed to add check on pre-upgradeand and post-upgrade steps.

* if mon and osd have same versions, than report to operator about this and return without any upgrade
* if mon and osd have different versions before upgrade, than start the upgrade
* if mon and osd have different versions after upgrade, than log error and raise exception about this to the operator.

Changed in fuel:
assignee: nobody → Sergey Abramov (sabramov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (master)

Fix proposed to branch: master
Review: https://review.openstack.org/365709

Changed in fuel:
status: New → In Progress
Ilya Kharin (akscram)
Changed in fuel:
importance: Undecided → Critical
status: In Progress → Confirmed
milestone: none → 9.1
milestone: 9.1 → next
Ilya Kharin (akscram)
Changed in fuel:
milestone: next → 10.0
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/367277

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (master)

Reviewed: https://review.openstack.org/365709
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=e7fad96f4125386df4e5c9b7f0744de185de6c33
Submitter: Jenkins
Branch: master

commit e7fad96f4125386df4e5c9b7f0744de185de6c33
Author: Sergey Abramov <email address hidden>
Date: Mon Sep 5 18:24:37 2016 +0300

    Osd upgrade failed if version not changed

    ceph deploy not raise exception if it doesn't upgrade osd version.

    Change-Id: Ifcddd822228d78166d59b2ba49852be2e51c79fc
    Closes-bug: 1620277

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/367652

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/mitaka)
Download full text (4.3 KiB)

Reviewed: https://review.openstack.org/367652
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=293b79e6490605d9e74d2e2b6825146b49fdecfb
Submitter: Jenkins
Branch: stable/mitaka

commit 9a4817174ed8b0702564baadc1728eaf8b778e28
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 01:04:10 2016 +0300

    Move a set of nodes at once

    Before this patch octane made separate calls to move nodes one by one.

    Change-Id: I999a98d57b3184d35972e4862fcb4f284a066e9e
    Related-Bug: #1616925

commit 763b69b97645752b4b08253751962687d50cf1be
Author: Alexander Tsamutali <email address hidden>
Date: Thu Sep 8 22:03:24 2016 +0300

    Cleanup %files section in spec

    Don't use --record during install, don't use -f in %files. Specify
    octane files/directories explicitly.

    Change-Id: I84f0d71a2a582b3a23fc048a331d6caae775e38b
    Closes-Bug: #1619319

commit e7fad96f4125386df4e5c9b7f0744de185de6c33
Author: Sergey Abramov <email address hidden>
Date: Mon Sep 5 18:24:37 2016 +0300

    Osd upgrade failed if version not changed

    ceph deploy not raise exception if it doesn't upgrade osd version.

    Change-Id: Ifcddd822228d78166d59b2ba49852be2e51c79fc
    Closes-bug: 1620277

commit 349073cbe399184daee672f75b4fe9941ae3c5da
Author: Sergey Abramov <email address hidden>
Date: Wed Sep 7 18:20:33 2016 +0300

    Remove patch pupet on upgrade osd

    this is not required after merge
    https://review.openstack.org/#/c/203639/

    Change-Id: I67fbcd77ab3437443219c34ee3ddaf7895b068ce
    Closes-bug: 1621436

commit 807c5166a8a0145623d41c3110007e33a1402c47
Author: Alexey Stepanov <email address hidden>
Date: Wed Sep 7 17:53:40 2016 +0300

    Stop waiting status change on node with stopped status

    Stopped status, if not expected, should be a reason for error on nodes
    Closes-bug: #1621069

    Change-Id: I0156d694ef20ece8603e3d840f085852a528e635

commit 5f689f3d6904e092998a0d8ba124280833452c30
Author: Anastasiya <email address hidden>
Date: Tue Sep 6 10:23:39 2016 +0300

    Backup/restore for admin networks

    * backup/restore for /etc/hiera/networks.yaml was added
    * configure dhcp after restore was added

    Change-Id: I5b1e3861589e1c56acbc37d0be569da5e55b8536
    Closes-Bug: #1616998

commit e4820087678d83d9477db4c1688137ee5ff66c3f
Author: Pavel Chechetin <email address hidden>
Date: Thu Aug 18 14:46:59 2016 +0300

    Graph-based upgrade-ceph. Python part.

    Change-Id: Icb4d543bd6801f21c6aca57415105f88a601c0c2

commit 0af1b6517a97589a401328f09f3bf56d84db9dbd
Author: Pavel Chechetin <email address hidden>
Date: Tue Sep 6 13:13:42 2016 +0300

    Graph-based upgrade-ceph. Puppet part.

     - Graphs and puppet manifests part
     - Delete lib from .gitignore
     - Augeas lens for Ceph is copied and pasted, should be switched to the
       version from the upstream when [0] is merged and published.

    [0] https://github.com/hercules-team/augeas/pull/401

    Change-Id: I639cbf786971fea8c56b4da6b2661477b3b12c41

commit 3315d741f295635303413ad839b3b66a0dac3282
Author: Alexey Stepanov <penguinolog@gmail....

Read more...

tags: added: team-upgrades
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (master)

Reviewed: https://review.openstack.org/367277
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=19ae8ddcdb0d96b77f941871cece9fe5afafe28c
Submitter: Jenkins
Branch: master

commit 19ae8ddcdb0d96b77f941871cece9fe5afafe28c
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 12:24:58 2016 +0300

    Add packages in preference generation

    Change-Id: I465dd959b50b855f6ddc9e81fe7605c80415b5b9
    Partial-Bug: 1620277

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/365787
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=12160e167694b81ea301700219e7f54a68faf827
Submitter: Jenkins
Branch: master

commit 12160e167694b81ea301700219e7f54a68faf827
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 13:08:18 2016 +0300

    Refactoring osd cmd

    * Use util/apt function for construct sources
    * Use util/apt function for contrstruct preferences

    Change-Id: I3bbdeb3a0bc31154be4cce7d8d354e05b0700084
    Partial-Bug: 1620277

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/368266

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/mitaka)

Reviewed: https://review.openstack.org/368266
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=e7e2e84e15b7c9559fc51e690a78f367fd5278e9
Submitter: Jenkins
Branch: stable/mitaka

commit 27531efb9567ba900cf5ba4c66704f9df0388d3d
Author: Ilya Kharin <email address hidden>
Date: Wed Sep 7 16:16:41 2016 +0300

    Use deploy_changes for sets of nodes

    The deploy_changes method involves additional nodes into deployment to
    perform necessary tasks to keep clustered software in a consistent
    state. A direct execution of tasks on a specific set of nodes does not
    involve other necessary nodes and it leads to some failures during
    deployments.

    Change-Id: Ibbdeb40e4287a3170b57c11d88f5aea0afa2b5c7
    Closes-Bug: #1619821

commit a9e7c1020f2426eb3805a2f0e487ac052751d68f
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 16:51:25 2016 +0300

    Don't set up duplicate repos

    This is required if seed and orig env repos has the same repos.
    Set up only that repos that not found in orig env.

    Change-Id: I6a953634e1482af1b89ab1f5da878f52152e7ada
    Closes-bug: 1621491

commit 12160e167694b81ea301700219e7f54a68faf827
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 13:08:18 2016 +0300

    Refactoring osd cmd

    * Use util/apt function for construct sources
    * Use util/apt function for contrstruct preferences

    Change-Id: I3bbdeb3a0bc31154be4cce7d8d354e05b0700084
    Partial-Bug: 1620277

commit 19ae8ddcdb0d96b77f941871cece9fe5afafe28c
Author: Sergey Abramov <email address hidden>
Date: Thu Sep 8 12:24:58 2016 +0300

    Add packages in preference generation

    Change-Id: I465dd959b50b855f6ddc9e81fe7605c80415b5b9
    Partial-Bug: 1620277

commit 1aca7e4a2b8ff7d57729d1421debd8be6553b282
Author: Nguyen Hung Phuong <email address hidden>
Date: Wed Sep 7 10:30:30 2016 +0700

    TrivialFix: Remove logging import unused

    This patch removes logging import unused in
    octane/commands/cleanup.py
    octane/commands/enable_release.py
    octane/handlers/backup_restore/astute.py

    Change-Id: I53b6f83d72be729215d840ebe0be18794440008e

tags: added: in-stable-mitaka
Revision history for this message
Vladimir Khlyunev (vkhlyunev) wrote :

snapshot 251:
thanks to this fix, got https://bugs.launchpad.net/fuel/+bug/1622543 while testing

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-octane ocata-eol

This issue was fixed in the openstack/fuel-octane ocata-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.