Bug #1616925 “[upgrade] Node can not be rebooted while upgrading...” : Bugs : Fuel for OpenStack

Revision history for this message

Vladimir Khlyunev (vkhlyunev) wrote on 2016-08-25:

#1

astute.log.gz Edit (1.7 MiB, application/octet-stream)

Dmitry Klenov (dklenov) on 2016-08-25

Changed in fuel:
milestone:	none → 7.0-updates
importance:	Undecided → Medium
tags:	added: area-python
Changed in fuel:
status:	New → Confirmed

Ilya Kharin (akscram) on 2016-09-01

Changed in fuel:
milestone:	7.0-updates → 8.0-updates

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-02: Related fix proposed to fuel-nailgun-extension-cluster-upgrade (stable/mitaka)

#2

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/365168

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-02: Related fix proposed to fuel-octane (master)

#3

Related fix proposed to branch: master
Review: https://review.openstack.org/365169

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-02: Related fix proposed to fuel-nailgun-extension-cluster-upgrade (master)

#4

Related fix proposed to branch: master
Review: https://review.openstack.org/365173

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-03: Change abandoned on fuel-nailgun-extension-cluster-upgrade (master)

#5

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/365173

Vladimir Khlyunev (vkhlyunev) on 2016-09-07

Changed in fuel:
importance:	Medium → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-07: Related fix merged to fuel-nailgun-extension-cluster-upgrade (stable/mitaka)

#6

Reviewed: https://review.openstack.org/365168
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-extension-cluster-upgrade/commit/?id=a4e2a67e3e5024b0ae65f445355965a1263fef73
Submitter: Jenkins
Branch: stable/mitaka

commit a4e2a67e3e5024b0ae65f445355965a1263fef73
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 02:42:07 2016 +0300

Add support to re-assign a set of nodes

    This patch adds an ability to re-assign a set of the given nodes at
    once. This feature was technically available but not exposed to the
    client. A groupped re-assigning allows to effectively re-provision nodes
    by creating an atomic task in Astute.

Change-Id: I4a7c7e35d844683ef73ad7f8459d1892e80e0a64
Related-Bug: #1616925

tags:

added: in-stable-mitaka

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-07: Related fix proposed to fuel-nailgun-extension-cluster-upgrade (master)

#7

Related fix proposed to branch: master
Review: https://review.openstack.org/367019

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-07: Change abandoned on fuel-nailgun-extension-cluster-upgrade (master)

#8

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/367019

Revision history for this message

Vladimir Khlyunev (vkhlyunev) wrote on 2016-09-08:

#9

wrong moving into "confirmed" - actually "fix released"
root@node-1:~# ssh cirros@10.109.18.129
cirros@10.109.18.129's password:
$ uptime
18:39:39 up 5:21, 1 users, load average: 0.00, 0.01, 0.03
$ Connection to 10.109.18.129 closed.

Revision history for this message

Vladimir Khlyunev (vkhlyunev) wrote on 2016-09-08:

#10

wrong bug, sorry, my bad

Ilya Kharin (akscram) on 2016-09-08

Changed in fuel:
assignee:	Fuel Octane (fuel-octane-team) → Ilya Kharin (akscram)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Related fix proposed to fuel-nailgun-extension-cluster-upgrade (master)

#11

Related fix proposed to branch: master
Review: https://review.openstack.org/367419

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Change abandoned on fuel-nailgun-extension-cluster-upgrade (master)

#12

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/367419

Tatyana Borukhovich (tatyanaborukhovich) on 2016-09-08

tags:

added: blocker-for-qa

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Related fix merged to fuel-nailgun-extension-cluster-upgrade (master)

#13

Download full text (14.8 KiB)

Reviewed: https://review.openstack.org/367019
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-extension-cluster-upgrade/commit/?id=e38d48cbc5d216ac0f853466ab48e5d6d9687521
Submitter: Jenkins
Branch: master

commit a4e2a67e3e5024b0ae65f445355965a1263fef73
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 02:42:07 2016 +0300

Add support to re-assign a set of nodes

    This patch adds an ability to re-assign a set of the given nodes at
    once. This feature was technically available but not exposed to the
    client. A groupped re-assigning allows to effectively re-provision nodes
    by creating an atomic task in Astute.

Change-Id: I4a7c7e35d844683ef73ad7f8459d1892e80e0a64
Related-Bug: #1616925

commit d87125662f2f55204244e43ec6522fe36c6bf21e
Author: Nikita Zubkov <email address hidden>
Date: Mon Aug 29 17:56:09 2016 +0300

Add test for vip transformer

Change-Id: I65124237604fe6718ad6c351825f192a98d470fb
(cherry picked from commit 55422ddce7b0e9e1b9976504500de8a1688a99d7)

commit 3cac9b551cdecaf9148800cc6fd9b81fbb3149c5
Author: Nikita Zubkov <email address hidden>
Date: Tue Jul 19 18:24:42 2016 +0300

Switch to upstream fuel-web repository

Change-Id: I994304bdc8eaf7e4da175981cb721d41a286fed0
Depends-On: Id0bc78478cf3f40767fed760cd54e487a934fa10

commit a78f4348f84dd470ba1f3b99d2c751fa2dd12d7a
Author: Anastasiya <email address hidden>
Date: Mon Aug 1 14:39:35 2016 +0300

Move change_env_settings function from octane to cluster upgrade extension

* change_env_settings function was moved to cluster upgrade extention
* merge generated attributes code was written

    Change-Id: I6d1e27b8b0c01f3251067bc88931cd2354feb5ce
    Partial-Bug: #1602587
    (cherry picked from commit dc2e3f930957b2c8af2d6c6a60bfcc6c5e6bb061)

commit 280fc4f08258f1e85ba099f74c4956233652e9a2
Author: Ilya Kharin <email address hidden>
Date: Wed Aug 10 17:28:17 2016 +0300

Add absent __init__.py to migrations/versions

Without the versions/__init__.py file versions was not identified as
a package and was not included in a distribution.

Change-Id: I67f152ebb9234df880c61d79d154b1aabc8828c6
Closes-Bug: #1611793

commit f7ebb08b46f5beb13701f7a6a71a1f4fea05f451
Author: Alexander Tsamutali <email address hidden>
Date: Mon Aug 1 15:47:58 2016 +0300

Add package spec

Change-Id: Id71764dff07a4b32851eb8ccf69c66dca4a7b6ab
Related-Bug: #1604492

commit b93ebedc49f79f6ba4a710a9d1715c9f965b3081
Author: Anastasiya <email address hidden>
Date: Fri Jul 15 10:24:11 2016 +0300

Correction of transformation for text_list

* added removing of space in text_list
* added test for merge_attributes

    Change-Id: I5582878fc7c524551593abf21dfd4ea45cd430c9
    Closes-bug: 1602607
    (cherry picked from commit fdd2a6226483c67ce8bc7adc8b2d354862125bac)

commit d4db5ba78ccefd08e465cd30116094678c5cb35f
Author: Nikita Zubkov <email address hidden>
Date: Wed Jul 13 13:43:58 2016 +0300

Fix package namespace

commit 443fc43da6e963cd0825880a80678e8e385c0a3a
Author: Nikita Zubkov <email address hidden>
Date: Wed J...

Reviewed:  https://review.openstack.org/367019
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-extension-cluster-upgrade/commit/?id=e38d48cbc5d216ac0f853466ab48e5d6d9687521
Submitter: Jenkins
Branch:    master

commit a4e2a67e3e5024b0ae65f445355965a1263fef73
Author: Ilya Kharin <akscram@gmail.com>
Date:   Sat Sep 3 02:42:07 2016 +0300

Add support to re-assign a set of nodes
    
    This patch adds an ability to re-assign a set of the given nodes at
    once. This feature was technically available but not exposed to the
    client. A groupped re-assigning allows to effectively re-provision nodes
    by creating an atomic task in Astute.
    
    Change-Id: I4a7c7e35d844683ef73ad7f8459d1892e80e0a64
    Related-Bug: #1616925

commit d87125662f2f55204244e43ec6522fe36c6bf21e
Author: Nikita Zubkov <nzubkov@mirantis.com>
Date:   Mon Aug 29 17:56:09 2016 +0300

Add test for vip transformer
    
    Change-Id: I65124237604fe6718ad6c351825f192a98d470fb
    (cherry picked from commit 55422ddce7b0e9e1b9976504500de8a1688a99d7)

commit 3cac9b551cdecaf9148800cc6fd9b81fbb3149c5
Author: Nikita Zubkov <nzubkov@mirantis.com>
Date:   Tue Jul 19 18:24:42 2016 +0300

Switch to upstream fuel-web repository
    
    Change-Id: I994304bdc8eaf7e4da175981cb721d41a286fed0
    Depends-On: Id0bc78478cf3f40767fed760cd54e487a934fa10

commit a78f4348f84dd470ba1f3b99d2c751fa2dd12d7a
Author: Anastasiya <atolochkova@mirantis.com>
Date:   Mon Aug 1 14:39:35 2016 +0300

Move change_env_settings function from octane to cluster upgrade extension
    
    * change_env_settings function was moved to cluster upgrade extention
    * merge generated attributes code was written
    
    Change-Id: I6d1e27b8b0c01f3251067bc88931cd2354feb5ce
    Partial-Bug: #1602587
    (cherry picked from commit dc2e3f930957b2c8af2d6c6a60bfcc6c5e6bb061)

commit 280fc4f08258f1e85ba099f74c4956233652e9a2
Author: Ilya Kharin <akscram@gmail.com>
Date:   Wed Aug 10 17:28:17 2016 +0300

Add absent __init__.py to migrations/versions
    
    Without the versions/__init__.py file versions was not identified as
    a package and was not included in a distribution.
    
    Change-Id: I67f152ebb9234df880c61d79d154b1aabc8828c6
    Closes-Bug: #1611793

commit f7ebb08b46f5beb13701f7a6a71a1f4fea05f451
Author: Alexander Tsamutali <atsamutali@mirantis.com>
Date:   Mon Aug 1 15:47:58 2016 +0300

Add package spec
    
    Change-Id: Id71764dff07a4b32851eb8ccf69c66dca4a7b6ab
    Related-Bug: #1604492

commit b93ebedc49f79f6ba4a710a9d1715c9f965b3081
Author: Anastasiya <atolochkova@mirantis.com>
Date:   Fri Jul 15 10:24:11 2016 +0300

Correction of transformation for text_list
    
    * added removing of space in text_list
    * added test for merge_attributes
    
    Change-Id: I5582878fc7c524551593abf21dfd4ea45cd430c9
    Closes-bug: 1602607
    (cherry picked from commit fdd2a6226483c67ce8bc7adc8b2d354862125bac)

commit d4db5ba78ccefd08e465cd30116094678c5cb35f
Author: Nikita Zubkov <nzubkov@mirantis.com>
Date:   Wed Jul 13 13:43:58 2016 +0300

Fix package namespace

commit 443fc43da6e963cd0825880a80678e8e385c0a3a
Author: Nikita Zubkov <nzubkov@mirantis.com>
Date:   Wed Jul 13 13:43:16 2016 +0300

Add nailgun extension entrypoint to setup.cfg

commit 030fda6fa4ca625b226faf7f2ec79211358d13a1
Author: Sergey Abramov <sabramov@mirantis.com>
Date:   Fri May 13 17:53:43 2016 +0300

Fix cluster attributes dns_list and ntp_list
    
    Settings in release change. Cloned cluster should have values valid for
    its release.
    
    dns_list and net_list were text values but in version mitaka-9.0 it
    changed to text_list
    
    Change-Id: Iac0aa42b7c36333e6d9c40b8a27a19df9efe36f5
    Closes-Bug: 1572179
    (cherry picked from commit 5ae4c7ebdcdbf1621ce617de4ec019ae2b5670c4)

commit 7cf79178e600027cfc1718d6ba86af8f4ed055f1
Author: Artur Svechnikov <asvechnikov@mirantis.com>
Date:   Wed May 4 13:29:17 2016 +0300

Check nodes roles before deployment
    
    Nodes roles should be checked in CheckBeforeDeploymentTask,
    because it's possible to deploy node with conflicting roles
    or with incompatible role. Roles release metadata will be
    used for roles checks, this metadata contains restrictions.
    
    Since `depends` is not used anymore, it's changed to
    `restrictions` in assignment validator.
    
    Change-Id: Ibba7951968cbafd59fff0d516e74f9dd9e454edc
    Closes-Bug: #1573006
    (cherry picked from commit 3482a5d7a9417c268a526cf511e114be94c884be)

commit e15ad4ab636fc391fd2e20296d28ffe057248dbf
Author: Artem Roma <aroma@mirantis.com>
Date:   Thu Feb 25 19:45:50 2016 +0200

Fix get_common_node_group behavior
    
    Now if common nodegroup has not been found for given nodes None as a
    result is returned instead of default node group.
    
    This directly affects procedure of VIP allocation as (e.g.) VIPs will
    not be allocated when cluster does not have assigned nodes.
    
    Tests updated accordingly
    
    Change-Id: Iaa94453f3f98cc9238f9810aab7311ffabbfa8b7
    Closes-Bug: #1549254

commit a72d2f68628fdbf12ba98891a507a6c638e259ee
Author: Artem Roma <aroma@mirantis.com>
Date:   Tue Mar 1 15:38:37 2016 +0200

Add handler for copying of VIPs
    
    New handler triggers copying of VIPs from given original cluster to new
    one. The reason for separation of this action from copying of network
    configuration is that accordion to [1] copying of existing VIPs/allocation
    of new ones will not take effect for new cluster unless it has assigned
    nodes. Thus in context of cluster upgrade procedure VIP transfer must be
    done after node reassignment, and as long as nodes are being operated on
    one by one it would be not efficient to call VIP copying method after
    each such reassignment.
    
    Tests updated accordingly.
    
    [1]: https://review.openstack.org/#/c/284841/
    
    Change-Id: I33670e8f2561be6fe18cec75bfc7ecc056ae2f6b
    Closes-Bug: #1552744

commit 6b5b5b8ad7935a158a83b7a90902a04ef6fa8f93
Author: Alexander Kislitsky <akislitsky@mirantis.com>
Date:   Tue Mar 22 09:31:58 2016 +0300

Performance of network manager operation improve
    
    We have extra SQLs generated in the NetworkManager when passing
    node_id instead already loaded SQLAlchemy node object.
    
    Additional changes:
    - Bulk insert used in IPs assiging process.
    - zip changed on six.moves.zip in the NetworkManager.
    - Removed unused function get_admin_ips_for_interfaces from NetworkManager.
    
    Co-Authored-By: Dmitry Guryanov <dguryanov@mirantis.com>
    
    Partial-Bug: #1498365
    Change-Id: I0518a5879c775d568de5652dbdd856a0cede80ce

commit 715da75c3b27068d89b1b79ca66025d628407d92
Author: Ilya Kharin <akscram@gmail.com>
Date:   Sun Feb 14 23:15:32 2016 -0600

Reassign nodes without reinstallation
    
    In some upgrade scenarios when shadow environments are used some of
    nodes should not be reprovisioned during this procedure. It is useful in
    combination when control plane nodes are reprovisioned and data plane
    nodes are updated in place.
    
    The update_cluster_assignment method of the objects.Node class was
    changed to accept roles and pending_roles for node during the
    reassignment. It allows to specify proper roles by the upgrade
    extention.
    
    The NodeReassignHandler handler accepts two additional parameters in the
    request body:
      - reprovision = True (default) - allows to skip the reprovision step
      - roles = [] (default) - allows to specify new roles or preserve the
                               current roles if empty
    
    Two additional methods were added to NailgunClusterAdapter and
    NailgunReleaseAdapter respectively.
    
    Change-Id: Iedb20a904e58f5b9a86eb47de8d8d317dc3cc61b
    Blueprint: upgrade-major-openstack-environment
    Closes-Bug: #1558655

commit 3522118cce643c64ac16b8584274f8ab6356b25a
Author: Ilya Kharin <akscram@gmail.com>
Date:   Mon Mar 7 21:04:04 2016 -0600

Use the 9.0 release in tests for cluster_upgrade
    
    This patch also renames names of variables in tests of the
    cluster_upgrade extension from more specific names that contain versions
    of releases to least specific names.
    
    Change-Id: Ic02b4426c1bb0bf38ba06cfeb54255043ebc8481
    Closes-Bug: #1555339

commit 9e4ead20e5095246ebe1c724d2ef1a3e0ef41b97
Author: Fedor Zhadaev <fzhadaev@mirantis.com>
Date:   Mon Feb 8 11:42:00 2016 +0300

Remove Mirantis-specific code from fuel-web repo
    
    Remove code related to registration in Mirantis tracking system and
    using Mirantis server to collect statistics.
    
    Change-Id: Ie1243a8b12368a0d61cd51da2d5ab6cce3eeea65
    Related to blueprint remove-vendor-code
    Depends-On: Id67d6201cb23e371dc42ec6e818bcbc2ca0fde31

commit 91d398c6a8ef2b75638cca1adbc96c03a350a77b
Author: Ryan Moe <rmoe@mirantis.com>
Date:   Tue Oct 27 14:54:43 2015 -0700

Move all db queries from network manager to objects
    
    All network-related database queries are moved into the appropriate
    object methods. This is being done to make it possible to have an external
    network management service.
    
    Blueprint: network-config-refactoring
    Change-Id: I4ce965f227c54577659e64f598ff5cdf4c868ed6

commit 0de1187a497b4bf571836ec95e6f4ea4f60a998b
Author: Sylwester Brzeczkowski <sbrzeczkowski@mirantis.com>
Date:   Tue Dec 22 11:22:08 2015 +0100

Nailgun extensions in stevedore
    
    The change introduces Nailgun extensions which use stevedore to
    plug into nailgun. Stevedore enables (not only) Fuel developers
    to write extensions and develop them independently in separate
    repos.
    
    Functions for loading and retriving extensions classes was
    moved to new file `manager.py`.
    
    Change-Id: I59015a28f460b1e45312b1c003aadec3cc396ad5
    Implements: blueprint stevedore-extensions-discovery

commit 4657f61259874a8ee57b5e4978260627c9df1c54
Author: Ivan Kliuk <ivan.kliuk@gmail.com>
Date:   Sat Dec 5 17:16:38 2015 +0200

New VIP-related fields in the database
    
    * Add 'is_user_defined' field to 'ip_addrs' table.
    * Rename 'vip_type' field to 'vip_name' of table 'ip_addrs'.
    * Add 'vip_namespace' field to 'ip_addrs' table.
    * Copy vip namespaces from plugin table network roles to 'vip_namespace' field
      according to the unique vip name.
    * Add database migrations.
    * Add unit test for migrations.
    
    Change-Id: Ia3e1d7f6e08dbebcb182de75eeaf58ddf6be4a8d
    Partial-bug: #1482399

commit 5c1319c48912cc0c58d4676b020592bd3194a789
Author: Oleg Gelbukh <ogelbukh@mirantis.com>
Date:   Thu Oct 22 17:57:23 2015 +0000

Change openstack_version to liberty-9.0 in openstack.yaml
    
    Set release name to 'Liberty' in display name and description of new release.
    Set openstack_version setting to 'liberty-9.0' to comply with the existing
    versions schema.
    Set opesntack_version to 'liberty-9.0' in tests.
    
    Closes-bug: 1503663
    Change-Id: Ifef952e18f08b98c430bbff9434984deaa68df81

commit 74226e20cc08691354913d1aa3cd4041f32fff1f
Author: Andrey Shestakov <ashestakov@mirantis.com>
Date:   Mon Nov 2 17:08:04 2015 +0200

Patch network_roles_metadata for 7.0 release where needed
    
    The patching mostly done for tests for nailgun components adhering to
    the release version. E.g. network serializers. Some tests are forced to
    create environments of 8.0 version. The change is needed to resolve
    possible issues with the tests when network roles metadata is changed
    drastically and is not compatible with mentioned components.
    
    Partial-Bug: #1517874
    Change-Id: I55607157ae7767ffdfd1d855b872630832706e8e

commit 0e73f2122d0a65ce052218ee0c3b62b7c56e83d8
Author: Oleg Gelbukh <ogelbukh@mirantis.com>
Date:   Fri Oct 30 13:54:28 2015 +0000

Rename ClusterUpgradeHandler
    
    Name ClusterUpgradeHandler should be used for handling uri
    /cluster/<id>/upgrade.
    
    Current handler performs clone of environment for upgrading. Thus, rename
    ClusterUpgradeHandler to ClusterUpgradeCloneHandler.
    
    Blueprint: upgrade-major-openstack-environment
    Change-Id: I6c98e0882c300d587caa32429b49b22baa9ad82f

commit 84ece08e252eb134ac218b30dcf6a8e3ebca29c9
Author: Maciej Kwiek <mkwiek@mirantis.com>
Date:   Tue Oct 20 11:40:25 2015 +0200

Replace release.is_deployable with release.state
    
    is_deployable doesn't really give any additional information for a
    release, it is removed. To make change API-backward-compatible, it is
    retained as a property dependent on release.state.
    
    New state is added - manageonly, for environments that are not able to
    be deployed, but can still be managed.
    
    Change-Id: I518a0114730a2f227c9ef035a376f9a90d3d5bbd
    Closes-bug: #1503303
    DocImpact

commit fb275424635aea72eeecace83c18b487c4d72d4b
Author: Ilya Kharin <akscram@gmail.com>
Date:   Tue Aug 18 14:37:25 2015 +0300

Set the node_id param as a required property
    
    The node_id property is a required property to perform the re-assigning
    a node with a given ID from one cluster to another.
    
    Change-Id: I2442442260f19833db239b01856d0ff4f63ecfbb
    Closes-Bug: #1483239

commit f7f28096f198644a31d94f22c0c88fedeb718e08
Author: Ilya Kharin <akscram@gmail.com>
Date:   Tue Aug 11 16:23:08 2015 +0300

Rename types of VIPs during upgrade to 7.0
    
    In the 7.0 release the networking templates were introduced. They use
    the ip.addrs.vip_type column as names of VIPs and these names differ
    from names of previous releases. To solve this we can renamed VIPs of
    older releases during upgrade to 7.0 accoring the rules:
    
        management: haproxy -> management
        public: haproxy -> public
        public: vrouter -> vrouter_pub
    
    Change-Id: Ia77d13ea90408a06896f2a49c6e43d44c6af1d0d
    Closes-Bug: #1482577

commit 6782ddfe16fad80da574f71d7b4f4b1ba906dab2
Author: Ilya Kharin <akscram@gmail.com>
Date:   Sun Jul 19 23:50:40 2015 +0300

Directly assign node to an upgrade cluster
    
    The patch adds method that assigns a node to an upgrade cluster without
    deleting it from DB. This allows to keep ID of the node and IP addresses
    assigned to it. The node is booted into the bootstrap image as soon as
    it moves to an upgrade cluster.
    
    Implements blueprint: nailgun-api-env-upgrade-extensions
    Co-Authored-By: Artur Svechnikov <asvechnikov@mirantis.com>
    Change-Id: If10fadd149a32317420778607146d9d12108d3f9

commit 29bbd906085e4d8c6089c657976fbda1a4b690e8
Author: Ilya Kharin <akscram@gmail.com>
Date:   Tue Jul 28 05:03:32 2015 +0300

Introduce extension to upgrade clusters
    
    The patch adds an extension which implements the procedure of upgrading
    clusters from one major release to another. As a first step of the
    procedure the extension provides an ability to create a seed cluster
    with the same settings as the original.
    
    Implements blueprint: nailgun-api-env-upgrade-extensions
    Change-Id: I22d51a3ffd51a7c88bdcbde0eef6f47b65def1c8

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Related fix merged to fuel-octane (master)

#14

Reviewed: https://review.openstack.org/365169
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=9a4817174ed8b0702564baadc1728eaf8b778e28
Submitter: Jenkins
Branch: master

commit 9a4817174ed8b0702564baadc1728eaf8b778e28
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 01:04:10 2016 +0300

Move a set of nodes at once

Before this patch octane made separate calls to move nodes one by one.

Change-Id: I999a98d57b3184d35972e4862fcb4f284a066e9e
Related-Bug: #1616925

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Related fix proposed to fuel-octane (stable/mitaka)

#15

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/367652

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-08: Related fix merged to fuel-octane (stable/mitaka)

#16

Download full text (4.3 KiB)

Reviewed: https://review.openstack.org/367652
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=293b79e6490605d9e74d2e2b6825146b49fdecfb
Submitter: Jenkins
Branch: stable/mitaka

commit 9a4817174ed8b0702564baadc1728eaf8b778e28
Author: Ilya Kharin <email address hidden>
Date: Sat Sep 3 01:04:10 2016 +0300

Move a set of nodes at once

Before this patch octane made separate calls to move nodes one by one.

Change-Id: I999a98d57b3184d35972e4862fcb4f284a066e9e
Related-Bug: #1616925

commit 763b69b97645752b4b08253751962687d50cf1be
Author: Alexander Tsamutali <email address hidden>
Date: Thu Sep 8 22:03:24 2016 +0300

Cleanup %files section in spec

Don't use --record during install, don't use -f in %files. Specify
octane files/directories explicitly.

Change-Id: I84f0d71a2a582b3a23fc048a331d6caae775e38b
Closes-Bug: #1619319

commit e7fad96f4125386df4e5c9b7f0744de185de6c33
Author: Sergey Abramov <email address hidden>
Date: Mon Sep 5 18:24:37 2016 +0300

Osd upgrade failed if version not changed

ceph deploy not raise exception if it doesn't upgrade osd version.

Change-Id: Ifcddd822228d78166d59b2ba49852be2e51c79fc
Closes-bug: 1620277

commit 349073cbe399184daee672f75b4fe9941ae3c5da
Author: Sergey Abramov <email address hidden>
Date: Wed Sep 7 18:20:33 2016 +0300

Remove patch pupet on upgrade osd

this is not required after merge
https://review.openstack.org/#/c/203639/

Change-Id: I67fbcd77ab3437443219c34ee3ddaf7895b068ce
Closes-bug: 1621436

commit 807c5166a8a0145623d41c3110007e33a1402c47
Author: Alexey Stepanov <email address hidden>
Date: Wed Sep 7 17:53:40 2016 +0300

Stop waiting status change on node with stopped status

Stopped status, if not expected, should be a reason for error on nodes
Closes-bug: #1621069

Change-Id: I0156d694ef20ece8603e3d840f085852a528e635

commit 5f689f3d6904e092998a0d8ba124280833452c30
Author: Anastasiya <email address hidden>
Date: Tue Sep 6 10:23:39 2016 +0300

Backup/restore for admin networks

* backup/restore for /etc/hiera/networks.yaml was added
* configure dhcp after restore was added

Change-Id: I5b1e3861589e1c56acbc37d0be569da5e55b8536
Closes-Bug: #1616998

commit e4820087678d83d9477db4c1688137ee5ff66c3f
Author: Pavel Chechetin <email address hidden>
Date: Thu Aug 18 14:46:59 2016 +0300

Graph-based upgrade-ceph. Python part.

Change-Id: Icb4d543bd6801f21c6aca57415105f88a601c0c2

commit 0af1b6517a97589a401328f09f3bf56d84db9dbd
Author: Pavel Chechetin <email address hidden>
Date: Tue Sep 6 13:13:42 2016 +0300

Graph-based upgrade-ceph. Puppet part.

     - Graphs and puppet manifests part
     - Delete lib from .gitignore
     - Augeas lens for Ceph is copied and pasted, should be switched to the
       version from the upstream when [0] is merged and published.

[0] https://github.com/hercules-team/augeas/pull/401

Change-Id: I639cbf786971fea8c56b4da6b2661477b3b12c41

commit 3315d741f295635303413ad839b3b66a0dac3282
Author: Alexey Stepanov <penguinolog@gmail....

Reviewed:  https://review.openstack.org/367652
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=293b79e6490605d9e74d2e2b6825146b49fdecfb
Submitter: Jenkins
Branch:    stable/mitaka

commit 9a4817174ed8b0702564baadc1728eaf8b778e28
Author: Ilya Kharin <akscram@gmail.com>
Date:   Sat Sep 3 01:04:10 2016 +0300

Move a set of nodes at once
    
    Before this patch octane made separate calls to move nodes one by one.
    
    Change-Id: I999a98d57b3184d35972e4862fcb4f284a066e9e
    Related-Bug: #1616925

commit 763b69b97645752b4b08253751962687d50cf1be
Author: Alexander Tsamutali <atsamutali@mirantis.com>
Date:   Thu Sep 8 22:03:24 2016 +0300

Cleanup %files section in spec
    
    Don't use --record during install, don't use -f in %files. Specify
    octane files/directories explicitly.
    
    Change-Id: I84f0d71a2a582b3a23fc048a331d6caae775e38b
    Closes-Bug: #1619319

commit e7fad96f4125386df4e5c9b7f0744de185de6c33
Author: Sergey Abramov <sabramov@mirantis.com>
Date:   Mon Sep 5 18:24:37 2016 +0300

Osd upgrade failed if version not changed
    
    ceph deploy not raise exception if it doesn't upgrade osd version.
    
    Change-Id: Ifcddd822228d78166d59b2ba49852be2e51c79fc
    Closes-bug: 1620277

commit 349073cbe399184daee672f75b4fe9941ae3c5da
Author: Sergey Abramov <sabramov@mirantis.com>
Date:   Wed Sep 7 18:20:33 2016 +0300

Remove patch pupet on upgrade osd
    
    this is not required after merge
    https://review.openstack.org/#/c/203639/
    
    Change-Id: I67fbcd77ab3437443219c34ee3ddaf7895b068ce
    Closes-bug: 1621436

commit 807c5166a8a0145623d41c3110007e33a1402c47
Author: Alexey Stepanov <astepanov@mirantis.com>
Date:   Wed Sep 7 17:53:40 2016 +0300

Stop waiting status change on node with stopped status
    
    Stopped status, if not expected, should be a reason for error on nodes
    Closes-bug: #1621069
    
    Change-Id: I0156d694ef20ece8603e3d840f085852a528e635

commit 5f689f3d6904e092998a0d8ba124280833452c30
Author: Anastasiya <atolochkova@mirantis.com>
Date:   Tue Sep 6 10:23:39 2016 +0300

Backup/restore for admin networks
    
    * backup/restore for /etc/hiera/networks.yaml was added
    * configure dhcp after restore was added
    
    Change-Id: I5b1e3861589e1c56acbc37d0be569da5e55b8536
    Closes-Bug: #1616998

commit e4820087678d83d9477db4c1688137ee5ff66c3f
Author: Pavel Chechetin <pchechetin@mirantis.com>
Date:   Thu Aug 18 14:46:59 2016 +0300

Graph-based upgrade-ceph. Python part.
    
    Change-Id: Icb4d543bd6801f21c6aca57415105f88a601c0c2

commit 0af1b6517a97589a401328f09f3bf56d84db9dbd
Author: Pavel Chechetin <pchechetin@mirantis.com>
Date:   Tue Sep 6 13:13:42 2016 +0300

Graph-based upgrade-ceph. Puppet part.
    
     - Graphs and puppet manifests part
     - Delete lib from .gitignore
     - Augeas lens for Ceph is copied and pasted, should be switched to the
       version from the upstream when [0] is merged and published.
    
    [0] https://github.com/hercules-team/augeas/pull/401
    
    Change-Id: I639cbf786971fea8c56b4da6b2661477b3b12c41

commit 3315d741f295635303413ad839b3b66a0dac3282
Author: Alexey Stepanov <penguinolog@gmail.com>
Date:   Wed Sep 7 09:38:04 2016 +0300

Colorize output on tty only
    
    Check ConsoleHandler stream for tty and colorize only if True.
    On CI colorized output is not processed and metacharacters
    simply reproduced in output as-is.
    
    Change-Id: I802e3d338c318d6fbb61dd556e13a2fad7eda121

commit 1d87cfefa8c7f93bd72ac8a0236d128ced1e8c0e
Author: Nguyen Hung Phuong <phuongnh@vn.fujitsu.com>
Date:   Wed Sep 7 11:46:45 2016 +0700

Clean imports in code
    
    In some part in the code we import objects. In the Openstack style
    guidelines they recommend to import only modules.
    
    http://docs.openstack.org/developer/hacking/#imports
    
    Change-Id: I5f5add3ad87c8e9b3a6d781c81c3bedf5b4e1c9c

commit dcb147e11b0fabcfe1acfbc44ca7503c5a717264
Author: Ilya Kharin <akscram@gmail.com>
Date:   Tue Sep 6 23:16:08 2016 +0300

Use graph_types to reflect changes in fuelclient
    
    The GraphClient.execute method was changed and this patch compensates
    such changes. The interface of GraphClient was changed by the next
    change request:
    
        Iafc59c60a1ebce4a360ce1faa00a27ab320fa90b
    
    Change-Id: I68523afe49de6da9f9792200419dcc35843b07d8
    Closes-Bug: #1620809

Ilya Kharin (akscram) on 2016-09-09

Changed in fuel:
status:	Confirmed → Fix Committed

Ilya Kharin (akscram) on 2016-09-13

Changed in fuel:
status:	Fix Committed → In Progress
milestone:	8.0-updates → 8.0-mu-4

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-26: Related fix proposed to fuel-web (stable/8.0)

#17

Related fix proposed to branch: stable/8.0
Review: https://review.openstack.org/376453

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-26: Related fix proposed to fuel-octane (stable/8.0)

#18

Related fix proposed to branch: stable/8.0
Review: https://review.openstack.org/376463

Ilya Kharin (akscram) on 2016-10-03

Changed in fuel:
assignee:	Ilya Kharin (akscram) → Sergey Abramov (sabramov)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-03: Related fix merged to fuel-web (stable/8.0)

#19

Reviewed: https://review.openstack.org/376453
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=390756d3658937b9cface7d30176ffd0e7ede9d3
Submitter: Jenkins
Branch: stable/8.0

commit 390756d3658937b9cface7d30176ffd0e7ede9d3
Author: Sergey Abramov <email address hidden>
Date: Mon Sep 26 15:57:09 2016 +0300

Add support to re-assign a set of nodes

    This patch adds an ability to re-assign a set of the given nodes at
    once. This feature was technically available but not exposed to the
    client. A groupped re-assigning allows to effectively re-provision nodes
    by creating an atomic task in Astute.

Change-Id: Ia239d37ec0497de73715640e62205f6d3252f61b
Related-Bug: #1616925

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-03: Fix merged to fuel-octane (stable/8.0)

#20

Reviewed: https://review.openstack.org/376463
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=58a9e6b2d3b1e268a995d523c395bf2d45bf051e
Submitter: Jenkins
Branch: stable/8.0

commit 58a9e6b2d3b1e268a995d523c395bf2d45bf051e
Author: Sergey Abramov <email address hidden>
Date: Mon Sep 26 16:07:30 2016 +0300

Move a set of nodes at once

Before this patch octane made separate calls to move nodes one by one.
Partial-Bug: #1616925

Change-Id: I999a98d57b3184d35972e4862fcb4f284a066e9e

Revision history for this message

Alexander Gordeev (a-gordeev) wrote on 2016-10-03:

#21

astute_actions.log Edit (1.7 KiB, text/plain)

Revision history for this message

Alexander Gordeev (a-gordeev) wrote on 2016-10-03:

#22

partial_astute.log Edit (42.8 KiB, text/plain)

Revision history for this message

Alexander Gordeev (a-gordeev) wrote on 2016-10-03:

#23

Guys, i've done with astute log analysis. Good thing you've got only 2 nodes, so it was relatively easy to track all astute actions what happened.

I've attached to file: one with briefly decribed actions and the corresponding part of more verbose log.

My initial thought was that you've stepped into https://bugs.launchpad.net/fuel/+bug/1626962 but in fact, no any certain evidences about that was found at astute logs for booting into broken cobbler profile.

This never happened.

There're only 2 thing i can suspect:
1) fake positive result of rebooting into bootstrap image, nodes weren't rebooted and remained in target operating system.
2) successful reboot into bootstrap image, but image didn't have fuel-agent package installed (so provisioning failed complaining about not found 'provision' binary)

Too bad, there's no logs from bootstrap image builder.

For the 1) astute could just ensure that after 'reboot_reprovisioned_nodes' nodes were loaded into bootstrap image and have speficic node type such as 'bootstrap' https://github.com/openstack/fuel-agent/blob/master/contrib/fuel_bootstrap/files/trusty/etc/nailgun_systemtype

Revision history for this message

Alexander Gordeev (a-gordeev) wrote on 2016-10-03:

#24

perhaps, we should probably add additional check for node type of 'bootstrap' somewhere after 'reboot_reprovisioned_nodes' : https://github.com/openstack/fuel-astute/blob/stable/mitaka/lib/astute/provision.rb#L536-L543

Revision history for this message

Alexander Gordeev (a-gordeev) wrote on 2016-10-03:

#25

partial astute log from scale-lab Edit (6.2 KiB, text/plain)

Just found the same traces of

> :stderr=>"flock: provision: No such file or directory\n",

at our lab too. So i can vouch that there's nothing wrong with bootstrap image. It did contain fuel-agent package with '/usr/bin/provision' entry-point srcipt.

So, we should definitely prepare a fix astute.

Ilya Kharin (akscram) on 2016-10-17

Changed in fuel:
status:	In Progress → Fix Committed

Revision history for this message

TatyanaGladysheva (tgladysheva) wrote on 2017-04-18:

#26

Verified on 8.0 + MU4 updates.

Changed in fuel:
status:	Fix Committed → Fix Released

Affects		Status	Importance	Assigned to	Milestone
	Fuel for OpenStack	Fix Released	High	Sergey Abramov	Fuel for OpenStack 8.0-mu-4
	Mitaka	Fix Released	High	Ilya Kharin	Fuel for OpenStack 9.1

Fuel for OpenStack

[upgrade] Node can not be rebooted while upgrading

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches