[upgrade 7-8] Ceph crash on upgrade with network template

Bug #1605655 reported by Sergey Abramov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Sergey Abramov

Bug Description

Detailed bug description:
Upgrade crash if cluster has custom network template.
Octane overwrite ceph monmap on upgrade ceph step even if cluster has specific storage network.

Steps to reproduce:
install 7.0 fuel
install cluster on 7.0
upgrade 7.0 to 8.0 master node
upgrade primary-controller
upgrade db
upgrade ceph
upgrade remaining controllers

Actual result:
all controllers error

Expected results:
all controllers ready

Reproducibility:
100%

summary: - [upgrade 7-8] Ceph crash with upgrade with network template
+ [upgrade 7-8] Ceph crash on upgrade with network template
Changed in fuel:
assignee: nobody → Sergey Abramov (sabramov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (master)

Fix proposed to branch: master
Review: https://review.openstack.org/346508

Changed in fuel:
status: New → In Progress
Changed in fuel:
milestone: none → 8.0-updates
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (master)

Reviewed: https://review.openstack.org/346508
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=18c276d33ad0b247e9cd461e03512ea3a3637aad
Submitter: Jenkins
Branch: master

commit 18c276d33ad0b247e9cd461e03512ea3a3637aad
Author: Sergey Abramov <email address hidden>
Date: Sun Jul 24 12:41:10 2016 +0300

    Don't overwrite ips in monmap on upgrade ceph

    Change only fsid in monmap on upgrade ceph. This is required if user set
    up sepcial network setting in network template.

    Closes-bug: 1605655
    Change-Id: Ib7b10ce604e1c16960a70a74a1cabcd983cca7ab

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/8.0)

Reviewed: https://review.openstack.org/346036
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=031887135b2be88bd161c43e6609b3cf75308359
Submitter: Jenkins
Branch: stable/8.0

commit 031887135b2be88bd161c43e6609b3cf75308359
Author: Sergey Abramov <email address hidden>
Date: Sun Jul 24 12:41:10 2016 +0300

    Don't overwrite ips in monmap on upgrade ceph

    Change only fsid in monmap on upgrade ceph. This is required if user set
    up sepcial network setting in network template.

    Closes-bug: 1605655
    Change-Id: Ib7b10ce604e1c16960a70a74a1cabcd983cca7ab
    (cherry picked from commit 18c276d33ad0b247e9cd461e03512ea3a3637aad)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (master)

Fix proposed to branch: master
Review: https://review.openstack.org/351721

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-octane (master)

Change abandoned by Ilya Kharin (<email address hidden>) on branch: master
Review: https://review.openstack.org/351721
Reason: This patch have to go in stable/mitaka instead of master.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-octane (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/351728

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-octane (stable/mitaka)
Download full text (17.8 KiB)

Reviewed: https://review.openstack.org/351728
Committed: https://git.openstack.org/cgit/openstack/fuel-octane/commit/?id=3101bf302ea0f2d501fd42a6ae6046c3587ac093
Submitter: Jenkins
Branch: stable/mitaka

commit a05d2ecf13e15c12f10e1517fae425e583e0d714
Author: Nikita Zubkov <email address hidden>
Date: Thu Aug 4 17:51:56 2016 +0300

    fix ssh connection auth

    According to spec [1] service user must use key authentication​ for ssh

    [1] https://github.com/openstack/fuel-specs/blame/ba4504/specs/9.0/fuel-nonroot-openstack-nodes.rst#L61-L64

    Change-Id: I461f659d4f0985283a43ec6a702bdd78af91a072

commit 7140398e501879a4e1f5605f3d1cc99195464850
Author: Sergey Abramov <email address hidden>
Date: Tue Aug 2 12:12:57 2016 +0300

    Setup upgrade levels for each fuel version

    Setup upgrade levels block in nova.conf for each fuel version,
    this is required for backward compatibility on RPC.

    Change-Id: I6ccbc34457d11616b015cdb1e46e733b0b49ce58

commit 9e2a419ac79ad591faee9afe16419bf4497eacad
Author: Vladimir Khlyunev <email address hidden>
Date: Mon Aug 1 21:31:56 2016 +0300

    Fix several typos

    Copypasting is evil, lets remove typos

    Change-Id: Icf4bec1a3de954911a469fa98380af2b6175da08

commit 013a7ded2275245a38ee58289dac2bc8b90fa37e
Author: Sergey Abramov <email address hidden>
Date: Wed Jul 20 16:05:10 2016 +0300

    Host evacuation using nova

    Evacuate instance from compute node using
    nova host-evacuate-live except host_evacuation.sh script

    Change-Id: I01a04cfa1ed2aafce7987e523e6b589dea21fbb0

commit 00a1d0dce1f7ae5db6706a59353403516f7913dc
Author: Sergey Abramov <email address hidden>
Date: Tue Jul 19 17:10:44 2016 +0300

    Change shutoff vm command

    * run nova commands using nova util function
    * host filter using get nova node handle method(
        hostname for fuel < 6.1 and fqdn for newer version)

    Change-Id: Ib5c373714e359982b13910a225787167db7b35e9

commit 49357c98313c5f551b13d0877581c5fb5e108db1
Author: Nikita Zubkov <email address hidden>
Date: Mon Jun 27 18:40:13 2016 +0300

    Add `stderr_log_level` parameter to popen

    Now posible to set logging level in popen for logging stderr

    fuel-bootstrap build comand stderr now logged with INFO level

    Change-Id: Ibaa2be54c68ce7a428e07052c917f54e99b95a5e

commit fb582b271dab4bdd10d27149a77c0e1f6a580b9e
Author: Sergey Abramov <email address hidden>
Date: Thu Jul 21 17:56:41 2016 +0300

    Add run_nova_cmd function

    This is required for running nova command on contoller without
    manipulating manipulating with command.

    It makes code easier and more readable

    Usage:

        from fuelclient.objects import environment as environment_obj

        from octane.util import env
        from octane.util import nova

        env_id = ...
        my_env = environment_obj.Environment(env_id)
        controller = env.get_one_controller(my_env)
        nova_cmd_just_run = [...]
        nova_cmd_with_output = [...]
        nova.run_nova_cmd(nova_cmd_just_run, controller, False)
        result = nova.run_nova_cmd(nova_cmd_w...

tags: added: in-stable-mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-octane ocata-eol

This issue was fixed in the openstack/fuel-octane ocata-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.