[Upgrade] Re-deployment of simple 5.1.1 cluster failed after upgrading to 6.0

Bug #1403560 reported by Andrey Sledzinskiy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Ihor Kalnytskyi
6.0.x
Fix Released
Critical
Ihor Kalnytskyi
6.1.x
Fix Released
Critical
Ihor Kalnytskyi

Bug Description

{

    "build_id": "2014-12-17_03-38-09",
    "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
    "build_number": "50",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "dc621c96e30f0dd284998e975f3b800afe9fc082",
    "production": "docker",
    "fuelmain_sha": "fb89876812320464ebad60e7ca34b433d6b1d815",
    "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
    "feature_groups": [
        "mirantis"
    ],
    "release": "6.0",
    "release_versions": {
        "2014.1.3-5.1.1": {
            "VERSION": {
                "build_id": "2014-12-03_01-07-36",
                "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
                "build_number": "48",
                "api": "1.0",
                "nailgun_sha": "500e36d08a45dbb389bf2bd97673d9bff48ee84d",
                "production": "docker",
                "fuelmain_sha": "7626c5aeedcde77ad22fc081c25768944697d404",
                "astute_sha": "ef8aa0fd0e3ce20709612906f1f0551b5682a6ce",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "5.1.1",
                "fuellib_sha": "a3043477337b4a0a8fd166dc83d6cd5d504f5da8"
            }
        },
        "2014.2-6.0": {
            "VERSION": {
                "build_id": "2014-12-17_03-38-09",
                "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
                "build_number": "50",
                "api": "1.0",
                "nailgun_sha": "dc621c96e30f0dd284998e975f3b800afe9fc082",
                "production": "docker",
                "fuelmain_sha": "fb89876812320464ebad60e7ca34b433d6b1d815",
                "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "6.0",
                "fuellib_sha": "484f53f586c084cc38c88615b0594d9c48882c5a"
            }
        }
    },
    "fuellib_sha": "484f53f586c084cc38c88615b0594d9c48882c5a"

}

Steps:
1. Create next fuel 5.1.1-48 cluster - Ubuntu, simple, Neutron Vlan, ceph for volumes, ceilometer, sahara, 1 controller+mongo, 2 compute nodes, 2 ceph nodes
2. Deploy cluster
3. Upload 6.0-50 upgrade script to master
4. Upgrade fuel to 6.0
5. Add 1 compute node to cluster
6. Re-deploy

Actual - re-deployment failed, error in puppet.log of node-6
Wed Dec 17 14:54:12 +0000 2014 Puppet (err): Invalid parameter other_nets on node node-6.test.domain.local

Logs are attached

Tags: docs upgrade
Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :
Changed in fuel:
status: New → Confirmed
importance: High → Critical
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

It looks like the puppet manifests for the upgraded env is correct 5.1.1, but the astute.yaml generated includes several extra parameters, including:
other_nets
default_gateway

It looks like we're using the wrong deployment serializer for deploying node-6.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/142518

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/6.0)

Fix proposed to branch: stable/6.0
Review: https://review.openstack.org/142519

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/142518
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=4e7a99598b2ced232a90c18af79ed80f02d540ca
Submitter: Jenkins
Branch: master

commit 4e7a99598b2ced232a90c18af79ed80f02d540ca
Author: Igor Kalnitsky <email address hidden>
Date: Wed Dec 17 19:43:16 2014 +0200

    Serialize default_gateway/other_nets for 6.0 only

    Both default_gateway and other_nets fields have been introduced in Fuel 6.0,
    but accidently were serialized for old envs too. This led to puppet
    errors when we tried to re-deploy node in one of the old envs.

    Change-Id: I7e4ba8f3a6a0ba822c47b9c14fe4d02b3523d658
    Closes-Bug: #1403560

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/6.0)

Reviewed: https://review.openstack.org/142519
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=5f91157daa6798ff522ca9f6d34e7e135f150a90
Submitter: Jenkins
Branch: stable/6.0

commit 5f91157daa6798ff522ca9f6d34e7e135f150a90
Author: Igor Kalnitsky <email address hidden>
Date: Wed Dec 17 19:43:16 2014 +0200

    Serialize default_gateway/other_nets for 6.0 only

    Both default_gateway and other_nets fields have been introduced in Fuel 6.0,
    but accidently were serialized for old envs too. This led to puppet
    errors when we tried to re-deploy node in one of the old envs.

    Change-Id: I7e4ba8f3a6a0ba822c47b9c14fe4d02b3523d658
    Closes-Bug: #1403560

tags: added: docs
Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

verified on {

    "build_id": "2014-12-18_01-32-01",
    "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
    "build_number": "56",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90",
    "production": "docker",
    "fuelmain_sha": "45caacadb878abfbd9d60e134d72229698b469c9",
    "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
    "feature_groups": [
        "mirantis"
    ],
    "release": "6.0",
    "release_versions": {
        "2014.1.3-5.1.1": {
            "VERSION": {
                "build_id": "2014-12-03_01-07-36",
                "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
                "build_number": "48",
                "api": "1.0",
                "nailgun_sha": "500e36d08a45dbb389bf2bd97673d9bff48ee84d",
                "production": "docker",
                "fuelmain_sha": "7626c5aeedcde77ad22fc081c25768944697d404",
                "astute_sha": "ef8aa0fd0e3ce20709612906f1f0551b5682a6ce",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "5.1.1",
                "fuellib_sha": "a3043477337b4a0a8fd166dc83d6cd5d504f5da8"
            }
        },
        "2014.2-6.0": {
            "VERSION": {
                "build_id": "2014-12-18_01-32-01",
                "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
                "build_number": "56",
                "api": "1.0",
                "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90",
                "production": "docker",
                "fuelmain_sha": "45caacadb878abfbd9d60e134d72229698b469c9",
                "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "6.0",
                "fuellib_sha": "73332192a257ea02c40a39885c502ad1ebdf3eda"
            }
        }
    },
    "fuellib_sha": "73332192a257ea02c40a39885c502ad1ebdf3eda"

}

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

verified on {

    "build_id": "2015-01-16_10-22-15",
    "ostf_sha": "92ad9f8e4c509c82e07ceb093b5d579205c76014",
    "build_number": "64",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "13e4fb6c9ec62982b4c5870b400e695af126b0ff",
    "production": "docker",
    "fuelmain_sha": "",
    "astute_sha": "82125b0eef4e5a758fd4afa8917812e09a1f7dac",
    "feature_groups":

[

    "mirantis"

],
"release": "6.1",
"release_versions":
{

    "2014.2-6.0":

{

    "VERSION":

{

    "build_id": "2014-12-26_14-25-46",
    "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
    "build_number": "58",
    "api": "1.0",
    "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90",
    "production": "docker",
    "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8",
    "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
    "feature_groups":

        [
            "mirantis"
        ],
        "release": "6.0",
        "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"
    }

},
"2014.2-6.1":
{

    "VERSION":

{

    "build_id": "2015-01-16_10-22-15",
    "ostf_sha": "92ad9f8e4c509c82e07ceb093b5d579205c76014",
    "build_number": "64",
    "api": "1.0",
    "nailgun_sha": "13e4fb6c9ec62982b4c5870b400e695af126b0ff",
    "production": "docker",
    "fuelmain_sha": "",
    "astute_sha": "82125b0eef4e5a758fd4afa8917812e09a1f7dac",
    "feature_groups":

                [
                    "mirantis"
                ],
                "release": "6.1",
                "fuellib_sha": "59af43598682f4f0c5aebf584a959ac730a4d86d"
            }
        }
    },
    "fuellib_sha": "59af43598682f4f0c5aebf584a959ac730a4d86d"

}

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.