Network verification failed after upgrading FUEL.

[Vasily Gorin]

version :
{"build_id": "2015-05-27_17-33-07", "build_number": "474", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-12-26_14-25-46", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "58", "api": "1.0", "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90", "production": "docker", "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"}}, "2014.2.2-6.1": {"VERSION": {"build_id": "2015-05-27_17-33-07", "build_number": "474", "api": "1.0", "fuel-library_sha": "05b59b7c9279222de734295535d86f53dd3d4225", "nailgun_sha": "ac8668cc06368fe22330e293c9ce8655d46846bd", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "4cd2fef040ae9e7645a6b17a7cb44d3cd8fbe0be", "release": "6.1", "fuelmain_sha": "6b5712a7197672d588801a1816f56f321cbceebd"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "05b59b7c9279222de734295535d86f53dd3d4225", "nailgun_sha": "ac8668cc06368fe22330e293c9ce8655d46846bd", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "e19f1b65792f84c4a18b5a9473f85ef3ba172fce", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "4cd2fef040ae9e7645a6b17a7cb44d3cd8fbe0be", "release": "6.1", "fuelmain_sha": "6b5712a7197672d588801a1816f56f321cbceebd"}

Steps:
1. Upgrade FEUL
2. Create environment with 3 controller, 2 compute and 1 Cinder
3. Verify networks

Expected result:
Verification should passed

Actual result:
Verification failed.
Method verify_networks. 9cdf78d0-71cb-4229-9e7f-85bff368c3b6: MCollective call failed in agent 'net_probe', method 'check_url_retrieval', failed nodes: ID: 5 - Reason: Attempted to call action 'check_url_retrieval' for 'net_probe' but it's not declared in the DDL ID: 6 - Reason: Attempted to call action 'check_url_retrieval' for 'net_probe' but it's not declared in the DDL ID: 4 - Reason: Attempted to call action 'check_url_retrieval' for 'net_probe' but it's not declared in the DDL . Inspect Astute logs for the details

[Łukasz Oleś (loles)]

You should reboot bootstrap nodes after upgrade

[Ihor Kalnytskyi (ikalnytskyi)]

Łukasz is right here. Bootstrap nodes have to be rebooted. That's by design.

Vasily, could you please confirm that you didn't reboot them?

[Ihor Kalnytskyi (ikalnytskyi)]

I think the best thing we can do is to show message after successful upgrade that you have to reboot bootstrap nodes, if you want to get new features.

[Evgeniy L (rustyrobot)]

Igor, yeah, agree, probably we can just show a message to ask user to reboot bootstrap nodes.

[Vasily Gorin (vgorin)]

I didn't reboot bootstrap nodes. I agree it will be good to notify users about it.

[Vasily Gorin (vgorin)]

Thanks. I did it and it solves this problem.

[Dmitry Pyzhov (dpyzhov)]

We should show a notification 'please reboot your bootstrap nodes in order to make sure that they use the latest bootstrap image' after upgrade.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/186698
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=3e9ef8b84cc510ed04e2af5a8b408e3da13af0b6
Submitter: Jenkins
Branch: master

commit 3e9ef8b84cc510ed04e2af5a8b408e3da13af0b6
Author: Maciej Kwiek <email address hidden>
Date: Fri May 29 11:16:01 2015 +0200

    Notice about restarting slave nodes after upgrade

    Change-Id: I0d4d42526764808e4d081118b35bf9a67eec5841
    Closes-Bug: #1459658

[Maksym Strukov (unbelll)]

Verified as fixed in 6.1-496

{"build_id": "2015-06-02_06-16-08", "build_number": "496", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-12-26_14-25-46", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "58", "api": "1.0", "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90", "production": "docker", "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"}}, "2014.2.2-6.1": {"VERSION": {"build_id": "2015-06-02_06-16-08", "build_number": "496", "api": "1.0", "fuel-library_sha": "5a2146c80186cd74021cf1a55449286c3c59a60b", "nailgun_sha": "3830bdcb28ec050eed399fe782cc3dd5fbf31bde", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "7413186490e8d651b8837b9eee75efa53f5e230b", "release": "6.1", "fuelmain_sha": "bcc909ffc5dd5156ba54cae348b6a07c1b607b24"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "5a2146c80186cd74021cf1a55449286c3c59a60b", "nailgun_sha": "3830bdcb28ec050eed399fe782cc3dd5fbf31bde", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "5d570ae5e03909182db8e284fbe6e4468c0a4e3e", "fuel-ostf_sha": "7413186490e8d651b8837b9eee75efa53f5e230b", "release": "6.1", "fuelmain_sha": "bcc909ffc5dd5156ba54cae348b6a07c1b607b24"}

Success message:

2015-06-02 15:50:59 INFO 9889 (upgrade) *** UPGRADING MASTER NODE DONE SUCCESSFULLY
2015-06-02 15:50:59 INFO 9889 (upgrade) *** PLEASE REBOOT YOUR BOOTSTRAP NODES IN ORDER TO MAKE SURE THAT THEY USE THE LATEST BOOTSTRAP IMAGE