After upgrading fuel network verification is failed on new env

[Vasily Gorin]

Was tested on KVM and installed using system tests.

Steps to reproduce:

    Before upgrade:
1. Create environment ( CentOS, Neutron GRE, 3 controller, 2 compute)
2. Change fuel admin password
3. Deploy the environment
Upgrade:
1. Load upgrade script to fuel and run it.

After upgrade:
1. Delete env and wait for running of bootstrapped nodes
2. Create new env(CentOS, NeutronVLAN, Install Ceilometer, 1 controller, 2 compute, 2 MongoDB)
3. Verify network

Expect result:
Verification is successful.

Actual result:
Verification failed.
These nodes: "13", "14", "11", "12", "15" failed to connect to some of these repositories: "http://mirror.fuel-infra.org/mos/centos-6/mos6.1/updates"

VERSION:

ISO before upgrade : fuel-6.0-58-2014-12-26_14-25-46.iso

After upgrade:

{"build_id": "2015-06-08_06-13-27", "build_number": "521", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-12-26_14-25-46", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "58", "api": "1.0", "nailgun_sha": "5f91157daa6798ff522ca9f6d34e7e135f150a90", "production": "docker", "fuelmain_sha": "81d38d6f2903b5a8b4bee79ca45a54b76c1361b8", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "fde8ba5e11a1acaf819d402c645c731af450aff0"}}, "2014.2.2-6.1": {"VERSION": {"build_id": "2015-06-08_06-13-27", "build_number": "521", "api": "1.0", "fuel-library_sha": "f43c2ae1af3b493ee0e7810eab7bb7b50c986c7d", "nailgun_sha": "4340d55c19029394cd5610b0e0f56d6cb8cb661b", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "7766818f079881e2dbeedb34e1f67e517ed7d479", "fuel-ostf_sha": "7c938648a246e0311d05e2372ff43ef1eb2e2761", "release": "6.1", "fuelmain_sha": "bcc909ffc5dd5156ba54cae348b6a07c1b607b24"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "f43c2ae1af3b493ee0e7810eab7bb7b50c986c7d", "nailgun_sha": "4340d55c19029394cd5610b0e0f56d6cb8cb661b", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "4fc55db0265bbf39c369df398b9dc7d6469ba13b", "astute_sha": "7766818f079881e2dbeedb34e1f67e517ed7d479", "fuel-ostf_sha": "7c938648a246e0311d05e2372ff43ef1eb2e2761", "release": "6.1", "fuelmain_sha": "bcc909ffc5dd5156ba54cae348b6a07c1b607b24"}

[Ihor Kalnytskyi (ikalnytskyi)]

@Vasily, I have two questions for you:

1/ did you run network verification after deployment or before?

2/ did you run network verification one more time? perhaps, there were some connectivity issues?

p.s: also, do you have this env? it would save my time if so.

[Vasily Gorin (vgorin)]

@ikalnitsky
1. Network verification was running before deployment.
2. I tried to run net verification more then 3 times.

I'm sorry, I don't have already this environment.

[Ihor Kalnytskyi (ikalnytskyi)]

Vasily, I can't reproduce this issue. Could you please try again and give an env?

It looks like you didn't have Internet access through master node (fueladmin network) at all.

[Ihor Kalnytskyi (ikalnytskyi)]

Got it. In Fuel 6.1 you need valid default gateway. When you do fresh install, you can choose it in fuelmenu and it's master node by default. When you do upgrade from previous release, we automatically setup it to master node.

Unfortunately, our sys tests can't deal with it out of box and our test setup host system as default gateway, not master node.

That's why you fail. You have to change default gateway to host system or fix the master node. I don't remember how, you can ask Artem Panchenko.

[Vasily Gorin (vgorin)]

Reproduced again. If you need the env, tell me.

[Ihor Kalnytskyi (ikalnytskyi)]

It's still the same. Please DO NOT use master node as default gateway in system tests. Use host system instead.