[library] Mongo node deployment fails on Ubuntu due to broken routing

[Artem Panchenko]

Fuel version (6.1 build #314): http://paste.openstack.org/show/204082/

Environment deployment with ceilometer fails on Ubuntu, because nodes with 'mongo' role can't be deployed:

http://paste.openstack.org/show/204081/

Steps to reproduce:

1. Create environemnt Ubuntu+NeutronGre+CephVolumes+Ceilometer
2. Add 3 Controller, 2 Compute, 2 Ceph and 1 Mongo nodes
3. Deploy changes

Expected result:

- cluster is deployed and passes OSTF

Actual:

- deployment of 'mongo' node failed, because puppet was unable to install some packages using apt-get from remote repositories

Here is routing details on mongo node (node-8):

http://paste.openstack.org/show/204080/

As you can see there are 2 default routes: 1 via mgmt VIP and 1 via master node. At the moment when 'mongo' node is deployed, mgmt VIP isn't available. The route via master node has lower priority due to its high metric. As node has a route to mgmt VIP via directly connected network it doesn't remove it from routing table, so default gw via admin network is always ignored. In my opinion it's much better to not add routes via VIP at all on any nodes, until VIPs are configured on some of controllers.

Diagnostic snapshot is attached.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/176665

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/176665
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=b9f1b048d5e5f73a7640c890e7499fc39ad4cca0
Submitter: Jenkins
Branch: master

commit b9f1b048d5e5f73a7640c890e7499fc39ad4cca0
Author: Stanislav Makar <email address hidden>
Date: Thu Apr 23 08:51:36 2015 +0000

    Reconfigure default route

    We have some roles which are deployed before vrouter deployment
    due to we need to reconfigure default route after vrouter is ready.

    Default route reconfiguration:
    * add new configure_default_route puppet function
    * add new configure_default_route task in post deploy
    * add tests

    Change-Id: I5db09111a9eecad058654e5463ffb13d077cee70
    Closes-bug: #1444962

[Anastasia Palkina (apalkina)]

Verified on ISO #412

"build_id": "2015-05-13_11-36-33", "build_number": "412", "release_versions": {"2014.2.2-6.1": {"VERSION": {"build_id": "2015-05-13_11-36-33", "build_number": "412", "api": "1.0", "fuel-library_sha": "86505a5b4aab91b16abacca9a0da4f4cf90d3da9", "nailgun_sha": "09a9f2726223b9f31075adf79898045cd269c00c", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "af6c9c3799b9ec107bcdc6dbf035cafc034526ce", "astute_sha": "484d5038b81b7260597be027b1265dc88091026b", "fuel-ostf_sha": "1ae384a28b42d5325bd5d2546aead1e9d502f6a8", "release": "6.1", "fuelmain_sha": "51b86bb24b27742a22b23e2ae3dfc850c47e5fbf"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "86505a5b4aab91b16abacca9a0da4f4cf90d3da9", "nailgun_sha": "09a9f2726223b9f31075adf79898045cd269c00c", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-6.1", "production": "docker", "python-fuelclient_sha": "af6c9c3799b9ec107bcdc6dbf035cafc034526ce", "astute_sha": "484d5038b81b7260597be027b1265dc88091026b", "fuel-ostf_sha": "1ae384a28b42d5325bd5d2546aead1e9d502f6a8", "release": "6.1", "fuelmain_sha": "51b86bb24b27742a22b23e2ae3dfc850c47e5fbf"