Re-deployment of controller fails if net04 name of network was changed and roter-net04 stays as was before

[Tatyanka]

Steps to reproduce:
1. Deploy ha vlan (Centos)
2. 3 controllers + 2 compute
3. As soon as deployment becomes ready - ssh on any controller
4. .Update keystone admin user and password
5. re-name net04 network
6. Delete 1 controller and 1 compute
7. Add new controller
8. Change interface configuration and dissk configuration
9. Press deploy changes

Expected Result:

Cluster is ready, OSTF tests are passed

Actual Result:
Nodes from step 6 was removed, new controller was provisioned and re-deployment of existing controller starts(those one that was primary. and finishes with error:
 Non-fatal error: "Execution of '/usr/bin/neutron net-create --format=shell --tenant_id=0810fd72bf2a4a3c8893d94a7f91c459 --provider:network_type=gre --provider:segmentation_id=2 --router:external=False net04' returned 1: Conflict (HTTP 409) (Request-ID: req-dae2bb05-0f34-480c-a2c1-8635f927b435)

2015-07-05 11:33:50 DEBUG

 Executing '/usr/bin/neutron net-create --format=shell --tenant_id=0810fd72bf2a4a3c8893d94a7

After re-naming of neutron net - networks looks like
[root@node-1 ~]# neutron net-list
+--------------------------------------+-----------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+-----------+-------------------------------------------------------+
| fd9d4e48-13bb-41d7-8e37-b9ecc601fa67 | net04_ext | fd662a47-9d3f-473c-81df-a3adb90ec22f 10.109.1.0/24 |
| f572f5d8-c3ca-4c77-b20f-978da4abd73d | admin1 | 0a62cfb2-4757-43ae-ab70-a8d7bba82f06 192.168.111.0/24 |
+--------------------------------------+-----------+-------------------------------------------------------+

[root@node-1 ~]# /usr/bin/neutron router-show --format=shell 7a827626-0f5c-4323-9921-7924f7738747
admin_state_up="True"
distributed="False"
external_gateway_info="{\"network_id\": \"fd9d4e48-13bb-41d7-8e37-b9ecc601fa67\", \"enable_snat\": true, \"external_fixed_ips\": [{\"subnet_id\": \"fd662a47-9d3f-473c-81df-a3adb90ec22f\", \"ip_address\": \"10.109.1.128\"}]}"
ha="False"
id="7a827626-0f5c-4323-9921-7924f7738747"
name="router04"
routes=""
status="ACTIVE"
tenant_id="0810fd72bf2a4a3c8893d94a7f91c459"
[root@node-1 ~]# /usr/bin/neutron router-show --format=shell 7a827626-0f5c-4323-9921-7924f7738747
admin_state_up="True"
distributed="False"
external_gateway_info="{\"network_id\": \"fd9d4e48-13bb-41d7-8e37-b9ecc601fa67\", \"enable_snat\": true, \"external_fixed_ips\": [{\"subnet_id\": \"fd662a47-9d3f-473c-81df-a3adb90ec22f\", \"ip_address\": \"10.109.1.128\"}]}"
ha="False"
id="7a827626-0f5c-4323-9921-7924f7738747"
name="router04"
routes=""
status="ACTIVE"
tenant_id="0810fd72bf2a4a3c8893d94a7f91c459"

after re-naming admin1 network back to neto4 - re-deployment passed, but we have 2 admin users and different openrc files (on re-deployed controller and added)
[root@node-1 ~]# cat openrc
#!/bin/sh
export LC_ALL=C
export OS_NO_CACHE='true'
export OS_TENANT_NAME='haGre'
export OS_USERNAME='haGre'
export OS_PASSWORD='haGre'
export OS_AUTH_URL='http://10.109.2.2:5000/v2.0/'
export OS_AUTH_STRATEGY='keystone'
export OS_REGION_NAME='RegionOne'
export CINDER_ENDPOINT_TYPE='internalURL'
export GLANCE_ENDPOINT_TYPE='internalURL'
export KEYSTONE_ENDPOINT_TYPE='internalURL'
export NOVA_ENDPOINT_TYPE='internalURL'
export NEUTRON_ENDPOINT_TYPE='internalURL'
export OS_ENDPOINT_TYPE='internalURL'
export MURANO_REPO_URL='http://storage.apps.openstack.org/'
[root@node-1 ~]# exit
logout
Connection to node-1 closed.
[root@nailgun ~]# ssh node-2
Warning: Permanently added 'node-2' (RSA) to the list of known hosts.
Last login: Sun Jul 5 21:02:35 2015 from 10.109.0.2
[root@node-2 ~]# cat openrc
#!/bin/sh
export LC_ALL=C
export OS_NO_CACHE='true'
export OS_TENANT_NAME='haGre'
export OS_USERNAME='haGre'
export OS_PASSWORD='haGre'
export OS_AUTH_URL='http://10.109.2.2:5000/v2.0/'
export OS_AUTH_STRATEGY='keystone'
export OS_REGION_NAME='RegionOne'
export CINDER_ENDPOINT_TYPE='internalURL'
export GLANCE_ENDPOINT_TYPE='internalURL'
export KEYSTONE_ENDPOINT_TYPE='internalURL'
export NOVA_ENDPOINT_TYPE='internalURL'
export NEUTRON_ENDPOINT_TYPE='internalURL'
export OS_ENDPOINT_TYPE='internalURL'
export MURANO_REPO_URL='http://storage.apps.openstack.org/'
[root@node-2 ~]#

[root@nailgun ~]# cat /etc/fuel/version.yaml
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "525"
  build_id: "2015-06-19_13-02-31"
  nailgun_sha: "dbd54158812033dd8cfd7e60c3f6650f18013a37"
  python-fuelclient_sha: "4fc55db0265bbf39c369df398b9dc7d6469ba13b"
  astute_sha: "1ea8017fe8889413706d543a5b9f557f5414beae"
  fuel-library_sha: "2e7a08ad9792c700ebf08ce87f4867df36aa9fab"
  fuel-ostf_sha: "8fefcf7c4649370f00847cc309c24f0b62de718d"
  fuelmain_sha: "a3998372183468f56019c8ce21aa8bb81fee0c2f"

[Tatyanka (tatyana-leontovich)]

https://drive.google.com/a/mirantis.com/file/d/0B_tSitrwrgvoWUh6eEVMZUVkd2M/view?usp=sharing

[Mike Scherbakov (mihgen)]

Why this was Medium? I bumped to High. I believe this is quite common for Cloud Administrator to change default network created by Fuel, or at least rename it (this "encryptive" net04 not really clear for users).

[Oleksiy Molchanov (omolchanov)]

@Dima, it seems to intersect with yours bug https://bugs.launchpad.net/fuel/+bug/1471244

[Aleksandr Didenko (adidenko)]

@Mike, I believe we should close this bug in documentation. If cluster admin does not want to use predefined networks, he/she should disable them in Fuel (https://bugs.launchpad.net/fuel/+bug/1471244). Otherwise Fuel relies on configuration data from astute.yaml and it will try to create those predefined networks on every re-deploy. So if you rename network, puppet won't be able to detect that and will try to create "net04" and "net04_ext" networks and attach them to "router04" router - and it may fail.

[Sergey Vasilenko (xenolog)]

I agree with @adidenko.
It is very rare case.

[Matthew Mosesohn (raytrac3r)]

mos-puppet team, can you look at making these resources "unmanaged"? If the user creates them and renames them, it should not update their parameters after deployment? We did a similar thing for keystone_user resource

[Sergey Kolekonov (skolekonov)]

Matthew,
I think it's not the same situation as with Keystone user and its password, or probably I misunderstood you.
The problem here is that when you rename the default network after deployment (net04, for example) and then re-run puppet modules (during scaling and so on), neutron_network provider checks that a network with the default name (net04) exists and tries to create it. As the network was renamed, in fact it doesn't exist anymore for neutron_network provider, so it tries to create it again.

[Ivan Berezovskiy (iberezovskiy)]

To prevent from such failures we should stop manage networks/subnets/routers by Fuel (we shoudn't create them during/right after deployment). But before removing manifests for network/routers creation, we need to adapt our OSTF tests, so they will be able to create those resources during each test execution. Also they should cleanup all their stuff after run. Once OSTF are fixed we can update fuel-library manifests. Fuel QA team, please take a look and share your thoughts.

[Sam Stoelinga (sammiestoel)]

It's common to modify/delete or recreate the default networks created by Fuel. Fuel should not try to recreate the net04 networks on a redeployment but leave the OpenStack resources as is.

@Vitaly Sedelnik: When we update from 9.0 to 9.1, we also execute redeployment, so in that case we may hit the same issue described here.