[nailgun] Nailgun dont handle cases when deployment data redefined

[Egor Kotko]

{"build_id": "2013-12-23_15-48-09", "ostf_sha": "2a59b9a850f5c612ef4e8617395a60c27305283f", "build_number":
"194", "nailgun_sha": "8d1e8f41106c72af441ab52bb909987d4db32774",
"fuelmain_sha": "025bfe2342120be347da255e4d68e283f54d7a60",
"astute_sha": "2ed6e2c122ae593c20f0e87156b32a468ccdbfd8", "release": "4.0",
"fuellib_sha": "2b61ec0e425bad9b016b6fa7f51b3d0f5b3a6f0a"}

Nailgun should return information about network ip ranges. But returns only vlan IDs instead.

"network_data": [
            {
                "dev": "eth1",
                "name": "floating",
                "vlan": 378
            },
            {
                "dev": "eth1",
                "name": "public",
                "vlan": 378

Steps to reproduce:
1. Deploy environment via cli (Centos, Nova Vlan manager)
2. Review json responce on the master node: curl http://localhost:8000/api/nodes?cluster_id=1 | python -m json.tool
3. Execute all Health Check tests

Expected result:
Health Check tests - should pass

Actual result:
fuel_health.config: DEBUG: Traceback (most recent call last):
  File "/opt/fuel_plugins/ostf/lib/python2.6/site-packages/fuel_health/config.py", line 498, in prepare_config
    self._parse_nodes_cluster_id()
  File "/opt/fuel_plugins/ostf/lib/python2.6/site-packages/fuel_health/config.py", line 546, in _parse_nodes_cluster_id
    ip = public_network['ip'].split('/')[0]
KeyError: 'ip'

[Evgeniy L (rustyrobot)]

So, I looked at logs. Problem is that we don't handle case when you have redefined deployment data which were getting not from default handler of this cluster.
I can suggest half-working workaround, just make
curl http://0.0.0.0:8000/api/v1/clusters/1/orchestrator/deployment/defaults?nodes=1,2,3
And maybe you will get ip address which will be assigned in right order for nodes.

But right usage is
1. get default deployment settings
2. change default deployment setting
3. upload them

When you use such functional as deployment data redefinition it's very easy to shot in leg.

[Aleksandr Didenko (adidenko)]

After changing public IP/networks via CLI and deployment, Nailgun returns old IPs/networks which is wrong and it breaks OSTF.

{"build_id": "2014-02-21_13-53-24", "mirantis": "yes", "build_number": "176", "nailgun_sha": "d528441c490fc53f26a9115fb866d41208727d1a", "ostf_sha": "e74dcb33134d4334786a4fa27107febf9a497f2b", "fuelmain_sha": "fe157dcc0c68a237b527cb9b6adf9101c815e374", "astute_sha": "3d43abeefb60677ce6cae83d31ebbba1ff3cdbe2", "release": "4.1", "fuellib_sha": "a964b37f7f96f62a2f92b117c06077b53386a3ae"}

Steps to reproduce:

1. Create CentOS simple env with Novanetwork+FlatDHCP
2. Add 3 nodes: 1 controller, 1 cinder, 1 compute.
3. On Fuel node run (may need to adjust env ID):
fuel deployment default --env-id=1
4. Change public network address and IPs to anything different from what you have.
5. On Fuel node run the following command to upload changes:
fuel deployment --upload --dir /root/deployment_1 --env 1
6. Deploy changes:
fuel deploy-changes --env 1
7. After deployment ends check public IPs via API:
curl -s http://127.0.0.1:8000/api/clusters/1/network_configuration/nova_network
You will get old public IPs/networks that were configured before changes made via CLI.
8. Run OSTF via Fuel web interface. It will fail all tests with "Keystone client is not available. Please, refer to OpenStack logs to fix this problem".

In "nailgun" DB I can see old IPs in "ip_addrs" and "ip_addr_ranges" tables.

Archive with diagnostic snapshot, DBs dumps and deployment yamls attached.

[Evgeniy L (rustyrobot)]

Ok, I see several solutions to solve this problem

1. documentation, we don't give good description how does cli works and why it can break env, it's a hack or workaround to skip nailgun's validation and data preparation and pass data almost directly to puppet, and if changes something there, not all services will work correctly

2. we can try to deserialize data, which user sends us back to database, but we will not be able to solve this problem in common way, because when user uses cli we assume that he can send us any data structure and we shouldn't validate it and make any additional preparation, purpose of it was flexibility.

Alexander, I have a question, what is the purpose of public ip changing? Maybe we should add ability to configure node more precisely, like exact IP addresses for each network.

[Egor Kotko (ykotko)]

In my case I change public ip range for making nodes accessible via local (office) network - there is a trunk interface, on the server, which transfer several vlans (with unique networks) I receive vlan on master node that’s why there should be custom ip range.

[Artem Roma (aroma-x)]

Ok, guys, I have done some digging into fuelclient and corresponding nailgun code and discovered some features of deployment settings processing for cli.

So only solution I may propose to you guys is using separate network configuration for env via cli. Workflow is similar to that with deployment data:
1) download default network configuration:
     fuel --env {env_id} network --download [--dir {path}]
2) change needed settings in downloaded yaml file
3) upload changed configuration
     fuel --env {env_id} network --upload [--dir {path}]

Few notations for this approach: it does make db data updated, so after deployment you get new values via corresponding requests; second - it doesn't work if you already have uploaded deployment configuration for whole env via cli, in such case network data will be taken from uploaded conf but not saved into db so afterwards you will get stale data for your requests; and third - AFAIU, not all data are updated (only section networks is using now) so keep that in mind.

[Egor Kotko (ykotko)]

Well I have got the point. But there are some cases when it is necessary to use several of yaml/json configuration options network settings and deployment. As I can conclude from previous comments that it is quite big design issue connected not only with cli and need to be discussed.

[Vladimir Sharshov (vsharshov)]

Very well known issue. Can be safely moving to 8.0

[Alexander Gordeev (a-gordeev)]

Adjusted status to "Won't fix" for 7.0

[Roman Prykhodchenko (romcheg)]

Moving this feature bug to Fuel Newton.