Network templates with bond can brake OSTF
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Invalid
|
High
|
Aleksandr Didenko | ||
7.0.x |
Confirmed
|
High
|
Aleksandr Didenko | ||
8.0.x |
Invalid
|
High
|
Aleksandr Didenko |
Bug Description
VERSION:
feature_groups:
- mirantis
production: "docker"
release: "7.0"
openstack_
api: "1.0"
build_number: "301"
build_id: "301"
nailgun_sha: "4162b0c15adb42
python-
fuel-agent_sha: "50e90af6e3d560
fuel-
astute_sha: "6c5b73f93e24cc
fuel-library_sha: "5d50055aeca1dd
fuel-ostf_sha: "2cd967dccd66cf
fuelmain_sha: "a65d453215edb0
Steps to reproduce
1. Create new environment (Neutron with VxLAN, use Ceph as backend)
2. Add 1 controller, 2 ceph-osd nodes
3. Create additional network group
# fuel --env=32 network-group --create --node-group 32 --name "cluster" --release 2 --cidr "10.50.107.0/24"
4. Create and upload network template (see attachment)
# fuel --env=32 network-template --upload --dir ./
5. Hit 'Deploy Changes' button
6. Wait till puppet configured network on Controller node (Task called 'netconfig'). All bonds should be configured once netconfig.pp has finished
7. Artificially put environment into Error state (e.g. kill puppet process multiple times). In real world this can happen due to various reasons: network issues, deployment misconfiguration, HW issues
8. Wait a minute or two (nailgun agent should report node informations to nailgun API)
9. Hit 'Deploy Changes' button one more time.
10. Wait till environment will become ready.
11. Execute 'Request flavor list' OSTF test
Expected result:
- Test pass
Actual result:
- Test failed
Diagnostic Snapshot: https:/
Problem appears to be related to node information update for nodes in Error state. Once bond created nailgun agent will report 3 interfaces with the same MAC address. Nailgun treats this information like node has lost one interface. Network template does not update node interfaces info and public network is assigned to 2nd interface by default. As a result nailgun will not return information regarding public network for node. This informations seems essential for OSTF.
[root@fuel ~]# cat 1.py
from nailgun import objects
import sys
n=objects.
for i in n.interfaces:
print i.name, i.assigned_networks
Before Deployment
[root@fuel ~]# python 1.py 94
2015-11-05 12:31:44.727 DEBUG [7f9d7089a700] (settings) Looking for settings.yaml package config using old style __file__
2015-11-05 12:31:44.727 DEBUG [7f9d7089a700] (settings) Trying to read config file /usr/lib/
2015-11-05 12:31:45.031 DEBUG [7f9d7089a700] (settings) Trying to read config file /etc/nailgun/
2015-11-05 12:31:45.050 DEBUG [7f9d7089a700] (settings) Trying to read config file /etc/fuel/
eth0 [{'id': 1, 'name': u'fuelweb_admin'}, {'id': 135, 'name': u'management'}, {'id': 136, 'name': u'storage'}, {'id': 137, 'name': u'private'}]
eth1 [{'id': 134, 'name': u'public'}]
eth2 []
eth3 []
eth4 []
After Node put into error state
[root@fuel ~]# python 1.py 94
2015-11-05 12:32:47.772 DEBUG [7f557a065700] (settings) Looking for settings.yaml package config using old style __file__
2015-11-05 12:32:47.772 DEBUG [7f557a065700] (settings) Trying to read config file /usr/lib/
2015-11-05 12:32:48.083 DEBUG [7f557a065700] (settings) Trying to read config file /etc/nailgun/
2015-11-05 12:32:48.107 DEBUG [7f557a065700] (settings) Trying to read config file /etc/fuel/
eth0 [{'id': 1, 'name': u'fuelweb_admin'}, {'id': 135, 'name': u'management'}, {'id': 136, 'name': u'storage'}, {'id': 137, 'name': u'private'}]
eth2 []
eth3 []
eth4 []
Changed in fuel: | |
milestone: | none → 8.0 |
assignee: | nobody → Fuel Library Team (fuel-library) |
tags: | added: area-library |
Changed in fuel: | |
importance: | Undecided → High |
status: | New → Confirmed |
tags: | added: feature |
tags: |
added: feature-network-template removed: feature |
tags: | added: team-network |
Changed in fuel: | |
assignee: | Fuel Python Team (fuel-python) → Aleksandr Didenko (adidenko) |
tags: | added: tricky |
Changed in fuel: | |
milestone: | 8.0 → 9.0 |
status: | Incomplete → New |
Changed in fuel: | |
status: | New → Confirmed |
status: | Confirmed → Incomplete |
Now NICs info is locked when node is deployed or under deployment. AFAIC, we need to lock NICs info when node is in error state as well.