[Image Based] Cloud init reset all node files after delete cluster and deploy another one on the same nodes
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Confirmed
|
Medium
|
Alexander Gordeev |
Bug Description
{
"build_id": "2014-11-
"ostf_sha": "82465a94eed4ef
"build_number": "504",
"auth_
"api": "1.0",
"nailgun_sha": "8d23d1b1bcd921
"production": "docker",
"fuelmain_sha": "8d4943d5ead7a8
"astute_sha": "65eb911c38afc0
"feature_
"mirantis"
],
"release": "6.0",
"release_
],
}
}
},
"fuellib_sha": "8a0ceff90777af
}
Steps:
1. Create and deploy next cluster - Ubuntu, HA, Neutron GRE, Image-based provisioning, 3 controller, 2 compute, 1 cinder node
2. After deployment delete cluster
3. Create new cluster - CentOS, HA, Neutron Vlan, Image-based provisioning, 3 controller, 2 compute nodes
4. Provision cluster
Expected - all nodes were successfully provisioned
Actual - 1 time out of 4 one of the nodes is provisioned but after node's restart and start up of cloud init it destroyed all node's files
Logs are attached
tags: | added: experimental |
Changed in fuel: | |
status: | New → Confirmed |
tags: | added: cloud-init |
Changed in fuel: | |
milestone: | 6.0 → 6.1 |
tags: | added: release-notes |
The root cause is still unknown. Stable and repeatable way of how to reproduce it even.
At first i thought it was a failure inside of the boothook script. Nope, the boothook script worked fine every time i'd tried. https:/ /review. openstack. org/#/c/ 138384/ <- patch for boothook scripts.
It might be cloud-init's semaphores issues. For unknown reason cloud-init log was full of messages showing that all config_* modules have been already run. The executor simply checks the semaphore and skips if it exists.
They stored in /var/lib/ cloud/instance/ sem/config_ *
I have only one strategy to follow. We need to disable automatic cloud-init start on boot (just removing links from /etc/rc.d/* should help) and then start cloud-init by hand under `strace` or other hardcore debug stuff and watch what will happen.
Sounds as very time consuming task.