[astute] in two roles node, second role was not applied

[Nastya Urlapova]

iso 3.2.1-#74
os - Ubuntu/Centos
cluster :
controller + ceph OSD
compute + ceph OSD
+ (volumes_ceph, images_ceph)

After deploy, systest create snapshot with deployed env.
Check ostf run on this snapshot.

Error in test:
pass: 17 should pass: 18

two ostf tests failed:
Launch instance
Launch instance, create snapshot, launch instance from snapshot

instances have error: "No valid host was found" status.

nova-manage list shows that there is no nova-compute service at all (was never registered)

[Nikolay Fedotov (nfedotov)]

two ostf tests failed:
Launch instance
Launch instance, create snapshot, launch instance from snapshot

instances have error: "No valid host was found" status.

[Nikolay Fedotov (nfedotov)]

diagnostic snapshot

[Mike Scherbakov (mihgen)]

Discovered from the environment that compute host successfully deployed ceph-osd role, then got astute.yaml with compute role however looks like puppet has never started there. Puppet log shows nothing for compute role. Vladimir is investigating.

[Mike Scherbakov (mihgen)]

How repeatable is this?

[Vladimir Sharshov (vsharshov)]

Could not reproduce this problem. Check this configurations:

* CentOS, Simple, 2 nodes: controller+ceph_osd, compute+ceph_osd (volumes_ceph, images_ceph);
* Ubuntu, HA, 5 nodes: 3 controller+ceph_osd, 2 compute+ceph_osd (volumes_ceph, images_ceph).

ISO #79 (3.2.1)

[Vladimir Kuklin (vkuklin)]

Reproduced this bug again. Snapshot comes next.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/61098

[Dmitry Borodaenko (angdraug)]

I have reported the same issue with a detailed log analysis in this bug:
https://bugs.launchpad.net/fuel/+bug/1260077

I've marked it as duplicate, but my log analysis may be useful. I have 100% reproducibility for this problem.

[Vladimir Sharshov (vsharshov)]

I have 100% reproducibility - sound cool!

Can you provide me access to this cluster? I already prepare fix for this bug, but could not repeat it in my development env.

Or you can try to test new ISO because we will merge today pull request with fix.

[Vladimir Sharshov (vsharshov)]

Some details: fix at now moment solves the problem with an extra load on the poll of node status, as well as incorrectly Interpretation of running processes for the old role as a force for new. But reason of interpretation running process as stopped&&succeed we need to be established.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/61098
Committed: https://git.openstack.org/cgit/stackforge/fuel-astute/commit/?id=1b8d6e18a3c75b4aaf61fe5709e4f83be36c8d6f
Submitter: Jenkins
Branch: master

commit 1b8d6e18a3c75b4aaf61fe5709e4f83be36c8d6f
Author: Vladimir <email address hidden>
Date: Tue Dec 10 14:03:24 2013 +0400

    Strong puppet hung check

    * add second round for puppetd_runonce check;
    * change hang message to warn status;
    * mark hung nodes as error (before was running);
    * remove unused 'change_node_status' param;
    * refactoring;
    * add tests.

    Change-Id: Iba7ac5d30a1e57842a44c95c3c68f19dd71bccff
    Closes-Bug: #1256244