Cluster stuck in CREATE_IN_PROGRESS while Heat shows CREATE_COMPLETE

[Jack Ivanov]

For some reason I can't create clusters in Queens. Clusters just stuck in CREATE_IN_PROGRESS, but Heat status is CREATE_COMPLETE. THe problem is floating, sometimes It becomes CREATE_COMPLETE, but sometimes hangs on CREATE_IN_PROGRESS for forever.

python-magnum-6.1.1-1002.el7.noarch
python2-magnumclient-2.9.1-1.el7.noarch
openstack-magnum-api-6.1.1-1002.el7.noarch
openstack-magnum-conductor-6.1.1-1002.el7.noarch
openstack-magnum-common-6.1.1-1002.el7.noarch
openstack-magnum-ui-4.0.0-1.el7.noarch

openstack coe cluster show - http://paste.openstack.org/show/728313/
openstack stack show - http://paste.openstack.org/show/728314/
magnum.conf - http://paste.openstack.org/show/728315/

no errors in logs.

Is this a bug? How to debug it?

[Rick Cano (canori01)]

I believe this was fixed in a later release of magnum. You can try the following workaround in magnum.conf:

[drivers]
send_cluster_metrics = False

Basically, this is crashing the periodic tasks which are responsible for checking on the status of the stack

[Jack Ivanov (gunph1ld)]

Hi Rick, Thanks for your reply! Do you know where it was fixed? Might be a bug number or a commit message. Thanks you!

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robert Duncan (rduncan-t)]

I can confirm that this is listed as a known issue in the release notes for magnum 6.0.1

the issue cites:
https://bugs.launchpad.net/magnum/+bug/1746510

however it's still broken in queens 6.1.0, at least as deployed by kolla-ansible.

I would add that this is difficult to troubleshoot as there is no error or warning with debugging turned up and initial troubleshooting would lead to heat signaling