| 2017-12-12 01:38:17 |
Shi Yan |
description |
I am using the latest pike version heat. But when creating a stack with multiple servers, sometimes (not always reproducible) the stack creation will be stuck in the progress status because of the OS::Heat::WaitCondition resource is hanging seemingly in progress forever.
I didn't find the suspect points in the logs, but below logs will repeat for the all time.
2017-12-12 12:15:58.312 23029 DEBUG heat.engine.scheduler [req-cf1adf2a-95c1-4d71-9e3c-ac80471d42ed - pt-914 - default default] Task create from HeatWaitCondition "worker-5326d53d-wc-waiter" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4-0-irw6e7fj7iip" [690808ee-910e-4b27-8e99-e56078d4fce7] running step /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-12-12 12:15:58.333 23029 DEBUG heat.engine.scheduler [req-cf1adf2a-95c1-4d71-9e3c-ac80471d42ed - pt-914 - default default] Task create from HeatWaitCondition "worker-5326d53d-wc-waiter" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4-0-irw6e7fj7iip" [690808ee-910e-4b27-8e99-e56078d4fce7] sleeping _sleep /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-12-12 12:15:58.561 23029 DEBUG heat.engine.scheduler [req-9e2f0a86-1f1d-4fdb-8eb9-21abb391b9af - - - - -] Task create from TemplateResource "0" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4" [b4b2c8c7-7d56-47c3-b644-dcc7aede08c1] running step /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-12-12 12:15:58.571 23029 DEBUG heat.engine.scheduler [req-9e2f0a86-1f1d-4fdb-8eb9-21abb391b9af - - - - -] Task create from TemplateResource "0" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4" [b4b2c8c7-7d56-47c3-b644-dcc7aede08c1] sleeping _sleep /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155 |
I am using the latest pike version heat. But when creating a stack with multiple servers, sometimes (not always reproducible) the stack creation will be stuck in the progress status because of the OS::Heat::WaitCondition resource is hanging in progress.
The problem is the instance is actually CREATE_COMPLETE and its corresponding OS::Heat::WaitConditionHandle resource is also in CREATE_COMPLETE, but WaitCondition doesn't get the status update.
I didn't find the suspect points in the logs, but below logs will repeat for the all time.
2017-12-12 12:15:58.312 23029 DEBUG heat.engine.scheduler [req-cf1adf2a-95c1-4d71-9e3c-ac80471d42ed - pt-914 - default default] Task create from HeatWaitCondition "worker-5326d53d-wc-waiter" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4-0-irw6e7fj7iip" [690808ee-910e-4b27-8e99-e56078d4fce7] running step /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-12-12 12:15:58.333 23029 DEBUG heat.engine.scheduler [req-cf1adf2a-95c1-4d71-9e3c-ac80471d42ed - pt-914 - default default] Task create from HeatWaitCondition "worker-5326d53d-wc-waiter" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4-0-irw6e7fj7iip" [690808ee-910e-4b27-8e99-e56078d4fce7] sleeping _sleep /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-12-12 12:15:58.561 23029 DEBUG heat.engine.scheduler [req-9e2f0a86-1f1d-4fdb-8eb9-21abb391b9af - - - - -] Task create from TemplateResource "0" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4" [b4b2c8c7-7d56-47c3-b644-dcc7aede08c1] running step /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-12-12 12:15:58.571 23029 DEBUG heat.engine.scheduler [req-9e2f0a86-1f1d-4fdb-8eb9-21abb391b9af - - - - -] Task create from TemplateResource "0" Stack "SCENATEST-9f359dc7fd7b8669-worker-5326d53d-wqhr5y5zxgi4" [b4b2c8c7-7d56-47c3-b644-dcc7aede08c1] sleeping _sleep /usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155 |
|
