Detailed bug description:
IBP provisioning requires specific file with provisioning data to be uploaded on every node before the actual provisioning script would be executed.
This upload data task is executed synchronously for every node in a row; one by one.
Unless the task accomplished for one node, it won't start to upload the data for next node.
If one node for some reasons becomes irresponsible, then it would take 11.5minutes to recognize the failure.
Steps to reproduce:
0. Emulate mcollective outage for some slave nodes: stop mcollective service on some of nodes.
1. Start provisioning of the nodes
Expected results:
Provisioning task provisions all nodes on which mcollective is still operatable. No significant delay due to some nodes are being unavailable for mcollective.
Actual result:
Provisioning task provisions all nodes on which mcollective is still operatable. Every unavailable node add additional 11.5mins of delay to provisioning. Eg.: for 8 unavailable nodes it would be 1.5H
Reproducibility:
Always
Workaround:
???
Impact:
UX, abnormally increased OS provisioning time on large scale could lead to provisioning task failure by timeout.
Description of the environment:
Fuel 9.0
Additional information:
Perhaps, somebody should add more shorter timeout for non-responding mcollective agents for UploadTask. Since UploadTask for provisioning usually takes few secs, therefore the timeout should be adjusted to a dozen of seconds. Not a dozen of minutes.
https://github.com/openstack/fuel-astute/blob/stable/mitaka/lib/astute/image_provision.rb#L48-L58
please provide update on proposed resolution?