[Scale] Receiverd process message too slow (1 in 2 seconds)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Committed
|
High
|
Vladimir Sharshov | ||
Mitaka |
Fix Released
|
High
|
Vladimir Sharshov |
Bug Description
Detailed bug description:
In big env ~ 200 nodes deployment was end by Astute log with error, but UI and Nailgun do not have such information after 5 hours.
After investigation was found that problem in Receiverd which processed message from Astute about progress and status very slow. (0,5 in second, expected behavior 15 in second). Also it took too much memory - 1,3 GB.
Steps to reproduce:
Use big cluster with many nodes (at least 100), create cluster and run deployment.
Expected results:
Deployment succeed. Information about deployment ending should be delivered and processed in real time.
Actual result:
Reproducibility:
Always in big env
Workaround:
Slice deployment on small part of nodes, for example, 10-20 and run it one by one.
Impact:
Scale
Description of the environment:
Fuel 9.0 #188, 200+ nodes
tags: | added: scale |
Changed in fuel: | |
assignee: | Vladimir Sharshov (vsharshov) → Igor Kalnitsky (ikalnitsky) |
Changed in fuel: | |
assignee: | Igor Kalnitsky (ikalnitsky) → Vladimir Sharshov (vsharshov) |
Changed in fuel: | |
status: | Fix Committed → In Progress |
tags: | added: 9.1-proposed |
Changed in fuel: | |
status: | In Progress → Fix Committed |
Fix proposed to branch: master /review. openstack. org/307361
Review: https:/