receiverd ACKs messages if it was interrupted
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Committed
|
Medium
|
Dima Shulyak |
Bug Description
[UPD]: receiverd sends ACKs for all messages even if it was interrupted in the middle of processing one. Because of that some messages do not get processed after receiverd is restarted. One of the consequences which happened in the test lab is the following:
During 100 nodes cluster deployment MOS/Fuel masternode didn't see that last controller was finished his deployment procedure.
Here is last lines of puppet logs from controller itself(about half hour ago):
Mon Oct 20 22:35:57 +0000 2014 Stage[main] (info): Evaluated in 0.36 seconds
Mon Oct 20 22:35:57 +0000 2014 Stage[glance-image] (info): Starting to evaluate the resource
Mon Oct 20 22:35:57 +0000 2014 Stage[glance-image] (info): Evaluated in 0.02 seconds
Mon Oct 20 22:35:57 +0000 2014 Stage[glance-image] (info): Starting to evaluate the resource
Mon Oct 20 22:35:57 +0000 2014 Stage[glance-image] (info): Evaluated in 0.01 seconds
Mon Oct 20 22:35:57 +0000 2014 Puppet (debug): Finishing transaction 70365384691300
Mon Oct 20 22:35:57 +0000 2014 Puppet (debug): Storing state
Mon Oct 20 22:35:57 +0000 2014 Puppet (info): Creating state file /var/lib/
Mon Oct 20 22:35:57 +0000 2014 Puppet (debug): Stored state in 0.16 seconds
Mon Oct 20 22:35:57 +0000 2014 Puppet (notice): Finished catalog run in 1270.24 seconds
Please find diagnostic snapshot and screenshot of Fuel UI in attachments.
Changed in fuel: | |
milestone: | none → 6.0 |
assignee: | nobody → Fuel Astute Team (fuel-astute) |
Changed in fuel: | |
status: | New → Confirmed |
importance: | Undecided → High |
Changed in fuel: | |
assignee: | Fuel Python Team (fuel-python) → Łukasz Oleś (loles) |
Changed in fuel: | |
assignee: | Łukasz Oleś (loles) → nobody |
assignee: | nobody → Fuel Python Team (fuel-python) |
Changed in fuel: | |
status: | In Progress → Confirmed |
assignee: | Dima Shulyak (dshulyak) → Roman Prykhodchenko (romcheg) |
Changed in fuel: | |
status: | Confirmed → Fix Committed |
milestone: | 6.1 → 6.0 |
It finally got deployed but after 35-40 minutes (just for 5 minutes which I was filing this bug +)