adding units to large deployments is slow
Bug #1746134 reported by
Paul Collins
This bug affects 2 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
NTP Charm |
Fix Released
|
High
|
Unassigned |
Bug Description
PS4.5 has approximately 90 ntp units, which upon add-unit seems to entail invoking the peer relation hook many times, possibly up to 180 times, which at ~4s per invocation is 12 minutes. This leads to a) the unit itself taking a long time to settle; and b) its hooks blocking other hooks on the same machine, slowing down adding new units overall. This became particularly acute thanks to LP:1746119, which resulted in nova-compute having to run its hooks a bunch more times than is typical in order to complete deploying.
Related branches
~paulgear/ntp-charm/+git/ntp-charm:master
- NTP charm developers: Pending requested
-
Diff: 22 lines (+4/-0)1 file modifiedconfig.yaml (+4/-0)
Changed in ntp-charm: | |
status: | New → Confirmed |
importance: | Undecided → High |
Changed in ntp-charm: | |
assignee: | nobody → Paul Gear (paulgear) |
Changed in ntp-charm: | |
status: | Confirmed → Fix Committed |
Changed in ntp-charm: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.
I'm trying to optimise this along with the changes for chrony support in
bionic. I've optimised out a relation-set from the relation-changed
hook, but I'm not hopeful of further performance improvements without
some abuse of update-status as a config-changed caching mechanism. Once
I have the new code in place I'll do some performance comparisons and
note the results here.