Juju model migration fails during validation phase

Bug #1882827 reported by Erlon R. Cruz on 2020-06-09
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
juju
High
Joseph Phillips

Bug Description

When trying to migrate a model between 2 controllers on Juju version
2.8.0, the migration fails due to error:

ERROR agents failed phase "validating" (machines: 1,2,3,5,8,9,10; units: calico/0,ceph-osd/1,containerd/0,containerd/1,etcd/0)
ERROR validating, some agents reported failure

This happens due to the rate-limiting on the target controller rejecting
the stampede of connections as all the agents start up again. This is
something model migration has not specifically been coded to handle.

To work around this issue, you need to set the following config on the
destination controller:

agent-ratelimit-max (default is 10): Should be set to a value at least equal
to the number of units (including subordinates) on the model being migrated.

Changed in juju:
status: New → Confirmed
importance: Undecided → High
Ian Booth (wallyworld) on 2020-06-10
Changed in juju:
milestone: none → 2.8-next
tags: added: model-migration
Salman (salmankh) wrote :

How "agent-ratelimit-max" can be changed, can't find it in the controller or model config?

Salman (salmankh) wrote :

Ok, found that `juju controller-config` does not show the whole list of config parameters. However, it could be set via `juju controller-config -c <controller> agent-ratelimit-max=150`. Once set the controller-config starts showing agent-ratelimit-max value.

Changed in juju:
assignee: nobody → Joseph Phillips (manadart)
milestone: 2.8-next → 2.8.8
status: Confirmed → Fix Committed
Joseph Phillips (manadart) wrote :

https://github.com/juju/juju/pull/12503 applies a retry strategy that better accommodates the default controller rate limiting.

Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers