Juju model migration fails during validation phase

Bug #1882827 reported by Erlon R. Cruz
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Joseph Phillips

Bug Description

When trying to migrate a model between 2 controllers on Juju version
2.8.0, the migration fails due to error:

ERROR agents failed phase "validating" (machines: 1,2,3,5,8,9,10; units: calico/0,ceph-osd/1,containerd/0,containerd/1,etcd/0)
ERROR validating, some agents reported failure

This happens due to the rate-limiting on the target controller rejecting
the stampede of connections as all the agents start up again. This is
something model migration has not specifically been coded to handle.

To work around this issue, you need to set the following config on the
destination controller:

agent-ratelimit-max (default is 10): Should be set to a value at least equal
to the number of units (including subordinates) on the model being migrated.

Pen Gale (pengale)
Changed in juju:
status: New → Confirmed
importance: Undecided → High
Ian Booth (wallyworld)
Changed in juju:
milestone: none → 2.8-next
tags: added: model-migration
Revision history for this message
Salman (salmankh) wrote :

How "agent-ratelimit-max" can be changed, can't find it in the controller or model config?

Revision history for this message
Salman (salmankh) wrote :

Ok, found that `juju controller-config` does not show the whole list of config parameters. However, it could be set via `juju controller-config -c <controller> agent-ratelimit-max=150`. Once set the controller-config starts showing agent-ratelimit-max value.

Changed in juju:
assignee: nobody → Joseph Phillips (manadart)
milestone: 2.8-next → 2.8.8
status: Confirmed → Fix Committed
Revision history for this message
Joseph Phillips (manadart) wrote :

https://github.com/juju/juju/pull/12503 applies a retry strategy that better accommodates the default controller rate limiting.

Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.