kill-controller removes machines from migrated model

Bug #1648063 reported by Nate Finch
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
John A Meinel
2.1
Fix Released
Critical
Christian Muirhead
juju-ci-tools
Fix Released
High
Christopher Lee

Bug Description

(Now monitored at: http://reports.vapour.ws/releases/issue/5893ebba749a565681529d1e)

migrate a model from lxd1 to lxd2
kill-controller lx1 -yt 0

machines in the migrated model get taken down with kill-controller

Not sure if this is unique to the lxd provider.

Note that I had previously tried to migrate the model back to lxd1 which failed, saying "ERROR target prechecks failed: model is being migrated out of target controller" (a known bug), which is why I killed lxd1. Not sure if that matters.

Changed in juju:
assignee: nobody → Alexis Bruemmer (alexis-bruemmer)
Revision history for this message
Curtis Hovey (sinzui) wrote :

Given that the model-migration test is take a long time to delete all the models that were created, we know we want to delete unneeded models as soon as they are unneeded. This issues shows that we should not only delete unused models, but verify only the old model is deleted; the new model is kept.

tags: added: gap
Changed in juju-ci-tools:
assignee: nobody → Christopher Lee (veebers)
status: New → Triaged
importance: Undecided → High
Changed in juju:
assignee: Alexis Bruemmer (alexis-bruemmer) → Christian Muirhead (2-xtian)
Revision history for this message
Christian Muirhead (2-xtian) wrote :

This is happening because in LXD and some other providers we tag the instances with the controller UUID, and when destroying the controller at the provider level we check all instances to see if they still refer to the controller and if they do we destroy them too. The migration code isn't updating the tags on migrated instances to point to the new controller.

The providers that do track instance -> controller are: azure, ec2, gce, lxd, maas and openstack.
I'm adding a MoveInstancesToController method to each provider's environ that will update the tags for the migrated machines.

I'll add the method to the Environ interface once all of the implementations are done, and then update the migrationmaster worker to call it when the migration minions switch over.

Revision history for this message
Christian Muirhead (2-xtian) wrote :

The GCE change turned out to be bigger than I expected, so I'm going to start doing each of the providers as separate PRs.

Revision history for this message
Christian Muirhead (2-xtian) wrote :

Note for myself - the rackspace provider is a thin wrapper around the openstack one, so it doesn't need the new method added (although the fake rackspace provider used in tests still does).

Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR to add a do-nothing implementation of the new method to controllers that don't track instance->controller tags: https://github.com/juju/juju/pull/6810
PR for gce provider: https://github.com/juju/juju/pull/6813

Changed in juju:
status: Triaged → In Progress
Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR for lxd: https://github.com/juju/juju/pull/6827

axw suggests AdoptInstances as a better name than MoveInstancesToController, so I'll do that.

Revision history for this message
Christian Muirhead (2-xtian) wrote :
Revision history for this message
Christian Muirhead (2-xtian) wrote :
Changed in juju:
milestone: 2.1.0 → 2.1-rc1
Revision history for this message
Christian Muirhead (2-xtian) wrote :
description: updated
Changed in juju:
milestone: 2.1-rc1 → 2.2.0-alpha1
assignee: Christian Muirhead (2-xtian) → nobody
importance: Critical → High
status: In Progress → Triaged
Revision history for this message
Christian Muirhead (2-xtian) wrote :
Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR to add AdoptResources to the migrationtarget facade: https://github.com/juju/juju/pull/6950

Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR to call AdoptResources from the migrationmaster worker: https://github.com/juju/juju/pull/6952

Revision history for this message
Christian Muirhead (2-xtian) wrote :

I've just seen a bug in the azure provider change - it updates the controller tags for everything except the resource group. Fixing it now.

Revision history for this message
Christian Muirhead (2-xtian) wrote :
Revision history for this message
Anastasia (anastasia-macmood) wrote :

Final fix committed to develop (2.2) as part of the merge https://github.com/juju/juju/pull/6968

Changed in juju:
status: Triaged → Fix Committed
assignee: nobody → John A Meinel (jameinel)
Curtis Hovey (sinzui)
Changed in juju:
status: Fix Committed → Fix Released
Changed in juju-ci-tools:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.