[2.1] Re-adding virsh chassis to discover new nodes powers down existing nodes

Bug #1637401 reported by Trent Lloyd on 2016-10-28
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Medium
Lee Trager
2.1
Medium
Lee Trager

Bug Description

When using "Add Chassis" to add an existing virsh chassis to discover new nodes, the existing deployed nodes are powered off. This could be dangerous for production environments virtualising juju state servers (not uncommon).

Steps to reproduce:

 (1) Install and configure MAAS (2.0.0+bzr5189-0ubuntu1~16.0) and libvirt-bin
 (2) Setup virsh with 2 VMs, labelled maas-node01 and maas-node02 on the MAAS network
 (3) Add a virsh chassis to MAAS with at least 2 VMs
     My configuration options were:
     URL: qemu+ssh://<email address hidden>/system
     Password: (none, ssh key setup in ~maas)
     Prefix: maas-
 (4) Deploy one of the nodes and ensure it's powered on. Don't think it needs to be even deployed, just the VM has to be on.
 (5) Add the same chassis again with the same parameters
 (6) Observe that the existing deployed VM gets powered off - takes a little bit for MAAS to poll and notice.

Does not seem necessary for there actually to be any new nodes. Log entries are like this for each node:
Oct 28 13:54:44 maas2 maas.region: [ERROR] A node with one of the mac addresses in ['52:54:00:3f:2c:0c', '52:54:00:4f:b4:e7', '52:54:00:54:62:3e', '52:54:00:d7:69:ce'] already exists.

Eventually MAAS polls and notices the VM is off, but it turns off almost straight away:
Oct 28 13:54:53 maas2 maas.power: [INFO] maas-node01: Power state has changed from on to off.

Related branches

Trent Lloyd (lathiat) wrote :

Looks like this is done intentionally in the code. I would suggest that it only powers it off if create_node succeeds.

src/provisioningserver/drivers/hardware/virsh.py#probe_virsh_and_enlist

for machine in conn.list():
(truncated)
    # Force the machine off, as MAAS will control the machine
    # and it needs to be in a known state of off.
    if state == VirshVMState.ON:
        conn.poweroff(machine)
(truncated)
    system_id = create_node(
        macs, arch, 'virsh', params, domain, machine).wait(30)

Mike Pontillo (mpontillo) wrote :

I don't think it's necessary to power the machine off. It will be in a known state the first time we issue a regularly-scheduled power query.

Lee Trager (ltrager) on 2016-11-29
Changed in maas:
importance: Undecided → Medium
assignee: nobody → Lee Trager (ltrager)
Changed in maas:
status: New → Fix Committed
summary: - Re-adding virsh chassis to discover new nodes powers down existing nodes
+ [2.1] Re-adding virsh chassis to discover new nodes powers down existing
+ nodes
Changed in maas:
milestone: none → 2.2.0
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers