Virsh Power address of VM changes by itself

[Mustafa Alam]

Hi,

So a little overview of the setup I am working on:

I have three baremetal nodes: NodeA, NodeB & NodeC
NodeA: MaaS v.2.2.1 installed on baremetal. This node hosts 6 VMs.
NodeB: 5 Mirtual Machines
NodeC: 6 Virtual Machines

-- I have enlisted all the VMs on MaaS GUI with the power options as below:
Power Type: Virsh (virtual systems)
Virsh Address: qemu+ssh://ubuntu@<Host_Node_IP>/system
Virsh Password: ubuntu
Virsh VM ID: <VM_Name_from_visrh_list>

-- Once I configure the power setting, I can see the MaaS communicating with the VM. By initially printing the message: Queried node's BMC - Power state queried: off. I can commission, deploy, power ON/OFF the VM. This verifies that the power settings are working for that particular VM.

-- The test scenario is where I have two VM (lets call them VM1 and VM2), one hosted on NodeA and other on NodeB. The power address for the VMS is as below:
VM1: qemu+ssh://ubuntu@<NodeA_IP_address>/system
VM2: qemu+ssh://ubuntu@<NodeB_IP_address>/system

-- If I commission and deploy OS on VM1, the VM2 goes in to error state and when I check the power settings of VM2, the power address has been changed to "qemu+ssh://ubuntu@<NodeA_IP_address>/system" by itself. Than I have to change it back to "qemu+ssh://ubuntu@<NodeB_IP_address>/system" and the VM now comes out of the error state.

-- I have reproduced this issue for VM2 as well, I have seen the same issue comes up for VM1 (goes in to error state and its power address is changed to "qemu+ssh://ubuntu@<NodeB_IP_address>/system"). I have also reproduced this issue for NodeC. I have observed same behaviour in the power address for VMs on NodeC as well. I have made sure that NodeA (where maas is installed) has password-less access to NodeB and NodeC.

[Andres Rodriguez (andreserl)]

Dear user,

This is an automated message.

We believe this bug report is no longer an issue in the latest version of MAAS. For such reason, we are making this issue as Won't Fix. If you believe this issue is still present in the latest version of MAAS, please re-open this bug report.

[Vijay M (vj.m)]

This bug still exists in current MAAS stable release 2.3.2-6485-ge93e044.

Changing a virsh address on one node is changing the virsh address on all the nodes, this only happens to virsh_address and not virsh VM ID.

Also, a warning message reads as below when editing the power parameters.

WARNING MESSAGE: This power controller manages 80 other nodes. Changing power parameters will affect these nodes.

[Andres Rodriguez (andreserl)]

Hi Vijay,

As per the warning message, if you have a VM in the same KVM host (determined by the power parameters), for which you want to update its address, this change will affect to all machines inside the same host.

This is not a bug, it is intended to work as per the warning message.

So, are you requesting this to be different?

[Vijay M (vj.m)]

Hi Andres,

I have VM's running on multiple KVM hosts, when I change virsh address of a VM running on KVM host A, it's also changing the virsh address for KVM host B VM's, and show all the VM's running on KVM B shows power error.

Thanks

[Scott Hussey (sh8121)]

Possibly related, after concurrently updating multiple VMs I seem to end up with the below stack trace when I attempt additional updates to power parameters some of the time. It seems this magic to attempt to apply the update to multiple nodes may suffer a race condition.

The below is w/ MAAS 2.3.0 from the 16.04 repos

Aug 07 14:46:29 maas-region-0 sh[98]: 2018-08-07 14:46:29 maasserver: [error] ################################ Exception: get() returned more than one BMC -- it returned 2! ################################
Aug 07 14:46:29 maas-region-0 sh[98]: 2018-08-07 14:46:29 maasserver: [error] Traceback (most recent call last):
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 132, in get_response
Aug 07 14:46:29 maas-region-0 sh[98]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 185, in view_atomic_with_post_commit_savepoint
Aug 07 14:46:29 maas-region-0 sh[98]: return view_atomic(*args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3.5/contextlib.py", line 30, in inner
Aug 07 14:46:29 maas-region-0 sh[98]: return func(*args, **kwds)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 59, in __call__
Aug 07 14:46:29 maas-region-0 sh[98]: response = upcall(request, *args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/django/views/decorators/vary.py", line 21, in inner_func
Aug 07 14:46:29 maas-region-0 sh[98]: response = func(*args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/piston3/resource.py", line 190, in __call__
Aug 07 14:46:29 maas-region-0 sh[98]: result = self.error_handler(e, request, meth, em_format)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/piston3/resource.py", line 188, in __call__
Aug 07 14:46:29 maas-region-0 sh[98]: result = meth(request, *args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 298, in dispatch
Aug 07 14:46:29 maas-region-0 sh[98]: return function(self, request, *args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 162, in wrapper
Aug 07 14:46:29 maas-region-0 sh[98]: return func(self, request, *args, **kwargs)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/api/machines.py", line 447, in update
Aug 07 14:46:29 maas-region-0 sh[98]: return form.save()
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/forms/__init__.py", line 1037, in save
Aug 07 14:46:29 maas-region-0 sh[98]: WithPowerTypeMixin.set_values(self, machine)
Aug 07 14:46:29 maas-region-0 sh[98]: File "/usr/lib/python3/dist-packages/maasserver/forms/__init__.py", line 386, in set_values
Aug 07 14:46:29 maas-region-0 sh[98]: params_field_name)
Aug 07 14:46:29 maas-region-0 sh[98]: ...

[Andres Rodriguez (andreserl)]

I think this issue may be the same issue as https://bugs.launchpad.net/maas/+bug/1793340

[Adam Collard (adam-collard)]

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

MAAS Team