[2.3, snap] Cannot delete a rack controller running from the snap
Bug #1711414 reported by
Andres Rodriguez
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Fix Released
|
Critical
|
Andres Rodriguez |
Bug Description
Node failed to be deleted, because of the following error: Operation failed: No such file or directory
Unfortunately, there are no longs whatsoever on the rack controller. On the region it shows:
2017-08-17 16:58:48 maasserver.
Related branches
~andreserl/maas:snap_rack_delete
Merged
into
maas:master
- Andres Rodriguez (community): Approve
- Mike Pontillo (community): Approve
-
Diff: 109 lines (+57/-11)2 files modifiedsrc/provisioningserver/rpc/clusterservice.py (+26/-9)
src/provisioningserver/rpc/tests/test_clusterservice.py (+31/-2)
tags: | added: snap |
summary: |
- [2.3] Cannot delete a rack controller running from the snap + [2.3, snap] Cannot delete a rack controller running from the snap |
Changed in maas: | |
milestone: | none → 2.3.0 |
Changed in maas: | |
assignee: | nobody → Andres Rodriguez (andreserl) |
importance: | Undecided → Critical |
status: | New → Triaged |
status: | Triaged → In Progress |
Changed in maas: | |
status: | In Progress → Fix Committed |
Changed in maas: | |
milestone: | 2.3.0 → 2.3.0alpha3 |
Changed in maas: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.
With the attached branch (https:/ /code.launchpad .net/~andreserl /maas/+ git/maas/ +merge/ 329275 ) MAAS should be able to correctly stop the rack controller that's running on a snap. What this does is effectively tell the MAAS snap to change the mode from "Rack Controller" to "None". Doing so, it stops all services. However
In the UI I see the following error:
Node failed to be deleted, because of the following error:
In the logs I see:
2017-08-18 22:02:17 maasserver. websockets. protocol: [critical] Error on request (632) controller.action:
Traceback (most recent call last): ver.rpc. exceptions. CannotDisableAn dShutoffRackd: rpc.regionservi ce: [info] Rack controller 'xh33gw' disconnected. 6,::ffff: 192.168. 122.42: [info] RegionServer connection lost (HOST:IPv6Addre ss(TCP, '::ffff: 192.168. 122.2', 5250) PEER:IPv6Addres s(TCP, '::ffff: 192.168. 122.42' , 55580))
Failure: provisioningser
2017-08-18 22:02:27 maasserver.
2017-08-18 22:02:27 RegionServer,
In the snap logs I see:
==> /var/snap/ maas/common/ log/supervisor. log <==
2017-08-18 22:02:16,955 WARN received SIGHUP indicating restart request
2017-08-18 22:02:16,958 INFO waiting for tgt, rackd to die
==> /var/snap/ maas/common/ log/rackd. log <== client: [info] Received SIGTERM, shutting down. ver.rpc. clusterservice: [info] Region not available: Connection was refused by other side: 111: Connection refused. (While requesting RPC info at b'http://[::ffff: 127.0.0. 1]:5240/ MAAS/rpc/ ').
2017-08-18 22:02:16 ClusterClient,
2017-08-18 22:02:17 provisioningser
[...]
==> /var/snap/ maas/common/ log/supervisor. log <==
2017-08-18 22:02:20,781 INFO waiting for tgt, rackd to die
2017-08-18 22:02:23,788 INFO waiting for tgt, rackd to die
2017-08-18 22:02:26,794 INFO waiting for tgt, rackd to die
2017-08-18 22:02:27,797 WARN killing 'rackd' (5537) with SIGKILL
2017-08-18 22:02:28,813 INFO stopped: rackd (terminated by SIGKILL)
2017-08-18 22:02:29,818 INFO waiting for tgt to die
2017-08-18 22:02:32,825 INFO waiting for tgt to die
2017-08-18 22:02:35,832 INFO waiting for tgt to die
2017-08-18 22:02:38,839 INFO waiting for tgt to die
2017-08-18 22:02:41,846 WARN killing 'tgt' (5728) with SIGKILL
2017-08-18 22:02:41,849 INFO stopped: tgt (terminated by SIGKILL)
2017-08-18 22:02:41,871 CRIT Supervisor running as root (no user in config file)
2017-08-18 22:02:41,874 INFO RPC interface 'supervisor' initialized
2017-08-18 22:02:41,877 INFO supervisord started with pid 5502
As you can see, this correctly stopped the snap and the rack controller is no longer running, however, the action failed above, which is what causes some of the extra logging. That said, if I click on "retry" in the UI, it now successfully removes the rack controller.