[2.1] MAAS cannot power query with Cisco UCSM power driver
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Fix Released
|
High
|
Newell Jensen | ||
2.1 |
Fix Released
|
High
|
Newell Jensen |
Bug Description
I'm trying to set up a blade in a Cisco chassis via UCSM. I'm using MAAS 2.0 and it looks like MAAS is failing to properly probe power.
tl;dr:
MAAS can successfully turn power on and off for nodes that are part of a UCSM controlled chassis. MAAS can NOT correctly get power status from UCSM so user has to manually use Power Off from action menu in MAAS to put node in the correct state.
Here's what happens (and I'll attach screen shots showing each step).
1: System not in MAAS. UCSM shows blade power OFF.
2: Power system on in UCSM. System boots, enlists, shuts off. UCSM shows power state OFF. MAAS shows system in New state with no power type set.
3: Configure power type with U/P, UUID and API URL and save changes
4: MAAS checks power state and shows System in New state with Power ON, evne though UCSM shows that the blade power state is OFF.
5: Clicking Check Now re-probes power and still shows as ON.
6: Manually set Power OFF in MAAS by selecting Power Off from the Action menu.
7: MAAS NOW shows power state as OFF.
8: Check Now also says power state is OFF
At this point, maas.log only shows this for the UCS node:
bladernr@
Aug 10 18:59:06 lucuma maas.api: [INFO] unfluffed-cherise: Enlisted new machine
Aug 10 20:11:12 lucuma maas.power: [INFO] Changing power state (off) of node: unfluffed-cherise (4y3h8r)
Aug 10 20:11:14 lucuma maas.power: [INFO] Changed power state (off) of node: unfluffed-cherise (4y3h8r)
The last two are where I manually set power OFF using the Action menu.
9: I click Commission from the Action menu.
10: MAAS shows power state as ON and node state as Commissioning.
11: UCSM confirms that power has turned on, so MAAS CAN control power.
12: Commissioning completes. UCSM shows power state OFF, MAAS shows system as Ready, with power state ON.
In the MAAS long, we see where maas turned the power on:
Aug 10 20:14:42 lucuma maas.node: [INFO] unfluffed-cherise: Status transition from NEW to COMMISSIONING
Aug 10 20:14:42 lucuma maas.power: [INFO] Changing power state (on) of node: unfluffed-cherise (4y3h8r)
Aug 10 20:14:42 lucuma maas.node: [INFO] unfluffed-cherise: Commissioning started
Aug 10 20:14:44 lucuma maas.power: [INFO] Changed power state (on) of node: unfluffed-cherise (4y3h8r)
13: MAAS changes node state to Ready, Power remains ON
maas.log shows this:
Aug 10 20:17:52 lucuma maas.interface: [INFO] enp7s0 (physical) on unfluffed-cherise: Observed connected to fabric-1 via 10.1.10.0/23.
Aug 10 20:17:53 lucuma maas.node: [INFO] unfluffed-cherise: Storage layout was set to flat.
Aug 10 20:17:53 lucuma maas.node: [INFO] unfluffed-cherise: Status transition from COMMISSIONING to READY
Note that UCSM shows the node as powered OFF, MAAS still shows the power ON. Once again, I have to manually select Power Off from the Action menu and then maas changes state to Power Off.
maas.log shows this when I manually tell it to turn the power off:
Aug 10 20:20:15 lucuma maas.power: [INFO] Changing power state (off) of node: unfluffed-cherise (4y3h8r)
Aug 10 20:20:17 lucuma maas.power: [INFO] Changed power state (off) of node: unfluffed-cherise (4y3h8r)
Now that MAAS thinks the power is off, I can begin a deployment. MAAS acquires the node correctly, changes state to Deploying, and sets the Power state to ON... UCSM shows the node ON. This appears in the MAAS log when I do the deployment:
Aug 10 20:20:30 lucuma maas.node: [INFO] unfluffed-cherise: Status transition from READY to ALLOCATED
Aug 10 20:20:30 lucuma maas.node: [INFO] unfluffed-cherise: allocated to user bladernr
Aug 10 20:20:30 lucuma maas.interface: [INFO] Allocated automatic IP address 10.1.11.7 for enp6s0 (physical) on unfluffed-cherise.
Aug 10 20:20:30 lucuma maas.node: [INFO] unfluffed-cherise: Status transition from ALLOCATED to DEPLOYING
Aug 10 20:20:30 lucuma maas.power: [INFO] Changing power state (on) of node: unfluffed-cherise (4y3h8r)
Aug 10 20:20:33 lucuma maas.power: [INFO] Changed power state (on) of node: unfluffed-cherise (4y3h8r)
Aug 10 20:22:21 lucuma maas.preseed: [INFO] unfluffed-cherise: custom network and storage options are only supported on Ubuntu. Using flat storage layout.
Related branches
- Blake Rouse (community): Approve
-
Diff: 121 lines (+13/-43)2 files modifiedsrc/provisioningserver/drivers/hardware/tests/test_ucsm.py (+7/-29)
src/provisioningserver/drivers/hardware/ucsm.py (+6/-14)
- Newell Jensen (community): Approve
-
Diff: 134 lines (+15/-43)3 files modifieddocs/changelog.rst (+2/-0)
src/provisioningserver/drivers/hardware/tests/test_ucsm.py (+7/-29)
src/provisioningserver/drivers/hardware/ucsm.py (+6/-14)
Changed in maas: | |
status: | New → Incomplete |
Changed in maas: | |
importance: | Undecided → High |
summary: |
- MAAS 2 does not properly probe power state on Cisco UCS systems + [2.0] MAAS cannot power query with Cisco UCSM power driver |
no longer affects: | maas/2.0 |
summary: |
- [2.0] MAAS cannot power query with Cisco UCSM power driver + [2.1] MAAS cannot power query with Cisco UCSM power driver |
Changed in maas: | |
status: | Fix Committed → Fix Released |
Here are the various screenshots showing the state on UCSM and MAAS