Failure powering up nodes

Bug #1484551 reported by Ricardo Bánffy
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
MAAS
Won't Fix
Critical
Newell Jensen

Bug Description

In the MAAS CI environment (running off trunk, r4185) we started getting power-up failures seemingly due to an extra space added to the node's IP address.

From cluster.log:

2015-08-13 04:26:25-0400 [-] Unhandled Error
 Traceback (most recent call last):
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 423, in errback
     self._startRunCallbacks(fail)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
     self._runCallbacks()
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1155, in gotResult
     _inlineCallbacks(r, g, deferred)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/drivers/power/__init__.py", line 278, in perform_power
     raise e
 provisioningserver.drivers.power.PowerFatalError: Failed to power on 10.245.143.105 : /usr/sbin/ipmi-chassis-config: invalid hostname

Full set of artifacts is attached to this bug report.

Revision history for this message
Ricardo Bánffy (rbanffy) wrote :
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Power parameters are set like:

    "node-4e750024-419f-11e5-99c0-deadbeef6bb3": {
        "power_driver": "",
        "mac_address": "",
        "power_address": "10.245.143.104 ",
        "power_pass": "ubuntu",
        "power_user": "root"
    }

My guess is the trailing whitespace after power_address. However, this is a regression.... the ipmi power driver should not fail due to a trailing whitespacE?

Changed in maas:
importance: Undecided → Critical
assignee: nobody → Newell Jensen (newell-jensen)
status: New → Confirmed
Revision history for this message
Gavin Panella (allenap) wrote :

There is a schema to validate power parameters when they're set. I suggest we first check that that is doing the right thing before patching over the problem in the driver.

Gavin Panella (allenap)
Changed in maas:
status: Confirmed → Triaged
Revision history for this message
Newell Jensen (newell-jensen) wrote :

Gavin,

Didn't we discuss that this would be handled by MAAS closer to the database so that valid data is saved and the power parameters doesn't have to deal with this?

Revision history for this message
Gavin Panella (allenap) wrote :

> Didn't we discuss that this would be handled by MAAS closer to the
> database so that valid data is saved and the power parameters doesn't
> have to deal with this?

Yep, I think that's the best approach. For reference, my comments in https://code.launchpad.net/~newell-jensen/maas/fix-1484551/+merge/268029 have my thoughts, and my conclusion was:

-> We should *not* fix-up white-space issues in the driver.

-> We should tighten up validation of power parameters to help prevent
   bad data from getting in.

-> We should fix validation in the case of power_driver.

Changed in maas:
status: Triaged → Invalid
Changed in maas:
status: Invalid → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.