Liberty release of deploy drivers break compat with out-of-tree power drivers

[aeva black]

The liberty release of the PXE and Agent deploy drivers were refactored to separate the BootInterface from the DeployInterface. While work was done in the ConductorManager class to retain backwards compatibility with previous drivers that did not include the new BootInterface, both the PXE and Agent deploy driver implementations ARE NOT backwards compatible.

I discovered this when attempting to use the "AMTTool" driver that I use in demos. After rebasing on Liberty, node validation and deploy fail with the following error:

2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher [req-bc6a5389-cd3e-4ea2-8f12-734bbe511a56 ] Exception during message handling: 'NoneType' object has no attribute 'validate'
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 142, in inner
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher return func(*args, **kwargs)
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 1506, in validate_driver_interfaces
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher iface.validate(task)
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/agent.py", line 194, in validate
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher task.driver.boot.validate(task)
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher AttributeError: 'NoneType' object has no attribute 'validate'
2015-10-05 16:40:39.062 16570 ERROR oslo_messaging.rpc.dispatcher
2015-10-05 16:40:39.066 16570 ERROR oslo_messaging._drivers.common [req-bc6a5389-cd3e-4ea2-8f12-734bbe511a56 ] Returning exception 'NoneType' object has no attribute 'validate' to caller

A quick work around was to instantiate the driver.boot interface in my driver's base class:

     def __init__(self):
         self.power = amttool.AMTPower()
         self.management = amttool.AMTManagement()
+ self.boot = pxe.PXEBoot()
         self.deploy = agent.AgentDeploy()
         self.vendor = agent.AgentVendorInterface()

This is also posted here:

  https://github.com/devananda/ironic/commit/f05b237bb4d2b7afde139cdeeb84462a0cf316ef

I don't consider this to be a good fix, however, since it requires updating out of tree drivers. A proper fix, which allows out of tree drivers to continue to work as-is should be implemented and then backported to Liberty.

[aeva black (tenbrae)]

Quick follow up with a slightly contrived experiment. If I deploy a node using the above fix, and then roll back the fix and attempt to delete the instance, I get a different traceback. This is a different, but also backwards-incompatible, code path:

2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager [-] Error in tear_down of node 98e7f8ae-92cf-4449-916d-c0ad42278e7c: 'NoneType' object has no attribute 'clean_up_ramdisk'
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager Traceback (most recent call last):
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager File "/usr/local/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 798, in _do_node_tear_down
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager task.driver.deploy.clean_up(task)
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager File "/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/agent.py", line 281, in clean_up
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager task.driver.boot.clean_up_ramdisk(task)
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager AttributeError: 'NoneType' object has no attribute 'clean_up_ramdisk'
2015-10-05 16:34:04.583 15864 ERROR ironic.conductor.manager
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
    result = function(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 806, in _do_node_tear_down
    task.process_event('error')
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/ironic/conductor/manager.py", line 798, in _do_node_tear_down
    task.driver.deploy.clean_up(task)
  File "/usr/local/lib/python2.7/dist-packages/ironic/drivers/modules/agent.py", line 281, in clean_up
    task.driver.boot.clean_up_ramdisk(task)
AttributeError: 'NoneType' object has no attribute 'clean_up_ramdisk'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/235026

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/235026
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=71e4e7440371b36694c4453f5b8637cf349b4e01
Submitter: Jenkins
Branch: master

commit 71e4e7440371b36694c4453f5b8637cf349b4e01
Author: Jim Rollenhagen <email address hidden>
Date: Wed Oct 14 17:15:04 2015 -0700

    Add note about driver API breakage

    This adds a note about the breakage we caused to our driver API behavior
    during Liberty, and resolutions that out of tree drivers may use to fix
    themselves.

    See also:
    http://lists.openstack.org/pipermail/openstack-dev/2015-October/076205.html

    Change-Id: I2671e7c8634f3fd3115697df6f0cab68c589d21d
    Closes-Bug: #1502980

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/ironic 4.3.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Devananda van der Veen (<email address hidden>) on branch: master
Review: https://review.openstack.org/231215