RFE: driver composition reform

[Dmitry Tantsur]

Our driver interface matrix becomes more and more complex. To top it all,
nowadays we have many interfaces that can be used for every driver. To name a
few:

* ``boot``: most drivers support PXE and iPXE, while some also support
  virtual media; support for *petitboot* bootloader is proposed.

* ``deploy``: two deploy approaches are supported: write image via iSCSI or
  write it directly from within the agent.

* ``inspect``: there is generic inspection using ironic-inspector, but some
  drivers allow also out-of-band inspection. This feature is optional, so we
  should provide a way to disable it.

Currently we ended up with a complex and really confusing naming schema.
Just an example:

* ``pxe_ipmitool`` uses PXE or iPXE for boot and iSCSI for deploy.

* ``agent_ipmitool`` actually also uses PXE or iPXE, but it does not use iSCSI.

* To top it all, ``pxe_ipmitool`` is actually using agent!

* So better names would probably be ``pxe_iscsi_ipmitool``,
  ``ipxe_iscsi_ipmitool``, ``pxe_direct_ipmitool``, ``ipxe_direct_ipmitool``.

* Now repeat the same with every power driver we have.

[Jim Rollenhagen (jim-rollenhagen)]

Spec is here: https://review.openstack.org/#/c/188370/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/188370
Committed: https://git.openstack.org/cgit/openstack/ironic-specs/commit/?id=98886ccd37bdb4824e6d62ea065ed01f915aad16
Submitter: Jenkins
Branch: master

commit 98886ccd37bdb4824e6d62ea065ed01f915aad16
Author: Dmitry Tantsur <email address hidden>
Date: Thu Jun 4 14:19:44 2015 +0200

    The driver composition reform

    This spec suggests revamping how we name and compose drivers from interfaces.
    It will allow having one vendor driver with options configurable per node
    instead of many drivers for every vendor.

    Partial-Bug: 1524745
    Change-Id: I2754f9bb810f5e05a9c1e4625afb8f2589eeef74

[Ruby Loo (rloo)]

The spec has been approved. Setting tag to rfe-approved.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/336626

[Sam Betts (sambetts)]

I have a concern that the hardware_type defaults imply a interface implementation is enabled, and therefore supported by a particular deployment of Ironic. This may not be the case for example:

hardware_type X:
supported_FOO_interfaces: [BAR, RAR]
default_FOO_interface: BAR

A deployer knows BAR doesn't work in their environment, but wants to support hardware_type X using RAR. So he enables hardware_type X. In the current spec enabling X automatically enables BAR because its the default even though it might not be supported.

This then means that someone using Ironic can create a node using BAR even though it is known not to work in this deployment. And worse than that is, if the person using Ironic doesn't override it the node will be created with BAR by default, resulting in a broken node by default.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/349510

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/349510
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=8c842ec50574cf72fe00cbfb7675cbdf826377f4
Submitter: Jenkins
Branch: master

commit 8c842ec50574cf72fe00cbfb7675cbdf826377f4
Author: Dmitry Tantsur <email address hidden>
Date: Mon Aug 1 15:40:08 2016 +0200

    Deprecate putting periodic tasks on a driver object

    This approach will no longer make sense when the driver composition is in effect.
    Anyway usually a better place to put a task is an interface.

    Change-Id: I7096e428ce9774d89ac624c2d38bb23984a4b842
    Related-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/393848

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/393867

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/396681

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/393848
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=565a0ed6b940df2d45172065840700bbc3802c99
Submitter: Jenkins
Branch: master

commit 565a0ed6b940df2d45172065840700bbc3802c99
Author: Dmitry Tantsur <email address hidden>
Date: Fri Nov 4 17:40:42 2016 +0100

    Simplify base interfaces in ironic.drivers.base

    Move abstract definitions of get_properties and validate to BaseInterface
    and make all interfaces inherit it. This has a side effect of them getting
    clean steps methods, but they anyway won't be wired in due to a whitelist
    of interfaces in the conductor.

    The RAIDInterface contained a mistake: it didn't have an ABCMeta metaclass,
    so its @abstractmethod definitions didn't work. I've implemented get_properties
    for it to avoid breaking implementers.

    Also added interface_type to all interfaces for consistency and for future
    use in the driver composition work.

    Change-Id: Ia6708247a99ecdcf73d83e66350e8c309090b7d4
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/393867
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=7591dcedfb1bc4e2f443af6445afb0319ec07ba5
Submitter: Jenkins
Branch: master

commit 7591dcedfb1bc4e2f443af6445afb0319ec07ba5
Author: Dmitry Tantsur <email address hidden>
Date: Fri Nov 4 17:43:17 2016 +0100

    Create dummy interfaces for use with hardware types

    A new module is created containing "noop" implementations of all
    interfaces we consider optional. These can be used as default
    implementation for hardware types that don't support a particular
    interface or just don't want it enabled by default.

    Partial-Bug: #1524745
    Change-Id: I2abe2ff5449ac2671020f309b27c4e738fa017b5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/397103

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tuan Luong-Anh (<email address hidden>) on branch: master
Review: https://review.openstack.org/397103
Reason: Only need for BareDriver. Thanks for your help

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/396681
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9e938f24668e49216b4cb135770899a50d627914
Submitter: Jenkins
Branch: master

commit 9e938f24668e49216b4cb135770899a50d627914
Author: Dmitry Tantsur <email address hidden>
Date: Fri Nov 11 16:40:46 2016 +0100

    Add node (database and objects) fields for all interfaces

    For the driver composition reform we need nodes to record hardware
    interfaces they're using. This change adds necessary fields.

    The new fields will have a default value of None, which is the expected
    value for nodes using classic drivers.

    The new fields are not added to notifications yet, as they're not wired in.

    Change-Id: Id7697de0276e9b2730ea62f9c562ed6e1d0f8a06
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/400678

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/400881

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/401311

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dmitry Tantsur (<email address hidden>) on branch: master
Review: https://review.openstack.org/400881
Reason: rolling into previous patch

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/401311
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=6e981156c83c4d7217e985c88553548d5924c601
Submitter: Jenkins
Branch: master

commit 6e981156c83c4d7217e985c88553548d5924c601
Author: Dmitry Tantsur <email address hidden>
Date: Wed Nov 23 16:17:58 2016 +0100

    Move interface validation from API to conductor side

    This change moves the check on network interface to driver_factory.py
    and moves node creation from API to conductor side. A new exception
    InterfaceNotFoundInEntrypoint (similar to DriverNotFoundInEntrypoint)
    was introduced for the case when invalid interface is requested either
    during creation or during updating.

    The current approach already duplicates the check in two places.
    With the driver composition in place it will not be quite possible:
    different conductors may have different interfaces enabled as long as
    no hardware type ends up with different interfaces on different
    conductors. Also when calculating the defaults, we'll need access
    to a real hardware type object, which may not be possible on API side.

    Also there is a demand for driver-side validations on creation
    (e.g. detecting duplicating IPMI addresses), which also requires
    creation to happen on conductor side. Such features, however, are
    out of scope for this change.

    A side effect of this change is that objects.Node.network_interface
    is now defined the same way as other interfaces (i.e. nullable string).

    Also added more clean ups to base unit test class, as otherwise newly
    added tests randomly break other tests.

    Change-Id: Id1da20ccd5bb50e61a82449ef3d2ffce91822d71
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/409812

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/336626
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=901171194bfecee9fc7233b9aafe7e721a201bd6
Submitter: Jenkins
Branch: master

commit 901171194bfecee9fc7233b9aafe7e721a201bd6
Author: Dmitry Tantsur <email address hidden>
Date: Fri Jul 1 17:52:11 2016 +0200

    Support defining and loading hardware types

    This change introduces a new module ironic.drivers.hardware_types
    with AbstractHardwareType class. It also updates driver_factory
    code to support loading hardware types and creating dynamic drivers.
    Interfaces validation code extended to cover hardware types.

    This change also introduces the FakeHardware class for testing.
    It is special-cased to bypass compatibility validation completely.

    No hardware types are loaded on conductor start up yet, as hardware
    types still do not participate in the hash ring. Thus, nodes with
    hardware types cannot still be created via HTTP API.

    Change-Id: If8e3342baf818a9e37aa82b43aec71898d48c29b
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tuan Luong-Anh (<email address hidden>) on branch: master
Review: https://review.openstack.org/397103
Reason: Not usefull

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/400678
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=2777814449b96310fd5a7d027e95b130834c454d
Submitter: Jenkins
Branch: master

commit 2777814449b96310fd5a7d027e95b130834c454d
Author: Dmitry Tantsur <email address hidden>
Date: Tue Nov 22 11:19:26 2016 +0100

    Introduce generic hardware types

    This change adds ironic.drivers.generic.GenericHardware which contains
    useful defaults. Other hardware types are recommended to derive from it.

    Add reasonable vendor-independent defaults to enabled_{deploy,raid,boot}_interfaces
    configuration options. Add an entry point for "inspector" inspect implementation.

    Also adding ironic.drivers.generic.ManualManagementHardware. This hardware
    type is mostly introduced to make unit testing of GenericHardware possible.
    However, it also has a potential use case: hardware not covered by
    an existing driver or when an existing driver misbehaves. I personally
    know people using fake_pxe currently in production for such use cases.

    Change-Id: I328191524eb71c2ef51fec02c9cb28087dcf663b
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/412631

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/416232

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/409812
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=4fa2412d749261b7b0aa8e69997d6376ae24bf07
Submitter: Jenkins
Branch: master

commit 4fa2412d749261b7b0aa8e69997d6376ae24bf07
Author: Dmitry Tantsur <email address hidden>
Date: Mon Dec 12 15:27:27 2016 +0100

    Create a table to track loaded interfaces

    A helper table conductor_hardware_interfaces was created to track which
    hardware types have which interfaces loaded on which conductors.

    Partial-Bug: #1524745
    Change-Id: Iae2f3fddb2441b6069c011cf8bf646943d34d0a4

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/417920

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/417970

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/418364

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/418573

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/418364
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=41da55ced24fced54c42ba9e6462aead960c2e8b
Submitter: Jenkins
Branch: master

commit 41da55ced24fced54c42ba9e6462aead960c2e8b
Author: Dmitry Tantsur <email address hidden>
Date: Tue Jan 10 13:33:18 2017 +0100

    Correct logging of loaded drivers/hardware types/interfaces

    Currently we log "Loaded the following drivers" for everything.
    This change modifies the string to log an actual type.

    Change-Id: I5e2a9f2282e03d7eddcf898ed7336040613cd311
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/419065

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/421464

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/421465

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/416232
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=56604b9d688d76b3d77e85c3c069ab2d0a08fb6a
Submitter: Jenkins
Branch: master

commit 56604b9d688d76b3d77e85c3c069ab2d0a08fb6a
Author: Dmitry Tantsur <email address hidden>
Date: Wed Jan 11 17:43:44 2017 +0100

    Do not fail in Inspector.__init__ if [inspector]enabled is False

    We already have a check in Inspector.create_if_enabled. This additional
    check prevents normal use of Inspector with hardware types, as it's not
    enough to add "inspector" to "enabled_inspect_interfaces" to use it.

    Change-Id: Iff29785002cd978702317d50ce52bdcebe06d0e3
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/421464
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=ac90e7372c6348dda2a04e8789caa625db331c5e
Submitter: Jenkins
Branch: master

commit ac90e7372c6348dda2a04e8789caa625db331c5e
Author: Jim Rollenhagen <email address hidden>
Date: Tue Jan 17 15:55:28 2017 +0000

    Add default column to ConductorHardwareInterfaces

    This will mark the default interface for each hardware type in use.
    This is the only piece of information about hardware interfaces that is
    not in the database, and so this will allow the API to fetch this
    information without an RPC call.

    Change-Id: I5b05853a64ebb2744f9dc87855fb31fbe5cefce6
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/418573
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=472abf43b03ee5ce8aaf25a89732a318cd9cf680
Submitter: Jenkins
Branch: master

commit 472abf43b03ee5ce8aaf25a89732a318cd9cf680
Author: Jim Rollenhagen <email address hidden>
Date: Tue Jan 10 21:20:03 2017 +0000

    Expose default interface calculation from driver_factory

    The API for this feature will need to be able to calculate the default
    interfaces for a given hardware interface. The code was already there to
    do it, but just needed a bit of refactoring and a public method. Move
    things around as we need them, and add a few tests to test the
    default_interface method directly.

    Change-Id: I993b76b7a2e6219cacc86558a612ecf0ba10e685
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422599

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422708

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/412631
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=b7e6b737d7aea841e718d66776894986dbcab2a2
Submitter: Jenkins
Branch: master

commit b7e6b737d7aea841e718d66776894986dbcab2a2
Author: Jim Rollenhagen <email address hidden>
Date: Mon Dec 19 17:12:01 2016 -0500

    Register/unregister hardware interfaces for conductors

    This registers the intersection of supported and enabled interfaces for
    each hardware type enabled in the conductor at conductor startup, and
    unregisters them at conductor shutdown. Validation is left as a todo for
    now.

    Change-Id: I14e88bfc304de9414de008d1cc8568dda9115ecc
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422599
Committed: https://git.openstack.org/cgit/openstack/ironic-specs/commit/?id=aecb9a75b717c0fd7d34e5152912ee7df80f3a37
Submitter: Jenkins
Branch: master

commit aecb9a75b717c0fd7d34e5152912ee7df80f3a37
Author: Jim Rollenhagen <email address hidden>
Date: Thu Jan 19 07:56:13 2017 -0500

    Driver composition: add default column to new table

    This was found to be needed, to allow the API to have all necessary data
    to return default and supported interfaces for hardware types.

    Change-Id: I6b3b74dd8407634ad079a3411375eb93a6fcb18d
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/417920
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3d22fa4270931edf344a5396f5333f73f4a5f0f2
Submitter: Jenkins
Branch: master

commit 3d22fa4270931edf344a5396f5333f73f4a5f0f2
Author: Jim Rollenhagen <email address hidden>
Date: Mon Jan 9 15:13:35 2017 +0000

    Enable manual-management hardware type in devstack

    This should prove that the loading and registration of hardware types
    works properly in CI.

    Change-Id: I6cd50fd94fe1a6ca5304f907a662fcc7c52d8fd8
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422862

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422863

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jim Rollenhagen (<email address hidden>) on branch: master
Review: https://review.openstack.org/421465
Reason: Accidentally rolled this into 419065, but I think it's cleaner there anyway.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/419065
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3c45f2fd1b8ca90ecc02ec32206eadab0333c7a1
Submitter: Jenkins
Branch: master

commit 3c45f2fd1b8ca90ecc02ec32206eadab0333c7a1
Author: Jim Rollenhagen <email address hidden>
Date: Wed Jan 11 16:59:13 2017 +0000

    Add hardware types to the hash ring

    This loads hardware types into the hash ring, along with the drivers
    that are currently there. This allows us to make RPC calls for nodes
    that are using a dynamic driver.

    Adds a dbapi method `get_active_hardware_type_dict`, similar to
    `get_active_driver_dict`, to get a mapping of hardware types to active
    conductors.

    Change-Id: I50c1428568bd8cbe4ef252d56a6278f1a54dfcdb
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422862
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d46ab0d0b2972e54ff5b66294bdfb38d2c9f3ecf
Submitter: Jenkins
Branch: master

commit d46ab0d0b2972e54ff5b66294bdfb38d2c9f3ecf
Author: Jim Rollenhagen <email address hidden>
Date: Thu Jan 19 20:42:25 2017 +0000

    Clean up driver_factory.enabled_supported_interfaces

    This had an extra variable (and related work) that was unnecessary,
    clean it up.

    Change-Id: I245946a36e891d305dd06dfd3944a6e623d731a4
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/423536

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/423537

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422863
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=1dc154033f294c34a4ea3f52d89522a59f31f26c
Submitter: Jenkins
Branch: master

commit 1dc154033f294c34a4ea3f52d89522a59f31f26c
Author: Jim Rollenhagen <email address hidden>
Date: Thu Jan 19 20:43:17 2017 +0000

    Turn NOTE into docstring

    conductor.base_manager._register_and_validate_hardware_interfaces had a
    note at the top about what exceptions might be raised. Turn this into a
    proper docstring.

    Change-Id: I60b3e864f4cfba38ed7d12caf3bf723d73ab9e39
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jim Rollenhagen (<email address hidden>) on branch: master
Review: https://review.openstack.org/423536
Reason: Squashed into https://review.openstack.org/#/c/417970/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jim Rollenhagen (<email address hidden>) on branch: master
Review: https://review.openstack.org/423537
Reason: squashed into https://review.openstack.org/#/c/417970/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/424720

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422708
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=797544316ebd07d2ce674ed9aa4f8f98fe7d894d
Submitter: Jenkins
Branch: master

commit 797544316ebd07d2ce674ed9aa4f8f98fe7d894d
Author: Dmitry Tantsur <email address hidden>
Date: Thu Jan 19 16:35:49 2017 +0100

    Add hardware type for IPMI using ipmitool

    Created a new Python module ironic.drivers.ipmi to contain all
    supported IPMI-related drivers (both new and old).
    Backward compatible aliases are left at old locations.

    Change-Id: I526b8505d08b25b668af76d3fcd8f2feab31e420
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/425873

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/426213

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jim Rollenhagen (<email address hidden>) on branch: master
Review: https://review.openstack.org/425873
Reason: Squashing into https://review.openstack.org/#/c/417970/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/417970
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=e77675781231409d16744ab9d1b13ffc92d358f6
Submitter: Jenkins
Branch: master

commit e77675781231409d16744ab9d1b13ffc92d358f6
Author: Jim Rollenhagen <email address hidden>
Date: Wed Jan 11 17:01:51 2017 +0000

    Add dynamic driver functionality to REST API

    This adds API version 1.30, which adds dynamic driver parameters
    and response fields to `GET /v1/drivers` and `GET /v1/drivers/<name>`.

    Changes RAID APIs to work for dynamic drivers.

    Also changes GET /v1/drivers/<name>/properties to work for dynamic
    drivers. It uses the calculated default implementation for each
    interface, when calculating the properties.

    Last, changes node and driver vendor passthru to work correctly
    for dynamic drivers. Similar to properties, driver vendor passthru
    will use the calculated default vendor implementation.

    Change-Id: If13e7e7fd368273e84d9a108be93b58150432fae
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/427322

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/427335

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/427389

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/424720
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=8570bee3d658522cc6118cf3e32750fa5211307a
Submitter: Jenkins
Branch: master

commit 8570bee3d658522cc6118cf3e32750fa5211307a
Author: Jim Rollenhagen <email address hidden>
Date: Tue Jan 24 16:09:12 2017 +0000

    Add dynamic interfaces fields to nodes API

    This adds version 1.31 of the REST API, which adds dynamic interface
    fields to the node object.

    Change-Id: Ic8398a6093189a65a7c1ab5cf7e682577dde3257
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/427651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427335
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=e0affc191ef5c9aa5bf3bb47037e44f43546a596
Submitter: Jenkins
Branch: master

commit e0affc191ef5c9aa5bf3bb47037e44f43546a596
Author: Ruby Loo <email address hidden>
Date: Tue Jan 31 17:48:19 2017 +0000

    Duplicated code in ..api.get_active_driver_dict()

    In the refactoring of some code in ironic/db/sqlalchemy/api,
    there was some duplication of code in get_active_driver_dict().
    This removes that duplication.

    This is a follow up to 3c45f2fd1b8ca90ecc02ec32206eadab0333c7a1.

    Change-Id: Id32280a6c020dce8e3580ea951b5822cf6a8eabf
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427389
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=94bd464ce88fdbd43aa46d61ad625aa06e1b5278
Submitter: Jenkins
Branch: master

commit 94bd464ce88fdbd43aa46d61ad625aa06e1b5278
Author: Ruby Loo <email address hidden>
Date: Tue Jan 31 20:13:45 2017 +0000

    Log reason for hardware type registration failure

    The log string 'Failed to register hardware types' doesn't
    provide much help as to what went wrong. This adds the reason
    (exception message) to the log.

    Change-Id: I941e35473f48c636134d5df31087d0ddbcacf44a
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427651
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d361d756add356253c474dd6affe8e5356ae470c
Submitter: Jenkins
Branch: master

commit d361d756add356253c474dd6affe8e5356ae470c
Author: Yuriy Zveryanskyy <email address hidden>
Date: Wed Feb 1 13:36:49 2017 +0200

    Add dynamic interfaces fields to base node notification

    This patch adds dynamic interface fields which are exposed via API
    to base node notification object.

    Change-Id: If761fb73357d940afe42b329f413ff0228434586
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/426213
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=0071c7e1ec991b8b150168216133390b854c759f
Submitter: Jenkins
Branch: master

commit 0071c7e1ec991b8b150168216133390b854c759f
Author: Jim Rollenhagen <email address hidden>
Date: Fri Jan 27 12:26:22 2017 +0000

    Improve conductor driver validation at startup

    This changes the driver loading validation in the conductor
    startup to check for at least one classic *or* dynamic driver.
    Previously the conductor would fail to start if no classic drivers
    were loaded. This allows the conductor to use only dynamic
    drivers, without loading any classic drivers.

    It also now checks classic driver names against dynamic driver
    names, and fails to start if there is a conflict there. This
    would totally break the hash ring and cause mass confusion,
    so we cannot allow it.

    Change-Id: Id368690697f90471d09f16eaa4925338dadebd0f
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/428219

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427322
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=6206c47720eb16719088540a48b53fb22a4eb999
Submitter: Jenkins
Branch: master

commit 6206c47720eb16719088540a48b53fb22a4eb999
Author: Jim Rollenhagen <email address hidden>
Date: Tue Jan 31 17:29:10 2017 +0000

    Fail conductor startup if invalid defaults exist

    This causes the conductor to fail to start up if a default interface
    implementation cannot be found for any dynamic driver. This avoids
    problems later where building a task object to operate on a node
    could fail for the same reason.

    This also removes a RAID interface test that turned out to be an
    invalid test, but we couldn't tell it was invalid until we had
    changed the start up behavior of the conductor.

    Note that this release note doesn't actually note a change between
    releases, but rather is mostly for my use when I come back to combine
    many of the release notes for this feature later.

    Change-Id: I39d3c30a6beda2e496ff85119281fdf4de191560
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/428475

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/428219
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=1f25244bf64d24444c488b4396b5402a0fd9d53d
Submitter: Jenkins
Branch: master

commit 1f25244bf64d24444c488b4396b5402a0fd9d53d
Author: Jim Rollenhagen <email address hidden>
Date: Thu Feb 2 15:50:45 2017 +0000

    Improve enabled_*_interfaces config help and validation

    This adds additional constraints to the help messages for the
    enabled_*_interfaces config options. It also checks if they are
    empty at conductor startup, and if any are empty, errors out
    with a better error message than previously provided.

    Change-Id: I97fc318ce00291d5e43b70423930981c2f5a2de0
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/429694

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/429694
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=e98b028a656111e8c60c1a4953f7e4e007af3029
Submitter: Jenkins
Branch: master

commit e98b028a656111e8c60c1a4953f7e4e007af3029
Author: Dmitry Tantsur <email address hidden>
Date: Mon Feb 6 14:12:06 2017 +0100

    [devstack] Allow using "ipmi" hardware type

    Currently only the default interfaces are supported.

    Also fail devstack if the deploy driver is neither in enabled_drivers
    not in enabled_hardware_types, as we no longer can easily detect which
    list to append it to automagically.

    Change-Id: I055497963f05833bf065ab396fff9b628357fc08
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/428475
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=0a7fc6059a70486793353f20fb7152782a3f398a
Submitter: Jenkins
Branch: master

commit 0a7fc6059a70486793353f20fb7152782a3f398a
Author: Ruby Loo <email address hidden>
Date: Thu Feb 2 22:27:01 2017 +0000

    exception from driver_factory.default_interface()

    This changes driver_factory.default_interface() so that instead
    of returning None if there is no calculated default interface,
    it raises exception.NoValidDefaultForInterface.

    This is a follow up to 6206c47720eb16719088540a48b53fb22a4eb999.

    Change-Id: I0c3d5d75b5a37af02f3660968cf3f2c669e52019
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/430435

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/430435
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d8bdbda03067842c13deda504d68670b4b955b66
Submitter: Jenkins
Branch: master

commit d8bdbda03067842c13deda504d68670b4b955b66
Author: Ruby Loo <email address hidden>
Date: Tue Feb 7 20:44:18 2017 +0000

    No node interface settings for classic drivers

    Classic drivers have their interfaces (except for
    network and storage) pre-determined. Unlike dynamic
    drivers, it is not possible to change these interfaces
    for nodes with a classic driver. If that is attempted
    (when creating or updating a node), an exception
    MustBeNone is raised (and HTTP status 400 is returned).

    So that we don't break existing ironic clusters that
    have nodes with classic drivers and interfaces specified,
    a warning is logged (instead of raising an exception)
    when a TaskManager is created.

    Change-Id: I290b10f735d0da9710d1ee3b50c3252f73956428
    Partial-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/432191

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/428460
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d33faedbacc6f25d4106b7738bd3f7aeafc839b1
Submitter: Jenkins
Branch: master

commit d33faedbacc6f25d4106b7738bd3f7aeafc839b1
Author: Mario Villaplana <email address hidden>
Date: Thu Feb 2 22:00:18 2017 +0000

    Update api-ref for dynamic drivers

    Updates reference for drivers API as well as nodes API to match the
    merged driver composition reform work.

    Change-Id: I89a7b1998a858d5c39f21c6653893aa8d7ad15bd
    Partial-Bug: 1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/432191
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=b85d0d7632d663d56f211d3e4e5534c513dd6012
Submitter: Jenkins
Branch: master

commit b85d0d7632d663d56f211d3e4e5534c513dd6012
Author: Vasyl Saienko <email address hidden>
Date: Fri Feb 10 11:14:18 2017 +0200

    Fix missed '_' import

    This patch fixes missed import of '_' in
    ironic/api/controllers/v1/drivers.py

    Change-Id: Ie076000b441ada7f45aff26e96fe627a961f7336
    Related-Bug: #1524745

[Vladyslav Drok (vdrok)]

Are there any items left to do here?

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/462151

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/463609

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/464046

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/462151
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=10decb88bb101986031710c2ad8aebfac19f6eba
Submitter: Jenkins
Branch: master

commit 10decb88bb101986031710c2ad8aebfac19f6eba
Author: Dmitry Tantsur <email address hidden>
Date: Wed May 3 15:31:34 2017 +0200

    Refactor install-guide: driver and hardware types configuration

    * Move PXE and iPXE configuration to a separate document
    * Move IPMI and iSCSI configuration to a separate document
    * Move bits mostly dealing with node-update to "advanced" section
    * Write detailed documentation on drivers, hardware types and interfaces
    * Drop outdated information, like custom DHCP file

    Change-Id: I6dfc8842fb262285308578f82fab701bd29289fa
    Related-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/463609
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=a8ca351edc5b768992c072458a1ac050bdee0bf3
Submitter: Jenkins
Branch: master

commit a8ca351edc5b768992c072458a1ac050bdee0bf3
Author: Dmitry Tantsur <email address hidden>
Date: Tue May 9 20:53:26 2017 +0200

    Refactor install-guide: update node enrollment

    * Document how to choose a driver and figure out its properties
      separately from the main flow for simplicity.
    * Rewrite the documentation to recommend using "enroll" state, as it's
      our feature, and it's more complex to use anyway.
    * Add documentation on changing hardware interfaces.
    * Use console coloring and $ prefix for command examples
      (for readability and consistency with other projects install guides).
    * Various small fixes.

    Related-Bug: #1524745
    Change-Id: I52c1a48829a04e5b497680ef25ed80e0af998cbd

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/466734

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/466741

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/466741
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d3e78688ca5bb87dfff915b4f450942635ec3cb3
Submitter: Jenkins
Branch: master

commit d3e78688ca5bb87dfff915b4f450942635ec3cb3
Author: Dmitry Tantsur <email address hidden>
Date: Mon May 22 16:22:45 2017 +0200

    [install-guide] explain the defaults calculation for hardware types

    Also add a missing note about configuring hardware types in a multi-conductor
    environment.

    Change-Id: Ib26786f443c3f30859affde37663aa3c5a60eea8
    Related-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/466734
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d3ea000db614487879f3c61f4e4554a37115574e
Submitter: Jenkins
Branch: master

commit d3ea000db614487879f3c61f4e4554a37115574e
Author: Dmitry Tantsur <email address hidden>
Date: Mon May 22 15:28:30 2017 +0200

    Update ipmitool installation and usage documentation

    * Mention the IPMI hardware type in usage docs
    * Cross-link between usage and install docs
    * Add missing ipmitool vendor interface example
    * Stop pretending that there is one "IPMItool driver"
    * Fix several small issues

    Change-Id: I3c8928abc319d04b84f8581226627860c57a1abc
    Related-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/464046
Committed: https://git.openstack.org/cgit/openstack/ironic-specs/commit/?id=2db98e494e3fe8b28852c10aab593c4934d7206b
Submitter: Jenkins
Branch: master

commit 2db98e494e3fe8b28852c10aab593c4934d7206b
Author: Dmitry Tantsur <email address hidden>
Date: Thu May 11 18:58:31 2017 +0200

    Define the future of the classic drivers

    This is a follow up to the driver composition reform.

    Change-Id: I1111742cee554e8f3b26e85dddfe81a1b563ca22
    Partial-Bug: #1690185
    Related-Bug: #1524745

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/486696

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/486651
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=fe295ffb5a29c004e17b0958b5969c0250789cff
Submitter: Jenkins
Branch: master

commit fe295ffb5a29c004e17b0958b5969c0250789cff
Author: Dmitry Tantsur <email address hidden>
Date: Mon Jul 24 16:59:41 2017 +0200

    Fix small issues in the installation documentation

    Fixes nits spotted on the following reviews:
    * https://review.openstack.org/#/c/461834/
    * https://review.openstack.org/#/c/462151/
    * https://review.openstack.org/#/c/466741/
    * https://review.openstack.org/#/c/466734/

    Also updated links in the touched files.

    Related-Bug: #1524745
    Change-Id: I12cfbbd4cbdd9423f86d0b3d9f4209427313c472

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/486696
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d39f9af92b635e5aa0eec8ae9c826023ab6ec38a
Submitter: Jenkins
Branch: master

commit d39f9af92b635e5aa0eec8ae9c826023ab6ec38a
Author: Dmitry Tantsur <email address hidden>
Date: Mon Jul 24 18:56:24 2017 +0200

    Guide on upgrading to hardware types

    Only covers the generic cases for now. More drivers should be added later.

    Related-Bug: #1524745
    Partial-Bug: #1690185
    Change-Id: Ic63c7ec7068db84931ac4ede712691327547810e