[RFE] Allow Traits to be set on Nodes

[John Garbutt]

Nova and placement are moving towards traits to describe qualitative resources.

There is a Nova spec to take traits from ironic and put them into placement:
https://review.openstack.org/#/c/507052/

The other side of that is allowing traits to be set on Ironic nodes, so the Ironic driver can read those and set them on placement.

This will enable use cases such as flavors mapping to specific subsets of nodes with a given ResourceClass, allowing for things like reserving certain nodes for particular projects, via the use of a private flavor, etc.

It is hoped, this will later be expanded to include driver specified traits, and the ability to reconfigure nodes based on the requested traits in instance_info.

[John Garbutt (johngarbutt)]

There is a spec in review for this here:
https://review.openstack.org/#/c/504531

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/504531
Committed: https://git.openstack.org/cgit/openstack/ironic-specs/commit/?id=c912844dc5b27e89ee284bc6fe1c7ba074db1db7
Submitter: Zuul
Branch: master

commit c912844dc5b27e89ee284bc6fe1c7ba074db1db7
Author: John Garbutt <email address hidden>
Date: Fri Sep 15 15:47:19 2017 -0600

    Traits on Ironic Nodes

    This spec describes how we set traits on ironic nodes and make use of
    that within Nova's placement API.

    Related-bug: #1722194

    Change-Id: Ia181b1bb3188042dc0d17e02c5f7628194c34dad

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/528238

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/530723

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/532268

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/532269

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528238
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=bbff319f0226af11af3c2864ad8f678abef6dcdb
Submitter: Zuul
Branch: master

commit bbff319f0226af11af3c2864ad8f678abef6dcdb
Author: Mark Goddard <email address hidden>
Date: Fri Dec 15 11:29:00 2017 +0000

    Node traits: Add DB API & model

    Adds a NodeTrait model to the DB and DBAPI methods for the following:

    * Set/unset node traits
    * Get node traits
    * Add/remove a single trait
    * Check for existence of a node trait

    All node traits are deleted when their owner node is deleted. Traits
    cannot be specified on node creation currently.

    Change-Id: I8b375a33dcb94d2cc73eaa93a5cc5aed86c27178
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/530723
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=88a6761503e9a31ff2b776f4f462d2cf175aa1b0
Submitter: Zuul
Branch: master

commit 88a6761503e9a31ff2b776f4f462d2cf175aa1b0
Author: Mark Goddard <email address hidden>
Date: Sat Dec 30 11:12:06 2017 +0000

    Join nodes with traits

    Join the node_traits table with the nodes table in some
    DB operations.

    Change-Id: Ic959b5c5b1d431edc55d9abe1c6c2d7b41d76fbd
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/535296

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/535482

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/535908

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/535482
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c3ed7dfb9e7e878f30fbff68f066a409879b7a3b
Submitter: Zuul
Branch: master

commit c3ed7dfb9e7e878f30fbff68f066a409879b7a3b
Author: Mark Goddard <email address hidden>
Date: Thu Jan 18 17:02:50 2018 +0000

    Add a version argument to traits DB API

    We need to save the Trait object version to the DB when creating traits.

    Change-Id: I2c43c27455de6e7017477b0f12b18873c66455ad
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/536384

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/536978

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/536979

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/536980

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mark Goddard (<email address hidden>) on branch: master
Review: https://review.openstack.org/536978
Reason: Rolled into https://review.openstack.org/#/c/532268.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/532268
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=2cd7232f142373b0744dd42aaf1e6ce0740a1567
Submitter: Zuul
Branch: master

commit 2cd7232f142373b0744dd42aaf1e6ce0740a1567
Author: John Garbutt <email address hidden>
Date: Tue Jan 9 16:48:57 2018 +0000

    Add RPC object for traits

    Adds a new traits object to expose traits DB operations to the API. It
    also adds a new traits field into the node object, with the appropriate
    version compatibility logic.

    get_node_by_port_addresses is modified to ensure we correctly join in
    the tags and traits in that DB call, this avoids a orphaned db object
    lazy load style failure.

    _set_from_db_object in the base object is modified such that the new
    traits object doesn't have to include the dictionary style compatibility
    mix-in.

    Change-Id: I69403b9875a020fab7a7975810b57bf646417953
    Partial-Bug: #1722194
    Co-Authored-By: Mark Goddard <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/537386

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/537637

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/537637
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d8a291a1e1c84df2f5f164f292c4fa80ed034078
Submitter: Zuul
Branch: master

commit d8a291a1e1c84df2f5f164f292c4fa80ed034078
Author: Mark Goddard <email address hidden>
Date: Wed Jan 24 20:11:59 2018 +0000

    Fix node update with PostgreSQL

    In the node traits RPC object change [1], the _do_update_node method in
    the sqlalchemy DB API was modified to use a select query joined with the
    node_tags and node_traits tables in the node update DB operation. This
    was done to ensure that the returned NodeModel had its tags and traits
    attributes populated rather than lazy loaded. This was found not to work
    on PostgreSQL, which presents the following error message [2]:

    Exception during message handling: DBNotSupportedError:
    (psycopg2.NotSupportedError) FOR UPDATE cannot be applied to the
    nullable side of an outer join

    This issue is resolved in this commit by reverting to the original query
    without joins for the node update, and performing a joined query after
    the update.

    [1] https://review.openstack.org/#/c/532268/13
    [2]
    http://logs.openstack.org/69/532269/22/check/ironic-tempest-dsvm-pxe_ipmitool-postgres/562b2ea/logs/screen-ir-cond.txt.gz?level=ERROR#_Jan_23_23_59_05_062967

    Change-Id: Ib0d49f7e781f1aa0c4496957b00f6c8e3da0ecc0
    Related-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/535296
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c1cce7eb452c228dc2633e80f2c98fd142574fa9
Submitter: Zuul
Branch: master

commit c1cce7eb452c228dc2633e80f2c98fd142574fa9
Author: Mark Goddard <email address hidden>
Date: Thu Jan 18 11:11:16 2018 +0000

    Add RPC API and conductor manager for traits

    Adds two new methods to the conductor RPC API, add_node_traits and
    remove_node_traits, and provides implementations for them in the
    conductor manager. add_node_traits can be used to add one or more traits
    to a node, or to replace all the traits for a node. remove_node_traits
    can be used to remove one or more traits from a node, or to remove all
    traits from a node.

    The conductor RPC API version is bumped to 1.44.

    Change-Id: I0181df6a41e603874677246066d84bf4ac4f433c
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/532269
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=864331a94bddfd1860e0db9c6763fbf4f3ab6507
Submitter: Zuul
Branch: master

commit 864331a94bddfd1860e0db9c6763fbf4f3ab6507
Author: John Garbutt <email address hidden>
Date: Tue Jan 9 17:19:58 2018 +0000

    API: Node Traits API

    Adds basic CRUD on traits, including set and delete of individual
    traits.

    Bumps the API microversion to 1.37.

    NOTE: This patch does not implement the advanced filtering outlined in
    the OpenStack API working group's tags guidelines[1]. That will be
    implemented in a separate microversion as a follow up patch.

    [1] http://specs.openstack.org/openstack/api-wg/guidelines/tags.html

    Change-Id: I313fa01fbf20bf0ff19f102ea63b02e72ac2b856
    Partial-Bug: #1722194
    Co-Authored-By: Mark Goddard <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/536384
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=223e9b1faac157d78fed38ec40ac700803e8cce5
Submitter: Zuul
Branch: master

commit 223e9b1faac157d78fed38ec40ac700803e8cce5
Author: Mark Goddard <email address hidden>
Date: Mon Jan 22 11:32:02 2018 +0000

    Add node traits to API reference

    The node traits API was added in API microversion 1.37.

    Change-Id: I8c48f2e7662ae9641ec823dc3802d5979a7ea03e
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/535908
Committed: https://git.openstack.org/cgit/openstack/ironic-tempest-plugin/commit/?id=bcdb740e5d40e3dedfc73a1e6fe6b80585621fc2
Submitter: Zuul
Branch: master

commit bcdb740e5d40e3dedfc73a1e6fe6b80585621fc2
Author: Mark Goddard <email address hidden>
Date: Fri Jan 19 15:08:21 2018 +0000

    Add node trait tests

    Adds admin API tests for the node traits API added in Bare Metal REST
    API microversion 1.37.

    Change-Id: Ied30a56c93c874d036ca82569321660d6aa23906
    Partial-Bug: #1722194
    Depends-On: I313fa01fbf20bf0ff19f102ea63b02e72ac2b856

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/538653

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/536979
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c9677cd43ba61e737d91294b94e7ed74c282dca5
Submitter: Zuul
Branch: master

commit c9677cd43ba61e737d91294b94e7ed74c282dca5
Author: Mark Goddard <email address hidden>
Date: Mon Jan 22 09:51:00 2018 +0000

    Add traits field to node notifications

    Adds a traits field to node notifications, and triggers notifications
    when node traits are added or removed. Node traits are emitted in
    notifications as a list of trait name strings.

    Bumps the following notification payload versions:

    NodePayload: 1.6
    NodeSetPowerStatePayload: 1.6
    NodeCorrectedPowerStatePayload: 1.6
    NodeSetProvisionStatePayload: 1.6
    NodeCRUDPayload: 1.4

    Change-Id: I4e0333173250a641b317d466e52742cf7728ed90
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/537386
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=ccf22acb53f4357047e4e0caa658bf9801d817c4
Submitter: Zuul
Branch: master

commit ccf22acb53f4357047e4e0caa658bf9801d817c4
Author: Mark Goddard <email address hidden>
Date: Tue Jan 23 12:06:49 2018 +0000

    Fix nits found in node traits

    Includes:

    * Return a location in HTTP response header for PUT
      /nodes/<node>/traits/<trait>.
    * Correct order of arguments to assertEqual() in conductor manager unit
      tests for traits.
    * Use () rather than \ for multiline formatting.
    * Set tags and traits fields on NodeModel to default of [] in DB node
      create after DB insert.

    Change-Id: Ia88f7ca69b777b82caca2871e44edab39f2a8310
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/536980
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=00daa18fc99516b0947d2695c030871acff6d6bd
Submitter: Zuul
Branch: master

commit 00daa18fc99516b0947d2695c030871acff6d6bd
Author: Mark Goddard <email address hidden>
Date: Mon Jan 22 11:41:00 2018 +0000

    Add documentation for node traits

    Updates the install and admin guides with information on node traits.

    Support for node traits was added in Bare Metal REST API version 1.37.

    Change-Id: I86e86320df4eb81aa30e5708dde9c82f50ba4d50
    Depends-On: https://review.openstack.org/#/c/532288/
    Partial-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/538653
Committed: https://git.openstack.org/cgit/openstack/ironic-tempest-plugin/commit/?id=c4d1c83c1efa477624faa54a50acdac850c61341
Submitter: Zuul
Branch: master

commit c4d1c83c1efa477624faa54a50acdac850c61341
Author: Mark Goddard <email address hidden>
Date: Sun Jan 28 18:42:34 2018 +0000

    Sort node traits in comparisons

    Don't assume that node traits in API responses will be sorted. While
    this seems to be the case with mysql, it is not guaranteed for
    postgresql.

    Change-Id: I8f2afe6a3194203230e3f1cd2fee22190ff3be9f
    Partial-Bug: #1722194

[Ruby Loo (rloo)]

There is one outstanding item from the spec: Ironic needs to validate that instance_info[‘traits’] is consistent with the list of traits currently associated with Ironic node... "

Some discussion in irc [1] as to whether we need to do this, why we need to do this, how to go about doing it...

[1] http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2018-02-08.log.html#t2018-02-08T17:59:06

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/543461

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/545118

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/545297

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/545370

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mark Goddard (<email address hidden>) on branch: master
Review: https://review.openstack.org/545118
Reason: No longer required - nova has merged the fix to bug 1748942.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/545370
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=c8ae2457c7fd86975d77d6bc67b50b155f602944
Submitter: Zuul
Branch: master

commit c8ae2457c7fd86975d77d6bc67b50b155f602944
Author: Mark Goddard <email address hidden>
Date: Fri Feb 16 15:59:23 2018 +0000

    Register traits on nodes in devstack

    This change adds support for assiging traits to nodes in the ironic
    devstack plugin. The set of traits added is configured by
    IRONIC_DEFAULT_TRAITS, which defaults to a single trait - CUSTOM_GOLD

    Using nodes with traits during testing ensures that we are exercising
    node traits code paths in the Ironic CI.

    Node traits are not used for upgrade testing, as the queens devstack
    plugin does not currently register traits. Once this change is
    backported to queens, we can enable traits during upgrade testing.

    Change-Id: I18b9dbf8a5b16e36e0cc14f290b12ffdd634364a
    Related-Bug: #1722194
    Related-Bug: #1750027

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/551938

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/552043

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/551938
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=6015890adee55582330256bc10e933cba0b84cc9
Submitter: Zuul
Branch: stable/queens

commit 6015890adee55582330256bc10e933cba0b84cc9
Author: Mark Goddard <email address hidden>
Date: Fri Feb 16 15:59:23 2018 +0000

    Register traits on nodes in devstack

    This change adds support for assiging traits to nodes in the ironic
    devstack plugin. The set of traits added is configured by
    IRONIC_DEFAULT_TRAITS, which defaults to a single trait - CUSTOM_GOLD

    Using nodes with traits during testing ensures that we are exercising
    node traits code paths in the Ironic CI.

    Node traits are not used for upgrade testing, as the queens devstack
    plugin does not currently register traits. Once this change is
    backported to queens, we can enable traits during upgrade testing.

    Change-Id: I18b9dbf8a5b16e36e0cc14f290b12ffdd634364a
    Related-Bug: #1722194
    Related-Bug: #1750027
    (cherry picked from commit c8ae2457c7fd86975d77d6bc67b50b155f602944)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/552077

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552077
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=cef19cbf0a4d7b179de5d4448e247e9db1a79831
Submitter: Zuul
Branch: master

commit cef19cbf0a4d7b179de5d4448e247e9db1a79831
Author: Mark Goddard <email address hidden>
Date: Mon Mar 12 16:18:51 2018 +0000

    Use node traits during upgrade

    Now that the queens branch has support for node traits in the devstack
    plugin [1], we can apply node traits when testing upgrades from queens
    or later.

    [1] https://review.openstack.org/551938

    Change-Id: I99b9487d849ecd5241657d6ca91c2e9f9f7dfcf8
    Related-Bug: #1722194
    Related-Bug: #1750027

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/543461
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d1cd215c66eaa205bc63a5405835e0ef28dadc20
Submitter: Zuul
Branch: master

commit d1cd215c66eaa205bc63a5405835e0ef28dadc20
Author: Mark Goddard <email address hidden>
Date: Mon Feb 12 13:47:09 2018 +0000

    Validate instance_info.traits against node traits

    The ironic node traits spec calls out that traits added to
    instance_info.traits should be validated against the node's traits. All
    traits in instance_info.traits should exist in the node's traits. This
    protects us against race conditions between traits being removed from a
    node in ironic, and the node's resource provider's traits being updated
    in placement.

    This change adds validation to do_node_deploy() and
    validate_driver_interfaces() in the conductor manager, ensuring that all
    instance traits are also node traits.

    Change-Id: I956f8285fe428b2bdf8822e4a308f5c2a1675836
    Closes-Bug: #1755146
    Related-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/545297
Committed: https://git.openstack.org/cgit/openstack/ironic-tempest-plugin/commit/?id=56399ccba1fdbff6ded31671af0d8e87175d776f
Submitter: Zuul
Branch: master

commit 56399ccba1fdbff6ded31671af0d8e87175d776f
Author: Mark Goddard <email address hidden>
Date: Fri Feb 16 13:37:25 2018 +0000

    Validate scheduling fields in basic ops scenario

    Currently there is no validation of node scheduling fields - resource
    class and traits - in the scenario tests. This change adds validation of
    these fields to the bare metal basic ops test.

    We query the flavor used to boot the instance, and extract all requested
    resources and traits from extra_specs. These are matched against the
    resource class and traits set on the bare metal node that was scheduled.

    Change-Id: I9ddc895ead61cf02c6967ead094d061cb7f558d8
    Depends-On: https://review.openstack.org/545370
    Related-Bug: #1722194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552043
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=f5605d13e67c71c46538bafa5f7ddc18fb7427a6
Submitter: Zuul
Branch: master

commit f5605d13e67c71c46538bafa5f7ddc18fb7427a6
Author: Mark Goddard <email address hidden>
Date: Mon Mar 12 14:42:56 2018 +0000

    Assume node traits in instance trait validation

    In Rocky, we can assume that node.traits is set, since the earliest
    version we may be pinned to is Queens, which supports node traits.

    Change-Id: Ib011122a72f31e73e7e2031cb7582057e1f33ef4
    Related-Bug: #1755146
    Related-Bug: #1722194