Ironic multinode-grenade upgrades nova without upgrading the API

[Julia Kreger]

As a result of the boot from volume work, and the necessary micro version minimum bump in Nova, we were able to observe failures with the multimode grenade tests.

In short, the multi-node grenade test does the following:
1) Installs ironic on two nodes
2) Upgrades _one_ of those nodes
3) Shuts down the upgraded node's API.
4) Redirects the upgraded node's API to the node that has not been upgraded.

Which, if that node's API micro version is below what is in nova's code base, then nova-compute fails to start, and we get a wonderful error along the lines of:

"Did a compute worker not boot correctly"

That worker failed to start up with the following error:

ERROR oslo_service.service [req-382988f0-2a47-42cd-b956-a0daa9237c09 - -] Error starting thread.: UnsupportedVersion: Requested API version 1.32 is not supported by the server or the
requested operation is not supported by the requested version.
Supported version range is 1.1 to 1.31

The bulk of the discussion in IRC about this can be found at:
http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/latest.log.html#t2017-07-25T13:12:27

[John L. Villalovos (happycamp)]

Fix proposed: https://review.openstack.org/487224

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/487224
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9d161f6922c48779cfd2356726a5449ccb253229
Submitter: Jenkins
Branch: master

commit 9d161f6922c48779cfd2356726a5449ccb253229
Author: John L. Villalovos <email address hidden>
Date: Tue Jul 25 14:33:29 2017 -0700

    grenade: For multi-node grenade, do not upgrade nova

    Multi-node grenade jobs will fail if we upgrade nova as the minimum
    required API for nova to communicate with ironic was upgraded.

    Our goal for multi-node grenade is to test rolling upgrades. The
    scenario we envision for rolling upgrades to be done is:

     1) Upgrade ironic-conductor services one by one. This means old
        ironic-api services communicating with new and old
        ironic-conductor services. This is what the multi-node grenade job
        tests.
     2) Upgrade ironic-api services.
     3) Upgrade nova

    In this upgrade procedure we should not be running updated nova with
    old ironic-api services. This patch fixes that.

    Closes-Bug: #1706418
    Change-Id: I85c607ae7d031e4cabcb505915aca7e88df166aa

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ironic 9.0.0 release.