It is not possible to define order when registering project to upgrade

[Vasyl Saienko]

Grenade doesn't allow to specify dependency between projects when registering project for upgrade. Especially when plugin should be upgraded before core service. Like ironic and nova.

The nova-compute won't start if ironic is not upgraded prior nova. It leads to https://bugs.launchpad.net/devstack/+bug/1660581

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/427248

[Castulo J. Martinez (castulo-martinez)]

@Vasyl Saienko, do you mind adding steps to recreate this bug please?

[Jim Rollenhagen (jim-rollenhagen)]

discussion in irc with perhaps a good/easy fix for this:

16:47:15 jroll | sdague: so we learned we are doing it wrong (tm) in our grenade jobs, but were getting lucky until recently, now it's broken. this hacks around it until we can really fix it: https://review.openstack.org/#/c/427248
16:47:52 jroll | the bug trail can be followed for details if you need more
16:52:26 sdague | jroll: if you are +1 on that, I'll +A it. Hopefully it unsticks you.
16:52:44 sdague | We do need to figure out how to deal with that in a more real way later
16:53:02 jroll | sdague: +1'd, yeah I'm good with it
16:53:06 sdague | I was really hoping we wouldn't have to build a full dependency graph
16:53:13 sdague | but... that might be dead hope
16:53:30 jroll | I guess there's the alternative of support upgrades in either order, but that makes other things harder
16:56:30 sdague | so... maybe we could cheat a little
16:57:18 sdague | register_project_for_upgrade is currently just adding things to a list
16:57:57 sdague | ensure_project_before ironic nova - that moves the ironic entry to be right before the nova entry
16:58:09 sdague | and we just do a list adjust early on
16:58:48 sdague | I'm trying to think about if that could go horribly wrong
17:06:08 jroll | sdague: oh, your other idea is interesting, we could play with it
17:07:24 sdague | jroll: yeh, if you or someone else on the ironic side wanted to implement that ensure_project_before function, which would be an explicit call after the register_project... call, that would be cool
17:07:35 sdague | we'll still land the other patch to fix you
17:07:48 sdague | but that would be the more correct way to handle things
17:09:03 jroll | sdague: yeah, thanks, I'll post in the bug about it and I think someone can get to it soon
17:09:10 sdague | jroll: cool
17:09:14 jroll | maybe next week, we're freezing this week

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427248
Committed: https://git.openstack.org/cgit/openstack-dev/grenade/commit/?id=99c8c9e998ea9a95845958fd6b1f8dbae9fadef1
Submitter: Jenkins
Branch: master

commit 99c8c9e998ea9a95845958fd6b1f8dbae9fadef1
Author: Vasyl Saienko <email address hidden>
Date: Tue Jan 31 17:16:36 2017 +0200

    Do not wait for nova-compute when ironic is enabled

    Ironic should be upgraded before nova according to requirements [0]
    using reverse order will lead to nova-compute start failure.
    This patch ensures that we do not check nova-compute status in the
    nova upgarde phase, when ironic is enabled, as it is expected behaviour.
    Ironic will restart nova-compute later.

    [0] http://docs.openstack.org/developer/ironic/deploy/upgrade-guide.html#general-upgrades-all-versions

    Change-Id: I48dc7649eedb6c6d6753da8ba66fcfccb09acd58
    Closes-Bug: #1660581
    Related-Bug: #1660646

[Vasyl Saienko (vsaienko)]

This bug also affect all networking-* drivers (neutron extensions). They should be upgraded before neutron. We need to find a permanent fix for this which looks like require building dependency graph between services.