OpenStack upgrade ordering needs better guidance in Appendix B

[Corey Bryant]

This page in the charm deployment guide needs better guidance on upgrade order: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-upgrade-openstack.html#known-openstack-upgrade-issues

There is service-specific guidance at the following that should be accounted for: https://docs.openstack.org/operations-guide/ops-upgrades.html#service-specific-upgrade-instructions.

The service-specific guidance above should be evaluated to determine any ordering changes.

It says:
"In terms of the upgrade order, begin with ‘keystone’. After that, the rest of the charms can be upgraded in any order."

It also has nova-compute and neutron-gateway listed as a 3 in the "Upgrade order" table.

For example, if nova-compute (ovs agent) is upgraded before neutron-gateway, the client side ovo version could be upgraded before the server side, causing a failure similar to the following: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1859649

Note that upgrade ^ from 14.0.0 to 14.0.3 was an unattended upgrade where the operator upgraded packages on the machine and the charm wasn't involved, but it is an example of a network outage caused by upgrading in the wrong order.

[Drew Freiberger (afreiberger)]

Important to note that when upgrading services landed on the same metal (such as nova-compute and ceph-osd) that unattended upgrades are disabled until both services have completed their source/openstack-origin updates.

[Peter Matulis (petermatulis)]

Drew, it's in there.

https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-upgrade-openstack.html#disable-unattended-upgrades

[Peter Matulis (petermatulis)]

This bug description mentions charm upgrade order ("keystone first...") but this is really about OpenStack upgrade order. So far: neutron-gateway should be upgraded before nova-compute. Please add any other changes in this bug.

[Peter Matulis (petermatulis)]

Second addition:

ceph-mon should be upgraded before ceph-osd

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/725687

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/725687
Committed: https://git.openstack.org/cgit/openstack/charm-deployment-guide/commit/?id=d8defee531873d166c348c6acd5ea34fc2adfa1c
Submitter: Zuul
Branch: master

commit d8defee531873d166c348c6acd5ea34fc2adfa1c
Author: Peter Matulis <email address hidden>
Date: Tue May 5 17:51:22 2020 -0400

    Improve order of OpenStack upgrades

    1. neutron-gateway before nova-compute
    2. ceph-mon before ceph-osd

    Closes-Bug: #1859990

    Change-Id: I645155b2f51ffaeb388dc3f8ce9fd125d3759d15