ffu: Missing Nova DB sync between O and P

[Lee Yarwood]

Description
===========
During a fast forward upgrade from N to Q there is no attempt to sync the main Nova DB during the O to P FFU steps. This is due to the only db sync command within the nova-api template being listed within an Ansible block that is only executed during the N to O set of FFU steps.

This results in the online DB migrations for O to P failing silently. This is however missed as the following major upgrade between P to Q, not using FFU steps, then calls for a db sync and additional online migration.

Steps to reproduce
==================
* FFU from N to Q
* Review nova-manage.log

Expected result
===============
Online migrations complete between O and P.

Actual result
=============
Online migrations fail between O and P.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

   N to Q FFU.

2. Which storage type did you use?
   (For example: Ceph, LVM, ...)
   What's the version of that?

   N/A

3. Which networking type did you use?
   (For example: Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

From the downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1585765 :

2018-06-04 15:52:26.307 424237 ERROR oslo_db.sqlalchemy.exc_filters [req-45a7f7d2-068f-4adc-b27e-f44a64b76d55 - - - - -] DBAPIError exception wrapped from (pymysql.err.InternalError) (1054, u"Unknown column 'services.uuid' in 'field list'") [SQL: u'SELECT services.created_at AS services_created_at, services.updated_at AS services_updated_at, services.deleted_at AS services_deleted_at, services.deleted AS services_deleted, services.id AS services_id, services.uuid AS services_uuid, services.host AS services_host, services.`binary` AS services_binary, services.topic AS services_topic, services.report_count AS services_report_count, services.disabled AS services_disabled, services.disabled_reason AS services_disabled_reason, services.last_seen_up AS services_last_seen_up, services.forced_down AS services_forced_down, services.version AS services_version \nFROM services \nWHERE services.deleted = %(deleted_1)s AND services.uuid IS NULL \n LIMIT %(param_1)s'] [parameters: {u'param_1': 50, u'deleted_1': 0}]: InternalError: (1054, u"Unknown column 'services.uuid' in 'field list'")

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/573740

[Matt Young (halcyondude)]

please update importance

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/573740
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=65b758330862df75f7ed2ddceb7debb89ce792a6
Submitter: Zuul
Branch: master

commit 65b758330862df75f7ed2ddceb7debb89ce792a6
Author: Lee Yarwood <email address hidden>
Date: Fri Jun 8 16:04:14 2018 +0100

    ffu: Add missing nova-manage db sync for Pike upgrade

    This would result in the failure of the online_data_migrations command
    during the Ocata to Pike upgrade. The following Pike to Queens upgrade
    would hide this failure by running both the sync and migrations again.

    Change-Id: I51f40254acee435ed4e60c0e97b5ced86fd67fc2
    Closes-bug: #1775868

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/578502

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/578502
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b544407dfbc5ab15397df0e8984803708c39caae
Submitter: Zuul
Branch: stable/queens

commit b544407dfbc5ab15397df0e8984803708c39caae
Author: Lee Yarwood <email address hidden>
Date: Fri Jun 8 16:04:14 2018 +0100

    ffu: Add missing nova-manage db sync for Pike upgrade

    This would result in the failure of the online_data_migrations command
    during the Ocata to Pike upgrade. The following Pike to Queens upgrade
    would hide this failure by running both the sync and migrations again.

    Change-Id: I51f40254acee435ed4e60c0e97b5ced86fd67fc2
    Closes-bug: #1775868
    (cherry picked from commit 65b758330862df75f7ed2ddceb7debb89ce792a6)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b4 development milestone.