Neutron db migration issue in master

[Matt Thompson]

The neutron db-related changes in https://review.openstack.org/#/c/199126/ need reworking as we end up writing junk to {{ neutron_system_home_folder }}/neutron-revision since the output we get from neutron-db-manage history | head -1 is not what we expect. Additionally, we only run neutron_db_setup.yml on groups['neutron_server'][0] however we're writing version history to that node and should subsequent runs get a different node for groups['neutron_server'][0] the migration may not happen as expected.

[Matt Thompson (mattt416)]

http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html#applying-database-migration-rules

[Kevin Carter (kevin-carter)]

@ matt to clarify, "on https://review.openstack.org/#/c/199126/23/playbooks/roles/os_neutron/tasks/neutron_db_setup.yml" line 60 we delegate to all nodes within the "neutron_all" group and write out the revision history. So a change / addition to neutron nodes should not be effected by a revision not existing on a node as it should always be on all nodes not just the first one.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/215584
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=092f606adcd7120195e9a71afe80893a9d512f03
Submitter: Jenkins
Branch: master

commit 092f606adcd7120195e9a71afe80893a9d512f03
Author: Matt Thompson <email address hidden>
Date: Fri Aug 21 12:04:30 2015 +0100

    Update how neutron migrations are handled

    We're seeing some neutron db sync failures in master branch (liberty).
    It looks like there may be a few issues with the way we're handling
    migrations:

    1. We're not capturing the correct information in the 'Check last DB
       revision' task as the first line returned by 'neutron-db-manage
       history' is a header (this also means that the 'Inspect on disk
       neutron DB revision' task will never do what we intend).
    2. We delegate 'Check last DB revision' to 'neutron_all' group, but in
       an ideal world this is probably only necessary on 'neutron_server'.
    3. When stopping neutron server to run 'Perform a Neutron DB Upgrade',
       we're only stopping it on groups['neutron_server'][0], while all
       other neutron server containers are up servicing requests.
    4. Perform a db stamp, which doesn't appear to be necessary.

    This change makes the following changes:

    1. Bumps neutron SHA to include 43c00a9, which introduces new
       --expand / --contract upgrade options (otherwise, you have to
       specify liberty_expand@head / liberty_contract@head which may no
       longer work when neutron gets bumped in the future)
    2. Checks if migrations have previously run, and if not runs a
       'neutron-db-manage upgrade heads'
    3. If migrations have previously run:
       a) it runs an online migration against expand alembic branch using
          'neutron-db-manage upgrade --expand'.
       b) it stops all neutron-server services
       c) it runs an offline migration against contract alembic branch
          using 'neutron-db-manage upgrade --contract'
       d) it starts all neutron-server instances
    4. It removes the temporary pin introduced in
       https://review.openstack.org/218572 as the SHA bump includes the
       upstream fix https://review.openstack.org/218723

    TODO: Currently, we upgrade expand and contract branches (shutting down
          neutron-server in the proceses), even if there are no pending
          migrations. We need to find a clean way to check what migration
          we are on and compare to alembic HEADS file to see if we're up to
          date. Unfortunately, 'neutron-db-manage current' doesn't indicate
          which migration is in which alembic branch, so you would have to
          further grep for each migration in the neutron migrations code to
          determine the branch.

    NOTE: Liberty introduces the split alembic branches for online/offline
          migrations, see [1] for more information.

    [1] http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html

    Change-Id: I1176b5fe12cad1ee732486ae179e76deea5623e1
    Closes-Bug: #1486593

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/219759
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=1c22ffa986a5270b444ea7af55231b97cfff1ba3
Submitter: Jenkins
Branch: master

commit 1c22ffa986a5270b444ea7af55231b97cfff1ba3
Author: Matt Thompson <email address hidden>
Date: Wed Sep 2 16:55:20 2015 +0100

    Add neutron_migrations_facts module

    This change does the following:

    1. Adds neutron_migrations_facts module, which creates a fact
       with the state of each neutron alembic migration branch
    2. Removes var neutron_db_revision since this is no longer used
    3. Removes the migration check task since this is now handled by the
       neutron_migrations_facts module
    4. Updates roles/os_neutron/tasks/main.yml so the init scripts are
       dropped before we handle db syncing, which means we can stop the
       neutron-server on a greenfield deploy

    DocImpact
    Closes-Bug: #1486593
    Change-Id: Ie0c5cb5abff61d11b7a9278de6771fa008b69218