Unsequenced alembic migration files block the gate.

[Allison Randal]

The gate was substantially delayed today by a handful of alembic db migration changes that hit the gate at the same time, and failed with the following error in logs/devstacklog.txt:

Only a single head is supported. The script directory has multiple heads (due to branching), which must be resolved by manually editing the revision files to form a linear sequence. Run `alembic branches` to see the divergence(s).

The current blockage was cleared by sequencing the db migrations to resolve alembic's branching conflict. This bug is linked to an elastic-recheck fingerprint, to track any future recurrence of the problem.

[Allison Randal (allison)]

Logstash query is: http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJmaWxlbmFtZTpsb2dzKmRldnN0YWNrbG9nLnR4dCBBTkQgbWVzc2FnZTpcIk9ubHkgYSBzaW5nbGUgaGVhZCBpcyBzdXBwb3J0ZWQuIFRoZSBzY3JpcHQgZGlyZWN0b3J5IGhhcyBtdWx0aXBsZSBoZWFkcyAoZHVlIHRvIGJyYW5jaGluZyksIHdoaWNoIG11c3QgYmUgcmVzb2x2ZWQgYnkgbWFudWFsbHkgZWRpdGluZyB0aGUgcmV2aXNpb24gZmlsZXMgdG8gZm9ybSBhIGxpbmVhciBzZXF1ZW5jZS4gUnVuIGBhbGVtYmljIGJyYW5jaGVzYCB0byBzZWUgdGhlIGRpdmVyZ2VuY2UocykuXCIiLCJ0aW1lZnJhbWUiOiI0MzIwMCIsImdyYXBobW9kZSI6ImNvdW50Iiwib2Zmc2V0IjowLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDA1NDYyNzM5OSwibW9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIifQ==

[Octavian Ciuhandu (ociuhandu)]

We seem to be hitting the same problem on the Hyper-V CI quite often in the last day. Here is a sample devstack log output:

http://64.119.130.115/78575/3/console.log.gz

[Allison Randal (allison)]

The fingerprint was merged into elastic-recheck in https://review.openstack.org/#/c/78435/

We're currently clear of these failures in the gate, but have a very high hit rate in check.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/79377

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/79377
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=36d85f831ae8eb21383806261bfc4c3d53dd1929
Submitter: Jenkins
Branch: master

commit 36d85f831ae8eb21383806261bfc4c3d53dd1929
Author: Mark McClain <email address hidden>
Date: Fri Mar 7 15:07:43 2014 -0500

    add HEAD sentinel file that contains migration revision

    This change wraps the command to generate a migration with code to
    update a file called HEAD. The HEAD file will contain the revision ID
    of the head of the migraton timeline. Additionally, check migrations has
    been altered to verify the contents of this file against the timeline
    head.

    This file helps the OpenStack gate detect potential migration branches
    without running alembic via git merge conflicts.

    Closes-Bug:1288427
    Change-Id: If382c57baea061753d3f4fcd6faec1a31fbfb7ed

[Sean Dague (sdague)]

Dropped from ER tracking.