heat_integrationtests..test_stack_tag is racy

[Angus Salkeld]

When convergence is enabled and you run the functional tests this test fails often.
But I added some LOG messages to the the functional test and the failure disappeared.

It seems like sometimes it runs too fast and gets stuck waiting for a state that is not going to happen.

Example:
http://logs.openstack.org/73/198273/9/check/gate-heat-dsvm-functional-convg-mysql/315b93c/console.html.gz#_2015-08-01_05_34_09_304

[Angus Salkeld (asalkeld)]

Seems we are hitting the case of the same update time:
https://github.com/openstack/heat/blob/master/heat_integrationtests/common/test.py#L253-L259

[Angus Salkeld (asalkeld)]

The update is happing too fast and the update time is within the same second. Note (not convergence specific).

[Angus Salkeld (asalkeld)]

I am going to give this a go:

http://stackoverflow.com/questions/29711102/sqlalchemy-mysql-millisecond-or-microsecond-precision

Column(DATETIME(fsp=6))

fsp = fractional seconds parameter

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/208276

[Angus Salkeld (asalkeld)]

I can confirm that with mysql the fractional parts are now getting stored and this functional test always passes.
(it was easy to hit previously)

It *seems* like postgres has fractional parts by default, but I have not confirmed this.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/208276
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=3b2d14ad993c1e605f0242f99eb2224bbc10c353
Submitter: Jenkins
Branch: master

commit 3b2d14ad993c1e605f0242f99eb2224bbc10c353
Author: Angus Salkeld <email address hidden>
Date: Tue Aug 4 09:38:47 2015 +1000

    Make sure all "updated_at" fields store fractional seconds

    If update happen really fast (subsecond) we can't distinguish
    one update from the next, so when using mysql turn on
    fractional second DATETIME.

    Just to be consistent, change the created_at fields at the same
    time.

    Change-Id: I949fedbfda58348b051072bba98ed59897824f6e
    Closes-bug: #1480739

[Steve Baker (steve-stevebaker)]

This will need to be re-proposed in a way which doesn't trigger bug 1488779

[Angus Salkeld (asalkeld)]

One option is to use the current_traversal (instead of update time) to make sure the update has happened.

[Steve Baker (steve-stevebaker)]

Another option would be to trigger a replacement on a RandomString resource and assert that the output has changed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/222947

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/222947
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=9e13a0c3bf752b6fa1d50d0b00d55ba745050022
Submitter: Jenkins
Branch: master

commit 9e13a0c3bf752b6fa1d50d0b00d55ba745050022
Author: Angus Salkeld <email address hidden>
Date: Mon Sep 14 11:12:20 2015 +1000

    Make the stack tags template non-empty

    This is to work around the update time not having fractional seconds.
    Closes-bug: #1480739

    Change-Id: Id5f9d318420e37525626c5a770c6126d8369b5e7