MultipleCreateTestJSON.test_multiple_create intermittently fails for cells v1 due to server name check change

[Matt Riedemann]

http://logs.openstack.org/75/486475/38/gate/nova-cells-v1/2c2a566/job-output.txt.gz#_2018-06-13_13_26_17_075329

2018-06-13 13:26:17.110798 | primary | Captured traceback:
2018-06-13 13:26:17.110851 | primary | ~~~~~~~~~~~~~~~~~~~
2018-06-13 13:26:17.110932 | primary | Traceback (most recent call last):
2018-06-13 13:26:17.111103 | primary | File "tempest/api/compute/servers/test_multiple_create.py", line 43, in test_multiple_create
2018-06-13 13:26:17.111218 | primary | self.assertEqual(set(['VM-1', 'VM-2']), server_names)
2018-06-13 13:26:17.111435 | primary | File "/opt/stack/new/tempest/.tox/tempest/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2018-06-13 13:26:17.111542 | primary | self.assertThat(observed, matcher, message)
2018-06-13 13:26:17.111758 | primary | File "/opt/stack/new/tempest/.tox/tempest/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2018-06-13 13:26:17.111830 | primary | raise mismatch_error
2018-06-13 13:26:17.111992 | primary | testtools.matchers._impl.MismatchError: set(['VM-2', 'VM-1']) != set([u'VM'])

This started failing after this change was made:

https://review.openstack.org/#/c/569199/

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22self.assertEqual(set(%5B'VM-1'%2C%20'VM-2'%5D)%2C%20server_names)%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d

8 hits in 24 hours

[Matt Riedemann (mriedem)]

This is likely the reason for the failure, cells v1 doesn't honor that multi-create instance name contract:

https://github.com/openstack/nova/blob/dd87118acc2bf57f235cb287ed0ee12736263ecb/nova/compute/api.py#L1419

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/575132

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/575132
Reason: Nevermind, will just blacklist the test for cells v1.

[Matt Riedemann (mriedem)]

Restored the tempest revert: https://review.openstack.org/#/c/575132/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/575132
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=1604a3854462df3386a5f9861ac5913e1ce97b37
Submitter: Zuul
Branch: master

commit 1604a3854462df3386a5f9861ac5913e1ce97b37
Author: Matt Riedemann <email address hidden>
Date: Wed Jun 13 14:39:11 2018 +0000

    Revert "Add server name check for multiple creations"

    This reverts commit b59f458f2823bfcfd8a530003d34e912110c2f3a.

    This breaks the nova-cells-v1 job since the cells v1 API
    does not honor the same multi-create instance name format.

    Furthermore, prior to Rocky [1], the
    multi_instance_display_name_template config option could be used to
    customize the display name for multiple servers in a single request.
    Since Tempest is branchless, this means deployments older than Rocky
    could now fail this test if they are customizing that option.

    Really, the change to assert the server names shouldn't be possible
    until the nova pike series enters extended maintenance mode since that
    would be the last branch that supports the
    multi_instance_display_name_template config option.

    [1] https://review.openstack.org/499612/

    Change-Id: Ia9273be6bf9fd55822dfe46f56503f70953c3ff3
    Closes-Bug: #1776684

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tempest 19.0.0 release.