Nova::Cell_v2::Simple_setup/Nova_cell_v2[cell0] fails on master promote

[John Trowbridge]

FS20 is failing in the promote pipeline for master during deploy. It looks like we have a regression in either nova or puppet-nova.

The error in the deploy logs is:

"Error: /Stage[main]/Nova::Cell_v2::Simple_setup/Nova_cell_v2[cell0]: Could not evaluate: Execution of '/usr/bin/nova-manage cell_v2 update_cell --cell_uuid 00000000-0000-0000-0000-000000000000 --transport-url none:/// --database_connection mysql+pymysql://nova:QWDqCAV6AmavmFxPPpxCJecd6@192.168.24.8/nova_cell0?read_default_group=tripleo&read_default_file=/etc/my.cnf.d/tripleo.cnf' returned 3: The specified transport_url and/or database_connection combination already exists for another cell with uuid 00000000-0000-0000-0000-000000000000.",

Full logs for the job at:

https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/12230cb

[John Trowbridge (trown)]

Logs of the last time this succeeded:

https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/5937f4f

which is built from:
https://github.com/openstack/nova/commit/3875eaae73eaf77186d6d3d0718e61310049336b

Logs of the first time this failed:
https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/de7cfba

which is built from:
https://github.com/openstack/nova/commit/6a1e0c295d0c4e4c5e2c142380922f8b716b1895

Likely this review is the root cause: https://review.openstack.org/518556

Still need further investigation to determine if this should be fixed in nova or puppet-nova.

[John Trowbridge (trown)]

Actually https://review.openstack.org/518556 is 100% the root cause. There was no exit code 3 from nova-manage before that commit.

Still need to determine where/how/why we are providing a transport_url or/and database_connection that are the same as another cell.

[Oliver Walsh (owalsh)]

That fix needs to omit the cell being updated from the cells that are inspected for conflicts.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/536546

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/536546
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0467b537bfcf0cb973eb74b49ff7207dafbdbdbc
Submitter: Zuul
Branch: master

commit 0467b537bfcf0cb973eb74b49ff7207dafbdbdbc
Author: Dan Smith <email address hidden>
Date: Mon Jan 22 11:57:53 2018 -0800

    Fix update_cell to ignore existing identical cells

    The recent cell duplication filter makes it impossible to do update_cell
    on an existing cell with identical details. If we're updating a cell by
    uuid and specify the exact same parameters, we shouldn't signal failure.

    Change-Id: I2faaec94444bc7d29fda1facd22e0da3bc6f01f9
    Closes-Bug: #1744780

[John Trowbridge (trown)]

Nova fix worked for TripleO:

https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/e58f823

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0b3 development milestone.