periodic OVB train centos 7 failing overcloud configuration (config download) Invalid cross-device link

[Marios Andreou]

At [1][2][3] (and likely more - all are from the latest centos-7 train buildset [4]) the ovb jobs are failing post deployment with trace like:

            * 2020-08-17 13:43:15 | The action raised an exception [action_ex_id=f3a29bfd-8d09Waiting for messages on queue 'tripleo' with no timeout.
    2020-08-17 13:43:18 | Host 10.0.0.5 not found in /home/zuul/.ssh/known_hosts
    2020-08-17 13:43:21 | Overcloud configuration failed.
            * 2020-08-17 13:43:21 | -4892-b7cc-e179e5f91021, msg='[Errno 18] Invalid cross-device link', action_cls='<class 'mistral.actions.action_factory.AnsibleGenerateInventoryAction'>', attributes='{}', params='{u'work_dir': u'/var/lib/mistral/overcloud', u'ansible_python_interpreter': None, u'ansible_ssh_user': u'tripleo-admin', u'undercloud_key_file': u'/var/lib/mistral/.ssh/tripleo-admin-rsa', u'plan_name': u'overcloud', u'ssh_network': u'ctlplane'}']

[1] https://logserver.rdoproject.org/openstack-periodic-integration-stable2-centos7/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-train/fa33575/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[2] https://logserver.rdoproject.org/openstack-periodic-integration-stable2-centos7/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-train-upload/8a9c52f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[3] https://logserver.rdoproject.org/openstack-periodic-integration-stable2-centos7/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-train/ff59195/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
[4] https://review.rdoproject.org/zuul/buildset/435b17a1c46d44b19362bf0c36d2f25a

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/746724

[chandan kumar (chkumar246)]

https://review.opendev.org/746724 needs to be cherry-picked for stable/ussuri and stable/train

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/746841

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/746842

[chandan kumar (chkumar246)]

Added the cherry-pick here https://review.opendev.org/#/q/fbd575cbe88400bad1ce4d37b914d7ae8a43e15c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746842
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=662b0acd91266d28bf3b0b7a4c68ba919182c56a
Submitter: Zuul
Branch: stable/train

commit 662b0acd91266d28bf3b0b7a4c68ba919182c56a
Author: Oliver Walsh <email address hidden>
Date: Tue Aug 18 14:31:18 2020 +0100

    Ensure atomic inventory file rename runs on the same mountpoint

    Linux rename system call is only supported on the same mountpoint [1].
    /tmp is often a tmpfs mount, so instead generate the temp inventory file in
    the same directory as the target inventory file.

    [1] See EXDEV in https://man7.org/linux/man-pages/man2/rename.2.html

    Change-Id: Id306243352f310f9054506577aa9bbce3790a7e5
    Closes-bug: #1892008
    (cherry picked from commit fbd575cbe88400bad1ce4d37b914d7ae8a43e15c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746724
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=fbd575cbe88400bad1ce4d37b914d7ae8a43e15c
Submitter: Zuul
Branch: master

commit fbd575cbe88400bad1ce4d37b914d7ae8a43e15c
Author: Oliver Walsh <email address hidden>
Date: Tue Aug 18 14:31:18 2020 +0100

    Ensure atomic inventory file rename runs on the same mountpoint

    Linux rename system call is only supported on the same mountpoint [1].
    /tmp is often a tmpfs mount, so instead generate the temp inventory file in
    the same directory as the target inventory file.

    [1] See EXDEV in https://man7.org/linux/man-pages/man2/rename.2.html

    Change-Id: Id306243352f310f9054506577aa9bbce3790a7e5
    Closes-bug: #1892008

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/746841
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=a036159701920d1c8dfe5bfe489c5a754eeadca5
Submitter: Zuul
Branch: stable/ussuri

commit a036159701920d1c8dfe5bfe489c5a754eeadca5
Author: Oliver Walsh <email address hidden>
Date: Tue Aug 18 14:31:18 2020 +0100

    Ensure atomic inventory file rename runs on the same mountpoint

    Linux rename system call is only supported on the same mountpoint [1].
    /tmp is often a tmpfs mount, so instead generate the temp inventory file in
    the same directory as the target inventory file.

    [1] See EXDEV in https://man7.org/linux/man-pages/man2/rename.2.html

    Change-Id: Id306243352f310f9054506577aa9bbce3790a7e5
    Closes-bug: #1892008
    (cherry picked from commit fbd575cbe88400bad1ce4d37b914d7ae8a43e15c)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/761530

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/763170

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 11.5.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common stein-eol release.