Sync between repo containers not working

[Andreas Scheuring]

The repo container sync is not working as the rsync command is copying some contents into the wrong directory [1].

The target of the sync is always "/var/www/repo". The source is determined along some variables. But this destination is not correct for most of them:

pools directory
---------------

rsync-source: /var/www/repo/pools/ubuntu-16.04-s390x
rsync-target: /var/www/repo

-> rsync creates the directory: /var/www/repo/ubuntu-16.04-s390x

-> But it should be: /var/www/repo/pools/ubuntu-16.04-s390x

This issue causes the wheel build on the repo master to fail, as symlinks in the "links" and "os-releases" directories point to not existing packages (to /var/www/repo/pools/ubuntu-16.04-s390x/package)

venv directory
--------------

rsync-source: /var/www/repo/venvs/15.1.2/ubuntu-16.04-s390x
rsync-target: /var/www/repo

-> rsync creates the directory: /var/www/repo/ubuntu-16.04-s390x

-> But it should be: /var/www/repo/venvs/15.1.2/ubuntu-16.04-s390x

os-releases directory
---------------------

rsync-source: /var/www/repo/os-releases/15.1.2/ubuntu-16.04-s390x
rsync-target: /var/www/repo

-> rsync creates the directory: /var/www/repo/ubuntu-16.04-s390x

-> But it should be: /var/www/repo/os-releases/15.1.2/ubuntu-16.04-s390x

[1] https://github.com/openstack/openstack-ansible-repo_build/blob/master/tasks/repo_package_sync.yml

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/484655

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/484655
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=e4c5ab4c6ebf33267d4a4cdf7ecff35465899167
Submitter: Jenkins
Branch: master

commit e4c5ab4c6ebf33267d4a4cdf7ecff35465899167
Author: Andreas Scheuring <email address hidden>
Date: Tue Jul 18 09:06:06 2017 +0200

    Sync repos to correct location on repo master

    The repo container sync task synchronises the following directory to a
    wrong location.

    pools
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/pools/{{ repo_build_os_distro_version }}

    As the links and other directories are synced successfuly, they point
    to invalid locations which will cause the repo build on the existing
    repo master to fail.

    venvs
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/venvs/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    os-releases
    -----------
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/os-releases/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    Change-Id: I9e5902ec91afbb2805663eb5678fc9b2b31f23b2
    Closes-Bug: #1704936

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/485631

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/485632

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/485631
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=a23424eb088715a690b8238ad7e36963fcb39ab2
Submitter: Jenkins
Branch: stable/ocata

commit a23424eb088715a690b8238ad7e36963fcb39ab2
Author: Andreas Scheuring <email address hidden>
Date: Tue Jul 18 09:06:06 2017 +0200

    Sync repos to correct location on repo master

    The repo container sync task synchronises the following directory to a
    wrong location.

    pools
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/pools/{{ repo_build_os_distro_version }}

    As the links and other directories are synced successfuly, they point
    to invalid locations which will cause the repo build on the existing
    repo master to fail.

    venvs
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/venvs/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    os-releases
    -----------
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/os-releases/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    Change-Id: I9e5902ec91afbb2805663eb5678fc9b2b31f23b2
    Closes-Bug: #1704936
    (cherry picked from commit e4c5ab4c6ebf33267d4a4cdf7ecff35465899167)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/485632
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=1c02e7906898c85a5469f4b053895e22e3cfdcac
Submitter: Jenkins
Branch: stable/newton

commit 1c02e7906898c85a5469f4b053895e22e3cfdcac
Author: Andreas Scheuring <email address hidden>
Date: Tue Jul 18 09:06:06 2017 +0200

    Sync repos to correct location on repo master

    The repo container sync task synchronises the following directory to a
    wrong location.

    pools
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/pools/{{ repo_build_os_distro_version }}

    As the links and other directories are synced successfuly, they point
    to invalid locations which will cause the repo build on the existing
    repo master to fail.

    venvs
    -----
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/venvs/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    os-releases
    -----------
    Wrongly created dir on target: /var/www/repo/{{ repo_build_os_distro_version }}
    Fixed dir on target: /var/www/repo/os-releases/{{ repo_build_release_tag }}/{{ repo_build_os_distro_version }}

    Change-Id: I9e5902ec91afbb2805663eb5678fc9b2b31f23b2
    Closes-Bug: #1704936
    (cherry picked from commit e4c5ab4c6ebf33267d4a4cdf7ecff35465899167)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 16.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/489188

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 15.1.8 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 14.2.8 release.