populate_downloads.sh doesn't clean/backup old content

Bug #1908297 reported by Scott Little
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Scott Little

Bug Description

Brief Description
-----------------
cgcs-root/stx/downloads accumulates a lot of links to tarballs that are no longer relevant to the build. populate_downloads.sh lacks a cleanup mechanism.

Severity
--------
minor

Steps to Reproduce
------------------
populate_downloads.sh
repo sync
populate_downloads.sh

Expected Behavior
------------------
cgcs-root/stx/downloads contains only links to current content

Actual Behavior
----------------
cgcs-root/stx/downloads retains links to old contant

Reproducibility
---------------
100% ... provided there has been an lst file change that obsoletes an old tarball.

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
2020-12-14

Last Pass
---------
N/A

Timestamp/Logs
--------------
ls -al $MY_REPO/stx/downloads/helm-charts-certmanager*
lrwxrwxrwx. 1 jenkins cgts 116 Sep 19 02:08 helm-charts-certmanager-1d6ecc9cf8d841782acb5f3d3c28467c24c5fd18.tar.gz -> /import/mirrors/CentOS/stx/CentOS//downloads/helm-charts-certmanager-1d6ecc9cf8d841782acb5f3d3c28467c24c5fd18.tar.gz
lrwxrwxrwx. 1 jenkins cgts 124 Apr 25 2020 helm-charts-certmanager-49e1a7a51c71307718d29c4275bf3916906011c2.tar.gz -> /import/mirrors/CentOS/stx-r1/CentOS/pike//downloads/helm-charts-certmanager-49e1a7a51c71307718d29c4275bf3916906011c2.tar.gz
lrwxrwxrwx. 1 jenkins cgts 124 Apr 15 2020 helm-charts-certmanager-6da95758a4751b20cf85b29a3252e993449660eb.tar.gz -> /import/mirrors/CentOS/stx-r1/CentOS/pike//downloads/helm-charts-certmanager-6da95758a4751b20cf85b29a3252e993449660eb.tar.gz
lrwxrwxrwx. 1 jenkins cgts 124 Apr 21 2020 helm-charts-certmanager-8bc03dc303d0e4267e44dab0a68607f35786919e.tar.gz -> /import/mirrors/CentOS/stx-r1/CentOS/pike//downloads/helm-charts-certmanager-8bc03dc303d0e4267e44dab0a68607f35786919e.tar.gz

Test Activity
-------------
Build

Workaround
----------
N/A

Ghada Khalil (gkhalil)
tags: added: stx.build
Revision history for this message
Scott Little (slittle1) wrote :
Changed in starlingx:
status: New → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.5.0
tags: removed: stx.5.0
Changed in starlingx:
assignee: nobody → Scott Little (slittle1)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to root (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/root/+/786129

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tools (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/tools/+/786144

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to root (f/centos8)

Reviewed: https://review.opendev.org/c/starlingx/root/+/786129
Committed: https://opendev.org/starlingx/root/commit/9453b4eb179f345084aeebd7973e921e536cc874
Submitter: "Zuul (22348)"
Branch: f/centos8

commit 9453b4eb179f345084aeebd7973e921e536cc874
Author: Scott Little <email address hidden>
Date: Thu Dec 17 14:02:40 2020 -0500

    Ignore downloads-backup* directories

    Tell git to ignore the stx/download-backup* directories
    that are now produced by populate_downloads.sh.

    Partial-bug: 1908297
    Signed-off-by: Scott Little <email address hidden>
    (cherry picked from commit a64a3078aa1ee2cb411f786e9b9c3526b0c7f36d)
    Change-Id: I7afcba760acd0ccfc6ffad67483515e944ab5b9a

tags: added: in-f-centos8
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tools (f/centos8)

Reviewed: https://review.opendev.org/c/starlingx/tools/+/786144
Committed: https://opendev.org/starlingx/tools/commit/e7ee5e2102c999cc763c3e70fbc753fbe2f8d6a7
Submitter: "Zuul (22348)"
Branch: f/centos8

commit e7ee5e2102c999cc763c3e70fbc753fbe2f8d6a7
Author: Scott Little <email address hidden>
Date: Tue Dec 15 14:17:34 2020 -0500

    tidy stx/downloads directory

    $MY_REPO/stx/downloads contains links to tarballs and other files
    relevant to the build. It is populated by populate_downloads.sh.
    On subsequent runs, new links might be added, but old links are
    not cleaned up.

    Adopt the convention used by generate-centos-repo.sh of moving
    the old content under a timestamp backup directory, leaving
    an empty directory to re-populate.

    Closes-bug: 1908297
    Signed-off-by: Scott Little <email address hidden>
    (cherry picked from commit 1fd6f3f636ec7b0780815120f10e1beb545bc07f)
    Change-Id: Ibed5e57054628cf147f0873c3008ef1e2feed924

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tools (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/tools/+/792229

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to root (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/root/+/792232

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to root (f/centos8)
Download full text (24.5 KiB)

Reviewed: https://review.opendev.org/c/starlingx/root/+/792232
Committed: https://opendev.org/starlingx/root/commit/7cb55d28111dbf7458ed3f01a5eda5c4dfdb124b
Submitter: "Zuul (22348)"
Branch: f/centos8

commit f457bd15b9ce0be512ec96abcb9b858b0bab2ea8
Author: Scott Little <email address hidden>
Date: Fri Apr 16 01:39:17 2021 -0400

    Improved branching tools

    create_branches_and_tags.sh:
    - Update the .gitreview files in branched git repos.
    - When updating a manifest, add the ability to update and
      use the default revision field.
    - Create two levels of manifest lockdown, soft and hard.
      Soft lockdown only sets sha revisions on unbranched projects
      that lack a revision, or set the revision to master.
      Hard lockdown applies to all unbranched projects.

    push_branches_tags.sh:
    - opendev no longer accepts 'git push' for the delivery of
      new branches with updates. Instead we must now
      use separate commands to deliver the tag, the branch,
      and any updates.

    Closes-Bug: 1924762
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I6d669ddc80cc9b3cb9e72d65a64589dbccf43ae3

commit 0babd33b6d851dd11492c26e92c8a6ac2c2557de
Author: Rafael Jardim <email address hidden>
Date: Wed May 12 11:25:54 2021 -0300

    Update stx-platformclients tag to stx.5.0-v1.4.3

    This commit updates the image with the updated clients.

    Test:
    Some normal commands
    Commands related with https dcmanager that wasn't working
    System application-upload that wasn't working when executed
    from remote cli

    Closes-Bug: 1928233
    Closes-Bug: 1928231
    Signed-off-by: Rafael Jardim <email address hidden>
    Change-Id: I8a0d12f699336a4412be5ff3c73cfb8d59038780

commit a163d7723e659e89c37ec933d1b0f9aa638a6a73
Author: Cole Walker <email address hidden>
Date: Wed May 5 09:39:17 2021 -0400

    Update image tag for notificationservice-base

    Update image tag to stx.5.0-v1.0.4 for notificationservice-base

    Closes-Bug: 1924201
    Closes-Bug: 1924197

    Signed-off-by: Cole Walker <email address hidden>
    Change-Id: Id863c4f154cc0b39e30ee8986fc07f9856a22826

commit 84c45f5e3a241237887af9db89d8c0aa1f8923e0
Author: Charles Short <email address hidden>
Date: Sun May 2 12:04:04 2021 -0400

    Fix wheels tarball generation

    In d7c5a54ab94bce6635b83d91a807d28f97836a81, django was dropped in favor
    of rfc3986. However the wheel was not added to the build-wheels
    generation which breaks the docker images. Also add the migrate wheel
    since it was mising as well. Add the required wheels in order to
    build the docker image properly.

    Test:
    - Build new centos-stable-wheels tarball
    - Build stx-keystone-api-proxy container

    Closes-Bug: 1926795

    Signed-off-by: Charles Short <email address hidden>
    Change-Id: Ib6f0abfdcc82ca14f92ebc5b45fe8df961e804ee

commit 062353b6de7d4b0e017203c3e6086891bd6b9213
Author: Cole Walker <email address hidden>
Date: Mon May 3 09:48:54 2021 -0400

    Update image tag for notificationclient-base...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tools (f/centos8)

Change abandoned by "Chuck Short <email address hidden>" on branch: f/centos8
Review: https://review.opendev.org/c/starlingx/tools/+/792229
Reason: Updated merge coming

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tools (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/tools/+/793627

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tools (f/centos8)
Download full text (30.4 KiB)

Reviewed: https://review.opendev.org/c/starlingx/tools/+/793627
Committed: https://opendev.org/starlingx/tools/commit/d701c6f896dfe440566cc942e3dd71be1f19ae5d
Submitter: "Zuul (22348)"
Branch: f/centos8

commit 7b5f3a45e663866a3c0ca3ca86eb3c92bc7f0210
Author: Scott Little <email address hidden>
Date: Wed May 5 09:56:33 2021 -0400

    fix bad flockflock url pt 2

    A stray '}' character found it's way into my prior update
    titled 'fix bad flockflock url' after testing. The result was
    the following error

    sed: -e expression #1, char 15: unexpected `}'

    This removes the unwanted '}', restoring the prior update
    to its intended form.

    Closes-bug: 1926987
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I48f4721ccaf121679916b01747243deedf5836cd

commit ac05493480f6df6f31d071d29380c1b4f35b70a9
Author: Scott Little <email address hidden>
Date: Tue May 4 12:42:36 2021 -0400

    fix git-review within docker build environment

    'tb create' fails to create a build environment since
    upstream git-review was updated of Apr 26.

    Fix is to install/update pbr ahead of git-review.

    Also, to reduce the likelyhood of this recurring, lock
    down specific versions of the pypi supplied tools we
    know to work.

    Closes-bug: 1927137
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: Ib9fe6fd33de4d637f254ac421cc0427ee6131b65

commit b96ebc83d859a4a7802a462504817ecec6182a7b
Author: Scott Little <email address hidden>
Date: Mon May 3 13:16:53 2021 -0400

    fix bad flockflock url

    download_mirror.sh fails due to a bad path containing
    ‘stx-tools/centos-mirror-tools/config/centos/flockflock’

    The path is constructed, and the trigger is when an EOL is missing
    from a centos_build_layer.cfg file, causing 'cat' to merge the last
    line of the offending file with the first line of the next file.

    Switch 'cat' to 'grep', which will always ensure an EOL is present.
    Along the way, we can filter out empty lines and comments.

    Closes-bug: 1926987
    Signed-off-by: Scott Little <email address hidden>
    Change-Id: I2404b3415f0f3e2f395c2bcb7a527aa01a488f61

commit 4c3ee114bcbff710c2049626044dd1ddc756cbd9
Author: Joe Slater <email address hidden>
Date: Tue Apr 27 18:50:53 2021 -0400

    screen: fix CVE-2021-26937 segfault

    Advance to screen-4.1.0-0.27.20120314git3c2946.el7_9.x86_64.rpm.

    Closes-bug: 1926372
    Change-Id: I41834e7b1e16153b0632751f59f7ac9f503389da
    Signed-off-by: Joe Slater <email address hidden>

commit e31e0dda7a4c09143d41cd518ab97ea6112d7fb5
Author: Li Zhou <email address hidden>
Date: Tue Apr 13 04:53:50 2021 -0400

    systemd: Upgrade to version 219-78.el7_9.3

    Refer the lst entries to the new version.

    Partial-Bug: #1924691
    Signed-off-by: Li Zhou <email address hidden>
    Change-Id: I557eff6a47f341cc67de02fd59024b28bb6cac84

commit 26db2859dd3a5c060c337b886fd16c4d2d9f93af
Author: Scott Little <email address hidden>
Date: Mon Apr 12 11:21:31 2021 -0400

    Replace basearch references in y...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.