build: jenkins: image "cadvisor" failed to compile (intermittent)

[Davlet Panech]

Brief Description
-----------------
Image "cadvisor" fails to compile causing Jenkins' daily build failures. The downloaded file's checksum doesn't match expectations (https://sourceforge.net/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz). This was observed once and seems to have been caused by an HTTP download hiccup. Suggest to add a retry to Dockerfile.

Severity
--------
Minor

Steps to Reproduce
------------------
Try to build cadvisor

Expected Behavior
------------------
Build succeeds

Actual Behavior
----------------
Build fails while downloading https://sourceforge.net/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz with a checksum error.

Reproducibility
---------------
Seen once

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
master/2023-09-06

Last Pass
---------
master/2023-09-05

Timestamp/Logs
--------------

From: https://mirror.starlingx.cengn.ca/mirror/starlingx/master/debian/monolithic/20230907T060000Z/logs/build-docker-images-stable.log.txt

[2023-09-07T13:03:35.297Z] Removing intermediate container 81f3437b49a7
[2023-09-07T13:03:35.297Z] ---> d7915ebb9bdc
[2023-09-07T13:03:35.297Z] Step 3/23 : RUN wget https://sourceforge.net/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz && echo "112bced9a67d565ff0ce6c2bb90452516d1183e5 libpfm-4.11.0.tar.gz" | sha1sum -c && tar -xzf libpfm-4.11.0.tar.gz && rm libpfm-4.11.0.tar.gz
[2023-09-07T13:03:35.297Z] ---> Running in cd08a9eb5ada
[2023-09-07T13:03:35.297Z] --2023-09-07 13:03:34-- https://sourceforge.net/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz
[2023-09-07T13:03:35.297Z] Resolving sourceforge.net (sourceforge.net)... 18.238.192.29, 18.238.192.76, 18.238.192.2, ...
[2023-09-07T13:03:35.297Z] Connecting to sourceforge.net (sourceforge.net)|18.238.192.29|:443... connected.
[2023-09-07T13:03:35.297Z] HTTP request sent, awaiting response... 302 Moved Temporarily
[2023-09-07T13:03:35.297Z] Location: https://sourceforge.net/#!/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz [following]
[2023-09-07T13:03:35.297Z] --2023-09-07 13:03:35-- https://sourceforge.net/
[2023-09-07T13:03:35.297Z] Reusing existing connection to sourceforge.net:443.
[2023-09-07T13:03:35.297Z] HTTP request sent, awaiting response... 200 OK
[2023-09-07T13:03:35.297Z] Length: 746 [text/html]
[2023-09-07T13:03:35.297Z] Saving to: 'libpfm-4.11.0.tar.gz'
[2023-09-07T13:03:35.297Z]
[2023-09-07T13:03:35.297Z] 0K 100% 242M=0s
[2023-09-07T13:03:35.297Z]
[2023-09-07T13:03:35.297Z] 2023-09-07 13:03:35 (242 MB/s) - 'libpfm-4.11.0.tar.gz' saved [746/746]
[2023-09-07T13:03:35.297Z]
[2023-09-07T13:03:35.297Z] libpfm-4.11.0.tar.gz: FAILED
[2023-09-07T13:03:35.297Z] sha1sum: WARNING: 1 of 1 computed checksums did NOT match
[2023-09-07T13:03:35.548Z] The command '/bin/sh -c wget https://sourceforge.net/projects/perfmon2/files/libpfm4/libpfm-4.11.0.tar.gz && echo "112bced9a67d565ff0ce6c2bb90452516d1183e5 libpfm-4.11.0.tar.gz" | sha1sum -c && tar -xzf libpfm-4.11.0.tar.gz && rm libpfm-4.11.0.tar.gz' returned a non-zero code: 1
[2023-09-07T13:03:35.548Z] cadvisor image build done
[2023-09-07T13:03:35.548Z] Error response from daemon: No such image: jenkins/cadvisor:master-debian-stable-build
[2023-09-07T13:03:35.548Z] Running: docker push starlingx/cadvisor:master-debian-stable-20230907T060000Z.0
[2023-09-07T13:03:35.798Z] An image does not exist locally with the tag: starlingx/cadvisor
[2023-09-07T13:03:35.798Z] The push refers to repository [docker.io/starlingx/cadvisor]
[2023-09-07T13:03:35.798Z] Command (docker) failed, attempt 1 of 3.

Test Activity
-------------
N/A

Workaround
----------
Try building again

[Davlet Panech (dpanech)]

It is evident from the logs the build script is attempting to push the image, even though "docker build" failed.

Actions:
(1) Add a retry to wget in cadvisor's Docker file
(2) Fix build-stx-images.sh to avoid a push if "docker build" failed

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/app-power-metrics/+/894225

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/app-power-metrics/+/894225
Committed: https://opendev.org/starlingx/app-power-metrics/commit/17ae181d29f3a20aa3cf4de984cc14e5d56dcea8
Submitter: "Zuul (22348)"
Branch: master

commit 17ae181d29f3a20aa3cf4de984cc14e5d56dcea8
Author: Davlet Panech <email address hidden>
Date: Thu Sep 7 16:09:43 2023 -0400

    cadvisor docker image: don't ignore build errors

    A recent Jenkins build failed because the custom build script for the
    cadvisor image falsely reported success, which prevented the master
    build script from retrying the build of this project.

    Solution: exit with non-zero if "docker build" fails

    TESTS
    ======================
    - Build image normally
    - Force build.sh to return non-zero and make sure the retry logic kicks
      in

    Closes-Bug: 2034733
    Change-Id: I12adbf41a8e64e1c8cec5cfde23fea1c454538f2
    Signed-off-by: Davlet Panech <email address hidden>