build-tools: debian: downloader sometimes leaves corrupted files behind

[Davlet Panech]

Brief Description
-----------------
The script "downloader.py" sometimes leaves corrupted files behind. We observed a 0-length .tar.gz file, which caused a package build to fail. It looks like this is caused by "wget" when a connection to the server breaks in the middle of download, and yet "wget" reports success (ie returns 0 status).

Severity
--------
Major

Steps to Reproduce
------------------
Delete all downloads, then run the downlader.py script, then build the packages.

Expected Behavior
------------------
Packages build succeeds.

Actual Behavior
----------------
Sometimes package build fails due to a corrupted download.

Reproducibility
---------------
Seen once

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
master/2022-08-16

Last Pass
---------
Unknown

Timestamp/Logs
--------------

2022-08-17 00:44:15,749 - debrepack - INFO: [ Run - "tar -tzf /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/openstack-helm-infra-c9d6676bf9a5aceb311dc31dadd07cba6a3d6392.tar.gz | awk -F "/" '{print $1}' | sort | uniq" ]
2022-08-17 00:44:15,759 - debrepack - INFO: [ Run - "tar -tzf /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/openstack-helm-infra-c9d6676bf9a5aceb311dc31dadd07cba6a3d6392.tar.gz | awk -F "/" '{print $2}' | sort | uniq" ]
2022-08-17 00:44:15,770 - debrepack - INFO: [ Run - "tar -xzf /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/openstack-helm-infra-c9d6676bf9a5aceb311dc31dadd07cba6a3d6392.tar.gz -C /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/armada-helm-toolkit-1.0" ]
2022-08-17 00:44:15,780 - debrepack - ERROR: gzip: stdin: unexpected end of file
2022-08-17 00:44:15,780 - debrepack - ERROR: tar: Child returned status 1
2022-08-17 00:44:15,780 - debrepack - ERROR: tar: Error is not recoverable: exiting now
2022-08-17 00:44:15,781 - debrepack - ERROR: [ Failed - "tar -xzf /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/openstack-helm-infra-c9d6676bf9a5aceb311dc31dadd07cba6a3d6392.tar.gz -C /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/armada-helm-toolkit-1.0" ]
2022-08-17 00:44:15,781 - debcontroller - ERROR: [ Failed - "tar -xzf /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/openstack-helm-infra-c9d6676bf9a5aceb311dc31dadd07cba6a3d6392.tar.gz -C /localdisk/loadbuild/jenkins/wrcp-master-debian/std/armada-helm-toolkit/armada-helm-toolkit-1.0" ]
2022-08-17 00:44:15,781 - debcontroller - ERROR: Failed to create needed dsc file, exit
2022-08-17 00:44:15,781 - debcontroller - INFO: Finished building packages in build_type std of layer distro
2022-08-17 00:44:16,034 - debcontroller - INFO: Start to build all packages in build_type rt of layer distro

Test Activity
-------------
N/A

Workaround
----------
Identify & delete the corrupted tarball in "mirrors/"; then re-run the downloader+build scripts

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/root/+/854344

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/root/+/854344
Committed: https://opendev.org/starlingx/root/commit/42044a6c173ea5dd1f1fcb275a018b0cd8fedd10
Submitter: "Zuul (22348)"
Branch: master

commit 42044a6c173ea5dd1f1fcb275a018b0cd8fedd10
Author: Yue Tao <email address hidden>
Date: Thu Aug 18 23:55:46 2022 -0700

    debrepack.py: add checksum upon download

    We use the return value of "wget" to check if download succeeds. However
    we observed "wget" returns 0 with a 0-length .tar.gz file. It looks like
    this is caused by "wget" when a connection to the server breaks in the
    middle of download, and yet "wget" reports success (ie returns 0
    status).

    Adding the checksum upon download to check if the download file is
    correct.

    Test Plan:
    Pass: downloader -s

    Closes-bug: 1986862
    Signed-off-by: Yue Tao <email address hidden>
    Change-Id: I5a64f695c5e60ef306eed309ee923556aaa0fd0a