stx-tools: download scripts leave corrupted files behind
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
High
|
Davlet Panech |
Bug Description
Brief Description
-----------------
Tarball & RPM download scripts in stx/tools sometimes leave corrupted files behind.
Details
-------
The problem is caused by the "wget" command, which doesn't handle download errors very well:
1) If the file can't be downloaded at all, *and* we use get with the -O flag, a zero-length output file is left behind. For example:
# wget -O foo https:/
--2021-11-05 16:30:09-- https:/
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.
HTTP request sent, awaiting response... 404 Not Found
2021-11-05 16:30:09 ERROR 404: Not Found.
# echo $?
8
# ls -al foo
-rw-r--r-- 1 dpanech users 0 Nov 5 16:30 foo
2) In some cases wget saves a partially downloaded file and yet returns 0 (success) to the shell. This happens if the download stalls mid-way, for example, *and* the web server doesn't support byte-ranges.
This is difficult to reproduce, but I observed it several times in the presence of a web proxy:
# wget http://
--2021-11-05 16:55:58-- http://
Resolving mirror.
Connecting to mirror.
HTTP request sent, awaiting response... 200 OK
Length: 33554674 (32M) [application/
Saving to: ‘v1.19.13.tar.gz’
55% [======
2021-11-05 17:11:15 (19.9 KB/s) - Read error at byte 18671712/33554674 (Connection timed out). Retrying.
--2021-11-05 17:11:16-- (try: 2) http://
Connecting to mirror.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
The file is already fully retrieved; nothing to do.
# echo $?
0
# ls -al
total 37792
drwxr-xr-x 2 dpanech users 4096 Nov 5 16:55 .
drwxrwxrwt. 754 root root 1339392 Nov 5 17:20 ..
-rw-r--r-- 1 dpanech users 18671712 Nov 5 16:49 foo
-rw-r--r-- 1 dpanech users 18671712 Nov 5 16:56 v1.19.13.tar.gz
Severity
--------
Major
Steps to Reproduce
------------------
N/A - See "Details" above
Expected Behavior
------------------
Failed downloads must not leave partial files behind
Actual Behavior
----------------
Failed downloads leave partial files behind.
Reproducibility
---------------
Intermittent
System Configuration
-------
N/A
Branch/Pull Time/Commit
-------
master/2021-11-05
Last Pass
---------
N/A
Timestamp/Logs
--------------
N/A
Test Activity
-------------
N/A
Workaround
----------
N/A
Changed in starlingx: | |
assignee: | nobody → Davlet Panech (dpanech) |
tags: | added: stx.build |
tags: | added: stx.6.0 |
Changed in starlingx: | |
importance: | Undecided → High |
status: | Fix Committed → Fix Released |
This is fixed: https:/ /review. opendev. org/c/starlingx /tools/ +/817049
I misspelled the "Closes-Bug" annotation in the commit message, that's why Gerrit didn't link the patch.