Add retry logic to snap-tool to make downloads more resilient
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
livecd-rootfs (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned | ||
Disco |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Impact]
* livecd-rootfs builds will fail immediately when a snap-tool invocation fails to contact the snapstore because of ephemeral connection problems or a transient error on the server side.
* The snap-tool script included with livecd-rootfs in Eoan has been enhanced to retry on connection errors and 5xx server errors reducing the likelihood of image builds breaking due to a flaky connection or a server hiccup.
[Test Cases]
* Download core, core18 snaps using both `snap download <snap-name>` and `snap-tool download <snap-name>` and make sure the downloads are identical.
* Invoke `snap-tool info <snap-name>` for a few snaps, e.g. review-gator, lpshipit, azure-cli and verify that all fields carry correct information.
* Test the backoff/retry logic using the following procedure:
Make netcat listen on port 12345
netcat -l -p 12345
Create a symlink from snaptool.py to snap-tool and import the ExpBackoffHTTPC
ln -s snap-tool snaptool.py
python3
from snaptool import ExpBackoffHTTPC
http_client = ExpBackoffHTTPC
request = http_client.get("http://
request.text()
Go back to the terminal where you invoked netcat and stop it. snap-tool should print the following and then fail:
WARNING: failed to open URL 'http://
Retrying HTTP request in 2 seconds...
WARNING: failed to open URL 'http://
Retrying HTTP request in 4 seconds...
Repeat the procedure above but instead of stopping netcat, paste the following response:
HTTP/1.1 503 Error
and hit enter twice. You should see
WARNING: failed to open URL 'http://
Retrying HTTP request in 2 seconds...
Repeat the above pasting "HTTP/1.1 404 Not found" instead. The snap-tool should fail immediately.
[Regression Potential]
* Tool logic and http request headers/body are unchanged, only the way connections are built has been modified. The expectation is that this will be more robust and testing in devel hasn't surfaced any bugs, but there is a slight risk that the tool's behavior has changed in unobvious corner cases that were missed during testing.
summary: |
- Backport snap-tool backoff/retry logic + Add retry logic to snap-tool to make downloads more resilient |
tags: | added: id-5d0a349876579b42ed84d920 |
Changed in livecd-rootfs (Ubuntu): | |
importance: | Undecided → Low |
Changed in livecd-rootfs (Ubuntu): | |
importance: | Low → High |
tags: |
added: verification-done-disco removed: verification-needed-disco |
MPs for disco: /code.launchpad .net/~tobijk/ livecd- rootfs/ +git/livecd- rootfs/ +merge/ 371794
https:/
and bionic: /code.launchpad .net/~tobijk/ livecd- rootfs/ +git/livecd- rootfs/ +merge/ 371795
https:/