maas-import-ephemerals crashes with "unexpected checksum 'sha256'" when using a proxy
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Invalid
|
Critical
|
Unassigned | ||
Ubuntu Cloud Archive |
Fix Released
|
High
|
Unassigned | ||
simplestreams |
Fix Released
|
High
|
Unassigned | ||
simplestreams (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Saucy |
Fix Released
|
High
|
Unassigned | ||
Trusty |
Fix Released
|
High
|
Unassigned |
Bug Description
== Begin SRU Info ==
[Impact]
maas-import-
This bug causes that download to fail in most cases with an http proxy between the user and maas.ubuntu.com. It is also reported to fail under cases where there is no http_proxy set (but may be a proxy in between).
[Test Case]
The bug could be reproduced by simply doing:
PROXY=http://
http_proxy=$PROXY python /usr/bin/
You would see the download start and then hang. A suitable proxy can be installed just by 'apt-get install squid3' and then using http://
[Regression Potential]
Regression potential should be very small. Previously each connection we made was doing an open and close and then open again. This was wreaking havoc with urllib3 as used by python-requests. Now, we're only opening and closing a single time during the mirror, and even then,
a.) the url that is being downloaded will change (by having a '/' inserted into its path) where previously it would be the same url
b.) step 'a' will only occur if the base url provided by the user did not already end in '/'.
== End SRU Info ==
Since 2013-10-11 we started seeing errors running the MAAS integration tests for saucy[1].
The checksum of the downloaded image wasn't matching the checksum expected[2]. maas-import-
Traceback (most recent call last):
File "/usr/sbin/
main(args)
File "/usr/lib/
target.
File "/usr/lib/
return self.sync_
File "/usr/lib/
self.
File "/usr/lib/
return self.sync_
File "/usr/lib/
self.
File "/usr/lib/
data, src, target, pedigree, contentsource)
File "/usr/lib/
size=
File "/usr/lib/
cksum.
Exception: unexpected checksum 'sha256' on precise/
}}}
Initially it was thought this could be caused by a misconfigured cache in the QA lab, but after re-installing and reconfiguring (and checking with wget that the cache was returning the correct objects for a given URL, as well as checking the checksum of the file matched the one in the json file), the error was still present.
Further investigation[3] raised the suspicion that the downloaded image is being handled by multiple processes. Turns out the code for simplestreams and import_
[1] http://
[2] http://
[3] http://
Related branches
- Julian Edwards (community): Disapprove
-
Diff: 348 lines (+339/-0)2 files modifiedsrc/provisioningserver/locking.py (+192/-0)
src/provisioningserver/tests/test_locking.py (+147/-0)
- Julian Edwards (community): Needs Fixing
-
Diff: 80 lines (+42/-2)2 files modifiedsrc/provisioningserver/import_images/ephemerals_script.py (+8/-2)
src/provisioningserver/locking.py (+34/-0)
Changed in maas: | |
status: | New → Triaged |
importance: | Undecided → Critical |
summary: |
- import_ephemerals.py script is not multi process safe + maas-import-ephemerals crashes with "unexpected checksum 'sha256'" when + using a proxy |
Changed in simplestreams: | |
status: | New → Confirmed |
importance: | Undecided → High |
description: | updated |
tags: |
added: verification-done removed: verification-needed |
Changed in maas: | |
status: | Triaged → Invalid |
fwiw, http:// celery. readthedocs. org/en/ latest/ tutorials/ task-cookbook. html#ensuring- a-task- is-only- executed- one-at- a-time
that provides a way to "run-one" (at a time).