| 2016-05-11 10:28:58 |
Robie Basak |
bug |
|
|
added bug |
| 2016-05-11 10:30:22 |
Robie Basak |
bug task added |
|
simplestreams |
|
| 2016-05-11 10:40:46 |
Launchpad Janitor |
branch linked |
|
lp:~racb/simplestreams/optimise-read_signed |
|
| 2016-05-12 16:30:52 |
Robie Basak |
nominated for series |
|
Ubuntu Xenial |
|
| 2016-05-12 16:30:52 |
Robie Basak |
bug task added |
|
simplestreams (Ubuntu Xenial) |
|
| 2016-05-12 16:47:54 |
Robie Basak |
description |
When running against http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.sjson, read_signed() in simplestreams/utils.py seems to take minutes (402 seconds in my instrumented timing) running 100% CPU. At the time of my test, download.sjson is 451737 bytes in 1870 lines.
This makes a uvtool sync virtually unusable on Xenial.
The culprit is the constant += of a string as it is read in. This is inefficient in Python as each += results in a new string that has to be allocated. Keeping the lines in a list and joining them at the end reduces the time to 0.127 seconds.
Merge proposal to follow. I'm filing this bug to track a Xenial SRU. |
[Impact]
When running against http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.sjson, read_signed() in simplestreams/utils.py seems to take minutes (402 seconds in my instrumented timing) running 100% CPU. At the time of my test, download.sjson is 451737 bytes in 1870 lines.
This makes a uvtool sync virtually unusable on Xenial.
[Development Fix]
The culprit is the constant += of a string as it is read in. This is inefficient in Python as each += results in a new string that has to be allocated. Keeping the lines in a list and joining them at the end reduces the time to 0.127 seconds.
[Stable Fix]
No change cherry-pick of fix committed upstream.
[Test Case]
On a machine with reasonable connectivity, run: uvt-simplestreams-libvirt sync release=xenial arch=amd64
Expected result: takes a few seconds with minimal CPU use.
Actual result: takes more than five minutes with CPU pegged at 100%.
Scott also has a more direct test case here: http://paste.ubuntu.com/16377613/
[Original Notes]
Merge proposal to follow. I'm filing this bug to track a Xenial SRU. |
|
| 2016-05-12 16:48:41 |
Robie Basak |
attachment added |
|
xenial.debdiff https://bugs.launchpad.net/ubuntu/+source/simplestreams/+bug/1580534/+attachment/4661385/+files/xenial.debdiff |
|
| 2016-05-12 16:58:21 |
Scott Moser |
simplestreams: status |
New |
Fix Committed |
|
| 2016-05-12 16:58:26 |
Scott Moser |
simplestreams: importance |
Undecided |
Medium |
|
| 2016-05-12 16:58:30 |
Scott Moser |
simplestreams (Ubuntu): status |
New |
Confirmed |
|
| 2016-05-12 16:58:33 |
Scott Moser |
simplestreams (Ubuntu): importance |
Undecided |
Medium |
|
| 2016-05-12 16:58:35 |
Scott Moser |
simplestreams (Ubuntu Xenial): status |
New |
Confirmed |
|
| 2016-05-12 16:58:38 |
Scott Moser |
simplestreams (Ubuntu Xenial): importance |
Undecided |
Medium |
|
| 2016-05-12 17:27:54 |
Robie Basak |
attachment removed |
xenial.debdiff https://bugs.launchpad.net/ubuntu/+source/simplestreams/+bug/1580534/+attachment/4661385/+files/xenial.debdiff |
|
|
| 2016-05-12 17:28:07 |
Robie Basak |
attachment added |
|
xenial.debdiff https://bugs.launchpad.net/ubuntu/+source/simplestreams/+bug/1580534/+attachment/4661405/+files/xenial.debdiff |
|
| 2016-05-12 17:46:19 |
Launchpad Janitor |
simplestreams (Ubuntu): status |
Confirmed |
Fix Released |
|
| 2016-05-18 08:28:42 |
Martin Pitt |
simplestreams (Ubuntu Xenial): status |
Confirmed |
Fix Committed |
|
| 2016-05-18 08:28:44 |
Martin Pitt |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2016-05-18 08:28:48 |
Martin Pitt |
bug |
|
|
added subscriber SRU Verification |
| 2016-05-18 08:28:50 |
Martin Pitt |
tags |
|
verification-needed |
|
| 2016-06-12 19:38:24 |
Mike Pontillo |
tags |
verification-needed |
verification-done |
|
| 2016-06-15 05:42:06 |
Launchpad Janitor |
simplestreams (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2016-06-15 05:42:10 |
Chris J Arges |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2017-09-14 13:47:21 |
Launchpad Janitor |
branch linked |
|
lp:~simplestreams-dev/simplestreams/artful |
|
| 2018-03-09 08:08:12 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~smoser/ubuntu/+source/simplestreams/+git/simplestreams/+merge/341205 |
|
| 2018-03-09 08:08:27 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~smoser/ubuntu/+source/simplestreams/+git/simplestreams/+merge/341206 |
|
| 2018-05-28 10:31:47 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~dmitriis/ubuntu/+source/simplestreams/+git/simplestreams/+merge/346958 |
|
| 2018-05-28 10:34:07 |
Launchpad Janitor |
merge proposal unlinked |
https://code.launchpad.net/~dmitriis/ubuntu/+source/simplestreams/+git/simplestreams/+merge/346958 |
|
|
| 2018-06-25 14:52:41 |
Scott Moser |
simplestreams: status |
Fix Committed |
Fix Released |
|
