Activity log for bug #1788518

Date Who What changed Old value New value Message
2018-08-22 23:43:24 Ryan Finnie bug added bug
2018-08-22 23:45:55 William Grant bug added subscriber William Grant
2018-08-22 23:48:26 Haw Loeung bug added subscriber The Canonical Sysadmins
2018-08-23 00:28:45 Dean Henrichsmeyer landscape-client: importance Undecided High
2018-08-23 07:24:10 Adam Collard bug added subscriber Adam Collard
2018-08-23 07:24:13 Adam Collard tags is
2018-08-23 16:19:41 Joel Sing bug added subscriber Joel Sing
2018-10-31 14:29:29 Adam Collard bug task added ubuntu
2018-10-31 14:29:34 Adam Collard bug task deleted ubuntu
2018-10-31 14:29:48 Adam Collard landscape-client: status New Triaged
2018-11-01 23:49:00 Ryan Finnie attachment added lp1788518-10percent-spread.patch https://bugs.launchpad.net/landscape-client/+bug/1788518/+attachment/5208028/+files/lp1788518-10percent-spread.patch
2019-02-13 15:04:38 Simon Poirier landscape-client: assignee Simon Poirier (simpoir)
2019-02-13 15:04:40 Simon Poirier landscape-client: status Triaged In Progress
2019-03-06 06:31:03 Junien F bug added subscriber Junien Fridrick
2019-03-06 06:32:09 Barry Price bug added subscriber Barry Price
2019-03-26 14:59:28 Simon Poirier landscape-client: status In Progress Fix Committed
2019-06-17 23:03:12 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/ubuntu/+source/landscape-client/+git/landscape-client/+merge/368945
2019-06-18 23:08:12 Simon Poirier description The various intervals appear to be statically determined from program start. As a practical example, a rebooted VM host with a large number of guests means all the guests start landscape-client at about the same time. Then every 30 minutes, package-monitor runs (which is a rather heavy operation). Multiply by a hundred or so and you've got a large flood on the host. When the client starts up, it would be good to have the initial schedule for each timer be ((interval / 2) + random(interval)), so the collective load is spread out over the entire interval. [Impact] * Restarting hosts/clouds of multiple VMs or containers managed by landscape leads to synchronized monitoring tasks. This can generate significant recurring load. * The fix adds a configurable randomized interval to scheduled tasks. [Test Case] sudo landscape-config --stagger-launch 0.5 --log-level debug # accept client in server interface sudo tail -f /var/log/landscape/monitor.log # Check for debug log entries mentioning monitors getting delayed [Regression Potential] * Although changing the scheduler for non-fixed loop intervals could have been slightly better at avoid load peaks on shared hosts, the current approach has been deemed safer as it simply adds an initial delay to tasks. The current logic of scheduling is unchanged. * External scripts expecting monitors to run instantly on landscape-client restart could be affected. I'm not aware of any such case, as the async nature of the code never made that guarantee. * The stagger interval is configurable with a flag and a configuration entry to work around regressions. [Original Description] The various intervals appear to be statically determined from program start. As a practical example, a rebooted VM host with a large number of guests means all the guests start landscape-client at about the same time. Then every 30 minutes, package-monitor runs (which is a rather heavy operation). Multiply by a hundred or so and you've got a large flood on the host. When the client starts up, it would be good to have the initial schedule for each timer be ((interval / 2) + random(interval)), so the collective load is spread out over the entire interval.
2019-06-28 13:28:09 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/ubuntu/+source/landscape-client/+git/landscape-client/+merge/369449
2019-06-28 17:33:40 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/ubuntu/+source/landscape-client/+git/landscape-client/+merge/369457
2019-06-28 17:34:09 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/ubuntu/+source/landscape-client/+git/landscape-client/+merge/369458
2019-07-09 16:37:33 Brian Murray bug task added landscape-client (Ubuntu)
2019-07-09 16:37:43 Brian Murray landscape-client (Ubuntu): status New Fix Released
2019-07-09 16:42:54 Brian Murray landscape-client (Ubuntu Disco): status New Fix Committed
2019-07-09 16:42:56 Brian Murray bug added subscriber Ubuntu Stable Release Updates Team
2019-07-09 16:42:58 Brian Murray bug added subscriber SRU Verification
2019-07-09 16:43:05 Brian Murray tags is is verification-needed verification-needed-disco
2019-07-09 16:53:02 Adam Collard removed subscriber Adam Collard
2019-07-12 20:29:57 Launchpad Janitor merge proposal linked https://code.launchpad.net/~simpoir/ubuntu/+source/landscape-client/+git/landscape-client/+merge/370075
2019-07-19 18:52:59 Brian Murray landscape-client (Ubuntu Bionic): status New Fix Committed
2019-07-19 18:53:06 Brian Murray tags is verification-needed verification-needed-disco is verification-needed verification-needed-bionic verification-needed-disco
2019-07-25 23:14:00 Simon Poirier tags is verification-needed verification-needed-bionic verification-needed-disco is verification-done-bionic verification-done-disco verification-needed
2019-08-13 20:24:25 Brian Murray landscape-client (Ubuntu Xenial): status New Fix Committed
2019-08-13 20:24:34 Brian Murray tags is verification-done-bionic verification-done-disco verification-needed is verification-done-bionic verification-done-disco verification-needed verification-needed-xenial
2019-09-05 21:04:04 Simon Poirier tags is verification-done-bionic verification-done-disco verification-needed verification-needed-xenial is verification-done verification-done-bionic verification-done-disco verification-done-xenial
2019-09-09 09:25:30 Ɓukasz Zemczak removed subscriber Ubuntu Stable Release Updates Team
2019-09-09 09:28:04 Launchpad Janitor landscape-client (Ubuntu Bionic): status Fix Committed Fix Released
2019-09-09 09:35:28 Launchpad Janitor landscape-client (Ubuntu Disco): status Fix Committed Fix Released
2019-11-13 00:08:20 Launchpad Janitor landscape-client (Ubuntu Xenial): status Fix Committed Fix Released
2021-11-19 18:40:46 Ryan Finnie removed subscriber Ryan Finnie