Don't run landscape-sysinfo if the load is high

Bug #608278 reported by Andreas Hasenack on 2010-07-21
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Landscape Client
Low
Thomas Herve
landscape-client (Ubuntu)
Undecided
Unassigned
Jaunty
Undecided
Unassigned
Karmic
Undecided
Unassigned
Lucid
Undecided
Unassigned
Maverick
Undecided
Unassigned

Bug Description

Chad reported that on a loaded machine, one about to die, landscape-sysinfo got in the way of him trying to login and do some corrective action. We don't do anything particularly heavy, but just starting the python interpreter and importing the landscape modules can be bad in an already bad situation.

Thomas' idea is to put a load checked in the wrapper and only call sysinfo if the load is low. For example, less than 1. if it's higher than 1, we can just echo something like "Sysinfo disabled due to high load".

Andreas Hasenack (ahasenack) wrote :

Another option is to just cache the output, as was done some time ago when -sysinfo was run by cron every 10min. But then the question of the usefulness of the data that is displayed comes to mind.

Thomas Herve (therve) on 2010-07-22
Changed in landscape-client:
assignee: nobody → Thomas Herve (therve)
milestone: 1.6.0 → 1.5.4
Thomas Herve (therve) wrote :

Committed in r275.

Changed in landscape-client:
status: New → Fix Committed
tags: added: needs-testing
Martin Pitt (pitti) on 2010-08-18
Changed in landscape-client (Ubuntu Maverick):
status: New → Fix Released
Changed in landscape-client:
status: Fix Committed → Fix Released

Accepted landscape-client into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Lucid):
status: New → Fix Committed
tags: added: verification-needed
Martin Pitt (pitti) wrote :

Accepted landscape-client into jaunty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Jaunty):
status: New → Fix Committed
Changed in landscape-client (Ubuntu Karmic):
status: New → Fix Committed
Martin Pitt (pitti) wrote :

Accepted landscape-client into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Andreas Hasenack (ahasenack) wrote :

Verified in Karmic:

andreas@nsn2:~$ ssh ubuntu@10.55.56.110
Warning: Permanently added '10.55.56.110' (RSA) to the list of known hosts.
Linux ip-172-56-125-36 2.6.31-22-server #60-Ubuntu SMP Thu May 27 03:42:09 UTC 2010 x86_64

To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/

 System information disabled due to load higher than 1 <<<<<<<<<<<<<<<<<<<<<<<<
---------------------------------------------------------------------
At the moment, only the core of the system is installed. To tune the
system to your needs, you can choose to install one or more
predefined collections of software by running the following
command:

   sudo tasksel --section server
---------------------------------------------------------------------

Last login: Fri Aug 20 17:27:38 2010 from 172.16.56.50

ubuntu@ip-172-56-125-36:~$ landscape-sysinfo
  System load: 1.94 Processes: 66
  Usage of /: 33.2% of 1.97GB Users logged in: 1
  Memory usage: 38% IP address for eth0: 172.56.125.36
  Swap usage: 0%

  Graph this data and manage this system at https://landscape.canonical.com/
ubuntu@ip-172-56-125-36:~$

ubuntu@ip-172-56-125-36:~$ apt-cache policy landscape-client
landscape-client:
  Installed: 1.5.4-0ubuntu0.9.10.0
  Candidate: 1.5.4-0ubuntu0.9.10.0
  Version table:
 *** 1.5.4-0ubuntu0.9.10.0 0
        500 http://archive.ubuntu.com karmic-proposed/main Packages
        100 /var/lib/dpkg/status
     1.5.2.1-0ubuntu0.9.10.0 0
        500 http://archive.ubuntu.com karmic-updates/main Packages
     1.3.2.4-0ubuntu0.9.10.0 0
        500 http://archive.ubuntu.com karmic/main Packages

Andreas Hasenack (ahasenack) wrote :

Confirmed working in Lucid:

andreas@nsn2:~$ ssh ubuntu@10.55.56.109
Warning: Permanently added '10.55.56.109' (RSA) to the list of known hosts.
Linux ip-172-56-125-35 2.6.32-21-server #32-Ubuntu SMP Fri Apr 16 09:17:34 UTC 2010 x86_64 GNU/Linux
Ubuntu 10.04.1 LTS

Welcome to the Ubuntu Server!
 * Documentation: http://www.ubuntu.com/server/doc

 System information disabled due to load higher than 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
---------------------------------------------------------------------
At the moment, only the core of the system is installed. To tune the
system to your needs, you can choose to install one or more
predefined collections of software by running the following
command:

   sudo tasksel --section server
---------------------------------------------------------------------

Last login: Fri Aug 20 17:42:17 2010 from 172.16.56.50

ubuntu@ip-172-56-125-35:~$ landscape-sysinfo
  System load: 1.07 Processes: 89
  Usage of /: 58.1% of 1.36GB Users logged in: 1
  Memory usage: 44% IP address for eth0: 172.56.125.35
  Swap usage: 0%

  Graph this data and manage this system at https://landscape.canonical.com/

ubuntu@ip-172-56-125-35:~$ apt-cache policy landscape-client
landscape-client:
  Installed: 1.5.4-0ubuntu0.10.04.0
  Candidate: 1.5.4-0ubuntu0.10.04.0
  Version table:
 *** 1.5.4-0ubuntu0.10.04.0 0
        500 http://archive.ubuntu.com/ubuntu/ lucid-proposed/main Packages
        100 /var/lib/dpkg/status
     1.5.2.1-0ubuntu0.10.04.0 0
        500 http://archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
     1.5.0.1-0ubuntu0.10.04.0 0
        500 http://archive.ubuntu.com/ubuntu/ lucid/main Packages
ubuntu@ip-172-56-125-35:~$

Andreas Hasenack (ahasenack) wrote :

Confirmed working in Jaunty:

andreas@nsn2:/virt$ ssh root@192.168.122.211
Warning: Permanently added '192.168.122.211' (RSA) to the list of known hosts.
root@192.168.122.211's password:
Linux jaunty-vm 2.6.28-11-server #42-Ubuntu SMP Fri Apr 17 02:48:10 UTC 2009 i686

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/

 System information disabled due to load higher than 1 <<<<<<<<<<<<<<<

82 packages can be updated.
117 updates are security updates.

Last login: Fri Aug 20 14:49:08 2010 from 192.168.122.1

root@jaunty-vm:~# landscape-sysinfo
  System load: 1.58 Processes: 61
  Usage of /: 7.4% of 9.38GB Users logged in: 1
  Memory usage: 11% IP address for eth0: 192.168.122.211
  Swap usage: 0%

  Graph this data and manage this system at https://landscape.canonical.com/

root@jaunty-vm:~# apt-cache policy landscape-client
landscape-client:
  Installed: 1.5.4-0ubuntu0.9.04.0
  Candidate: 1.5.4-0ubuntu0.9.04.0
  Version table:
 *** 1.5.4-0ubuntu0.9.04.0 0
        500 http://br.archive.ubuntu.com jaunty-proposed/main Packages
        100 /var/lib/dpkg/status
     1.5.2.1-0ubuntu0.9.04.0 0
        500 http://br.archive.ubuntu.com jaunty-updates/main Packages
     1.0.29.1-0ubuntu0.9.04.0 0
        500 http://br.archive.ubuntu.com jaunty/main Packages
root@jaunty-vm:~#

Martin Pitt (pitti) on 2010-08-20
tags: added: verification-done
removed: verification-needed
tags: removed: needs-testing
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.5.4-0ubuntu0.10.04.0

---------------
landscape-client (1.5.4-0ubuntu0.10.04.0) lucid-proposed; urgency=low

  * New upstream version (LP: #610744):

    - The Eucalyptus management plugin reports the output of the
      'euca-describe-availability-zones verbose' command, which includes
      information about the available instance types and the maximum
      number of each instance type that the cloud can support (LP: #599338)

    - Check if the package directory exists before trying to check the
      package changer lock in the dbus-proxy. This fixes a bug when upgrading
      a dbus-landscape which never registered (LP: #603514).

    - Allow an LDS server to bootstrap new cloud instances with its own CA,
      which is picked up by the client, written to a file on the instance, and
      used in subsequent exchanges with the server (LP: #605079).

    - Skip loopback interface when reporting device info (LP: #608314)

    - Disable landscape-sysinfo when load is more than 1 (LP: #608278)
 -- Free Ekanayaka <email address hidden> Wed, 28 Jul 2010 08:14:02 +0200

Changed in landscape-client (Ubuntu Lucid):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.5.4-0ubuntu0.9.04.0

---------------
landscape-client (1.5.4-0ubuntu0.9.04.0) jaunty-proposed; urgency=low

  * New upstream version (LP: #610744):

    - The Eucalyptus management plugin reports the output of the
      'euca-describe-availability-zones verbose' command, which includes
      information about the available instance types and the maximum
      number of each instance type that the cloud can support (LP: #599338)

    - Check if the package directory exists before trying to check the
      package changer lock in the dbus-proxy. This fixes a bug when upgrading
      a dbus-landscape which never registered (LP: #603514).

    - Allow an LDS server to bootstrap new cloud instances with its own CA,
      which is picked up by the client, written to a file on the instance, and
      used in subsequent exchanges with the server (LP: #605079).

    - Skip loopback interface when reporting device info (LP: #608314)

    - Disable landscape-sysinfo when load is more than 1 (LP: #608278)
 -- Free Ekanayaka <email address hidden> Wed, 28 Jul 2010 08:14:02 +0200

Changed in landscape-client (Ubuntu Jaunty):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.5.4-0ubuntu0.9.10.0

---------------
landscape-client (1.5.4-0ubuntu0.9.10.0) karmic-proposed; urgency=low

  * New upstream version (LP: #610744):

    - The Eucalyptus management plugin reports the output of the
      'euca-describe-availability-zones verbose' command, which includes
      information about the available instance types and the maximum
      number of each instance type that the cloud can support (LP: #599338)

    - Check if the package directory exists before trying to check the
      package changer lock in the dbus-proxy. This fixes a bug when upgrading
      a dbus-landscape which never registered (LP: #603514).

    - Allow an LDS server to bootstrap new cloud instances with its own CA,
      which is picked up by the client, written to a file on the instance, and
      used in subsequent exchanges with the server (LP: #605079).

    - Skip loopback interface when reporting device info (LP: #608314)

    - Disable landscape-sysinfo when load is more than 1 (LP: #608278)
 -- Free Ekanayaka <email address hidden> Wed, 28 Jul 2010 08:14:02 +0200

Changed in landscape-client (Ubuntu Karmic):
status: Fix Committed → Fix Released
Andrew LeCody (andrewlecody) wrote :

This fix doesn't take into account multi-processor systems. I suggest that the system information be suppressed if the load average divided by the number of CPUs is greater than 1. As it stands right now landscape-sysinfo is disabled 100% of the time for my 8 core server, even though a load average of 1 would be considered low. I have attached a patch with my suggested change.

I know that counting processors in /proc/cpuinfo isn't accurate when HyperThreading is being used, but I couldn't think of a simple and effective way to determine the number of real cores. So hopefully someone much smarter than me will come up with a snippet of bash to resolve the issue.

CvB (cvb-kruemel) wrote :

I agree with the previous comment - as it currently is, I often don't get system information on my multi-core system, even though load is far from high. It's just above 1.

Andreas Hasenack (ahasenack) wrote :

Thanks for the comments and the patch. I opened bug #643565 for this new issue and assigned it for the 1.5.6 client release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers