Ubuntu

High load average

Reported by Frantique on 2012-04-19
350
This bug affects 67 people
Affects Status Importance Assigned to Milestone
Xubuntu
Confirmed
Undecided
Unassigned
linux (Ubuntu)
Medium
Unassigned

Bug Description

With the same configuration and usage Ubuntu 11.10 used to have a load average of 0.2, now it is between 1.2 and 2.5 all the time!
Please help investigating this issue!

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: compiz 1:0.9.7.6-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-23.36-generic-pae 3.2.14
Uname: Linux 3.2.0-23-generic-pae i686
.tmp.unity.support.test.0:

ApportVersion: 2.0.1-0ubuntu5
Architecture: i386
CompizPlugins: [core,bailer,detection,composite,opengl,compiztoolbox,decor,mousepoll,resize,move,gnomecompat,place,imgpng,grid,wall,vpswitch,regex,snap,session,commands,unitymtgrabhandles,animation,workarounds,expo,fade,ezoom,scale,unityshell]
CompositorRunning: compiz
Date: Thu Apr 19 15:39:37 2012
DistUpgraded: 2012-04-16 12:16:02,137 DEBUG enabling apt cron job
DistroCodename: precise
DistroVariant: ubuntu
DkmsStatus: vboxhost, 4.1.12, 3.2.0-23-generic-pae, i686: installed
GraphicsCard:
 Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (primary) [8086:2a02] (rev 0c) (prog-if 00 [VGA controller])
   Subsystem: Hewlett-Packard Company Compaq 6710b [103c:30c0]
   Subsystem: Hewlett-Packard Company Compaq 6710b [103c:30c0]
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release i386 (20100429)
MachineType: Hewlett-Packard HP Compaq 6710b (GR684EA#AKC)
PackageArchitecture: all
PccardctlIdent:
 Socket 0:
   no product info available
PccardctlStatus:
 Socket 0:
   no card
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.2.0-23-generic-pae root=UUID=ade9955a-e4e1-4e92-b49e-5823c06e19e3 ro reboot=pci quiet splash ipv6.disable=1 vt.handoff=7
SourcePackage: compiz
UpgradeStatus: Upgraded to precise on 2012-04-16 (3 days ago)
dmi.bios.date: 01/15/2009
dmi.bios.vendor: Hewlett-Packard
dmi.bios.version: 68DDU Ver. F.15
dmi.board.name: 30C0
dmi.board.vendor: Hewlett-Packard
dmi.board.version: KBC Version 71.2E
dmi.chassis.asset.tag: CNU7451TFQ
dmi.chassis.type: 10
dmi.chassis.vendor: Hewlett-Packard
dmi.modalias: dmi:bvnHewlett-Packard:bvr68DDUVer.F.15:bd01/15/2009:svnHewlett-Packard:pnHPCompaq6710b(GR684EA#AKC):pvrF.15:rvnHewlett-Packard:rn30C0:rvrKBCVersion71.2E:cvnHewlett-Packard:ct10:cvr:
dmi.product.name: HP Compaq 6710b (GR684EA#AKC)
dmi.product.version: F.15
dmi.sys.vendor: Hewlett-Packard
version.compiz: compiz 1:0.9.7.6-0ubuntu1
version.libdrm2: libdrm2 2.4.32-1ubuntu1
version.libgl1-mesa-dri: libgl1-mesa-dri 8.0.2-0ubuntu3
version.libgl1-mesa-dri-experimental: libgl1-mesa-dri-experimental N/A
version.libgl1-mesa-glx: libgl1-mesa-glx 8.0.2-0ubuntu3
version.xserver-xorg-core: xserver-xorg-core 2:1.11.4-0ubuntu10
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev 1:2.7.0-0ubuntu1
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:6.14.99~git20111219.aacbd629-0ubuntu2
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.17.0-1ubuntu4
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:0.0.16+git20111201+b5534a1-1build2

Frantique (undernetangel) wrote :
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in compiz (Ubuntu):
status: New → Confirmed
Chen Tao (pro711) wrote :

Same problem here. Load average never drops even if I am doing nothing. Ubuntu 12.04 beta2 with NVIDIA driver.

Besides, if there is something causing frequent screen redraw (e.g. a spinning icon), compiz cpu usage soars.

nuttzo33 (slucas1979) wrote :

I have uploaded a video showing what i think is high compiz usage this what you guys are getting? I't happens with both nvidia proprietary and nouveau drivers, some times the launcher freezes for seconds and windows are slow to minimize/maximize.I have a core i5-2500k cpu.

nuttzo33 (slucas1979) wrote :
ivant (ivan-toshkov) wrote :

I'm not sure compiz is to blame here. On my system (core i3 / ati radeon / 64bit ubuntu) the load avarage is high (see below) even when runnign other window managers / desktops, e.g. gnome-shell (including classic and without acceleration), unity-2d, xfce.

I logged in at the text cosole and stopped X (service lightdm stop) and the load dropped to 0.03. Just starting X (/usr/bin/X) made the load go to 0.1 - 0.17. Starting lightdm got it to 0.3 - 0.4 and it was the same with xdm as well.

I tried this with fglrx, fglrx-updates, and without any of them. The results were generally the same. So I think X.org is the culprit.

P.S. Strangely, it seems that neither the CPU nor the disk usage is high when these loads occur.

P.P.S. Another strange thing, that I've noticed is, that if the display goes blank after some time of inactivity the CPU usage is gets higher and just waking it up brings the CPU to previous levels.

nuttzo33 (slucas1979) wrote :

Here is my load average over the last 15 mins just browsing web doing nothing much
0.73 0.79 0.78 1/411 6235
is that high?

I have a core i5-2500k

nuttzo33 (slucas1979) wrote :

Here is my load average over last 40 mins
20:47:20 up 40 min, 1 user, load average: 0.37, 0.61, 0.72

If i have a 4 core cpu does it mean that it maxes out at 4.0 or is it 1.0 maximum?

ivant (ivan-toshkov) wrote :

It depends, but generally, if the machine isn't doing anything, the load should be zero or close to zero (e.g. less than 0.1). At least that's what it was with previous versions of ubuntu on the same machine. And it seems that this load is keeping my machine hot all the time.

For a conscious description check out the wikipedia article: http://en.wikipedia.org/wiki/Load_%28computing%29

asasoft (asasoft) wrote :

Almost Idle after login

16:36:29 up 28 min, 1 user, load average: 0.60, 0.81, 0.80

3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz

Mem: 3826 1543 2283 0 174 588
-/+ buffers/cache: 780 3046
Swap: 509 0 509

nuttzo33 (slucas1979) wrote :

I have gone back to 11.10 and here is load average:

10:30:00 up 2:55, 1 user, load average: 0.12, 0.13, 0.09

There seems to be a problem with 12.04 at the moment.

Junior_sampa (juniorperes-ig) wrote :

Same problem here with load average:

0.63, 0.70, 0.53

Before with Lucid it was near 0.1

perriman (chuchiperriman) wrote :

These screenshots are in gnome shell and ubuntu sessions. Maybe the problem is not compiz, I don't know...

perriman (chuchiperriman) wrote :

The ubuntu session screenshot (Unity)

perriman (chuchiperriman) wrote :

Screenshot after 30 minutes only with chromium (gmail and launchpad)

skunk (tiobra666) wrote :

same here...high load average even when in idle.

pschroth (philip-schroth) wrote :

Same here my laptop is constantly hot. Load is very high compared to ubuntu 11.10..

pschroth (philip-schroth) wrote :

I tried Xubuntu and the load is also much to high.

philip schroth (info-schroth) wrote :

This affects my system as well. Core I7 with a load around 1 to 2.5 Ubuntu 12.04

Mickael-goetz (mickael-goetz) wrote :

I'm also affected, NVidia graphic card, core 2 Duo CPU and 8GB RAM. The load average is high in Ubuntu with Unity+Compiz 12.04 & Xubuntu 12.04 with Xfce+Compiz

Mickael-goetz (mickael-goetz) wrote :

Here is my Compiz profile, exported using CCSM (the default value are omitted).

It's seems it related to X-server.
I have a high load (0.8) even in login screen, but when I stop lightdm, load drops to near zero.
Note that highl load occures all the time, but cpu usage is low (few percent)
I googled this problem, the problem may be with kernel drivers. I used power top and here is the results.

Jorenko (jorenko) wrote :

Same issue here. My machine is doing nothing but running a few gnome-terminal sessions, and the load average is:
    1.12, 1.00, 1.05

The top few items in top are almost always:

 1094 root 20 0 327m 136m 15m R 11 6.8 45:20.30 Xorg
 1754 jamesj 20 0 1329m 85m 37m R 5 4.3 51:14.45 compiz
 1879 jamesj 20 0 576m 23m 11m S 3 1.2 34:37.32 unity-panel-ser

UPDATE: My bug fixed with reveting back kernel to 3.0.0-19

Tim Savage (tim-savage) wrote :

I am also experiencing this bug with 12.04 server. Therefore, I don't believe that compiz is the source, at least not for me, since there is no x server installed on this machine.

no longer affects: compiz-core
luca (l-savio) wrote :

@taha-jahangir I workarounded installing kernel from http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3.4-precise/ : maybe I will revert to 3.2 when this bug will be officially fixed! :)

Sergei Garmash (vpupkin) wrote :

Appears to be a problem with kernel. I had the same high average load problem with idle pc running only terminal with top (around 1.0 or even higher). After installing 3.3.4 load average under same conditions is like this: 0.03, 0.09, 0.08. Thanks for the hint, luca!

Daniel van Vugt (vanvugt) wrote :

Load average is a measure of runnable processes and not CPU usage. Probably a good way to find the cause of high load average is to run "ps auxw" and look for processes with STATus "R" (running).
[http://en.wikipedia.org/wiki/Load_(computing)#Unix-style_load_calculation]

affects: compiz (Ubuntu) → ubuntu
Junior_sampa (juniorperes-ig) wrote :

I also confirm the problem with kernel. I also installed the v3.3.4 from mainline to test and the problem was solved. The load average returned to "lucid" conditions like 0.01, 0.07, 0.06.

However I uninstalled it because it is not prepared for restricted drivers like my wireless broadcom.

It should be changed from compiz to kernel issue.

Soh Kam Yung (sohkamyung) wrote :

On Xubuntu 12.04, I tried the following kernels:

3.2.0-20-generic - load avg dropped close to 0.0
3.2.0-22-generic - load avg hovers close to 1.0

Could the high load avg be due to changes introduced between 20-generic and 22-generic?

pschroth (philip-schroth) wrote :

I can confirm that it is kernel related.

3.2.0-24 high load
3.2.16-030216 normal load (from kernel ppa)

Pse change this bug to the kernel.

ivant (ivan-toshkov) wrote :

I narrowed it down to a change in kernel 3.2.0-22. With 3.2.0-21 the load is normal

Douglas Leeder (ubuntu-leeder) wrote :

Looking at the 3.2.0-22 changelog: '* Revert "sched: tg->se->load should be initialised to tg->shares"' looks to me like it could be a candidate for the cause?

luca (l-savio) wrote :

@douglas: googling a bit that change upstream seem to be related with the "200 lines patch" ... anyway, could someone change properly the tag in this bug? I still seems a compiz bug ..

tags: added: kernel
removed: compiz-0.9
affects: ubuntu → linux (Ubuntu)
Changed in xubuntu-desktop:
status: New → Confirmed
Firat Birlik (fbirlik) wrote :

I can confirm that following commit is responsible: "sched: Fix nohz load accounting"
SHA: 3a50863f6706ece7719a68be0ae57957164a0f0c

It is introduced between following tags:
* Ubuntu-3.2.0-21.34
* Ubuntu-3.2.0-22.35

All tags after Ubuntu-3.2.0-22.35 are affected and reverting this commit fixes the high load behaviour. I built following tags with this commit reverted and verified the high load behaviour goes away:
* Ubuntu-3.2.0-22.35
* Ubuntu-3.2.0-23.36
* Ubuntu-3.2.0-24.38 (currently latest tag)

I couldn't reproduce/observe shortened battery life, but if anybody is interested I can provide binary/source deb's with this commit reverted for quick testing. (ppa can't hold full kernel package build)

Guy (guy-b) wrote :

Same problem here on Xubuntu 12.04 without compiz installed and kernel: 3.2.0-24-generic, Phenom x6.

The load everage (iddle) in htop: 1.02 0.92 0.95 and it jump to 1.80 with just chromium window opened! Far away from compiz bug!

Doug Smythies (dsmythies) wrote :

See also Launchapd bug 838811
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/838811

I wish I had known about this bug report before now.

Reverting the above mentioned commit will fix this issue yes, but then break the massively low reported load averages under other conditions.

Doug Smythies (dsmythies) wrote :

I only know of the related commit number as: c308b56b5398779cd3da0f62ab26b0453494c3d4

I would argue that the commit results in more accurate reported load averages, overall.

One has to look at the reported load averages over the entire range of operation. Meaning over all frequencies of cpu's entering and exiting the idle state, and all per cpu loads. An idle desktop environment is but one point in that two dimensional space, and yes, it seems to be too high.

Without the commit code changes, all cpu enter/exit idle frequencies above 25 hertz will result in a reported load average of 0.00, and will scale linearly from 0.00 at 25 hertz to accurate at 0 hertz. (for systems with a basic 250 hertz clock, which is the default)
Reference: https://launchpadlibrarian.net/93077784/load_freq_s15.png

With the commit code changes, and for medium to higher actual load averages the results are much better.

I am working on aquiring more data for low actual load situations, both with control samples using kernels compiled with CONFIG_NO_HZ=n (i.e. the old tick way) and for CONFIG_NO_HZ=y (the newer tickless way) with and without the commit. Reported load averages are very noisey, and take a long time to settle for given conditions, so each test takes a long time. I will post here or on the above referenced bug when done (it'll be a few days).

Doug Smythies (dsmythies) wrote :

The attached file should be viewed with zoom set to 1:1 and then scroll down as you read and view the graphs.

sds (sds-gnu) wrote :

additionally, the battery dies very quickly - can this be related?

Changed in linux (Ubuntu):
importance: Undecided → Medium
tags: added: kernel-da-key
Changed in linux (Ubuntu):
status: Confirmed → Incomplete
Changed in linux (Ubuntu):
status: Incomplete → Triaged
tags: added: kernel-fixed-upstream
tags: added: kernel-bug-exists-upstream
removed: kernel-fixed-upstream
26 comments hidden view all 106 comments
Doug Smythies (dsmythies) wrote :

In Kernel 3.5 RC2 there were changes in the load calc area of the code. However, no difference with respect to this bug report.
There is another proposed patch, that actually makes things worse, at least with respect to this bug report.
The maintainers of this area of code upsteam at kernel.org are aware of the issue.

Guy (guy-b) wrote :

Thanks Doug, good to know that the maintainers are informed, let see how long we will have to wait for a fixe...

sdblepas (sdblepas) wrote :

Hi
I have the same problem here
I have Core i5 quad cores 8G Ram and a GeForce 9500 GT (1G).
My load average is about ~0.62 whe I don't do anything, If I'm watching a movie I get 2.8 minimum
Thanks

sdblepas (sdblepas) on 2012-06-20
Changed in linux (Ubuntu):
status: Triaged → Confirmed
Changed in linux (Ubuntu):
status: Confirmed → Triaged
Doug Smythies (dsmythies) wrote :

A fix for continuing issues with incorrect Reported Load Averages (both too high for light loads, and it turns out, still too low for very high loads and high frequencies) is being tested now. To try to cover all the senarios, I would anticipate about another week of testing. The proposed solution doesn't have a commit number yet, but it would first end up in the 3.5 series kernels. I do not know how long after that it would take to propegate back to other kernel streams.

There have been a few upstreams threads on this. For those that are interested, the main thread making good progress has been: http://thread.gmane.org/gmane.linux.kernel/1310462

Frantique (undernetangel) wrote :

Doug: thank you for the information. We hope that the issue will be resolved soon.

luca (llucax) wrote :

Experiencing high loads with this CPU too, even using nohz=off, kernel Ubuntu kernel 3.4.0 from mainline builds:

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 42
model name : Intel(R) Core(TM) i5-2467M CPU @ 1.60GHz
stepping : 7
microcode : 0x23
cpu MHz : 800.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips : 3192.94
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

Doug Smythies (dsmythies) wrote :

The kernel run time parameter "nohz=off" is not the same as a kernel compiled with CONFIG_NO_HZ=n. The actual related code that is executed in the two cases is different. Yes, the last time I tested it, which was in about February/March, the run time parameter "nohz-off" had no effect .

For those that might be interested, test results, for my part of it, of the possible pending solution are at:
http://www.smythies.com/~doug/network/load_average/peter35.html

Hi,

same problem here. What seems interesting to me is the loads uniform up and down which you can see on this munin chart, one wave every 2-3 hours: http://paste.opensuse.org/71588251

Special here is, that my ubuntu runs as a virtual machine.

Perhaps special too: I also have an FreeNX server installed, but i think the load thing is the same, even if the NX server is not running. I can't say this for sure because i still don't undrestand NX very well, and i'm not sure wether the server is running or not (can't see any NX processes))

Kernel: 3.2.0-25-generic-pae

ps tree:

init-+-/usr/sbin/munin
     |-NetworkManager-+-dnsmasq
     | `-2*[{NetworkManager}]
     |-accounts-daemon---{accounts-daemon}
     |-acpid
     |-at-spi-bus-laun-+-dbus-daemon
     | `-3*[{at-spi-bus-laun}]
     |-at-spi2-registr---{at-spi2-registr}
     |-atd
     |-avahi-daemon---avahi-daemon
     |-bluetoothd
     |-colord---2*[{colord}]
     |-console-kit-dae---64*[{console-kit-dae}]
     |-cron
     |-cupsd
     |-2*[dbus-daemon]
     |-dconf-service---2*[{dconf-service}]
     |-gconfd-2
     |-geoclue-master
     |-6*[getty]
     |-gnome-settings----2*[{gnome-settings-}]
     |-gvfs-fuse-daemo---3*[{gvfs-fuse-daemo}]
     |-gvfsd
     |-indicator-datet---2*[{indicator-datet}]
     |-indicator-sessi---2*[{indicator-sessi}]
     |-indicator-sound---{indicator-sound}
     |-lightdm-+-Xorg
     | |-lightdm-+-lightdm-greeter---unity-greeter---3*[{unity-greeter}]
     | | `-{lightdm}
     | |-lightdm
     | `-2*[{lightdm}]
     |-modem-manager
     |-polkitd---{polkitd}
     |-pulseaudio-+-gconf-helper
     | `-2*[{pulseaudio}]
     |-pulseaudio-+-gconf-helper
     | `-{pulseaudio}
     |-rsyslogd---3*[{rsyslogd}]
     |-rtkit-daemon---2*[{rtkit-daemon}]
     |-sshd---sshd---sshd---bash---pstree
     |-system-service-
     |-ubuntu-geoip-pr
     |-udevd---2*[udevd]
     |-udisks-daemon-+-udisks-daemon
     | `-{udisks-daemon}
     |-upowerd---2*[{upowerd}]
     |-upstart-socket-
     |-upstart-udev-br
     `-whoopsie---{whoopsie}

Dominik

Guy (guy-b) wrote :

After this morning kerner update "3.2.0-26-generic", still high load.

Doug Smythies (dsmythies) wrote :

The proposed patch, mentioned in a couple of postings above, now has a Commit I.D. 5167e8d5417bf5c322a703d2927daec727ea40dd
As mentioned above, that patch is for the 3.5 series kernels and hopefully will be in 3.5-rc6, hopefully Sunday.
All of my testing was done with that patch (and two other previous related patches) backported to kernel 3.2.0-26 #41 Ubuntu
What I do not know how to do or help with is accelerating the process of propagation of the required patches to the 3.2 series kernels.

Guy (guy-b) wrote :

Thanks a lot for your efforts Doug, because it whas not easy!!! Now Ubuntu distro have all the keys in theyr hands, hope they will move the patch to the kernel in use... Once again, thank you for your efforts!

Doug Smythies (dsmythies) wrote :

The patch is not in kernel 3.5-rc6. I assume it missed some cut-off time for rc6. Hopefully it will be in rc7, perhaps in one week.

Sudhir Khanger (donniezazen) wrote :

I do see some significant reduction in both temperature and power. The load is still around or over 1.

Doug Smythies (dsmythies) wrote :

The patch mentioned in posting # 76 is included in kernel 3.5-rc7, available here: http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.5-rc7-quantal/
It would be good if someone with a 12.04 desktop edition could try this kernel and comment.

Myself, I am not interested in what the 1 minute reported load average is, as it will be all over the place.
I am intertested in the 15 minute load average, after about an hour.
Example "top" from my computer, where the 15 minute load average for 3 processes at 15 % each, 60 hertz sleep frequency used to be reported as about 2. Now it is about 0.46, which is correct:

top - 08:16:15 up 7:42, 3 users, load average: 0.14, 0.38, 0.46
Tasks: 134 total, 1 running, 133 sleeping, 0 stopped, 0 zombie
Cpu0 : 14.7%us, 0.0%sy, 0.0%ni, 85.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 15.0%us, 0.0%sy, 0.0%ni, 85.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 14.7%us, 0.0%sy, 0.0%ni, 85.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7956116k total, 1250208k used, 6705908k free, 449068k buffers
Swap: 8294396k total, 0k used, 8294396k free, 401092k cached

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 2451 doug 20 0 4156 92 0 S 15 0.0 68:32.59 waiter
 2452 doug 20 0 4156 92 0 S 15 0.0 68:33.63 waiter
 2453 doug 20 0 4156 92 0 S 15 0.0 68:32.48 waiter
    1 root 20 0 24420 2404 1344 S 0 0.0 0:00.88 init
    2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
    3 root 20 0 0 0 0 S 0 0.0 0:00.14 ksoftirqd/0
    4 root 20 0 0 0 0 S 0 0.0 0:00.04 kworker/0:0
     ...

doug@s15:~/temp-3.5rc7/linux-3.5-rc7/kernel/sched$ uname -a
Linux s15 3.5.0-030500rc7-generic #201207142035 SMP Sun Jul 15 00:35:57 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Sudhir Khanger (donniezazen) wrote :

Hi doug,

I am not using Ubuntu 12.04 but Arch Linux. Load average is significantly down.

[donnie@arch ~]$ uptime
 09:37:29 up 6:44, 1 user, load average: 0.20, 0.16, 0.13

[donnie@arch ~]$ uname -a
Linux arch 3.5.0-rc7-mainline #1 SMP PREEMPT Sun Jul 15 07:24:17 UTC 2012 x86_64 GNU/Linux

BUT temperature and power usage is still horrible for any 3.4 series onward kernel to be usable. My idle temperature is at least 10 degrees higher than what I get on 3.3 series. Power usage jumps between to 15-20W compared to 3.3's less than 10W.

Quinta Helmer (qhelmer) wrote :

I am willing to test the patch mentioned in post #80, but I have no idea how to install it (without risking to break everything because of doing something wrong). My load on idle is aroud 15 (4 cores).

Guy (guy-b) wrote :

After this morning kernel update to 3.2.0-27-generic, still high load 1.20 1.22 1.14.

Doug Smythies (dsmythies) wrote :

Hi Guy: Thanks for your attention to this one. No, this patch has not migrated to Ubuntu yet. At the upstream level (kernel.org) people have been testing the patch backported to the 3.2 series kernels. I think (but am not sure) that the patch will be reviewed for possible inclusion in kernel 3.2.24. After that it would eventually be included in an Ubuntu update (3.2.0-XX).

Quinta: A load of 15 on a 4 core system, is some other problem. I don't think this patch will fix your troubles.

Guy (guy-b) wrote :

Thank you Doug for the informations. I see, and problably the only way to include the patch in the current ubuntu kernel would be to compile the kernel with the patch, isn't it?

Quinta Helmer (qhelmer) wrote :

The reboot after yesterday's kernel update seems to have fixed my problem (load is now below 1). I am glad about that; trying unreleased kernel patches on a machine I need for work is a bit scary.

luca (llucax) wrote :

Seems to be fixed in 3.2.0-29.45 for me, or at least improved a lot, after booting, without opening any application, after some minutes the load dropped to ~0.15. With the system using about 25% of one core (I have 2 cores, 4 threads), the load is between 0.3 and 0.5, which seems fairly reasonable.

Doug Smythies (dsmythies) wrote :

Yes, the patch was backported to upstream kernel 3.2.24, which in tern was included in Ubuntu as mentioned. See also bug #1029431 (search for "load-ave") and https://launchpad.net/ubuntu/precise/+source/linux/3.2.0-29.46 (seach for "load-ave")
I don't think it is generally released yet.

Douglas Leeder (ubuntu-leeder) wrote :

Any idea when it will go for general release? I'm looking forward to having useful load averages again.

Guy (guy-b) wrote :

After this morning kernel update to 3.2.0-29-generic, the load still not perfect but much better: 0.36 0.40 0.31 with chromium and gedit open.

Doug Smythies (dsmythies) wrote :

Guy: How do you know the reported load average is not perfect? What is your actual load average?

The method for determining reported load average is extremely undersampled. Accuracy expectations should be within about 20% for the 15 minute reported load average. The 1 minute reported load average will be all over the place.

The older kernels, from before this bug report, had broken reported load averages reporting too low. Therfore you can not compare to them, as for the same conditions that gave the complaints of this bug report they were wrong.

Guy (guy-b) wrote :

You're right Doug, in fact the point that I used to compare is in reality only my experience with the previous release but not an objective way. With Gnome 2 it was near by 0.02 - 0.08 (15 min). Sincerely I was allway surprised by so low charge and I have not the knowlege and the utils to confirm what is wrong or right.

Right now my load are: 0.07 0.18 0.21 which after your explanations seems for me to be right and more realistic then the 0.02....

Anyway, once again, thank you for your efforts in solving this bug and your kindness!

sds (sds-gnu) wrote :

When the computer is "idle", load should be 0. This is the source of its usefulness.
I.e., non-0 loadavg means that
either the comp is thrashing,
or backup is running,
or R is partitioning a graph,
or emacs is being compiled
or a file db is being rebuilt
or ....
"normal" ui operation (editing a file, browsing the web &c) should not affect loadavg.
this is precisely the usefulness of loadavg: it tells me that the computer is busy,
and, unless I know it is busy because I told it to do something,
I should investigate: maybe a virus is trying to break into the NSA :-)
right now my loadavg is at 1.3 while only one single-threaded computation is running.
This is wrong.

It would be nice if there were a tool which would tell me that the loadavg consists of.
e.g.: loadavg=4.2 = 1 from R + 1.5 from Matlab + 1.0 from gcc + 0.7 from a no-longer-available NFS-mounted FS.
it is alleged that "atop" is supposed to be able to do that,
but even its adepts have not been able to produce such a split reliably.

Gonzalo Palarea (gpalarea) wrote :

3.2.0-29 has fixed it for me!

Sudhir Khanger (donniezazen) wrote :

I am wondering if kernel developers are aware of increased power consumption or if this bug only concerns load average. I am stuck with 3.3 or pre-3.4 kernel. Please let me know what information is required to sort this out. Thanks.

With the last kernel update (to 3.2.0-29-generic-pae) problem is fixed for me. System monitor chart shows it: http://susepaste.org/view/raw/97494002

sds (sds-gnu) wrote :

no good:
3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64
load average: 0.49, 0.66, 0.46

Doug Smythies (dsmythies) wrote :

Sudhir: I do not know anything about real increased power consumption. I only know about the reported load averages issue. However, I have observed many comments about real increased power consumption on other launchpad bug reports and various forums threads.

sds: You claim the kernel 3.2.0-29-46 is still reporting incorrect load averages. How do you know? For the example you gave, what is the real load average? If, as per your post #93, your expectation is that the reported load average be 0, I think that is incorrect for a desktop type ubuntu computer, as they have a lot more stuff running. Myself, I only use ubuntu server edition, and my background "idle" reported load average is 0, expect for the occasional cron related blib.

sds (sds-gnu) wrote :

load average: 0.37, 0.35, 0.41
I am not running anything, so I expect it to be 0.

"they have a lot more stuff running"
what is the full list of the "stuff" running this very second?
when I see non-0 loadavg, I want to be able to determine,
quickly and painlessly, the exact list of applications which contributed to the non-0 loadavg.

Doug Smythies (dsmythies) wrote :

Run top to see what is running and what CPU it is using and total CPU it has used over time.
I do not know the list of tasks that typically run with the desktop edition, as I don't use it.
If you are not doing anything with the computer, it still has operating system overhead to deal with. The load average code runs in the heart of the kernel, and it doesn't know anything other than if a CPU is in use or not busy at the 5 second sample time.

sds (sds-gnu) wrote :

top does __NOT__ explain the loadavg (if it did, I would not have complained):
non-0 CPU users account to 10-15% at most.
(this, however, is absurd in itself! why does compiz take 3-4% of CPU on a 2.7GHz system
when twm/fvwm/gwm/scwm never took more than 1% on a 200MHz system?!)

moreover, top __CANNOT__ always explain loadavg.
(e.g., `ls /mnt/foo` will be at 1 loadavg if /mnt/foo is NFS-mounted and the server is down).

"The load average code runs in the heart of the kernel, and it doesn't know anything other than if a CPU is in use or not busy at the 5 second sample time."

first of all, loadavg is not about CPU:

"the number of processes in the system run queue averaged over various periods of time."

there should be a way to get __WHICH__ processes occupied the the system run queue, in addition to _how many_ of them.
otherwise this loadavg metric is next to useless to the users,
even if the developers are convinced that it is "correct" in some way.

Doug Smythies (dsmythies) wrote :

sds: "first of all, loadavg is not about CPU:"
Yes, thanks for pointing that out. The issues were in the is the cpu busy or idle counting portion of the code, but the overall load average includes pending processes and I/O wait.

And yes, top does not always explain loadavg. Sorry for misleading this thread. There are many senarios where it will not. I have just found it to be, generally, useful to try to get an approximation of the actual load for otherwise idle desktop systems on other bug reports or forum threads (when the poster has provided the information).

Let me just repeat an off-topic comment:

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 2295 sds 20 0 1486m 52m 14m S 9 0.7 43:10.59 compiz
 2398 sds 20 0 527m 21m 5632 S 3 0.3 24:08.33 unity-panel-ser
 1340 root 20 0 281m 24m 4468 S 5 0.3 20:33.28 Xorg
 2312 sds 20 0 360m 4084 2676 S 0 0.1 11:11.56 pulseaudio
 2400 sds 20 0 617m 5348 2084 S 1 0.1 8:43.45 hud-service
 2267 sds 20 0 27976 2852 612 S 1 0.0 6:23.61 dbus-daemon
 6551 sds 20 0 360m 66m 7868 S 1 0.8 6:04.17 emacs

Consuming more TIME/CPU than the editor is an outrage.

I understand X & pulseaudio, but none of the others (compiz, unity, hud,
dbus) have any right to be so resource hungry.

Is there already a bug where I can vent my frustration with this? :-)

--
Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000
http://www.childpsy.net/ http://thereligionofpeace.com http://www.memritv.org
http://americancensorship.org http://truepeace.org http://pmw.org.il
Bus error -- please leave by the rear door.

Doug Smythies (dsmythies) wrote :
Download full text (4.7 KiB)

sds: Perhaps you could use "vmstat" to check your reported load average. Your system was reporting 0.41 (post #99), but only 0.20 is accounted for in your post above.

Example: I have 3 processes running at a load of 0.44 each (1.32 total (the 5 and 15 minute reported averages have not had time to settle in the below capture)):

top - 14:11:58 up 4 days, 6:29, 4 users, load average: 1.31, 0.80, 0.40
Tasks: 141 total, 4 running, 137 sleeping, 0 stopped, 0 zombie
Cpu(s): 17.2%us, 0.0%sy, 0.0%ni, 82.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7956412k total, 1694448k used, 6261964k free, 669132k buffers
Swap: 8294396k total, 0k used, 8294396k free, 505488k cached

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20631 doug 20 0 4156 92 0 R 44 0.0 3:05.54 waiter
20632 doug 20 0 4156 92 0 R 44 0.0 3:05.51 waiter
20633 doug 20 0 4156 92 0 R 44 0.0 3:05.48 waiter
  255 root 20 0 0 0 0 S 0 0.0 2:43.92 kworker/6:1
 1798 root 20 0 15972 692 512 S 0 0.0 0:45.75 irqbalance
 1641 mysql 20 0 480m 41m 6984 S 0 0.5 0:42.53 mysqld
 1006 syslog 20 0 243m 1440 1136 S 0 0.0 0:12.59 rsyslogd
  100 root 20 0 0 0 0 S 0 0.0 0:06.54 kworker/1:1
  101 root 20 0 0 0 0 S 0 0.0 0:06.01 kworker/3:1

and vmstat reports:

doug@s15:~$ vmstat -n 10 50
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r b swpd free buff cache si so bi bo in cs us sy id wa
 1 0 0 6262632 669060 505488 0 0 0 0 3 4 0 0 100 0
 0 0 0 6262880 669060 505488 0 0 0 5 32 80 0 0 100 0
 0 0 0 6262716 669060 505488 0 0 0 0 542 749 13 0 87 0
 0 0 0 6262584 669060 505488 0 0 0 0 657 947 17 0 83 0
 0 0 0 6262536 669064 505484 0 0 0 5 667 1032 17 0 82 0
 2 0 0 6262536 669064 505488 0 0 0 0 646 878 17 0 83 0
 0 0 0 6262536 669064 505488 0 0 0 0 660 921 17 0 83 0
 0 0 0 6262504 669068 505488 0 0 0 3 651 921 17 0 83 0
 0 0 0 6262512 669068 505488 0 0 0 0 635 878 17 0 83 0
 3 0 0 6262568 669068 505488 0 0 0 0 649 935 17 0 83 0
 3 0 0 6262464 669068 505488 0 0 0 0 653 878 17 0 83 0
 3 0 0 6262464 669068 505488 0 0 0 0 656 886 17 0 83 0
 3 0 0 6262464 669068 505488 0 0 0 0 643 933 17 0 83 0
 3 0 0 6262464 669076 505488 0 0 0 1 653 900 17 0 83 0
 3 0 0 6262464 669076 505488 0 0 0 0 632 882 17 0 83 0
 0 0 0 6262464 669084 505488 0 0 0 4 653 891 17 0 83 0
 0 0 0 6262520 669084 505488 0 0 0 4 657 867 17 0 83 0
 0 0 0 6262528 669084 505488 0 0 0 0 655 882 17 0 83 0
 3 0 0 6262528 669088 505488 0 0 0 3 639 903 17 0 83 0
 0 0 0 6262528 669096 505488 0 0 0 1 649 892 17 0 83 0
 1 0 ...

Read more...

Guy (guy-b) wrote :

Hi,

With today's kernel update: 3.2.0-32.51 it looks much better: 0.04 0.07 0.08 after 2-3 idle minutes with few open softs.

Doug Smythies (dsmythies) wrote :

As of early August, this bug report can be set to "fix released". I will do so now.

Changed in linux (Ubuntu):
status: Triaged → Fix Released
Displaying first 40 and last 40 comments. View all 106 comments or add a comment.