100% CPU usage with "initctl emit plymouth-ready" process

[anatoly techtonik]

root 977 99.1 2.7 98668 93964 ? R 12:59 453:31 initctl emit plymouth-ready

this makes my CPU very hot, I don't know how to debug deeper. Any ideas?

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: upstart 1.10-0ubuntu7
ProcVersionSignature: Ubuntu 3.11.0-13.20-generic 3.11.6
Uname: Linux 3.11.0-13-generic i686
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu2.1
Architecture: i386
Date: Wed Nov 27 20:46:04 2013
InstallationDate: Installed on 2012-03-12 (625 days ago)
InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Beta i386 (20120301)
MarkForUpload: True
ProcKernelCmdline: BOOT_IMAGE=/boot/vmlinuz-3.11.0-13-generic root=UUID=d0d8bf63-cf4c-479f-9091-1f14156a902f ro quiet splash
SourcePackage: upstart
UpgradeStatus: Upgraded to saucy on 2013-10-21 (37 days ago)
UpstartBugCategory: System
UpstartRunningSessionVersion: init (upstart 1.10)
UpstartRunningSystemVersion: init (upstart 1.10)
modified.conffile..etc.at.deny: [inaccessible: [Errno 13] Permission denied: '/etc/at.deny']
modified.conffile..etc.init.d.networking: [modified]
mtime.conffile..etc.init.d.networking: 2013-08-27T01:19:11
upstart.upstart-file-bridge.log:
 Job got added /com/ubuntu/Upstart/jobs/unicast_2dlocal_2davahi
 Job got added /com/ubuntu/Upstart/jobs/update_2dnotifier_2dcrash
 Job got added /com/ubuntu/Upstart/jobs/update_2dnotifier_2drelease

[James Hunt (jamesodhunt)]

Hi Anatoly - thanks for reporting this issue. Presumably, lightdm started correctly before you noticed this issue?

If the process is still running, please do:

sudo cat /proc/$(pidof initctl)/stack > /tmp/initctl.stack

... and attach "/tmp/initctl.stack" to this bug. Then, to stop initctl (by forcibly crashing it, which might provide some more clues):

sudo kill -ABRT $(pidof initctl)

This second command should cause an apport window to pop up. Just follow the prompts and raise a new bug, then reference that new bug here.

Thank you!

[anatoly techtonik (techtonik)]

I found some glue on the net that `kill -9` helps, so I did this and CPU is quiet now. I believe that issue persisted between reboots before that. So I am closing this issue for now, but if it pop up again, I'll be more prepared thanks to your instructions.

[James Hunt (jamesodhunt)]

Hi Anatoly - do you now have a /var/crash/_sbin_initctl.0.crash file? If so, please attach it to this bug so we can investigate.

Not that my instructions in comment #2 would have both quietened your CPU and provided us with the required debug we need to investigate this issue further.

[anatoly techtonik (techtonik)]

My speaker was beeping, so when the night came I had to do something about it. I took a shotgun of less known chinese company "sudo" and killed it. Nine times in a row.

Today I investigated the crime scene. Unfortunately there is no variable evidence in crash site. Only these:

1322162 Nov 27 20:10 _opt_google_chrome_chrome.1000.crash
22391528 Nov 25 18:04 _usr_bin_compiz.1000.crash
2290032 Nov 27 12:58 _usr_bin_Xorg.0.crash

Could the reason be installed https://github.com/lwfinger/rtl8188eu driver?

[Garth Hitchens (ghitchens)]

I have the same problem... ps reports....
root 1093 64.7 4.2 78016 75796 ? R 2013 23219:55 initctl emit plymouth-ready

After doing sudo cat /proc/$(pidof initctl)/stack > /tmp/initctl.stack, I get...

garth@hat:~$ cat /tmp/initctl.stack
[<c1637936>] work_resched+0x5/0x20
[<ffffffff>] 0xffffffff
garth@hat:~$

I was logged in via remote ssh so the kill -ABRT didn't allow me to submit another bug...

Fully up-to-date Ubuntu 13.10. Problem may have started during a recent apt-get upgrade, Im' not sure.

[Garth Hitchens (ghitchens)]

Oh, one more thing, there is nothing in /var/crash after the above other than a 0 byte .lock file

[Matthew Eaton (meaton)]

Hi guys,

I ran into this bug on an Ubuntu 14.04.1 system (see attached screenshot).

I ran kill -ABRT and submitted the crash report but I don't know what happened to it? I have the files in /var/crash if you need them.

[Matthew Eaton (meaton)]

Attaching my /var/crash files just in case...

[Daniel Défago (daniel-defagordi)]

Affect also on Ubunutu 14.1 64bit

sudo cat /proc/$(pidof initctl)/stack > /tmp/initctl.stack

$ cat /tmp/initctl.stack
[<ffffffffffffffff>] 0xffffffffffffffff

[Daniel Gonzalez (gonvaled)]

Affects also my system: laptop Acer Aspire E17 E5-771G-7868 (with a replaced HD, by an SSD disk, in case it matters). It happens quite often (several times a week?):

» uname -a
Linux polyphemus.wavilon.net 3.13.0-45-generic #74-Ubuntu SMP Tue Jan 13 19:36:28 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

» lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

» sudo cat /proc/1195/stack
[<ffffffffffffffff>] 0xffffffffffffffff

[Jean-Paul Calderone (exarkun)]

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty
$ cat /tmp/initctl.stack
[<ffffffff817298bc>] retint_careful+0x14/0x32
[<ffffffffffffffff>] 0xffffffffffffffff

[Jean-Paul Calderone (exarkun)]

Happened again, same system. Stack is even shorter this time:

$ cat /tmp/initctl.stack
[<ffffffffffffffff>] 0xffffffffffffffff

[Bryce Harrington (bryce)]

Possibly dupe with #1173915?

I too am seeing 100% cpu on initctl, after a nearly three month uptime on 14.04:

Linux dorset.bryceharrington.org 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

# cat /proc/1327/stack
[<ffffffffffffffff>] 0xffffffffffffffff

# uptime
 17:24:15 up 87 days, 13:54, 9 users, load average: 1.00, 1.01, 1.05

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

# ls -l /proc/1327/fd
total 0
lrwx------ 1 root root 64 Feb 11 02:39 0 -> /dev/null
lrwx------ 1 root root 64 Feb 11 02:39 1 -> /dev/pts/24
lrwx------ 1 root root 64 Feb 11 02:39 2 -> /dev/pts/24
lrwx------ 1 root root 64 Feb 11 02:39 3 -> socket:[9674]

# lsof | grep 9674
lsof: WARNING: can't stat() nfs4 file system /srv/Backup
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
initctl 1327 root 3u unix 0xffff8807ef6f2680 0t0 9674 socket

# strace -p 9674
...
poll([{fd=3, events=POLLIN}], 1, 1023162120) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162112) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162103) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162095) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162087) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162078) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN}], 1, 1023162070) = 1 ([{fd=3, revents=POLLIN}])
...

[kevin.liu (kevin-liu)]

Hi,

I am consistently watching this issue from my side with Ubuntu 14.04.1 LTS after 24 hours running, CPU is fully occupied by initctl emit plymouth-ready process.

This bug seems not responsive from Canonical side, anyone can tell me how could we WA or fix this issue.

[kRyTiCaL (krytical)]

I experience the same issue, on Ubuntu 14.04.4

I just wanted to add that I only noticed this after I had installed quite a few 32 bit packages on my 64 bit OS.

[Felix Moreno (info-justdust)]

Same problem in Ubuntu 16.10...

[Haim Daniel (hdaniel)]

Ditto - Ubuntu 14.04.05

[Peter Schüller (schueller-p)]

Same here on Ubuntu 14.04 after installing google chrome via download from google website and software manager (not manually adding ppa).

[Peter Schüller (schueller-p)]

Sorry, cannot come from google chrome because I installed < 48 h ago and the process already has 344 CPU hours.

Same strace as #14 here, and stack is sometimes

# cat /proc/1898/stack
[<ffffffffffffffff>] 0xffffffffffffffff

sometimes

# cat /proc/1898/stack
[<ffffffff8173587c>] retint_careful+0x14/0x32
[<ffffffffffffffff>] 0xffffffffffffffff

next to initctl emit plymouth-ready i also have /sbin/initctl emit indicator-services-start

but this is sleeping and has stack
# cat /proc/4370/stack
[<ffffffff811d6579>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d7b68>] do_sys_poll+0x428/0x540
[<ffffffff811d7d55>] SyS_poll+0x65/0x100
[<ffffffff8173dddd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff

[Peter Schüller (schueller-p)]

I now did kill -ABRT as described in #2 and the result is https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1664838.