dbus invoked while in recovery mode

[Removed by request]

I'm using Ubuntu 14.10 dev with thermald 1.2-1 and if the system is started in the recovery mode it is not possible to unmount /dev/sda1 because thermald is running. Interestingly lsof hasn't even showed that something has a file descriptor on /dev/sda1 open but executing "stop thermald" has worked as a workaround.
---
ApportVersion: 2.14.7-0ubuntu2
Architecture: amd64
DistroRelease: Ubuntu 14.10
EcryptfsInUse: Yes
NonfreeKernelModules: fglrx
Package: upstart 1.13.2-0ubuntu1
PackageArchitecture: amd64
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdline: BOOT_IMAGE=/boot/vmlinuz-3.16.0-18-generic root=UUID=af27feae-4c9e-4b5f-a1e1-900c0e71c5cb ro elevator=cfq
ProcVersionSignature: Ubuntu 3.16.0-18.25-generic 3.16.3
Tags: utopic
Uname: Linux 3.16.0-18-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UpstartBugCategory: System
UpstartRunningSystemVersion: init (upstart 1.13.2)
UserGroups:

_MarkForUpload: True
modified.conffile..etc.lxdm.lxdm.conf: [modified]
mtime.conffile..etc.init.control.alt.delete.conf: 2012-05-05T17:10:11.434470
mtime.conffile..etc.init.tty2.conf: 2014-05-29T05:14:16.791093
mtime.conffile..etc.init.tty3.conf: 2012-05-05T17:10:49.238469
mtime.conffile..etc.init.tty4.conf: 2012-05-05T17:10:58.066468
mtime.conffile..etc.init.tty5.conf: 2012-05-05T17:11:02.938468
mtime.conffile..etc.init.tty6.conf: 2012-05-05T17:11:09.442468
mtime.conffile..etc.lxdm.lxdm.conf: 2012-12-23T19:04:26.948844

[Colin Ian King (colin-king)]

I'm unabled to reproduce this, thermald is not running when I booted into recovery mode on 14.10. Can you reproduce this every time? If so, can you report which processes are running on the system.

[Removed by request (removed3425744)]

> Can you reproduce this every time?

I forgot to say that this happens only on my laptop but not on my desktop machine (but it is always reproducible on my laptop).

> If so, can you report which processes are running on the system.

/sbin/init
upstart-udev-bridge --daemon
/lib/systemd/systemd-udevd --daemon
dbus-daemon --system --fork
thermald --no-daemon --dbus-enable

[Colin Ian King (colin-king)]

OK, so it's starting because dbus has started, where as on my test machines dbus is not running in recovery mode. I wonder why that is so.

[Colin Ian King (colin-king)]

Can you inform me what the following shows when in recovery mode:

ls -al /proc/$(pidof thermald)/fd

Thanks

[Removed by request (removed3425744)]

This is the output:

total 0
dr-x------ 2 root root 0 Jul 28 19:22 .
dr-xr-xr-x 9 root root 0 Jul 28 19:21 ..
lrwx------ 1 root root 64 Jul 28 19:22 0 -> /dev/null
lrwx------ 1 root root 64 Jul 28 19:22 1 -> /dev/pts/8
lrwx------ 1 root root 64 Jul 28 19:22 2 -> /dev/pts/8
lrwx------ 1 root root 64 Jul 28 19:22 3 -> anon_inode:[eventfd]
lrwx------ 1 root root 64 Jul 28 19:22 4 -> socket:[11007]
lr-x------ 1 root root 64 Jul 28 19:22 5 -> pipe:[11023]
l-wx------ 1 root root 64 Jul 28 19:22 6 -> pipe:[11023]
lrwx------ 1 root root 64 Jul 28 19:22 7 -> socket:[11035]

[Colin Ian King (colin-king)]

Thermald has been updated since this bug was reported. Is it still an issue?

[Removed by request (removed3425744)]

Yes, the issue does still exist with thermald 1.3-4.

[Colin Ian King (colin-king)]

Just for clarification, what is /dev/sda1 mounted as?

[Colin Ian King (colin-king)]

I'm inclined to think that in single user and/or recovery mode, dbus is generally not running, and thermald depends on this. So perhaps we should only start thermald on run levels 2,3,4,5. Can you change the /etc/init/thermal.conf file start line to be:

start on runlevel [2345] and started dbus

And let me know if that helps

[Removed by request (removed3425744)]

> start on runlevel [2345] and started dbus

This does indeed prevent thermald from running in recovery mode (and I'm able to unmount /dev/sda1) but I don't think this is the way to go because:

> OK, so it's starting because dbus has started, where as on my test machines dbus is not running in recovery mode. I wonder why that is so.

I think we should try to figure out why dbus starts in recovery mode as this seems to be incorrect too (and it would also automatically solve the thermald/unmounting issue).

[Colin Ian King (colin-king)]

Indeed, this does seem to be an upstart/init config issue rather than a thermald issue per se. I'll add that to the "affects" part of the bug report.

[Colin Ian King (colin-king)]

BTW, I'm going to slip the run level changes into thermald to address another thermald bug.

[Colin Ian King (colin-king)]

@James, any idea why dbus is starting?

[James Hunt (jamesodhunt)]

I too am unable to recreate this problem - dbus should certainly not be running in recovery mode because dbus is 'start on local-filesystems', but that event will not have been emitted since mountall is not run in recovery mode.

Can you confirm you are entering recovery mode from the grub menu (Advanced options -> recovery)?

Can you also check the following:

1) Enter recovery mode.
2) Select the root shell option.
3) Run "set|grep UPSTART".

What you should see is:

UPSTART_EVENTS=recovery
UPSTART_INSTANCE=
UPSTART_JOB=friendly-recovery

Can you also check that you don't have an /etc/init/dbus.override and that your /etc/init/dbus.conf is unmodified - running 'apport-collect -p upstart 1348784' will do this for you.

Attaching the output of 'ps -efwww' and 'initctl list' from recovery mode could also be useful.

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

apport information

[Removed by request (removed3425744)]

> Can you confirm you are entering recovery mode from the grub menu (Advanced options -> recovery)?

Yes.

> Can you also check the following:
>
> 1) Enter recovery mode.
> 2) Select the root shell option.
> 3) Run "set|grep UPSTART".

I'm assuming you are meaning with "root shell option" the login question for root that automatically appears. Here is the output:

UPSTART_EVENTS=runlevel
UPSTART_INSTANCE=
UPSTART_JOB=rcS

> Can you also check that you don't have an /etc/init/dbus.override and that your /etc/init/dbus.conf is unmodified - running > 'apport-collect -p upstart 1348784' will do this for you.

/etc/init/dbus.override doesn't exist and I have reinstalled dbus to make sure that there are no changes on /etc/init/dbus.conf.

> Attaching the output of 'ps -efwww' and 'initctl list' from recovery mode could also be useful.

ps -efwww:

UID PID PPID C STIME TTY TIME CMD
root 1 0 0 13:20 ? 00:00:01 /sbin/init
root 2 0 0 13:20 ? 00:00:00 [kthreadd]
root 3 2 0 13:20 ? 00:00:00 [ksoftirqd/0]
root 4 2 0 13:20 ? 00:00:00 [kworker/0:0]
root 5 2 0 13:20 ? 00:00:00 [kworker/0:0H]
root 6 2 0 13:20 ? 00:00:00 [kworker/u8:0]
root 7 2 0 13:20 ? 00:00:00 [rcu_sched]
root 8 2 0 13:20 ? 00:00:00 [rcuos/0]
root 9 2 0 13:20 ? 00:00:00 [rcuos/1]
root 10 2 0 13:20 ? 00:00:00 [rcuos/2]
root 11 2 0 13:20 ? 00:00:00 [rcuos/3]
root 12 2 0 13:20 ? 00:00:00 [rcu_bh]
root 13 2 0 13:20 ? 00:00:00 [rcuob/0]
root 14 2 0 13:20 ? 00:00:00 [rcuob/1]
root 15 2 0 13:20 ? 00:00:00 [rcuob/2]
root 16 2 0 13:20 ? 00:00:00 [rcuob/3]
root 17 2 0 13:20 ? 00:00:00 [migration/0]
root 18 2 0 13:20 ? 00:00:00 [watchdog/0]
root 19 2 0 13:20 ? 00:00:00 [watchdog/1]
root 20 2 0 13:20 ? 00:00:00 [migration/1]
root 21 2 0 13:20 ? 00:00:00 [ksoftirqd/1]
root 22 2 0 13:20 ? 00:00:00 [kworker/1:0]
root 23 2 0 13:20 ? 00:00:00 [kworker/1:0H]
root 24 2 0 13:20 ? 00:00:00 [khelper]
root 25 2 0 13:20 ? 00:00:00 [kdevtmpfs]
root 26 2 0 13:20 ? 00:00:00 [netns]
root 27 2 0 13:20 ? 00:00:00 [khungtaskd]
root 28 2 0 13:20 ? 00:00:00 [writeback]
root 29 2 0 13:20 ? 00:00:00 [ksmd]
root 30 2 0 13:20 ? 00:00:00 [khugepaged]
root 31 2 0 13:20 ? 00:00:00 [crypto]
root 32 2 0 13:20 ? 00:00:00 [kintegrityd]
root 33 2 0 13:20 ? 00:00:00 [bioset]
root 34 2 0 13:20 ? 00:00:00 [kblockd]
root 35 2 0 13:20 ? 00:00:00 [ata_sff]
root 36 2 0 13:20 ? 00:00:00 [khubd]
ro...

[James Hunt (jamesodhunt)]

> UPSTART_EVENTS=runlevel
> UPSTART_INSTANCE=
> UPSTART_JOB=rcS
This is the cause of your problem - you are not actually in recovery mode. What is your /proc/cmdline? Have you modified /etc/default/grub or any of the initramfs configuration?

[Removed by request (removed3425744)]

> What is your /proc/cmdline?

BOOT_IMAGE=/boot/vmlinuz-3.16.0-18-generic root=UUID=af27feae-4c9e-4b5f-a1e1-900c0e71c5cb ro single nomodeset

> Have you modified /etc/default/grub or any of the initramfs configuration?

I think not that I have changed something of initramfs but there are some changes in /etc/default/grub:

- I have replaced "GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"" with "GRUB_CMDLINE_LINUX_DEFAULT="elevator=cfq"".
- I have added the line "GRUB_GFXPAYLOAD_LINUX=1366x768-24" (but I'm seeing the higher resolution only if I'm booting normally - in the recovery mode it is still very low (it looks like 640x480)).

[James Hunt (jamesodhunt)]

Your cmdline shows that you are actually in single-user-mode, not recovery mode. On a standard system, selecting "recovery mode" from the grub boot menu will add 'recovery' to the boot command line, which will trigger recovery mode to be started.

[Removed by request (removed3425744)]

On booting I'm holding shift to open the GRUB menu and selecting "Erweiterte Optionen für Ubuntu" (translated probably "Advanced options for Ubuntu") and then "Ubuntu, with Linux 3.16.0-18-generic (recovery mode)". It seems something is preventing here that the recovery mode is being entered.

[James Hunt (jamesodhunt)]

With your cursor on "Ubuntu, with Linux 3.16.0-18-generic (recovery mode)", press 'e' and you should see 'recovery' specified somewhere on the line that starts 'linux'. If not, your grub config is incorrect for some reason.

To test if recovery mode works, again having pressed 'e', remove 'single' and replace it with 'recovery' then CTRL+x or F10 to boot into recovery mode.

[Removed by request (removed3425744)]

> With your cursor on "Ubuntu, with Linux 3.16.0-18-generic (recovery mode)", press 'e' and you should see 'recovery' specified somewhere on the line that starts 'linux'.

I have made a look on my main computer and my laptop and I'm seeing on both systems there "single".

> To test if recovery mode works, again having pressed 'e', remove 'single' and replace it with 'recovery' then CTRL+x or F10 to boot into recovery mode.

On both systems it boots until "Begin: Running /scripts/init-bottom ... done." and then nothing happens anymore.

[James Hunt (jamesodhunt)]

It sounds like you may have modified files below /etc/initramfs-tools/ and/or /usr/share/initramfs-tools/, or created additional scripts that are somehow modifying the boot path. Further, I think you need to understand why single is being specified on the boot command-line as that is non-standard.

I would try installing a fresh 14.10 system (maybe in a VM) and comparing those directories in your fresh environment with your main system environment.

[Removed by request (removed3425744)]

> It sounds like you may have modified files below /etc/initramfs-tools/ and/or /usr/share/initramfs-tools/, or created additional scripts that are somehow modifying the boot path.

I have os-prober installed but removing it and updating initramfs and grub doesn't solve the issue.

> I would try installing a fresh 14.10 system (maybe in a VM) and comparing those directories in your fresh environment with your main system environment.

I have now installed the daily ISO of Ubuntu 14.10 dev but have same troubles with it. After entering the password in the login window nothing happens. The login window simply gets stuck but I'm still able to use the mouse and the few menu entries (like to reboot the system). Also it seems that my USB storage device isn't working with VirtualBox so I can't copy the related files for comparing.

[Removed by request (removed3425744)]

I have now made a look which files in /etc/initramfs-tools and /usr/share/initramfs-tools belongs to which package. Here is the result:

sworddragon@ubuntu:~$ (find /etc/initramfs-tools -type f; find /usr/share/initramfs-tools -type f) | xargs dpkg -S
initramfs-tools: /etc/initramfs-tools/initramfs.conf
dpkg-query: no path found matching pattern /etc/initramfs-tools/modules
dpkg-query: no path found matching pattern /etc/initramfs-tools/conf.d/resume
initramfs-tools: /etc/initramfs-tools/update-initramfs.conf
cryptsetup: /usr/share/initramfs-tools/conf-hooks.d/cryptsetup
mountall: /usr/share/initramfs-tools/hooks/mountall
kmod: /usr/share/initramfs-tools/hooks/kmod
cryptsetup: /usr/share/initramfs-tools/hooks/cryptkeyctl
cryptsetup: /usr/share/initramfs-tools/hooks/cryptgnupg
cryptsetup: /usr/share/initramfs-tools/hooks/cryptroot
plymouth: /usr/share/initramfs-tools/hooks/plymouth
keyboard-configuration: /usr/share/initramfs-tools/hooks/console_setup
initramfs-tools: /usr/share/initramfs-tools/hooks/framebuffer
cryptsetup: /usr/share/initramfs-tools/hooks/cryptpassdev
udev: /usr/share/initramfs-tools/hooks/udev
cryptsetup: /usr/share/initramfs-tools/hooks/cryptopenct
initramfs-tools: /usr/share/initramfs-tools/hooks/fixrtc
initramfs-tools: /usr/share/initramfs-tools/hooks/thermal
cryptsetup: /usr/share/initramfs-tools/hooks/cryptopensc
initramfs-tools: /usr/share/initramfs-tools/hooks/compcache
kbd: /usr/share/initramfs-tools/hooks/kbd
initramfs-tools: /usr/share/initramfs-tools/hooks/busybox
dmsetup: /usr/share/initramfs-tools/hooks/dmsetup
initramfs-tools: /usr/share/initramfs-tools/hooks/klibc
initramfs-tools: /usr/share/initramfs-tools/hook-functions
mountall: /usr/share/initramfs-tools/event-driven/upstart-jobs/mountall.conf
initramfs-tools: /usr/share/initramfs-tools/modules
initramfs-tools: /usr/share/initramfs-tools/scripts/nfs
initramfs-tools: /usr/share/initramfs-tools/scripts/local
cryptsetup: /usr/share/initramfs-tools/scripts/local-top/cryptroot
cryptsetup: /usr/share/initramfs-tools/scripts/local-top/cryptopensc
cryptsetup: /usr/share/initramfs-tools/scripts/local-bottom/cryptopensc
plymouth: /usr/share/initramfs-tools/scripts/init-bottom/plymouth
udev: /usr/share/initramfs-tools/scripts/init-bottom/udev
initramfs-tools: /usr/share/initramfs-tools/scripts/functions
plymouth: /usr/share/initramfs-tools/scripts/init-top/plymouth
initramfs-tools: /usr/share/initramfs-tools/scripts/init-top/keymap
keyboard-configuration: /usr/share/initramfs-tools/scripts/init-top/console_setup
initramfs-tools: /usr/share/initramfs-tools/scripts/init-top/framebuffer
udev: /usr/share/initramfs-tools/scripts/init-top/udev
initramfs-tools: /usr/share/initramfs-tools/scripts/init-top/blacklist
initramfs-tools: /usr/share/initramfs-tools/scripts/init-top/all_generic_ide
plymouth: /usr/share/initramfs-tools/scripts/panic/plymouth
initramfs-tools: /usr/share/initramfs-tools/scripts/panic/keymap
keyboard-configuration: /usr/share/initramfs-tools/scripts/panic/console_setup
initramfs-tools: /usr/share/initramfs-tools/scripts/local-premount/resume
initramfs-tools: /usr/share/initramfs-tools/scripts/local-premount/fixrtc
initramfs-tools: /usr/share/initramfs-t...

[Colin Ian King (colin-king)]

Did we conclude this is more of a misconfiguration rather than a thermald issue pe se, if so, I may remove thermald from the bug and rename it

[Removed by request (removed3425744)]

This seems to be plausible as thermald would not start if dbus would not start (which is the bug here (or probably a configuration issue)).