reboot hangs at 'Reached target Shutdown'

Bug #1464917 reported by Steve Langasek on 2015-06-13
412
This bug affects 85 people
Affects Status Importance Assigned to Milestone
systemd
Invalid
Undecided
Unassigned
systemd (Ubuntu)
High
Unassigned

Bug Description

This bug is resolved. Do not follow up to this bug. Shutdown bugs are specific to the software installed and its configuration. If you are experiencing a shutdown hang, please file a separate bug report and follow the debugging instructions described in the "Debugging boot/shutdown problems" section of /usr/share/doc/systemd/README.Debian.gz to check if there are any hanging jobs at shutdown. Capturing a screen photo of "journalctl -b" in the rescue shell might be enlightening.

[Original report]
I rebooted my 15.04 system, and found it hanging indefinitely at the plymouth shutdown screen. Hitting esc to reveal the console showed a normal set of shutdown messages, ending with 'Reached target Shutdown' (paraphrased from memory).

The system never rebooted on its own. I had to use SysRq to finish the shutdown.

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: systemd 219-7ubuntu6
ProcVersionSignature: Ubuntu 3.19.0-20.20-generic 3.19.8
Uname: Linux 3.19.0-20-generic x86_64
ApportVersion: 2.17.2-0ubuntu1.1
Architecture: amd64
CurrentDesktop: Unity
Date: Sat Jun 13 11:53:10 2015
InstallationDate: Installed on 2010-09-24 (1723 days ago)
InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64 (20100816.1)
MachineType: LENOVO 2306CTO
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.19.0-20-generic root=/dev/mapper/hostname-root ro quiet splash vt.handoff=7
SourcePackage: systemd
UpgradeStatus: Upgraded to vivid on 2014-12-06 (189 days ago)
dmi.bios.date: 10/25/2013
dmi.bios.vendor: LENOVO
dmi.bios.version: G2ET97WW (2.57 )
dmi.board.asset.tag: Not Available
dmi.board.name: 2306CTO
dmi.board.vendor: LENOVO
dmi.board.version: Not Defined
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvrG2ET97WW(2.57):bd10/25/2013:svnLENOVO:pn2306CTO:pvrThinkPadX230:rvnLENOVO:rn2306CTO:rvrNotDefined:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 2306CTO
dmi.product.version: ThinkPad X230
dmi.sys.vendor: LENOVO

Steve Langasek (vorlon) wrote :
tags: added: systemd-boot
Martin Pitt (pitti) wrote :

Your system isn't too happy because of bug 1452644. This is fixed in vivid-proposed, but this hasn't made it into -updates yet. This causes network mounts etc. to fail. That might be related to the shutdown problems. To get this out of the way, could you install the -proposed setserial and re-test?

Secondly, I suppose you actually used "reboot" (or the corresponding indicator action), and not "halt"?

Can you please run "sudo systemctl start debug-shell", then shut down/reboot, and when it hangs do Ctrl+Alt+F9 look at "systemctl list-jobs" (hanging jobs), or "dmesg" if you see any hanging jobs/errors there? Do you get some meaningful errors at the end of "journalctl"? You could try booting without "quiet splash" and with "debug" instead to increase verbosity what's going on, shut down, and capture what's being logged?

Changed in systemd (Ubuntu):
status: New → Incomplete
Launchpad Janitor (janitor) wrote :

[Expired for systemd (Ubuntu) because there has been no activity for 60 days.]

Changed in systemd (Ubuntu):
status: Incomplete → Expired
Niklas Keller (kelunik) on 2015-08-15
Changed in systemd (Ubuntu):
status: Expired → Confirmed
Niklas Keller (kelunik) wrote :

I can confirm that updating to vivid-proposed fixes that behavior.

Niklas Keller (kelunik) wrote :

Unfortunately, it only fixed it in the first reboot. The one now had that problem again, even with vivid-proposed.

Martin Pitt (pitti) on 2015-08-16
Changed in systemd (Ubuntu):
status: Confirmed → Incomplete
xtrchessreal (xtrchessreal) wrote :

I see this intermittently mostly after an internet session and I see at least 2 questions with total 5 upvotes on askubuntu.com concerning this hang on reboot or shut-down on 15.04 specifically. Most are hard booting after the hang. I found doing Ctrl+Alt+F1 followed by Ctrl+Alt+Del will continue the reboot. I will attempt to follow @Martin Pitt instructions and gain some log data if it is helpful to you. Please let me know if this is something you would like me to do. I referred to this bug#1464917 to those on askubuntu.com Just trying to help. :)

xtrchessreal (xtrchessreal) wrote :

So far I am unable to debug this. I need to some help to determine how. Is there a "shutdown log" I can view or post here. The problem is persistent now. Ctrl+Alt+F9 only worked once and I do not know how to capture this data. In every case I have to reboot using Sysrq REISUB and then immediately shutdown again to get laptop to shutdown properly. As I stated before it seems to only affect me when I have had an internet session.

kcdtv (kcdtv78) wrote :

same problem here with toshiba satellite C55 and Xubuntu 15.04 It happens just after a BIOS update (from windowzzzz)
I had some issues in the past fater updating BIOS, it generaly fucked my GRUB and my dual-boot
may it be related to X server?
 <code>
[ 14193.953] (II) UnloadModule: "evdev"
[ 14193.964] (WW) xf86CloseConsole: KDSETMODE failed: Input/output error
[ 14193.964] (WW) xf86CloseConsole: VT_GETMODE failed: Input/output error
[ 14193.964] (EE)
Fatal server error:
[ 14193.964] (EE) xf86CloseConsole: VT_ACTIVATE failed: Input/output error
[ 14193.965] (EE)
[ 14193.965] (EE)
Please consult the The X.Org Foundation support
  at http://wiki.x.org
 for help.
[ 14193.965] (EE) Please also check the log file at "/var/log/Xorg.1.log" for additional information.
[ 14193.965] (EE)
[ 14193.965] (EE)
[ 14193.965] (EE) Backtrace:
[ 14193.965] (EE) 0: /usr/bin/X (xorg_backtrace+0x56) [0x7f7c27653556]
[ 14193.965] (EE) 1: /usr/bin/X (0x7f7c274a0000+0x1b7749) [0x7f7c27657749]
[ 14193.965] (EE) 2: /lib/x86_64-linux-gnu/libc.so.6 (0x7f7c25166000+0x352f0) [0x7f7c2519b2f0]
[ 14193.965] (EE)
[ 14193.965] (EE) Segmentation fault at address 0x0
[ 14193.965] (EE)
FatalError re-entered, aborting
[ 14193.965] (EE) Caught signal 11 (Segmentation fault). Server aborting
[ 14193.965] (EE)
kcdtv@pr0fesoraBubbleVanAppletrudell:~$
</code>

libc.so.6 was installed no so long ago... could it be a problem with this library?

Matt Thomason (mthomason) wrote :

So, my 15.04 does exactly the same thing when I

sudo shutdown now -r

Like I have for years, leaving me with a server I have to run to and manually reboot.
However, after poking at the systemd docs I noticed it has its own shutdown+reboot command

sudo systemctl reboot (or sudo systemctl shutdown to just shut down)

Which doesn't hang with the dreaded "reached target shutdown" ad infinitum.

So, are people with problems all using the old "shutdown" command, or are they using the new "systemctl shutdown"

Matt Thomason (mthomason) wrote :

Ugh, sorry, I should have said

sudo systemctl poweroff - not sudo systemctl shutdown

xtrchessreal (xtrchessreal) wrote :

I can't speak for all but I have noticed a few including me using the shutdown from the drop down at top right. Interesting note for my case - since I was not able to get debug info I decided to try a different browser so I switched from Chromium to Firefox and oddly have not had the problem since that switch. Firefox is just a temp for now, I am going to switch back and see if the problem persists with Chromium. If so Maybe Chromium has a background app that is hanging. I have not installed ANY of the Ubuntu extra feature Web Apps in Firefox where as Chromium has all my google apps integrated.

How to capture the shutdown/poweroff dump data for debug?

Dagur (dagurp) wrote :

Still happens in 15.10

Tőkés Attila (tokes-atti) wrote :

I can confirm that happens with 15.10 too.

Attached a screenshot that shows a successful shutdown process (without splash screen). The red line marks the moment where the process hangs if the shutdown fails.

The workaround with the SysRq key presented here worked for me:
http://askubuntu.com/a/674209/93513

Gregory Kramida (algomorph) wrote :

Martin Pitt, I've tried all the recipes, here is what booting with "debug" shows after shutdown (attached image).

Unfortunately, for me, SysRq doesn't work, the and the keyboard is completely unresponsive. Is there any way to dig out what happened from system logs post-factum, after next login?

I have the following in syslog:
"Dec 5 09:31:21 deepblue systemd[1811]: shutdown.target: Fixing conflicting jobs shutdown.target/stop,shutdown.target/start by deleting job shutdown.target/stop"
Could that be the issue?

I'll attach the part of the syslog that I think may be relevant to the next message.

Gregory Kramida (algomorph) wrote :

Syslog starting with initiating shutdown from gui and ending right before hard shutdown.

xtrchessreal (xtrchessreal) wrote :

You held down alt+SysRq and then typed REISUB one key at a time, and that didn't work?
This: "Dec 5 09:31:21 deepblue systemd[1811]: shutdown.target: Fixing conflicting jobs shutdown.target/stop,shutdown.target/start by deleting job shutdown.target/stop" Looks like something is wrong but I am not expert - you might be onto something. I'm gonna do what you did and see what I get in Syslog.
This still affects me in 15.04 and only when I run Chromium but every time I run Chromium SMH - I hate intermittent issues. Firefox does not cause this - go figure.

How is this resolved? I'm having same problem but with ubuntu15.10 with kernel 4.2. Reboot works fine but when I try shutting down, I get to the ubuntu purple splash screen with text saying reboot: powering down or something like that and it just freeze. I waited like 4 hours and it's stuck. I had to manually power it off by holding the power button on the laptop. Need help with this problem. :(

Changed in systemd (Ubuntu):
status: Incomplete → Confirmed
Gregory Kramida (algomorph) wrote :

I have a hunch that this bug only occurs on systems upgraded from a previous Ubuntu version, perhaps with replacement of upstart with systemd, but not on fresh Ubuntu 15.04 or 15.10 installations.
Everyone who is affected: is there anyone who started having this problem on a fresh 15.04 or later installation? Please let us know within a week. This may shed insight onto how this bug may be reproduced.

I've changed status to 'Confirmed' since it looks like all the information that could be provided has been provided. Developers and enthusiasts: if there is anything else we can do to help diagnose this, please let us know.

Gregory Kramida (algomorph) wrote :

@xtrchessreal, yes, alt+SysRq+ REISUB didn't work. It did help me out in some other strange glitches. I think in this case, the USB drivers were already shut off, and since I have a USB keyborad, it was unresponsive. Maybe an analog / laptop keyboard would have worked.

xtrchessreal (xtrchessreal) wrote :

My Ubuntu 15.04 is fresh installed using the frugal install method. Its a Sony Vaio VGN-FW250j and the bios is locked out, which is why I had to use the Frugal method. Its a dual boot system with Windows 7. The two share a Data drive for storage only. I wish I knew how to capture the shutdown data.
I have not upgraded to 15.10 since someone confirmed its still doing it on that as well. The one odd thing for me is I tried to use a different browser than Chromium. I started using Firefox and the Target Reached Shutdown hang only hangs for a few seconds and then shutsdown on its own. But every time I bring up Chromium it hangs on shutdown or reboot and I have to use SYSRQ+alt+REISUB to reboot then wait until its up again just so I can shutdown properly. So I don't understand that and so far no one has been able to explain or repeat that to confirm it. It is very repeatable for me.
My hunch is some background google app that is embedded with Chromium is the culprit, at least in my case. It may just be the beginning of a chain of events that happen to cause the hang but I don't know that because I can't capture the shutdown data. I need a forensics tool.

John Center (john-center) wrote :

My Dell workstation is a fresh install of 15.04, also. I'm having the same problem. I've attached some screen shots that show the last messages typically seen after I try REISUB.

John Center (john-center) wrote :
Gregory Kramida (algomorph) wrote :

@xtrchessreal, if Chromium background processes are the culprit on your platform, this would mean we are suffering from different issues. Indeed, my symptoms are actually a bit different from what's described in the original bug report -- I'm seeing the "Reached target Shutdown" when I actually try to shut down, not reboot, and the system doesn't fully shut down. My reboot works fine.

You can try to disable Chromium background processes from running in the background when Chromium is shut down (settings --> Show advanced settings --> uncheck "Continue running ... "), and close chromium before you reboot.

@Gregory Kramida, I'll run a test with disabled background apps and reply
with results...Thanks for the info

On Sat, Dec 19, 2015 at 2:44 PM, Gregory Kramida <email address hidden>
wrote:

> @xtrchessreal, if Chromium background processes are the culprit on your
> platform, this would mean we are suffering from different issues.
> Indeed, my symptoms are actually a bit different from what's described
> in the original bug report -- I'm seeing the "Reached target Shutdown"
> when I actually try to shut down, not reboot, and the system doesn't
> fully shut down. My reboot works fine.
>
> You can try to disable Chromium background processes from running in the
> background when Chromium is shut down (settings --> Show advanced
> settings --> uncheck "Continue running ... "), and close chromium before
> you reboot.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1464917
>
> Title:
> reboot hangs at 'Reached target Shutdown'
>
> Status in systemd package in Ubuntu:
> Confirmed
>
> Bug description:
> I rebooted my 15.04 system, and found it hanging indefinitely at the
> plymouth shutdown screen. Hitting esc to reveal the console showed a
> normal set of shutdown messages, ending with 'Reached target Shutdown'
> (paraphrased from memory).
>
> The system never rebooted on its own. I had to use SysRq to finish
> the shutdown.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 15.04
> Package: systemd 219-7ubuntu6
> ProcVersionSignature: Ubuntu 3.19.0-20.20-generic 3.19.8
> Uname: Linux 3.19.0-20-generic x86_64
> ApportVersion: 2.17.2-0ubuntu1.1
> Architecture: amd64
> CurrentDesktop: Unity
> Date: Sat Jun 13 11:53:10 2015
> InstallationDate: Installed on 2010-09-24 (1723 days ago)
> InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64
> (20100816.1)
> MachineType: LENOVO 2306CTO
> ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.19.0-20-generic
> root=/dev/mapper/hostname-root ro quiet splash vt.handoff=7
> SourcePackage: systemd
> UpgradeStatus: Upgraded to vivid on 2014-12-06 (189 days ago)
> dmi.bios.date: 10/25/2013
> dmi.bios.vendor: LENOVO
> dmi.bios.version: G2ET97WW (2.57 )
> dmi.board.asset.tag: Not Available
> dmi.board.name: 2306CTO
> dmi.board.vendor: LENOVO
> dmi.board.version: Not Defined
> dmi.chassis.asset.tag: No Asset Information
> dmi.chassis.type: 10
> dmi.chassis.vendor: LENOVO
> dmi.chassis.version: Not Available
> dmi.modalias:
> dmi:bvnLENOVO:bvrG2ET97WW(2.57):bd10/25/2013:svnLENOVO:pn2306CTO:pvrThinkPadX230:rvnLENOVO:rn2306CTO:rvrNotDefined:cvnLENOVO:ct10:cvrNotAvailable:
> dmi.product.name: 2306CTO
> dmi.product.version: ThinkPad X230
> dmi.sys.vendor: LENOVO
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1464917/+subscriptions
>

xtrchessreal (xtrchessreal) wrote :

So far "unchecking running..." has not helped the hang situation for me, it was already unchecked. I unchecked the "use Hardware ..." too. I also closed all the peripheral apps like google voice, Hangouts, Unity Web apps. first. At some point I'll hit a breakthrough.

Acetilamino (poubellace) wrote :

The same problem on 15.10 willy fresh install full format HDD :/ Sysrq does not work for me too... Any idea please?
Thanks for your help!

Gary Letth (gary-letth) wrote :

Same here. Fresh install, 15.10 server on Intel RST raid 1 fakeraid.

Karo (stdevyan) wrote :

I also have the same problem
After install ubuntu 15.04, I can't reboot / shutdown
It hangs on Reached target Shutdown

xtrchessreal (xtrchessreal) wrote :

I ran
dpkg --get-selections | grep linux-image
 to find old installations of the kernel images such as: linux-image3.19.0.15-generic the purged them with this command

sudo apt-get purge linux-image-3.19.0-15-generic

I did this for all old linux-images in the list from the dpkg --etc command above
I deleted all but the current and the next most current, there were two linux-image-extra...etc that needed to be purged as well. These were listed with a "deinstall" parameter in the dpkg --etc list. I am not sure why those needed special attention but the rest auto purged the linux-image-extra...etc packages with the single: sudo apt-get purge linux-image-3.19.0-xx-generic command

I then upgraded to 15.10 and I no longer have the "Reached target Shutdown" hang issue...at this time, caveat it hangs for a few seconds but not indefinitely for me.

These are the only images currently installed now:

dpkg --get-selections | grep linux-image
linux-image-3.19.0-49-generic install
linux-image-4.2.0-27-generic install
linux-image-extra-3.19.0-49-generic install
linux-image-extra-4.2.0-27-generic install
linux-image-generic

I would be interested if others have tried this?

Alex (identityinvisible) wrote :

@xtrchessreal I just tried a variant of what you said (I deleted everything except the most recent kernel, and didn't keep the second most latest) and it fixed it. I am now able to reboot without having this bug any more.

Gregory Kramida (algomorph) wrote :

@xtrchessreal, I followed the recipe and removed everything but the most recent kernel. However, my problem didn't go away. Seeing as the symptoms are slightly different, I am probably suffering from a different bug. Still no clue how to diagnose or fix it.

Becky Depp (deppgirl87) wrote :

I have the same problem with Lubuntu 15.10 and it was a fresh install, however my computer hangs on shut down and reboot. The alt sysRq only works to reboot and it doesn't work if i've already reached the "reached target shutdown" as my keyboard isn't responsive at that point.

i did the following above about the linux image kernels and i have the following:

linux-image-4.2.0-16-generic install
linux-image-4.2.0-27-generic install
linux-image-extra-4.2.0-16-generic install
linux-image-extra-4.2.0-27-generic install
linux-image-generic install

I am assuming those are the most recent?

ggalli (jack-ughetta) wrote :

I have the same problem with Ubuntu 15.10, fresh install on a Lenovo B50-10: the system got stuck on shutdown or reboot (the last trace on screen, when I press Esc, is "Reached target shutdown").
I've tried waiting a few minutes (at least 10 min), but nothing changed... Only way to power off is to keep pressed power button (I don't think it is a good thing for the pc, but I have no other way to power off it)
Don't even know what to check to debug this (very annoying) issue.

Adam (adziurda) wrote :

I have similar problem.

I found workaround that usually allows me to shutdown system.
on debug-shell I run:

swapoff /dev/sda7

HDD start working and system poweroff few seconds later.

When system wasn't using swap before shutdown or i turn off swap before shutdown it went off almost instantly.
It just hangs when swap is used before poweroff.

I could do some debug if someone tell me what to check.

Adam (adziurda) wrote :
flurbius (flurbius) wrote :

Im also having this problem on 15.04 a fresh install

ggalli (jack-ughetta) wrote :

Tried Adam's workaround: if I do

swapoff <swap partition>

and then shutdown, the system poweroff complete successfully.

xtrchessreal (xtrchessreal) wrote :

The issue came back for me a few shutdowns after I purged old kernel images as explained above.

I was gonna wait until I've had more time before calling it successful at least for me as a work around but I tried Adam's idea too and so far it is working. When you 'sudo swapoff -a' it takes time on my machine for the system to do the switch. I hear the cpu fan run higher and the terminal screen is delayed in giving me the prompt back while the command is processed. I'm sure totally normal behavior. Then I shut down as usual and there is no delay at all, I see the message for a second maybe and then my machine is off.

I first played around with the swappiness=0 but that made my machine hang for normal operations so I set it to 50 and then tried Adam's idea just before shutdown. I think a nice script would be cool to make until this issue is fixed officially.

Tőkés Attila (tokes-atti) wrote :

I can confirm too that the issue is probably something to do with the swap usage.

My first guess was that the problem has something to do with the system's update, but seems that real cause is the swap usage. The shutdown usually goes fine when the system is used only for a short time and fails after a couple of hours of usage, when the system accumulates some data in the swap.

I did today some test and I got the following result:
~10 hours uptime (idle), 45 MB swap usage - the showdown went fine
~50 min uptime, 2.2 GB swap usage - the showdown hangs

xtrchessreal (xtrchessreal) wrote :

Tokes,
Your description of behavior is exactly as mine. The longer up time the more it is likely to hang. Also when it hangs the cpu begins to crunch a lot of operations and the fan starts higher and higher rpm. I get the same fan behavior when I 'sudo swapoff -a' as stated above prior to my shutdown as I do when I simply try to shutdown without running 'sudo swapoff -a' command.
The difference is the shutdown completes as it should if I take the time to run 'sudo swapoff -a' and wait for it to complete.
Swap management does seem to be pointing toward the direction of the issue.

Tőkés Attila (tokes-atti) wrote :

I'm just wondering if we could set some timeout for the shutdown in the systemd configuration files.

I tried to add the following to the /usr/lib/systemd/user/shutdown.target config:
JobTimeoutSec=30
JobTimeoutAction=poweroff-force

but unfortunately it has no effect.

xtrchessreal (xtrchessreal) wrote :

I am no expert but I looked into the systemd shutdown.target as well and found no parameters that can be changed. I also performed a swap test where I loaded the swap partition to over 54% roughly 4.2 gig while my ram was at 87% full. I have 3.7 gig ram available and 8.1 gig swap on my machine. I loaded the system using Gimp opening and entire folder of photos, opening a browser and streaming a youtube movie. I then closed all applications and waited for 20 minutes with the monitor open. The swap file did lose half of its stored data to about 2 gig but then it seemed to pause at that point for a long time, longer than I was willing to wait. I then ran:
sudo swapoff -a && systemctl poweroff

The swapoff process took another 10+ minutes and then the shutdown took place without the hang.

It is my belief that the swap management app/function built into the OS is the issue or at least part of it.

ggalli (jack-ughetta) wrote :

I did the "swapoff workaround" a number of times and not always it worked...

Probably the swap handling is only part of the problem...

dino99 (9d9) on 2016-03-08
tags: added: xenial
dino99 (9d9) wrote :

Xenial is also affected: reboot & shutdown, the lock usually ends by itself 10_20 seconds later, but sometimes it does not ends at all.

Looks like its due to "lockfile-progs" which is described as:

*******
These programs use liblockfile to perform the file locking and
unlocking, so they are guaranteed compatible with Debian's
file locking policies.
*******

Is there a Debian/Ubuntu file locking policies difference explaining that lock ?

Changed in systemd (Ubuntu):
importance: Undecided → High
Alex (identityinvisible) wrote :

This bug has reappeared for me, even after removing the older kernels.

xtrchessreal (xtrchessreal) wrote :

Have you tried @Adam idea turn your swap off just prior to shutdown, allow the command to process its a work around that seems to work for many. It may take 10+ minutes if a lot of data was being held in swap but it should shutdown.
sudo swapoff -a && systemctl poweroff
The amount of data held in swap must be lower than the amount available in Ram or the command will fail FYI.

Steve Langasek (vorlon) wrote :

On Tue, Mar 08, 2016 at 09:09:46AM -0000, dino99 wrote:
> Xenial is also affected: reboot & shutdown, the lock usually ends by
> itself 10_20 seconds later, but sometimes it does not ends at all.

> Looks like its due to "lockfile-progs" which is described as:

lockfile-progs is a set of tools that lets you manage lockfiles. It is not
the thing that is causing locks. Something else must be doing that. So
what on your system is using lockfile-progs during system shutdown?

For that matter, why do you think lockfile-progs is interfering with
shutdown at all? The kinds of lockfiles that lockfile-progs manages do not
result in open file descriptors and should not prevent shutdown.

Adam (adziurda) wrote :

I guess i have found cause and solution. It's timeout.
In logs after running system with systemd.log_level=debug I have:

Mar 09 21:45:25 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Forked /sbin/swapoff as 4403
...
And exactly 90s later:

Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Deactivation timed out. Stopping.
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Changed deactivating -> deactivating-sigterm
Mar 09 21:46:55 misio systemd[1]: Received SIGCHLD from PID 4403 (swapoff).
Mar 09 21:46:55 misio systemd[1]: Child 4403 (swapoff) died (code=killed, status=15/TERM)
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Child 4403 belongs to dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Swap process exited, code=killed status=15
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Changed deactivating-sigterm -> failed
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Job dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap/stop finished, result=done
Mar 09 21:46:55 misio systemd[1]: Deactivated swap /dev/disk/by-uuid/f7d9f751-c42a-40cf-bfb1-218c90c07b10.
Mar 09 21:46:55 misio systemd[1]: dev-disk-by\x2duuid-f7d9f751\x2dc42a\x2d40cf\x2dbfb1\x2d218c90c07b10.swap: Unit entered failed state.

Systemd has commented out ShutdownWatchdogSec=90s in /etc/systemd/system.conf.
I guess that means default value.

xtrchessreal (xtrchessreal) wrote :

Sweet!! nice find Adam!

My system (15.10 with full updates) has

#ShutdownWatchdogSec=10min

(and systemd-system.conf man page confirms the 10min default timeout)
and I still have the issue.

turning swap off before reboot
    _OR_
sudo systemctl reboot

works without any issues, and I get an immediate reboot.

xtrchessreal (xtrchessreal) wrote :

I wonder if the so called Watchdog Hardware is on my machine at all. I looked for the conf files in /dev/watchdog and found no such directory. According to the website: https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html The manual reads as though its assumed watchdog hardware is there. It states that, "These settings have no effect if a hardware watchdog is not available."

I wonder if anyone assumed the case if it was not there?

It is still my contention the swap management system is at fault here. I have run several tests loading my 3.7 gig ram and 8.1 gig swap then shutting all apps off. Monitoring with system monitor RAM returns to acceptable levels around .9 to 1.4 gig, however the swap never gets below 2 gig even after 30 mins of no load conditions, all apps closed. A swap management system appears to be absent minded.

Turning swap off is the only way I get a clean reboot or shutdown

sudo systemctl reboot or poweroff hang unless swap is cleared first and swapoff fails if data held in swap is larger than available RAM. I am tempted to use SyReq +REISUB just to reboot, why wait for the hang? I check system monitor before running the command below to shutdown to calculate whether the swapoff command will fail then decide how to proceed.

sudo swapoff -a && systemctl poweroff

Adam (adziurda) wrote :

Sorry my mistake.
#ShutdownWatchdogSec=10min
#DefaultTimeoutStopSec=90s

Thanks Daniel for pointing out my mistake.

ggalli (jack-ughetta) wrote :

Hi All,
did anyone find at least a workaround to automate the shutdown process (while waiting for the issue to be fixed) ?

Thanks

dino99 (9d9) wrote :

This problem is gone with xenial on my system. This was a transitional issue.

tags: removed: xenial
Tőkés Attila (tokes-atti) wrote :

@dino99:
The problem is present in Xenial. Nothing changed after I upgraded to Ubuntu 16.04 LTS preview.

tags: added: xenial
Rik Mills (rikmills) wrote :

Now have this present in Xenial

MarcVG (marc-vangiel) wrote :

I have the same problem on power off (not reboot) when upgraded to 16.04 LTS. Was not an issue in 15.10.

Tom (tom-tomhek) wrote :

Did not have this on 15.10, but I am having this issue on 16.04.

xtrchessreal (xtrchessreal) wrote :

I'm still using "sudo swapoff -a && systemctl poweroff" as a work around
Does anyone know how this bug gets ASSIGNED to a task group?

dino99 (9d9) wrote :

i highly suspect some old setting/config left behind:
- use gtkorphan to purge possible unused package(s)
- clean the system with bleachbit (as root; but optin carefully the settings)

and as a reminder, always 'purge' to get rid of old settings which disturb the system and give unexpected troubles.

Martin Pitt (pitti) wrote :

Steve, do you still have this problem on later Ubuntu versions? Please follow "Debugging boot/shutdown problems" in /usr/share/doc/systemd/README.Debian.gz and check if there are any hanging jobs at shutdown. Capturing a screen photo of "journalctl -b" in the rescue shell might also be enlightening.

Other reporters: Such bugs are highly specific to a particular installation or hardware (and in that case are in the kernel domain), so it's usually better to file new reports.

Changed in systemd (Ubuntu):
status: Confirmed → Incomplete
Mario Limonciello (superm1) wrote :

I've definitely reproduced this as recently as today on a real machine on
16.04 (fully up to date) with similar symptoms described as reaching the
shutdown target and then just hanging for a while.

I also uses sysrq to recover at that point.

I've also noticed this in Virtualbox, particularly after installation is
complete that it won't reboot.

On Tue, Apr 26, 2016, 14:31 Martin Pitt <email address hidden> wrote:

> Steve, do you still have this problem on later Ubuntu versions? Please
> follow "Debugging boot/shutdown problems" in
> /usr/share/doc/systemd/README.Debian.gz and check if there are any
> hanging jobs at shutdown. Capturing a screen photo of "journalctl -b" in
> the rescue shell might also be enlightening.
>
> Other reporters: Such bugs are highly specific to a particular
> installation or hardware (and in that case are in the kernel domain), so
> it's usually better to file new reports.
>
> ** Changed in: systemd (Ubuntu)
> Status: Confirmed => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1464917
>
> Title:
> reboot hangs at 'Reached target Shutdown'
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/systemd/+bug/1464917/+subscriptions
>

Martin Pitt (pitti) wrote :

> I've also noticed this in Virtualbox, particularly after installation is complete that it won't reboot.

That sounds like it would affect the live system with ubiquity, rather than an installed system? Or did I misunderstand this? Shutdown on the live system has always been brittle particularly in VMs, as the "press enter after removing the install medium" plymouth message often gets eaten.

Steve Langasek (vorlon) wrote :

> Steve, do you still have this problem on later Ubuntu versions?

No, this issue is not reproducible for me in xenial. Marking as fixed.

Changed in systemd (Ubuntu):
status: Incomplete → Fix Released
nand (p-ucuntu-t) wrote :

This or an identical-looking bug affects all of our test machines running Ubuntu 16.04. The machines are also fully patched.

It's not consistent, the bug seems to happen sporadically. It must be some sort of race condition. I can either reproduce it or not, on the same machine, as the same user, running the same series of instructions (boot up, log in, open shell, type `systemctl reboot`).

I can't give you a clear list of what the last few services that got stopped were since it seems inconsistent. But sometimes, it's seemingly triggered by lvm2-monitor being stopped (perhaps too late? I know that lvm2 being stopped too late can definitely trigger some indefinite hangs).

winnie (winnie-) wrote :

Please reopen this issue. I can reproduce this on several sligthly different boxes which are installed from scratch. I didn't find a good way to reproduce this reliably. It does however happen regulary on reboot/shutdown. My guess is about every 5th time.
I am able to switch to tty1 but typing anything is not possible.

Systemd doesn't offer much for shutdown debugging (even though a lot of debugging-tweaks exist for the boot-process). Has anybody an idea how to debug this properly?

Steve Langasek (vorlon) wrote :

On Wed, Apr 27, 2016 at 04:49:37PM -0000, winnie wrote:

> Please reopen this issue.

File a new bug for your issue, as Martin indicated.

> Systemd doesn't offer much for shutdown debugging

It does, as documented in the README that Martin pointed out in this bug.
This would perhaps be more obvious if that information were not being buried
in a discussion on a metabug where everyone is me-too'ing the same symptoms.
Which is another reason you should file a new bug report for your issue.

description: updated
Tőkés Attila (tokes-atti) wrote :

This still happens in 16.04 LST! As it was described several times above the shutdown process somehow blocks/gets in a deadlock when un-mounting the swap partitions. I did not understood exactly what happens but the recovery console clearly shows that the swap space is still on and vailable and there are four .swap jobs "running" (although I have a single swap partition).

PS: The "Fix Released" status suggest for me that some work was done to fix this issue, which supposes that the issue was clearly understood. This is obiviously not the case here.

Tőkés Attila (tokes-atti) wrote :
Martin Pitt (pitti) wrote :

This could be https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=788303 then. But really, please file new bugs -- the same symptom ("long shutdown" or similar) might apear the same, but over time we've seen wildly different root causes.

Changed in systemd:
status: New → Invalid
ggalli (jack-ughetta) wrote :

Will it be ported (systemd 229-5, that should fix swap issue) to Ubuntu 16.04 ?

xtrchessreal (xtrchessreal) wrote :

according to the changelog: http://metadata.ftp-master.debian.org/changelogs//main/s/systemd/systemd_229-5_changelog

"systemd (229-5) unstable; urgency=medium"

"* On shutdown, unmount /tmp before disabling swap. (Closes: #788303)"

is supposed to fix the hang on shutdown. Provided that we are all experiencing the "same hang" issue.
Reading this bug #1464917 then referenced #788303 which references #2930 https://github.com/systemd/systemd/issues/2930 which was merged with #3087 https://github.com/systemd/systemd/pull/3087

Just need to figure out when systemd 229-5 is stable released to Ubuntu

xtrchessreal (xtrchessreal) wrote :

229-5 is set to release with 16.10 "Yakkety Yak"

Martin Pitt (pitti) wrote :

Again: If you are affected by a shutdown hang, due to the swap issue or otherwise, please file a new bug.

Martin Pitt (pitti) wrote :

> "* On shutdown, unmount /tmp before disabling swap. (Closes: #788303)"

This fix is in xenial-proposed now, for the record.

Gregory Kramida (algomorph) wrote :

Dear all, turns out, my system was suffering from completely different bug with similar symptoms. It was a problem with the motherboard BIOS. I upgraded the BIOS and the bug went away. I'm taking myself off the mailing list for this bug.

If you're suffering from a similar issue even after the fix in Xenial, try upgrading your BIOS. If that doesn't help and you're on a desktop, try clearing CMOS with the jumper (not through the menu) after the successful BIOS upgrade.

Lauren Weinstein (lauren4321) wrote :

Thanks. Will it backport to 16.04?

L

On Thu, May 5, 2016 at 8:04 PM, Martin Pitt <email address hidden> wrote:

> > "* On shutdown, unmount /tmp before disabling swap. (Closes: #788303)"
>
> This fix is in xenial-proposed now, for the record.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1464917
>
> Title:
> reboot hangs at 'Reached target Shutdown'
>
> Status in systemd:
> Invalid
> Status in systemd package in Ubuntu:
> Fix Released
>
> Bug description:
> This bug is resolved. Do not follow up to this bug. Shutdown bugs
> are specific to the software installed and its configuration. If you
> are experiencing a shutdown hang, please file a separate bug report
> and follow the debugging instructions described in the "Debugging
> boot/shutdown problems" section of
> /usr/share/doc/systemd/README.Debian.gz to check if there are any
> hanging jobs at shutdown. Capturing a screen photo of "journalctl -b"
> in the rescue shell might be enlightening.
>
> [Original report]
> I rebooted my 15.04 system, and found it hanging indefinitely at the
> plymouth shutdown screen. Hitting esc to reveal the console showed a
> normal set of shutdown messages, ending with 'Reached target Shutdown'
> (paraphrased from memory).
>
> The system never rebooted on its own. I had to use SysRq to finish
> the shutdown.
>
>
>
> ProblemType: Bug
> DistroRelease: Ubuntu 15.04
> Package: systemd 219-7ubuntu6
> ProcVersionSignature: Ubuntu 3.19.0-20.20-generic 3.19.8
> Uname: Linux 3.19.0-20-generic x86_64
> ApportVersion: 2.17.2-0ubuntu1.1
> Architecture: amd64
> CurrentDesktop: Unity
> Date: Sat Jun 13 11:53:10 2015
> InstallationDate: Installed on 2010-09-24 (1723 days ago)
> InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64
> (20100816.1)
> MachineType: LENOVO 2306CTO
> ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.19.0-20-generic
> root=/dev/mapper/hostname-root ro quiet splash vt.handoff=7
> SourcePackage: systemd
> UpgradeStatus: Upgraded to vivid on 2014-12-06 (189 days ago)
> dmi.bios.date: 10/25/2013
> dmi.bios.vendor: LENOVO
> dmi.bios.version: G2ET97WW (2.57 )
> dmi.board.asset.tag: Not Available
> dmi.board.name: 2306CTO
> dmi.board.vendor: LENOVO
> dmi.board.version: Not Defined
> dmi.chassis.asset.tag: No Asset Information
> dmi.chassis.type: 10
> dmi.chassis.vendor: LENOVO
> dmi.chassis.version: Not Available
> dmi.modalias:
> dmi:bvnLENOVO:bvrG2ET97WW(2.57):bd10/25/2013:svnLENOVO:pn2306CTO:pvrThinkPadX230:rvnLENOVO:rn2306CTO:rvrNotDefined:cvnLENOVO:ct10:cvrNotAvailable:
> dmi.product.name: 2306CTO
> dmi.product.version: ThinkPad X230
> dmi.sys.vendor: LENOVO
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/systemd/+bug/1464917/+subscriptions
>

Danny Yates (mail4danny) wrote :

Since there hasn't been an answer for a month, I'll ask the question again...

Is there any chance that this will be backported for Xenial? At the moment, folks on an LTS release are struggling to cleanly shut their machines down.

Lauren Weinstein (lauren4321) wrote :

This is not a trivial matter. It needs to be backported.

On Mon, Jun 13, 2016 at 1:25 AM, Danny Yates <email address hidden>
wrote:

> Since there hasn't been an answer for a month, I'll ask the question
> again...
>
> Is there any chance that this will be backported for Xenial? At the
> moment, folks on an LTS release are struggling to cleanly shut their
> machines down.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1464917
>
> Title:
> reboot hangs at 'Reached target Shutdown'
>
> Status in systemd:
> Invalid
> Status in systemd package in Ubuntu:
> Fix Released
>
> Bug description:
> This bug is resolved. Do not follow up to this bug. Shutdown bugs
> are specific to the software installed and its configuration. If you
> are experiencing a shutdown hang, please file a separate bug report
> and follow the debugging instructions described in the "Debugging
> boot/shutdown problems" section of
> /usr/share/doc/systemd/README.Debian.gz to check if there are any
> hanging jobs at shutdown. Capturing a screen photo of "journalctl -b"
> in the rescue shell might be enlightening.
>
> [Original report]
> I rebooted my 15.04 system, and found it hanging indefinitely at the
> plymouth shutdown screen. Hitting esc to reveal the console showed a
> normal set of shutdown messages, ending with 'Reached target Shutdown'
> (paraphrased from memory).
>
> The system never rebooted on its own. I had to use SysRq to finish
> the shutdown.
>
>
>
> ProblemType: Bug
> DistroRelease: Ubuntu 15.04
> Package: systemd 219-7ubuntu6
> ProcVersionSignature: Ubuntu 3.19.0-20.20-generic 3.19.8
> Uname: Linux 3.19.0-20-generic x86_64
> ApportVersion: 2.17.2-0ubuntu1.1
> Architecture: amd64
> CurrentDesktop: Unity
> Date: Sat Jun 13 11:53:10 2015
> InstallationDate: Installed on 2010-09-24 (1723 days ago)
> InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release amd64
> (20100816.1)
> MachineType: LENOVO 2306CTO
> ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.19.0-20-generic
> root=/dev/mapper/hostname-root ro quiet splash vt.handoff=7
> SourcePackage: systemd
> UpgradeStatus: Upgraded to vivid on 2014-12-06 (189 days ago)
> dmi.bios.date: 10/25/2013
> dmi.bios.vendor: LENOVO
> dmi.bios.version: G2ET97WW (2.57 )
> dmi.board.asset.tag: Not Available
> dmi.board.name: 2306CTO
> dmi.board.vendor: LENOVO
> dmi.board.version: Not Defined
> dmi.chassis.asset.tag: No Asset Information
> dmi.chassis.type: 10
> dmi.chassis.vendor: LENOVO
> dmi.chassis.version: Not Available
> dmi.modalias:
> dmi:bvnLENOVO:bvrG2ET97WW(2.57):bd10/25/2013:svnLENOVO:pn2306CTO:pvrThinkPadX230:rvnLENOVO:rn2306CTO:rvrNotDefined:cvnLENOVO:ct10:cvrNotAvailable:
> dmi.product.name: 2306CTO
> dmi.product.version: ThinkPad X230
> dmi.sys.vendor: LENOVO
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/systemd/+bug/1464917/+subscriptions
>

xtrchessreal (xtrchessreal) wrote :

FYI I filed a new Bug #1582011 as requested.

Carlos Pineda (cpinedagt) wrote :

I had the same problem but it started after I uncomment the following line in the /etc/default/grub file:

#GRUB_TERMINAL=console.

I commented it again and the PC start to shutdown again without problems.

I commented this line at first because I had a problem with my screen resolution mode... it was always as 640x400 so I commented that line and my system let me choose another mode. Of course when I commented the line again the resolution started again in 640x400, but I changed the line

GRUB_GFXMODE=640x400 to GRUB_GFXMODE=1024x768 and it works fine in this mode... I will try other modes later but maybe this can help you all.

The GRUB_TERMINAL line is related to the graphical terminal and the line GRUB_GFXMODE is related to the Mode of the graphical terminal.

Lauren Weinstein (lauren4321) wrote :

This bug appears to now be fixed for me, at least in initial testing.

Gautam Bhat (mindentropy) wrote :

This bug is still present in Xenial. I have 4 different machines which are affected by this bug.

Prashant Gupta (prashu20) wrote :

Hi all

I am running Ubuntu 16.04 which I have upgraded from ubuntu 15.10 and I am still facing the same issue is after shutdown,the system hangs and I have to manually do hard shutdown.I have tried lots of solutions then came to this bug ,will try swap off but I guess this is not the permanent solution.Please reopen this bug

VinDSL (perfect-pecker) wrote :

This bug is affecting my Dell 7010 desktop machine, after a fresh install of Ubuntu 16.10. Never had this problem during testing. Smells like a regression...

David (dkahn10) wrote :

I am experiencing the same problem on a new install of 16.10 AMD 64 Gateway laptop. I did report it as a separate bug but I am adding my name to this one also.
Thanks,
It may have something to do with the timing or swap. I hope someone takes notice and replies :)
 thanks,

David

Changed in systemd:
status: Invalid → Incomplete
dino99 (9d9) on 2016-11-12
Changed in systemd:
status: Incomplete → Invalid
monochromec (monochromec) wrote :

Hi David,

As this issue is still persisting (and none of the above work-arounds worked for me): What's the bug # of the new bug you filed a few days ago?

Thanks for your help!

VinDSL (perfect-pecker) wrote :

@monochromec,

David's bug appears to be here: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1640200

NoOp (glgxg) wrote :

Still present in 16.04:

4.4.0-51-generic #72-Ubuntu SMP Thu Nov 24 18:29:54 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Sebastian Geiger (lanoxx) wrote :

Present in 16.10.

bigsicret (bigsicret) wrote :

Same problem with debian 9. I had no problem with debian 8.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.