No login screen unless "quiet splash" removed from boot line

[Barry Warsaw]

I'm not entirely sure this is a Plymouth bug, but here is what's going on.

MacBook Pro 1,1 running latest Quantal.

This machine was running 12.04 just fine, then I upgraded it to 12.10. Now, whenever I boot it, I just end up with a black screen, no login screen, no apparent cursor, no response to keyboard.

If, at the grub screen, I edit the boot line and remove "quiet splash", the machine boots to its login screen, although it sometimes takes quite a long time. I don't see much on the console output that indicates what's going on though.

[Steve Langasek (vorlon)]

Can you check whether downgrading to the 12.04 version of the plymouth packages fixes this issue? There are a lot of moving parts here, it could just as well be a kernel regression as a plymouth one.

[Barry Warsaw (barry)]

On Jun 21, 2012, at 06:18 PM, Steve Langasek wrote:

>Can you check whether downgrading to the 12.04 version of the plymouth
>packages fixes this issue? There are a lot of moving parts here, it
>could just as well be a kernel regression as a plymouth one.

I'll give it a try. I should note too that while I can't tell where boot is
hung, it doesn't even respond to ping, so it's somewhere north of network
start up.

[Barry Warsaw (barry)]

More data points:

* arch is i386
* at least half the time I don't even get to the grub screen
* I think removing "quiet" or "splash" is a red herring - if I get to grub the machine boots regardless of whether I edit the boot line, leave it alone, or just let grub timeout, etc. obviously if I don't get to grub it doesn't boot.

[Barry Warsaw (barry)]

I do think the "works half the time" behavior of grub safe boot is coming into play. It's not exactly half the time but whenever I get a hang, I definitely get the grub screen afterward. Occasionally I get the grub screen more than once in a row though.

I downgraded libplymouth2 plymouth plymouth-theme-ubuntu-logo plymouth-theme-ubuntu-text and plymouth-label (the only installed *plymouth* packages).

It still hangs about half the time.

Probably not relevant, but reported here for completeness. I got these warnings at the end of the downgrade.

cryptsetup: WARNING: failed to detect canonical device of /dev/sda3
cryptsetup: WARNING: could not determine root device from /etc/fstab

(FWIW, I re-updated to keep the number of variables low).

[James Hunt (jamesodhunt)]

Hi Barry - when it hangs, does pressing Escape show anything? Also, I've updated the Plymouth wiki page so it would be interesting to know if Plymouth is runnable via the initramfs and the live system:

https://wiki.ubuntu.com/Plymouth#Checking_Plymouth_Can_Run_in_the_Initramfs
https://wiki.ubuntu.com/Plymouth#Checking_Plymouth_Can_Run_Early_in_Boot

Also, if you've got a serial cable, you could try running plymouthd with '--debug-file=/dev/ttyS0' or similar to see if it shows anything interesting.

[James Hunt (jamesodhunt)]

Might also be useful to get a plymouth debug log (along with syslog, etc) even with "quiet splash" removed.

[Steve Langasek (vorlon)]

Since downgrading plymouth to the precise version doesn't fix it, it doesn't sound like plymouth itself is the culprit here. Reassigning to the kernel.

Barry, have you checked whether setting GRUB_GFXPAYLOAD_LINUX=text makes the boot reliable?

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1016170

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Joseph Salisbury (jsalisbury)]

Barry,

Can you test the latest mainline kernel[0], to see if this bug is already fixed upsteram?

Also, does the issue go away if you boot back into the Precise kernel?

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.5-rc4-quantal/

[Barry Warsaw (barry)]

On Jun 27, 2012, at 05:11 PM, Steve Langasek wrote:

>Barry, have you checked whether setting GRUB_GFXPAYLOAD_LINUX=text makes
>the boot reliable?

It does not.

[Barry Warsaw (barry)]

On Jun 27, 2012, at 04:19 PM, James Hunt wrote:

>Hi Barry - when it hangs, does pressing Escape show anything?

Hi James. Nothing happens if I press esc (or really, anything) during the
hang.

>Also, I've updated the Plymouth wiki page so it would be interesting to know
>if Plymouth is runnable via the initramfs and the live system:
>
>https://wiki.ubuntu.com/Plymouth#Checking_Plymouth_Can_Run_in_the_Initramfs

I can't get this one to work. I can do the chroot, see the warnings, but the
first plymouth command seems to do nothing but produce usage/help output. If
I `plymouth --help` it doesn't seem to show all those options

>https://wiki.ubuntu.com/Plymouth#Checking_Plymouth_Can_Run_Early_in_Boot

I can't get very far with this one either. I can get to the /bin/sh warning
and a hash prompt, but the system is completely unresponsive to keystrokes.

>Also, if you've got a serial cable, you could try running plymouthd with
>'--debug-file=/dev/ttyS0' or similar to see if it shows anything
>interesting.

This is a MacBook Pro so no serial cable. I'm sure there's a way to get a
terminal on a USB or Firewire port, but I'm not sure I want to spend the time
to do that just yet. :)

Thanks!

[Barry Warsaw (barry)]

On Jun 27, 2012, at 06:08 PM, Joseph Salisbury wrote:

Hi Joseph.

>Can you test the latest mainline kernel[0], to see if this bug is
>already fixed upsteram?
>
>Also, does the issue go away if you boot back into the Precise kernel?

So, I set GRUB_DEFAULT=saved then chose the 3.2.0-25-generic kernel that was
still there from pre-upgrade Precise, but this made no difference. It still
hangs every other time, but I've verified that when it does boot (i.e. after
seeing the grub screen), it does boot into 3.2.0-25-generic.

>[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.5-rc4-quantal/

I tried it with the i386 versions of 3.5.0-030500rc4-generic and it made no
difference.

[Joseph Salisbury (jsalisbury)]

So this bug still exists if you boot the Precise kernel on Quantal, but it did not exist at all booting the precise kernel on Precise?

An interesting test would be to boot the Quantal kernel on this machine with Precise installed. Is this a test machine? Do you have the option to re-install Precise to test the Quantal kernel there?

[Steve Langasek (vorlon)]

On Wed, Jun 27, 2012 at 07:26:33PM -0000, Joseph Salisbury wrote:
> So this bug still exists if you boot the Precise kernel on Quantal, but
> it did not exist at all booting the precise kernel on Precise?

> An interesting test would be to boot the Quantal kernel on this machine
> with Precise installed. Is this a test machine? Do you have the
> option to re-install Precise to test the Quantal kernel there?

Note that the bootloader could also be the key here, so it's important to
check which bootloader is actually in use when booting for each test.

There doesn't seem to be any new version of refit in quantal vs. precise, so
this would only be a question of grub-efi.

[Barry Warsaw (barry)]

On Jun 27, 2012, at 07:26 PM, Joseph Salisbury wrote:

>So this bug still exists if you boot the Precise kernel on Quantal, but
>it did not exist at all booting the precise kernel on Precise?

Correct.

>An interesting test would be to boot the Quantal kernel on this machine
>with Precise installed. Is this a test machine? Do you have the
>option to re-install Precise to test the Quantal kernel there?

That's a bit tough. I do use this machine in production. I'd hate to wipe
it, but if we exhaust all other options, I could probably make a snapshot of
the partition, wipe it, then re-install later.

[Joseph Salisbury (jsalisbury)]

Barry, there are some additional things that can be done to gather further debug information. Details can be found at:
https://wiki.ubuntu.com/DebuggingKernelBoot

It looks like you already tried setting GRUB_GFXPAYLOAD_LINUX=text

I'm going to add this bug to the kernel team hotlist to get some additional folks to review it.

[Barry Warsaw (barry)]

On Jun 27, 2012, at 08:22 PM, Joseph Salisbury wrote:

>Barry, there are some additional things that can be done to gather further
>debug information. Details can be found at:
>https://wiki.ubuntu.com/DebuggingKernelBoot

It seems like the problem is in the handoff from refit to grub, since I cannot
get to a grub screen by holding down the shift when the machine is in "I want
to hang" mode. No shift is necessary after the failed boot because the grub
screen automatically comes up, and when it does, it will always boot.

So I'm not sure the recommendations on that wiki page are useful, since they
all assume you can get to the grub screen.

>It looks like you already tried setting GRUB_GFXPAYLOAD_LINUX=text
>
>I'm going to add this bug to the kernel team hotlist to get some
>additional folks to review it.

Thanks!

FWIW, from the "About rEFIt" page:

rEFIt Version 0.14
Running on:
    EFI Revision 1.10
    Platform x86 (32 bit)
    Firmware: Apple 8192.01
    Screen Output: UGA Draw (EFI 1.10), 1440x900

So this should even be the latest version of refit.

[Steve Langasek (vorlon)]

On Wed, Jun 27, 2012 at 09:23:39PM -0000, Barry Warsaw wrote:
> On Jun 27, 2012, at 08:22 PM, Joseph Salisbury wrote:

> >Barry, there are some additional things that can be done to gather further
> >debug information. Details can be found at:
> >https://wiki.ubuntu.com/DebuggingKernelBoot

> It seems like the problem is in the handoff from refit to grub, since I cannot
> get to a grub screen by holding down the shift when the machine is in "I want
> to hang" mode. No shift is necessary after the failed boot because the grub
> screen automatically comes up, and when it does, it will always boot.

> So I'm not sure the recommendations on that wiki page are useful, since they
> all assume you can get to the grub screen.

Note that holding down the shift key to get to the grub screen assumes a
BIOS that will handle the shift key. I have no idea if this works at all in
an EFI environment.

You can force the grub menu to always be shown, with or without a previous
failure and with or without a shift key, by setting GRUB_TIMEOUT=-1 (I
think).

[Seth Forshee (sforshee)]

Barry: Can you modify /etc/defaults/grub to remove "quiet splash" and set GRUB_GFXPAYLOAD_LINUX=text, then run update-grub, and see if you still get a hang? If so, what do you get on the screen?

I think forcing the boot menu to always be shown is a good idea as well, to give another point of reference regarding when the hang happens.

[Barry Warsaw (barry)]

On Jun 27, 2012, at 09:42 PM, Steve Langasek wrote:

>You can force the grub menu to always be shown, with or without a previous
>failure and with or without a shift key, by setting GRUB_TIMEOUT=-1 (I
>think).

According to `info grub` -1 waits indefinitely, 0 boots immediately. However,
even setting this to 0 doesn't help. After selecting Linux from the rEFIt
menu, the machine just goes directly from the tux image to black unresponsive
screen.

Something does seem to be going on though because I can hear the fan revving
up, but other than that, the machine is completely frozen, even to ssh, until
I power cycle it.

[Seth Forshee (sforshee)]

One of the things I'm interested in knowing is if you force the grub menu to displayed every time you boot (so any non-zero value of GRUB_TIMEOUT) does the hang happen before the boot menu is displayed. If so then it's pretty conclusive that it's happening in grub. Based on your comments I'm thinking that's probably the case.

[Barry Warsaw (barry)]

On Jun 27, 2012, at 10:32 PM, Seth Forshee wrote:

>One of the things I'm interested in knowing is if you force the grub
>menu to displayed every time you boot (so any non-zero value of
>GRUB_TIMEOUT) does the hang happen before the boot menu is displayed. If
>so then it's pretty conclusive that it's happening in grub. Based on
>your comments I'm thinking that's probably the case.

So, with

GRUB_TIMEOUT=1
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_GFXPAYLOAD_LINUX=text

it still hangs right before the boot menu is displayed. So yeah, it's
probably grub.

[Colin Watson (cjwatson)]

It may be worth putting 'set debug=all' at the top of grub.cfg and seeing how far it gets. (Warning: this can potentially take a *long* time to boot. In extremis you might even need to boot from a live CD or similar to remove it.)

[Barry Warsaw (barry)]

Colin, I must have missed your comment #24; I'll try that. In the meantime, note that if I let rEFIt boot to OS X, it will *always* boot successfully to Ubuntu the next time.

[Barry Warsaw (barry)]

Setting debug=all doesn't provide anything useful. When it freezes, it still freezes before there's any output. When it doesn't freeze, I get all the expected debugging output to the console.

About the only other bit of information I can provide is that when it does freeze, the keyboard stays backlit. When it doesn't freeze, the keyboard backlight goes out.

[Barry Warsaw (barry)]

I got the opportunity to show Colin the behavior in person, and he had a key insight. He asked whether the freeze happens on the reboot after a shutdown, as opposed to an explicit reboot command (e.g. choosing Reboot from the greeter menu).

In my very limited testing so far, it seems to be true that a shutdown->boot does not freeze, while an explicit-reboot does freeze. I'll have to test a few more times to verify, but this is consistent with the previously observed behavior because obviously after a freeze is observed, leaning on the power button results in a successful boot.

Colin suggests looking at reboot quirks and fiddling with the reboot parameters to see if something's changed or if there is a workaround. A quick google didn't find anything immediately relevant, but I may do some more (very) background investigation. OTOH, this machine is fairly old and the h/w is mostly limping along as it is, so I'm not particularly motivated to spend much more time on this.

[Joseph Salisbury (jsalisbury)]

Making some changes to this bug due to issues with kernel hot team report.