init: shouldn't log a warning/error when a job fails with a status in "normal exit"

Bug #522197 reported by Scott James Remnant (Canonical) on 2010-02-15
320
This bug affects 111 people
Affects Status Importance Assigned to Milestone
upstart
Low
Scott James Remnant
upstart (Ubuntu)
Undecided
Unassigned

Bug Description

When a job exits with a non-zero exit status or with a signal, this is logged with a high priority (warning/error). If the syslog daemon isn't running yet, this ends up on the system console.

If the exit status or signal is a "normal exit" reason for the process, they shouldn't be anywhere near as high priority and certainly shouldn't spam the console.

Changed in upstart:
importance: Undecided → Low
status: New → Triaged

Correct. The "flicker-free" startup sure doesn't happen to me because of this :)

Chris (cdekker) wrote :

I have the additional problem that whenever the ureadahead exit messages appear on the screen, the boot process hangs at that point until I hit the ENTER key in which case the login screen appears immediately. No idea whether its due to this bug or even related, though. It's just that usually I see no other messages on the screen. I am currently using an Ubuntu 10.04 snapshot from the repositories as of March 09, 2010.

tags: added: iso-testing

Plus I think a "error" message should not be shown for a mount point that isn't even mounted at boot time. For example: I have two NFS shares on another computer, but that computer isn't always on. So in /etc/fstab, those two nfs mounts use the "noauto" option. Despite this, there is console output for these.

Valieff (skymaster-fwr) wrote :

I got same problem on my MSI VR321X laptop with VIA C7M CPU. I tried to install from alternate cd and desktop cd but it's not a solution. (sorry my English)

Ernst (ernst-blaauw) wrote :

I stil have the problem with latest upstart (0.6.5-5) on a fully up to date Lucid. Will iso testing add more info? Then I can do that, but I think a fully updated Lucid has the same behavior?

I have ubuntu lucid x64 with latest updates and nvidia proprietary driver, and i this is my beautiful plymouth ^

nullservice (itanium128) wrote :

is this resovled? my box can not boot!!!Not even single user mode!

My box is dell precision 340, I am jumping upgrading from 8.04 to 10.04 with ubuntu i686

the system hangs after a full screen of udev warnings(after I tried my method 3 in the following, before is the same as Daniel Añez Scott 's screen shot! )

my system is upto today's upstart package and ureadahead package.
I have separated /boot / /tmp on different partions as
sda1 /boot
sda2 swap
sda3 /tmp
sda5 /
sda6 /home

I tried the following, none of them work

1,change /etc/ureadahead.con and made it mount on localfilesystem
2, disable non critical partitions in /etc/fstab while only keep /boot / partitions
3 change /etc/ureadahead.conf to normal exit with status 0 and 4

nullservice: are you sure there isn't a kernel panic occurring or the kernel cannot mount the root device?

nullservice (itanium128) wrote :

jordanwb, Thank you for your prompt reply!

I am sure it's not kernel panic,because I am on x86 but didnt not see any oops message and call stack.

I am sure kernel finished loading and initramfs is finished because I already saw udev warnings pulse a fsck output like
sda1 xxxxxx/xxxxx blocks.

if you keep /etc/ureadahead.conf as default
like
normal exit 0

then I will see the "init: ureadahead PID terminated with status 4", sth similar to this

then I modify /etc/ureadahead.conf by live CD to
normal exit 0 4
(added 4 as the normal exit status if I understand it right)

and the "terminated with status 4" message is gone, and the last message is the "XXXX blocks " one

another thing make me believe it is NOT kernel panic is
while I was sending ALT+DEL+CTRL for rebooting , the system respones with the menu looks like

continue normal booting
ctrl+D for a maitainance shell
....some other options......

but before you can make a choice, the system reboots.

So ,I dont know what exactly blocks the system(or the init process) from booting. init in upstart? ureadahead?usplash?GDM?I am using
recovery option in grub with "single" as the kernel option, should go into rc1.S which is not related to GDM. right?
all the above are reported from a 2.6.32-18-generic kernel

I tried to remove ureadahead but found ubuntu-minimal depends on it, if I proceed, I probably will remove all the system?

I tried to delete the config files in /etc/ureadahead/*,
the same.

nullservice (itanium128) wrote :

BTW, I already had 2 other vmware running ubuntu x86 jumping upgraded from 7.04/7.10 to 10.04, with single partition as the disk,
both of them are working pretty now.

some one already suggested that this is a multiple partition on different mounting point issue, but I dont know how to solve it

On Thu, 2010-04-01 at 02:19 +0000, nullservice wrote:

> is this resovled? my box can not boot!!!Not even single user mode!
>
Your box being unable to boot is completely unrelated to this bug.

Scott
--
Have you ever, ever felt like this?
Had strange things happen? Are you going round the twist?

Ben Gamari (bgamari) wrote :

Scott,

How are you planning on handling this issue? It seems like it's going to be quite difficult to distinguish between normal non-zero exit codes and actual failures (which _should_ go to the console, no?). Perhaps the task description ought to supply upstart with some indication of the severity of a non-zero exit code? It would be nice to eliminate this noise from the otherwise quite clean boot process in Lucid.

On Sun, 2010-04-11 at 18:03 +0000, Ben Gamari wrote:

> How are you planning on handling this issue? It seems like it's going to
> be quite difficult to distinguish between normal non-zero exit codes and
> actual failures (which _should_ go to the console, no?). Perhaps the
> task description ought to supply upstart with some indication of the
> severity of a non-zero exit code? It would be nice to eliminate this
> noise from the otherwise quite clean boot process in Lucid.
>
Upstart already allows a job to use the "normal exit" stanza to specify
which of its exit codes aren't actual failures, e.g.:

 normal exit 1 2 SIGINT

The ureadahead jobs use those, the bug is that Upstart still logs the
error to the console even though it does.

Scott
--
Have you ever, ever felt like this?
Had strange things happen? Are you going round the twist?

Ben Gamari (bgamari) wrote :

Ahh, yep. Fair enough.

Paul (p37307) wrote :

Here is my boot log showing this error.

fsck from util-linux-ng 2.17.2
fsck from util-linux-ng 2.17.2
fsck from util-linux-ng 2.17.2
fsck from util-linux-ng 2.17.2
fsck from util-linux-ng 2.17.2
init: cryptdisks-enable main process (620) killed by TERM signal
fsck: fsck.ntfs: not found
fsck: Error 2 while executing fsck.ntfs for /dev/sda5
/dev/sda9: clean, 289996/2146304 files, 1538175/8576694 blocks
files: clean, 954963/15269888 files, 35581284/61049000 blocks
/dev/sda6: clean, 65/321280 files, 111278/1281175 blocks
init: ureadahead-other main process (649) terminated with status 4
init: ureadahead-other main process (654) terminated with status 4
home: clean, 98222/6651904 files, 10514934/13297795 blocks
init: ureadahead-other main process (666) terminated with status 4
init: ureadahead-other main process (669) terminated with status 4

I don't think the ureadahead "error" is displayed anymore.

jjungo (j-jungo) wrote :

HI,

I have same problem with Ubuntu 10.04 lucid lynx RC

I am also not able to boot normally, only getting the same error that nullservice gets. However, if I wait for a really long time, then it passes and boots, sometimes. Other times, I have to manually mount the home partition, but then it boots nicely. I've installed upgrades today. I think this is a showstopping bug.

michael brenden (mike-brenden) wrote :

Same show-stopping bug here. Just upgraded from 9.10 to 10.4 and found this nasty surprise. Boot appears normal until final message is printed to terminal:

init:ureadahead-other main process (1004) terminated with status 4

and that's it. Flashing cursor underneath. Can't type anything. SSH not enabled.

I'm totally locked out. Single boot doesn't work. Old kernels don't work.

ALT-F1 and ALT-F7 show the same output. F2 thru F6 show blank screens.

CTRL-ALT-DEL causes a reboot after showing that a few running services are terminating (all remaining processes...deconfiguring network interfaces...)

michael brenden (mike-brenden) wrote :

Is there a way to pass kernel parms or other parms to GRUB to shut off services that were added against my wishes, without my knowledge, and certainly without my consent ? There sure as hell should be. I'd sure love to turn off the "features" Playmouth, AppArmor, ureadahead and sreadahead.

What a stain on the previously excellent "dist-upgrade" track record of ubuntu ! This is very microsoft-ish behavior, to install extra crap, turn it all on, and totally screw what was a previously flawless box: Unauthorized services were added that have now downed a production server. Why can't dist-upgrade be left alone, and not add all this additional crap, none of which is guaranteed or even tested, and all of which will certainly cause user-specific failures.

michael brenden (mike-brenden) wrote :

This debacle has cost my company so far three days of downtime, over the weekend, $250 for new memory (separate bug, related to 2.6.32 kernel failing with ECC memory, incredibly...and this is a "server" OS?...) $350 for new hard drives, delivery/pickup, and hours of time. And we're still nowhere close, other than a long string of denials and 'this bug doesn't belong here' redirection nonsense. Sorry, but the harsh reality is that the last thing to print an output string on a failing terminal is the bug that people will search for. Get it?

Gabe Gorelick (gabegorelick) wrote :

@michael
I assure you that your issues with Ubuntu are taken seriously, but this bug report is not the place to voice them. This bug has a very limited scope, namely that upstart logs unnecessary warnings to the console that mar the boot process. If you have bugs with booting, file a separate bug report, if one for your problem hasn't already been filed, and people will be glad to help you.

michael brenden (mike-brenden) wrote :

Thanks. Found a fix, figured I'd report back here and hopefully address the problems people are googling to get to this page. I'd removed the "quiet splash" from the kernel parms in /boot/grub/menu.lst. Unable to find easy way to remove Plymouth completely I readded those parms via grub. System booted and graphic splash screen reported that three drives were missing, each requiring S to skip or M to fix manually. That was apparently the problem holding everything up: Missing drives from /etc/fstab and removed kernel parms (in attempt to turn off graphic splash), so plymouth couldn't get required keyboard input (S or M). I'd suggest a timeout period before auto-choice of "S" and also an easy way to ditch plymouth and other unwanted additions. Three days and several hundred bucks later. HIH

michael brenden (mike-brenden) wrote :

Found this just now, possibly related. http://swiss.ubuntuforums.org/showthread.php?p=9143980

Paul (p37307) wrote :

@Michael

 I don't see how your "solution" deals with this issue with logging normal exits messages to the boot terminal?

michael brenden (mike-brenden) wrote :

It doesn't. But people googling for "ureadahead-other main process terminated with status 4" are sent to https://bugs.launchpad.net/ubuntu/+source/ureadahead/+bug/484677 which sends them here. That's how I got here. Pretty clear it's not your bug, but it's also clear that many people are reporting that error and googling for and landing here. Maybe all this will help get them to the proper place.

More info
http://blog.fpmurphy.com/2009/09/project-plymouth.html

Paul (p37307) wrote :

People with "ureadahead-other main process terminated with status 4" would be coming to the correct place then. The issues with your hard drives and splash and Plymouth is not related to "ureadahead-other main process terminated with status 4". The bug is "ureadahead-other main process" should not be sending a message to the screen nor the log as it exits normally.

To put it simply, you have another error causing the problems with your hard drive. It is not this bug.

If the system didn't incorrectly log "ureadahead-other main process terminated with status 4" to the screen or log you would assume what ever message before that is the error.

Do you understand? This error did not have anything to do with your system problems. Not everyone who has this error has boot problems. It is completely unrelated. If the error would have read "because the sky is blue process terminated with status 4" then you would presume because the "sky is blue error" caused the other failures of your system.

Therefore, people googling "ureadahead-other main process terminated with status 4" would come to the right place to report and find out what is going on.

I am interested in what happened to your system and what the cause was. But can't and won't address them at this bug. If you don't want to create a new bug, maybe create a new question in the answer section here and others can discuss it and then create a bug report.

@michael brenden & Paul

I have the same issue, came here as described by your discussion above. I however have three mdadm arrays that are on controllers that spin up the drives sequentially. I have used 10.04 for about 3 days before seeing this issue.... I don't know why it surfaced now... I don't know if there was a recent update.

Michael how did you get in to your system to even make this change?

I can't do anything.

This machine has 25TiB of data on it that people need access to immediately.

As a note, maybe this could be my issue.

I originally had my system set to have my home in /home on the main drive (encrypted). I recently moved my home to my mdadm partition. I believe this is the change that made my system not boot.

Gabe Gorelick (gabegorelick) wrote :

@andrew.dunn
This is not the place to discuss any bugs you get during boot. As stated many time above, this bug is only about the fact that upstart logs warnings that shouldn't be logged. Please file a separate bug for your issues, if one hasn't been filed already. Thanks in advance.

Tell me where to log the bug, I am happy to be constructive but I think you will find more and more people coming to this spot for the issue that Michael and I are describing.

Gabe Gorelick (gabegorelick) wrote :

@andrew.dunn
The issue is that the ureadahead warning about terminating with status 4 has nothing to do with any error in system booting (see bug 484677, particularly comment 5). So it's impossible to tell without more information if your bug is the same as michael brenden's. This goes for anyone else who comes to this bug thinking that the ureadahead warning has some relevance to their bug; there's no way to point them to one overarching bug that encompasses their problems, as all their issues are potentially caused by different bugs. For now, just file a separate bug report and people can triage it from their.

I for sure get these messages related to my mdadm arrays spinning up and booting.

I am on a new format now and I believe my issue is not related to this bug... however it should be noted that the spin up of arrays causes the system to hang on boot for a bit and display these messages.

Well, something is wrong with this messages. How come that both my systems (Ubuntu 10.04 and Mint 9) are failing at the same time with the same error on the screen? I'm not saying that ureadahead is responsible for the system hang but it might be the effect of some other problem. The problem is that in all the bug reports opened the answer is the same: the ureadahead does not cause the hang problem, the problem are the messages on the screen. Come on guys, who cares about the messages? The system hang IS the problem! Two systems failing within the same day IS the problem! A LTS failing like this IS the problem! Now, how can we describe the problem so you can consider resolving the main issue? If we open a bug that says "The system hangs with the ureadahead message" are we going to have the same response? Please, don't behave like the obsd guys.

Gabe Gorelick (gabegorelick) wrote :

> Come on guys, who cares about the messages? The system hang IS
> the problem!
I assure you that we in the Ubuntu community care about all bugs reported, especially serious ones like your system hanging. However, THIS bug report only cares about the messages. In general, it is very important to limit the scope of bug reports to one issue and the issue this report is concerned with is the messages. I assure you that if you file a separate bug report documenting your issues, people will do their best to help you.

OK Gabe. You may be right. I was not completely serious when I said no one cares about the messages. I know it is important to solve every issue but what I was trying to say was that most of the people end up in this bu report because they have a major issue like the machine not booting. I checked almost all the bugs with a similar description and everywhere the developers are concentrating more on the message than on the boot failure. Anyway, I did not comment again for this reason. To whom it may concern I managed to boot in Ubuntu but not in Mint with the following procedure, even though I can not explain why it worked:

boot with init=/bin/bash
fsck /dev/sda6 (root partition, still ro)
mount -a
cd /etc
cd rcS.d
for i in `ls S*`; do ./$i start;done
cd ../rc2.d/
for i in `ls S*`; do ./$i start;done
startx

OF COURSE IT FAILED but

when I reset the system it booted normally.

I tried the same with Mint but it did not work. It even brought the problem to the Ubuntu again.
I tried again with the Ubuntu and it worked again. Now I'm working on it.

I can not explain why it is working. The / filesystem is ro so it can not change anything there and I don't have separate /usr or something. I have only two other partitions, one ext3 and one ntfs for keeping data (not system data).

So, in my opinion this could be a hardware (low chances) or a mounting issue. I think the later should be the real problem. Or it might be pure luck but I don't think so.

starslights (starslights) wrote :

Fixed for me on kubuntu Lucid 10.04 LVM on x86 64

Linux xxxxxxxxx 2.6.34-999-generic #201005251006 SMP Tue May 25 09:10:01 UTC 2010 x86_64 GNU/Linux

Christoph Roeder (brightdroid) wrote :

The Problem seems to be LVM related, because when I move my /tmp and /var from LVM to / (xfs) every boot works fine!

Karl Ljungkvist (k-ljungkvist) wrote :

Although it has been made pretty clear that this is not the cause for the problems michael, paul, etc have been having, I still wanted to give my experience of the issue. Many end up here (including me).

I also had the same problem with boot being halted during the fsck, with this error message being the last thing printed. It turned out the problem was due to /etc/fstab having entries which were not connected (i.e. a usb drive).

Nick Sarten (gen-battle) wrote :

Also just got this issue with my Lucid 10.04 x64 install. Personally i think it is related to a dodgy realtek 8168/8169 chipset/driver that i've had alot of problems with:
http://ubuntuforums.org/showthread.php?p=9354274#post9354274
http://ubuntuforums.org/showthread.php?t=538448

Whent he problem ocurred i tried rolling back kernels, booting in safe mode, all sorts. I even tried booting into my windows 7 install on another partition, but the computer restarted halfway through boot. I shut the machine down and unplugged the power for a full minute to hard reset it and managed to get into windows. I'll attempt to boot into Ubuntu now and see if the problem had been resolved completely by the hard reset. If it has, I will update here.

Nick Sarten (gen-battle) wrote :

Still can't boot into Ubuntu. Because of an issue with the ubuntu installer and my hardware, I can't reinstall 10.04 x64 from CD or USB. My only option is to install 9.10 and upgrade from there -_-

Cosmin L (lcosmin) wrote :

Not really related to this bug , but to the boot hang problem people are experiencing; some of you might have a different issue, but here it goes:

When I installed Ubuntu I used a RAID 1 configuration (3 devices, md0 - boot, md1- / , md2 - swap).
At a later point I also created another RAID 1 (md3) on top of which I created a LVM.

Now the boot problem was due to the fact that I added a partition from the LVM to /etc/fstab but forgot to add the new raid (md3) to /etc/mdadm/mdadm.conf, so it wasn't up at boot.

I used a LiveCD to get into the system and add the missing entry to /etc/mdadm/mdadm.conf . Everything seems to be back to normal now. So it seems an invalid line in /etc/fstab (/dev/mapper/lvm-storage in my case) was the culprit. HTH

@Karl: Your hint on the USB-entry in the fstab saved my day.

I added my entry for USB support in VirtualBox:
none /proc/bus/usb usbfs devgid=130,devmode=664 0 0

@Ubuntu team: I came here via Google for the boot hang bug, glad to have found Karl's answer here.

Nicolas Nobelis (nobelis) wrote :

What has been said above is perfectly right (cf Karl's post) : entries in the fstab that are not marked as 'noauto' and for which the device is not connected cause this hang.
I think that the error message is in fact never displayed on the TTY.

The solution is to press the key 'm', you will then drop to a shell. Launch 'busybox sh' for more confort and then edit you fstab file (/etc/fstab) to comment out the guity line.

It did the trick for me.

michael brenden (mike-brenden) wrote :

Removal of the "quiet splash" kernel options prevent Upstart from showing its graphical page...and prevent Upstart from showing the "press C to continue" (instead of fscking disks), 'press S to skip missing drive, M to drop to manual' and other key options. Upstart should show these options on the text screen, too, not just its goofy graphic screen. Most servers (and servers admins I know) always disable all graphics and go for rawest, simplest and most detailed possible text-only booting. The problem is, with Upstart, any problems anywhere in the boot process are not obviously visible (because no Upstart graphics screen to display the errors on), and we're left, usually, with something about ureadahead exiting with status 4, or something similar, that we then immediately google (with quotes), leading us to this page, where the authors and maintainers insist (correctly) that it is not their bug. This of course is infuriating, as infuriating as encountering totally non-helpful, mechanistic "customer service" reps, such as at a post office.

My solutions have been to (1) boot into grub, edit the kernel line, and re-add the "quiet splash" options, then rebooting. This turns on Upstart's graphics page, which can then show error messages and keyboard options for getting thru or around them, and getting to boot. I HATE that I can't easily and promptly get to a single user shell without Upstart graphic screen.

Solution (2) has been to install CDROM and boot into the live/recovery CD, and work on the system that way. Nothing has been uglier since old Fedora installs (and the primary reason I came to Ubuntu!) That I'm thinking of CentOS and RHEL and even Fedora again, because of this Upstart inanity, should throw up huge warning flags to key people at Ubuntu Inc. Clearly dozens have come here and even voiced complaints. Think of the tens of thousands of people who are likely having these problems but who say nothing, if they even get here. Think of the cost to humanity. And to the entire Ubuntu project. Upstart is much much bigger and far more important than butterfly wings proverbially whipping up a distant tsunami.

Solution (3) has been a re-install, and I've had to do this on two different servers. I'm curious to know how other people, particularly those experiencing graphic card problems, X problems, and other insidious problems, are able to cope with the blockade obstacle put in the way by Upstart.

It is absolutely inexcusable to be locked out from single user shell by Upstart or anything else. Does anyone disagree?

PLEASE -FIX- THIS BY SHOWING (AND ALLOWING) EVERYTHING IN TEXT WHEN UPSTART GRAPHICS ARE DISABLED.

Pietro (pietro) wrote :

I have two concerns. One is about the ureadahead error message, which started showing up during my boot sequence only when I updated to Meerkat (Ubuntu 10.10). My second concern is the way that Ubuntu developers seem to be treating this page

I was referred here from https://bugs.launchpad.net/ubuntu/+source/ureadahead/+bug/484677, because this was supposed to be the page where I will find the fix so that minor, irrelevant ureadahead error messages will not be posted to my screen before the Ubuntu splash screen shows up. I do not see the fix here. Furthermore, Scott James Remnant is quite rude and condescending on the previous page, but got he and Gabe seemed to have abandoned this page as of May 2010 without providing a step-by-step diagnostic and fix; and the problem has emerged new for me as of October 2010.

Furthermore Gabe has repeatedly given vague advice here about how other normal, non *nix-head users are supposed to go set up their own bug pages. That is less than helpful: it is dismissive. What I would recommend is: at the head of this page, whoever is entitled to edit the first posting should do the following:
1. add the following message to the first post:
"If you are experiencing boot failures after a unreadahead error message appears on your screen, the boot failure is another problem. Please go to the following page [link] to begin diagnosing the problem."
2. An experienced user of Ubuntu who has actually diagnosed boot problems should set up the new bug page. It should begin with a similar title, so that search-engine users will find it quickly as well. Then you should advise all us regular chickens about what to do next. The fact that the boot failure happens shortly after the ureadahead error-message posting should be at least a clue as to what is happening. But for people who are freaking out because they suddenly cannot boot their machine, what are you going to tell them? Probably:
A. How to boot from a CD or other partition, and then:
B. get the log files from the failing partition to look at them? Maybe:
C. refer them to postings around the Ubuntu site so that they can pursue their own diagnostic decision-trees?

At least get them started and then let them work things out through their own postings on that page. That would be helpful, rather than condescending.
Meanwhile, is there any progress in correcting upstart so that it does NOT post the following message from ureadahead?
"init: ureadahead main process (271) terminated with status 5"

This bug is critical. If no graphical splash screen is set up (e.g. most servers), warnings (e.g. ureadahead) hide real error prompt messages preventing boot from completing.

Thus the user is in front of a useless screen, trying to answer to hidden error messages by guessing what they could be. Working attempts include pressing Ctrl-C, ENTER (Comment 2) or M or S (Comment 23 and Comment 45). Of course such attempts may harm your system as you don't know to which question you will answer.

Scott James Remnant (scott) wrote :
Changed in upstart:
assignee: nobody → Scott James Remnant (scott)
milestone: none → 1.1
status: Triaged → Fix Committed
Changed in upstart:
status: Fix Committed → Fix Released
Dan Quade (danquade) wrote :

When will this fix be available in updates?

You have no idea how annoying it is having to reboot your laptop 10 times before it finally boots up.

Clint Byrum (clint-fewbar) wrote :

Marking Triaged since this has been fixed upstream, and should land in Oneiric as soon as the delta between Ubuntu and Upstream is resolved.

For those who cannot boot, I'm afraid this isn't the actual reason your system cannot boot, so please don't expect *this* issue to be fixed in previous stable releases. As annoying as it is, its not eating data or anything like that, so IMO it is not suitable for a stable release update.

Changed in upstart (Ubuntu):
status: New → Triaged
Steve Langasek (vorlon) wrote :

1.1 is included in oneiric.

Changed in upstart (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers