UNEXPECTED INCONSISTENCY: fails to boot ("last mount time is in the future")

[Daniel Hahler]

Binary package hint: mountall

I get "UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY" on booting, then it fails to start or provide me a shell.

I had this happening since some days already: the first boot would tell me that fsck failed, but a reboot (straight after pressing Ctrl-D) went through without any problems.

Now (IIRC) Ctrl-D did not do anything. I've used Alt-SysRq-K, which caused a kernel panic ("trying to kill init").

I'm not sure if all that FAIL is caused by "Superblock last mount time is in the future". It's mentioned in the e2fsprogs changelog at least:
  * E2fsck will now print much fuller information when the last mount
    time or last written time is in the future, since most people can't
    seem to believe their distribution has buggy init scripts, or they
    have a failed CMOS/RTS clock battery.

WORKAROUND:
Adding the following to "[options]" in /etc/e2fsck.conf, running update-initramfs and rebooting fixed it for me:
buggy_init_scripts = 1

[Daniel Hahler (blueyed)]

Running fsck manually:
ubuntu@ubuntu:/mnt$ sudo fsck /dev/md0
fsck 1.40.8 (13-Mar-2008)
e2fsck 1.40.8 (13-Mar-2008)
/dev/md0 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/md0: 68/50200 files (10.3% non-contiguous), 90611/200704 blocks

[Scott James Remnant (Canonical) (canonical-scott)]

It's hard to figure out what you think the problem is here, due to the number of different things you report.

From the pictures you attached, it looks like mountall is correctly running fsck on each of your disks, and correctly identifies that one of them has a problem.

It looks like it *waits* until the fsck on the other disks is complete (as intended) before exiting, and it looks like it exits with the correct value.

It also looks like the mountall-shell.conf job starts, and you get a recovery shell.

Is the problem just that ^D does nothing?

[Daniel Hahler (blueyed)]

I get no recovery shell - or I have not realized that I'm in there already. (wouldn't I have to give a root password, which I do not have?!)

The main problem appears to be that I get the fsck failure _all the time_ since a few days, but it only appears on the first boot. Normally, the second try would run fsck actually, but do not report any problems (AFAICS).

But now things are different: I do not get a second try (AFAICS).

I'm not reporting a lot of different things - I just don't know what the root problem is.

After all, the "superblock last mount time is in the future" shouldn't cause a "UNEXPECTED INCONSISTENCY" - I've mounted the device from a live CD just before rebooting. Therefore, it's all related to the changelog entry of e2fsprogs I've quoted in the description maybe.
But, it should not "crash" so hard.

Also, I've done a manual fsck from the LiveCD, and rebooted. The result has been the same (at least in its outcome).

[Glenn Brumfield (gbrumfield)]

Ubuntu Karmic alpha 6 (encrypted hard drive) updated to 18 Sep 09 in VBox 3.0.6 on Ubuntu 3.04.3 updated to 18 Sep 09.

Same error message after todays updates. We've had the superblock issue since late alpha 5, latest updates in clean alpha 6 install won't even allow manual fsck because it passes right by the maintenance shell and attempts to mount the encrypted hard drive.

See attached screenshot.

[Daniel Hahler (blueyed)]

Let me guess, Glenn: your timezone is GMT+4?!

Mine is GMT+2 and it always says that it has been mounted ~2h hours ago.

This appears to be an issue with timezone settings being applied to late or not recognized correctly.

Probably no bug in mountall, I've filed it again by mistake anyways.

Scott (or any other), please assign it to the correct package.

[Daniel Hahler (blueyed)]

Assigning to e2fsprogs, that's where the message comes from at least.

I'm trying http://<email address hidden>/msg388846.html now.

[Louis Simard (louis-simard-deactivatedaccount)]

Daniel Hahler's bug appears to be an instance of bug 423247, from the messages appearing on the screen; Glenn Brumfield's appears to be another bug completely, an ecryptfs-related one. Glenn, I would suggest trying to find a bug related to ecryptfs and filing a new one if you find none; your new bug, if applicable, should be a repost of your comment #10.

Thanks for the report, and keep on testing! :)

[Louis Simard (louis-simard-deactivatedaccount)]

Re-reading the text in Glenn's image, this looks like yet another instance of bug 423247.

The workaround is to type 'fsck /dev/THEDEVICEINERROR' at the recovery shell (don't type Ctrl+D, type your root password), and reboot.

If you don't have a root password, press Ctrl+D to continue booting if you can, then get another terminal and type:

sudo -s
passwd
<same password twice>
init 6

to get back to the boot process and its failed fsck.

[Daniel Hahler (blueyed)]

Adding the following to options in /etc/e2fsck.conf, running update-initramfs and rebooting fixed it for me:
buggy_init_scripts = 1

So, neither the fix for bug 423247 (first time reboot) nor bug 427822 (the more general one) have fixed it for, but this workaround does.

Finally, I'm in my KDE4 again.

[Daniel Hahler (blueyed)]

Not an e2fsprogs issue. Please assign a matching package, I don't know.

[Jeffrey Baker (jwbaker)]

It seems to me that the problem here is with the new upstart/initscripts and the ordering of hwclock, tz, and mount. Every time I reboot my Karmic system the clock goes back another 6 hours (my timezone is 6 hours from GMT) and every time I boot it complains that the superblock was last mounted in the future. The bug appeared when the new upstart/init system was uploaded.

Also "medium" hardly seems to characterize this bug. It prevents systems from booting.

[udippel (udippel)]

Please check if this is similar to bug 433829 that I just encountered after uprade to Alpha 6, and reported immediately.

[Scott James Remnant (Canonical) (canonical-scott)]

Can people stop reassigning bugs around to random packages?

Daniel, the screenshots you provided CLEARLY show that the console is asking for your root password to fix the filesystem problem.

Your report claims otherwise, thus I'm marking this Invalid.

[udippel (udippel)]

Scott James Remnant wrote:
>
> Daniel, the screenshots you provided CLEARLY show that the console is
> asking for your root password to fix the filesystem problem.
>

Is it? Sure? In my case the messages are flashing by, for all the 5 or
so partitions, in circles. If I gave you a screenshot, you'd read the
same. Except that when I type the password, it doesn't stop the flashing
by, and does not give me root shell. It just keeps flashing by all those
consistencies.
But I don't know about Daniel's case, tried to read and understand, but
could not make up my mind if we hit the same snag.

Uwe

[Daniel Hahler (blueyed)]

> Daniel, the screenshots you provided CLEARLY show that the console is
> asking for your root password to fix the filesystem problem.
>
> Your report claims otherwise, thus I'm marking this Invalid.

a) I have no root password/user setup (this is the default!)
b) the "UNEXPECTED INCONSISTENCY" can be worked around with the
"buggy_init_scripts = 1" in e2fsck.conf

This sounds like it's not really invalid.

Anyway, I agree that the report is messy - I just described all symptoms I've
seen.

The root problem here is still there probably, though.

Thanks,
Daniel

--
http://daniel.hahler.de/

[Jonne (jonathan-vanherpe)]

I'm seeing the same thing in a virtualbox image of Karmic. I could try and do a screencast if you really need me to. It asks me to do an fsck every time I reboot. I'm running Karmic on a netbook too, but on there I don't see this.

[udippel (udippel)]

> I'm seeing the same thing in a virtualbox image of Karmic. I could try
> and do a screencast if you really need me to. It asks me to do an fsck
> every time I reboot. I'm running Karmic on a netbook too, but on there I
> don't see this.

Yes, amazing. I myself have the same experience, on the netbook
everything was and is fine, all the time and any time (knock on wood).
Same situation, same repository, on my desktop was and is a pile of
c***. I was thinking it might have to make with the partitioning: on
the netbook it is just '/' and swap; on the desktop I have some 5
partitions.
But how could this affect the 'mount -a'? Or fsck?

Uwe

[Daniel Hahler (blueyed)]

Re-opening. It's clearly a bug (but I have not checked, if it is solved, i.e. the workaround (buggy_init_scripts = 1) is still needed - because of bug 444563).
Assigning to mountall, but I'm not sure about it.

[David Gibson (dwg)]

It seems like there are two main issues here. First, what is getting the dates wrong so that fsck reports the inconsistency on the filesystem. It sounds like that is covered in one of the other bugs referenced.

Second problem, is why is mountall just jamming up and not dropping back to a shell to allow manual fscks. I have defeintely seen this behaviour. For some reason when I hit the timestamp inconsistency on / and /boot, mountall died on the failing fsck and prompted for a root shell. However on another (custom) mount from fstab, it seemed like it jammed up mountall's fsck queue. mountall kept running, did not drop to a shell, but never attempted to fsck anything after the thing which failed - including /home.

[udippel (udippel)]

David Gibson wrote:
> It seems like there are two main issues here. First, what is getting
> the dates wrong so that fsck reports the inconsistency on the
> filesystem. It sounds like that is covered in one of the other bugs
> referenced.
>
> Second problem, is why is mountall just jamming up and not dropping back
> to a shell to allow manual fscks. I have defeintely seen this
> behaviour. For some reason when I hit the timestamp inconsistency on /
> and /boot, mountall died on the failing fsck and prompted for a root
> shell. However on another (custom) mount from fstab, it seemed like it
> jammed up mountall's fsck queue. mountall kept running, did not drop to
> a shell, but never attempted to fsck anything after the thing which
> failed - including /home.
>
>

Finally someone got it right. Since I filed my bug, most seem to close
an eye or two, and tell me and us that everything is hunky dory (after
some kernel update). It isn't. Asking how many hours one is off UTC is
the wrong question. Seriously. The wrong mindset. It must not matter at
all. Let it be 01-01-1970, and it still needs to come back either with a
proper check (why not, by the way?), or at least try mountall, and drop
to a console. It does neither. The latter is a serious bug, pointing to
badly audited code, and a failure to degrade gracefully. The former is
still a bug, because the file system isn't too corrupt to easily recover
the journal: The previous shutdown was clean. Therefore, there is zero
reason to not come back easily properly on its own, without even
dropping to a rescue console.

So we have three bugs actually: The third one is the getting wrong of
the dates.

[Glenn Brumfield (brumfield-glenn)]

Ubuntu Karmic beta (encrypted hard drive) updated to 11 Oct 09 in VBox 3.0.8 on Ubuntu 3.04.3 updated to 11 Oct 09.

Update:
Still having the same error message. We've had the superblock issue since late alpha 5, latest updates in clean beta install won't even allow manual fsck because it passes right by the maintenance shell and attempts to mount the encrypted hard drive, then hangs.

If I wait long enough to get past the incorrect "last mount time", the boot proceeds normally and everything works well. If I reboot, the reboot seems to go normally. However if I shut down and power off, I must again wait until I am past the erroneous "last mount time". Manual fsck from Live-CD does not fix the issue.

The screenshot from my previous post (#10) appears identical to what I am seeing on a failed boot.

[Luke (lukekuhn)]

  There is another workaround for this, though you will have to have a root password for a fsck run at the end of daylight savings time: Set the BIOS clock to local time, run sudo dpkg-reconfigure tzdata, set the timezone to Universal or whatever they call it (I forgot the exact designation), making sure that local time and GMT are listed as the same time. Then, set the machine time as shown in gnome-panel to current local time(otherwise the hardware clock might be the one that changes, as you will see a wrong local time). Now, there is only one time in the machine-and no clock changes during the boot process to foul things up.

You will have to set the clock back and forth for Daylight Saving Time if used in your area, and there are a few things like astronomical and navigation work for which this is not suited, but it worked for me when nothing else would and I did not know about the "buggy init scripts" workaround.

  On my Atom laptop, this bug also was leaving me with the clock set hours ahead unless I clicked on the clock to open the calender anyway, so this fix solved that problem as well. This indicates the bug is NOT in mountall but somewhere else, as the clock should not be getting moved after boot to a wrong time anyway.

[udippel (udippel)]

On Mon, Oct 12, 2009 at 9:38 AM, Luke <email address hidden> wrote:

> so this fix solved that problem as well. This indicates the bug is NOT
> in mountall but somewhere else, as the clock should not be getting moved
> after boot to a wrong time anyway.

Read the whole thread, please. There is more than one bug.
Of course, there is a bug in mountall (or any program calling it),
because it cycles. That's fundamentally wrong; graceful degradation
prescribes that it ends at a rescue shell; not at infinite cycling.

You are correct, though, with your notion that it is yet another bug
that the clock gets moved after boot.

Uwe