Need to not display hibernation in UI if do not have enough swap space

Bug #599352 reported by Jerone Young on 2010-06-28
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OEM Priority Project
High
Unassigned
Session Menu
Invalid
Low
Matthew Paul Thomas
indicator-session (Ubuntu)
Low
Unassigned
upower (Ubuntu)
Medium
Martin Pitt

Bug Description

Currently we have logic in the UI to not expose hibernation if there is NO swap. Though what we have been finding is that hibernation will fail at some point if you do not have enough swap. This is especially true for big memory systems.

Basically hibernation should not be exposed unless there is enough TOTAL MEMORY + 4 MB swap space on the system.

I need to figure out where the extra 4MB comes from. But we have found that if you do not have enough swap .. over time as more RAM gets use hibernation will fail.

For a system with 16GB of memory & a SSD drive this can be come an issue as you would need 16GB + 4MB of swap space for hibernation to work 100% of the time.

Related branches

Jerone Young (jerone) wrote :

Also to add we have found that at times the installer (partman) will not create enough space even on larger harddisks.

Jerone Young (jerone) wrote :

This bug is somewhat related (not exactly). Though this is what we are doing to work around the issue when seen:
https://bugs.launchpad.net/ubuntu/+source/upower/+bug/432598

James Ferguson (jamesf) wrote :

Surely the size needed is not an exact science. You need the amount of memory /currently in use/ (excluding caching) plus 4Mb, not the amount of RAM present. If the machine is using swap then total memory + 4Mb may not be enough. IIRC, the algorithm is that cache it thrown out, and everything remaining in memory is copied to swap.

Therefore no swap size is guaranteed to allow hibernation. Even if you have 4GB RAM and 32GB swap, if you happen to have 33GB virtual memory in use, you cannot hibernate.

The best solution would be a real-time check that the system at present is using less memory than the swap size (+4Mb) - even better it should have a message in it's place, like "Hibernation not available - too much memory in use. Shut down applications and retry".

Colin Ian King (colin-king) wrote :

There are two things to consider: physical memory to build an image before writing to disk and also available swap to do the write.

The kernel generates an memory image of the frozen system before writing it to disk.

Any pages in the zone regions that can be freed are freed. I believe this includes pages that are cached but not dirty (i.e. don't need writing back to disk).

Next, each zone is scanned an the number of present pages are totalled up, excluding the "nosave memory" (see later).
You can see the zone info in /proc/zoneinfo, the "present" field has the number of total pages present. The total excludes the "nosave memory" regions - this info can be seen by:

dmesg |grep "PM: Registered nosave memory:"

.. this tells you the nosave regions. These are pages that are marked in specific regions that do not need saving. These need to be turned into page sizes and taken from the "present" field from the zone info to give you an idea of how many pages need dumping in the final hibernate image.

However, the kernel calculates all of this in two sweeps, first the low-mem pages then the high-mem pages. It's rather complex to say the least.

Also, one needs to add in page meta information. This is calculated as: number of image pages * 8 and rounded up to the nearest page. In other words, for every 1M, we have 256 pages, and hence 2K of meta page info. So for a 1GB image, we have 2M of meta data too.

That's the image size. However, one needs to be able to allocate a hunk of memory for this image in physical memory before dumping to disk, and also have enough page space on swap for this image too.

Cheat Mode:

Instead of getting the calculator out and figuring out how big the swap needs to be, one can get a feel for the size by just doing a successful syspend and grepping through the output from dmesg and look for: "PM: Need to copy". This will tell you how many pages needed to be copied. However, this is only sensible for the desktop scenario you just hibernated. Some users may overload their system with many apps, in which case the hibernation image will be huge, in fact, there may not be enough free physical memory to generate an internal image before dumping it to swap.

The kernel will report the size of the hibernation image too, which can be grepped for. Look for "PM: Hibernation image created".

There may be mileage in shrinking the working set down by forcibly dropping pages, e.g.:

sync
sleep 1
sync
# Freeing the page cache:
echo 1 > /proc/sys/vm/drop_caches
# Free dentries and inodes:
echo 2 > /proc/sys/vm/drop_caches
# Free the page cache, dentries and the inodes:
echo 3 > /proc/sys/vm/drop_caches

But this maybe redundant. I've not experimented. And dropping caches is painful and if you don't sync it may cause data loss.

Changed in oem-priority:
assignee: nobody → Canonical Platform QA Team (canonical-platform-qa)
Jerone Young (jerone) on 2010-06-28
visibility: private → public
Pedro Villavicencio (pedro) wrote :

Marking the report as Triaged and assigning the tasks to the teams that are taking care of those package, thanks Jerone.

Changed in indicator-session (Ubuntu):
assignee: nobody → Canonical Desktop Experience Team (canonical-dx-team)
importance: Undecided → Low
status: New → Triaged
Changed in upower (Ubuntu):
assignee: nobody → Canonical Desktop Team (canonical-desktop-team)
importance: Undecided → Low
status: New → Triaged
Changed in oem-priority:
assignee: Canonical Platform QA Team (canonical-platform-qa) → nobody
Martin Pitt (pitti) wrote :

The indicator just asks upower whether it's able to hibernate. Why was that task added, doesn't it show the error message that upower returns? I recently tested it in XFCE, and there that works, so upower is sending an appropriate error.

upower itself already does have logic to check whether we have enough swap space. Currently it says "yes" if there is at least (active memory) * 0.8 available swap space. For the corner cases where this isn't enough, the UI should return an appropriate error message. Also, I regularly hibernate with less swap than I have memory, just because most memory is used by caches. This seems to do fine for most cases. Is there some practical scenario where this fails? We could certainly bump the 0.8 to 1, in which case it should almost always succeed.

Ted Gould (ted) wrote :

It seems to me that indicator-session is doing the right thing here. We're not currently displaying a dialog if it fails as we couldn't figure out what it would say. "Failed to Hibernate" -- well, the fact that the display is on probably told you that already :)

Changed in indicator-session:
status: New → Incomplete
Changed in indicator-session (Ubuntu):
assignee: Canonical Desktop Experience Team (canonical-dx-team) → nobody
Changed in upower (Ubuntu):
assignee: Canonical Desktop Team (canonical-desktop-team) → nobody
Changed in indicator-session (Ubuntu):
status: Triaged → Incomplete
Martin Pitt (pitti) on 2010-07-04
Changed in upower (Ubuntu):
assignee: nobody → Canonical Desktop Team (canonical-desktop-team)
status: Triaged → Incomplete
Jerone Young (jerone) wrote :

@Martin
             Yes we need to move it from .8 to 1 . This has to work 100% of the time and not expose the ability to hibernate to the users if the system knows it can't do it.

@Ted
        Having the failure message is part of the issue. If you know hibernate is going to fail do not expose it to the user.

Changed in indicator-session:
status: Incomplete → Confirmed
Changed in indicator-session (Ubuntu):
status: Incomplete → Confirmed
Changed in upower (Ubuntu):
status: Incomplete → Confirmed

Jerone Young [2010-07-04 22:09 -0000]:
> Having the failure message is part of the issue. If you know
> hibernate is going to fail do not expose it to the user.

We don't know in advance that hibernate is going to fail. We only know
that after the fact.

@Martin
         What I was saying is if we can calculate to know that we will or will not have enough space then that would prevent from failing, at least from not having enough swap space.

On Sun, 2010-07-04 at 22:09 +0000, Jerone Young wrote:
> Having the failure message is part of the issue. If you know hibernate is
> going to fail do not expose it to the user.

What should the error message say? Obviously a message saying
"Hibernate failed" wouldn't provide new information. We don't really
know why it failed. I don't think we provide a way to dynamically
adjust swap space, so there is no way to fix it.

The only thing I can think of is that we'd run apport so that a bug
could be filled. But, I'm not sure how we'd distinguish between a
hibernate that failed an a hibernation that was really short -- seems
possible though.

  affects indicator-session
  status incomplete

  affects ubuntu/indicator-session
  status incomplete

Changed in indicator-session:
status: Confirmed → Incomplete
Changed in indicator-session (Ubuntu):
status: Confirmed → Incomplete

As for the reason, it should be mentioned in /var/log/pm-hibernate.log, or at least in /var/log/kern.log. But those are mostly suitable for attaching to bug reports. As for an error message for display, upower should provide that through DBusErrors.

Martin Pitt (pitti) wrote :

Colin, thanks for your detailled explanation in comment 4. You didn't mention compression, though. Isn't the image written to disk compressed? If so, it's rather hard to predict, of course, but it might explain why upstream chose that "0.8" factor. Google isn't very helpful in rsolving this question, I'm afraid.

Once we can ascertain whether or not native Linux hibernation uses compression, I can take this discussion to the upstream ML.

Colin Ian King (colin-king) wrote :

To clarify, the kernel does not compress the image during hibernation.

Martin Pitt (pitti) wrote :

Discussed with upstream, I pushed the fix to upstream trunk:
http://cgit.freedesktop.org/upower/commit/?id=fc7c6e003a626367558300ce8566e346758ad8e7

Changed in upower (Ubuntu):
assignee: Canonical Desktop Team (canonical-desktop-team) → Martin Pitt (pitti)
status: Confirmed → Fix Committed
Martin Pitt (pitti) wrote :

Jerone, do we need this backported to lucid in OEM projects (the common PPA)?

Changed in oem-priority:
assignee: nobody → Martin Pitt (pitti)
status: New → In Progress
Martin Pitt (pitti) wrote :

I just checked the code again, and it was actually the other way around. The previous 80% was a bug in the wrong direction, it meant that you can only hibernate if your active memory is up to 80% of available swap space (not the inverse, as I originally thought). So this severely erred on the side of caution, and I'll fix it upstream to be 99% instead.

I also tested a hibernate attempt with not having enough swap, and confirm that upower fails with a D-Bus error message right away:

$ dbus-send --system --print-reply --dest=org.freedesktop.UPower --reply-timeout=60000 /org/freedesktop/UPower org.freedesktop.UPower.Hibernate
Error org.freedesktop.UPower.GeneralError: Not enough swap space

However, this isn't shown anywhere in the UI.

Now, the thing which is actually wrong in upower is that it checks available swap/used memory at the time when it starts up, instead of when you attempt to hibernate. I'll fix that.

Changed in upower (Ubuntu):
importance: Low → Medium
status: Fix Committed → In Progress
Martin Pitt (pitti) wrote :

Fixed testing for swap space upstream now:
http://cgit.freedesktop.org/upower/commit/?id=79c7b27332721a8ef6f24db585939de200927f07

It's planned to do a new release tomorrow, which I'll upload to sid and maverick.

Changed in upower (Ubuntu):
status: In Progress → Fix Committed
Jerone Young (jerone) wrote :

@Martin
          Not really for 10.04. Only moving forward for 10.10 and up. Having it in Lucid is a "nice to have". Thanks for looking into this Martin!

Jerone Young (jerone) wrote :

@Martin
          Was upower the only thing that needed this change? I plan to test out with build of 10.10 this week.

Jerone Young (jerone) on 2010-07-12
Changed in oem-priority:
importance: Undecided → High

Jerone Young [2010-07-12 0:18 -0000]:
> Was upower the only thing that needed this change?

The upower change should make hibernation robust enough so that it
hardly ever fails after trying. If you don't have swap (or encrypted
swap etc.), the indicator now does not offer hibernation at all. It
will fail immediately now if you have very little swap/too much memory
usage. That error message is not displayed anywhere, though (the
indicator task).

> I plan to test out with build of 10.10 this week.

It's not yet in Maverick, just committed upstream so far. I asked
upstream for a new release, and it's planned for today.

This bug was fixed in the package upower - 0.9.5-1

---------------
upower (0.9.5-1) unstable; urgency=low

  * New upstream release:
    - Dynamic testing for enough hibernate swap. (LP: #599352)
    - Port UPower to libusb1 to avoid unfixable crashes. (LP: #522827)
  * debian/rules: Build a PO template during package build, for translation
    systems. (LP: #538321)
  * debian/control: Switch libusb-dev build dependency to libusb-1.0-0-dev,
    and add libimobiledevice-dev to support the new backend.
  * debian/control: Bump Standards-Version to 3.9.0; no changes necessary.
  * Drop transitional devicekit-power-doc package. It's been in testing long
    enough, and we do not want to carry it into the squeeze release.
 -- Martin Pitt <email address hidden> Mon, 12 Jul 2010 14:43:12 +0200

Changed in upower (Ubuntu):
status: Fix Committed → Fix Released
Martin Pitt (pitti) wrote :

I did the upower part now. Not sure how much you need the error displaying in the indicator.

Changed in oem-priority:
assignee: Martin Pitt (pitti) → nobody
status: In Progress → Triaged
Jerone Young (jerone) wrote :

@Maritn
         Thanks for the work again. No need to display the error. Just need to make sure it doesn't show up in the indicator & session menu.

Martin Pitt (pitti) wrote :

It's not shown if there is too little swap for hibernation when the session is started. However, since we cannot constantly monitor the active RAM, it can happen that hibernate fails if right after login the active memory is less than available swap, but over time available swap decreases, active memory increases, and memory becomes greater than swap.

Martin Pitt (pitti) wrote :

According to comment 24 it seems that Jerone is happy with this now?

Changed in oem-priority:
status: Triaged → Fix Released
Ted Gould (ted) on 2010-09-28
Changed in indicator-session:
status: Incomplete → Confirmed
importance: Undecided → Low
Changed in indicator-session (Ubuntu):
status: Incomplete → Confirmed
Tony Espy (awe) wrote :

@Ted

Just curious why you rated this as Low?

If a user initiates hibernate from the UI, and it fails, there's no error displayed to a user. Couldn't this indirectly lead to loss of a user's data?

On Tue, 2010-09-28 at 22:46 +0000, Tony Espy wrote:
> Just curious why you rated this as Low?
>
> If a user initiates hibernate from the UI, and it fails, there's no
> error displayed to a user. Couldn't this indirectly lead to loss of a
> user's data?

I guess that I feel the cases that would lead to the loss of data the
better indicator is the lack of the screen being on. That's a huge
notification that the hibernate failed, a notification would just give
more information on why it failed not any better indication that it
failed.

Ted Gould (ted) on 2010-10-22
Changed in indicator-session:
assignee: nobody → Matthew Paul Thomas (mpt)
summary: - [Maverick] Need to not display hibernation in UI if do not have enough
- swap space
+ Need to not display hibernation in UI if do not have enough swap space
Changed in indicator-session (Ubuntu):
status: Confirmed → Triaged
Omer Akram (om26er) on 2012-01-23
Changed in indicator-session:
status: Confirmed → Triaged
Matthew Paul Thomas (mpt) wrote :

I don't understand why this is filed against indicator-session. If Hibernate fails, the error message you get (if any) should be the same regardless of whether you triggered it from indicator-session, from the power button dialog, from the battery getting critically low, or any other way. So it doesn't belong in the indicator-session code.

In any case, this bug is fixed in that the menu no longer shows Hibernate when there isn't enough memory. Then again, it doesn't show Hibernate in any other case, either. If we reintroduce the Hibernate feature later, we may need to revisit this.

Changed in indicator-session:
status: Triaged → Invalid
Changed in indicator-session (Ubuntu):
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Related blueprints