Temporary /tmp on a tmpfs

Bug #18661 reported by John Moser
48
This bug affects 5 people
Affects Status Importance Assigned to Milestone
debian-installer (Ubuntu)
Confirmed
Wishlist
Unassigned

Bug Description

Because temporary files are temporary, there is no need to eventually write them
to disk. Tasks which rapidly create, alter, and remove temporary files will
have extra write-back cache flushed to disk needlessly (i.e. journals?), and
will incur checks in the file system driver to locate enough allocation space.
This can slow down tasks relying on heavy short term usage of /tmp.

I find that a more optimal configuration seems to be to use a tmpfs on /tmp and
/var/tmp, to avoid disk usage for short-lived files such as files being used by
apt or synaptic. The space on a tmpfs is treated similarly to disk cache,
except that it gets swapped out rather than invalidated; the decision to swap is
made under the disk cache rules.

Using a tmpfs, rapidly created, altered, and deleted files stay in memory until
removed, and space allocation is the much simpler task of checking how much
memory is available and how much is used in the tmpfs; in the worst case, longer
lived file data will swap out. Swapping is less intrusive than managing a file
system, so it is negligibly faster, though not noticibly. The avoided disk
access is nice, but also /tmp self-cleans on reboot and is almost absolutely
unrecoverable; a crypted swap partition would make it impossible to recover.

Below are fstab lines that set up a tmpfs on any amount of ram (1M, 256M, 4G,
etc). I always set it as 2G in size because that much will never be used; it
uses the same amount of memory no matter the value of size.

# temporary file systems
none /tmp tmpfs size=2G,nr_inodes=200k,mode=01777 0 0
none /var/tmp tmpfs size=2G,nr_inodes=200k,mode=01777 0 0

UPDATE:
/tmp could be on a tmpfs, but doing the same for /var/tmp would violate the FHS because /var/tmp is expected to persist after reboot and a tmpfs would not do so.

Revision history for this message
John Moser (nigelenki) wrote :

Remarking this as enhancement and changing package to debian-installer

Revision history for this message
John Moser (nigelenki) wrote :

Found a design flaw: if you try to burn a DVD with this scheme, nautilus tries to make a multi-gigabyte file and starts flushing all ram into swap, which hangs the machine due to massive swap thrashing.

I'm going to file this as a bug against Nautilus. Oddly enough, if you happen to have a 4 gig / (ubuntu installs in 1.8G; I have a 6G / with 3.7G used) and you try to burn a full 4.2G DVD, it will fill up / and fail.

Revision history for this message
Vassilis Pandis (pandisv) wrote :

This is not a problem just with burning DVDs. I assume that writing 10 mp3s as an audio disc would also be problematic. IMVHO, it's unwise to expect people to have that much RAM for tmpfs.

The issue you describe with nautilus is well known. Temporary file handling in Gnome isn't ideal and affects several programs, not just nautilus. There are more problems: a user tries to compile openoffice for example, or KDE - again unless there's loads of RAM around, they'll be in trouble.

John, do you agree with rejecting this? Thank you for taking time to report it.

Revision history for this message
Sascha Silbe (sascha-ubuntu-launchpad) wrote :

I'm running ALL of my systems successfully with a tmpfs as /tmp. Applications that need loads of space usually can be configured to use a different directory (on my system /cdtemp, which is a separate LV so it won't disturb anything else).
Systems with a "real", but small /tmp (like all my systems before I switched to tmpfs) will run into exactly the same issues you mentioned, so that's not an argument against tmpfs.
The installer doesn't need to default to using tmpfs for /tmp, it just has to support it, so users who like it (and know how to configure or even fix their applications) can choose it at installation time, instead of having to boot single-user mode to change this (or reboot after changing fstab, which isn't much better).

Revision history for this message
John Moser (nigelenki) wrote :

I believe that you may find other issues. You can fill up / this way too; I use a several gigabyte / and a separarte /home, because I reinstall ubuntu a lot (due to me breaking it with hackery) and it does NOT SUPPORT WIPING SYSTEM DIRECTORIES BUT LEAVING /home ALONE. (we can ignore running 32/64 bit systems etc etc, the typical user may want to reinstall without backing up say a 400 gig hard drive?)

Interestingly this issue doesn't excatly have a solution. /tmp is designed to do what /tmp does; and linking /tmp/$USER to ~/.tmp or such and changing a variable (there is a pam tmpdir module! We should really get Ubuntu working with it some day) will only migrate the problem, ASSUMING the user even uses a separate /home (most ubuntu users won't because we don't tell them to).

I used a separate /tmp because 1) Ubuntu never clears /tmp on boot (so if it gets big it stays big; and it fills with junk); 2) stuff in /tmp may be stuff you want to die when the system is turned off (temporary cached mysql passwords..?); 3) It uses the swap file and RAM, so it's theoretically faster "by magic" than using the disk. We can deal with (1); (2) is really where you need tmpfs because a sudden system crash (power drop) won't clear sensitive data from /tmp (we can boot a livecd and...); (3) is a negligible non-issue that only picky people like me care about.

Considering the issue further, I make the following recommendations:

 - Do not make this a typical installer option. Perhaps a "power user" option or the like from the expert install (do we still have that?).
 - Consider creating a "Power User's Control Panel" for Ubuntu (I will file a bug on this), with this as an option.
 - In the mean time, WIPE /tmp on each boot AND shutdown. If we wipe it on shutdown, then wiping it on boot is a no-op; if the system goes down (power outage), we can get it on reboot.
 - /var/tmp is probably temporary as well, but I'm not honestly sure how temporary. I know /tmp is considered dead after application run ends. Possibly wipe this too anyway (rm -rf /tmp/* /var/tmp/*).
 - Figure a way for certain things-- meaning Nautilus CD burner-- to use pipes for CD/DVD burning and buffer the data to be burned before starting. We don't need / ballooning by 5 gigs at a time just because someone decided to burn a DVD, not to mention dual-layer DVDs (8 gigs).
 - When we support Video DVD burning these problems are going to pop up again; these decode the video and re-encode it to DVD size, so the space usage on disk doubles. Dual-layer DVD, 16 gigs space needed? Umm.

There's no one-size-fits-all here.

Revision history for this message
Sascha Silbe (sascha-ubuntu-launchpad) wrote :

Having sensitive data in swap is no better than having it on a normal partition. If you employ encryption, you can do it the same way for both swap and regular file systems (you could even use random keys for /tmp and create the filesystem on boot), so nothing's gained (w.r.t. confidentiality) by using tmpfs for /tmp.

Regarding /var/tmp: A usual policy is to clean files older (mtime) than 7 days. AFAIR this has been default on Debian potato.

Revision history for this message
Vineet Kumar (vineet) wrote :

I don't think tmpfs is appropriate for /var/tmp. Having it disappear after reboot violates the FHS, and along with it, many admins' expectations.

/var/tmp : Temporary files preserved between system reboots
Purpose

The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp.

Files and directories located in /var/tmp must not be deleted when the system is booted. Although data stored in /var/tmp is typically deleted in a site-specific manner, it is recommended that deletions occur at a less frequent interval than /tmp.

http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARYFILESPRESERVEDBETWEE

Revision history for this message
Motin (motin) wrote :

So, basically if the Nautilus temporary directory can be set manually - using tmpfs for /tmp (but not /var/tmp) is a reasonable setup for most desktops?

Sounds especially important for laptops - where frequent disk access significantly reduces battery and hard drive strain.

Revision history for this message
gwern (gwern0) wrote :

I actually ran into a situation where a ramfs /tmp would have been an absolute life-saver. As it was, my experience was very unpleasant until I managed to wipe and reinstall Ubuntu.

---

So I went and installed 8.04 on my desktop. It was fairly difficult as I was using a CD-ROM drive which is ancient and generally unrecognized by Linux livecds. But eventually with a little .iso and Grub hackery, I managed to install onto the normal 3 partitions - /, /home, and swap. And everything was good: I installed xmonad and zsh and basically got everything back up to speed. And I was pleased with the strides Ubuntu had made since I left it for Gentoo lo those many years ago.

And then 2 or 3 days ago, it rebooted while I was gone and got stuck. Some sort of disk error (if you are curious what the error looked like, see attached). And what does Ubuntu do by default when there are disk errors? It mounts / read-only. Now, the disk was perfectly alright more or less. I could still mount /home, / was still perfectly readable. The disk errors were pesky, but I am convinced it was the fault of some update (as those errors never manifested under Gentoo, but did on every boot of the now-corrupted Ubuntu install).

The *real* kicker here was that I could not even run X or do anything useful. Why? Because most big apps want to create stuff in /tmp, and /tmp is by default in /, and / was now determinedly read-only. OK, so I go to edit /etc/fstab. Whoops. It is read-only too! OK, so I go to unmount / and mount rw. Except... yes, / cannot be unmounted because it is busy! OK, so I go to use a livecd to edit fstab to remove this fscking ro thing - and then the CD-ROM drive thing I mentioned bit me.

I ordered a new DVD drive from Newegg which was having a sale, and it arrived today, and so I could reinstall. It has worked so far. (The disk error did not manifest again, further convincing me one of the updates was screwy.)

But the fundamental problem was that I was stuck in the console and could hardly do anything because /tmp was on-disk. Very unpleasant.

Revision history for this message
Kevin Hunter (hunteke) wrote :

> OK, so I go to edit /etc/fstab. Whoops. It is read-only too! OK,
> so I go to unmount / and mount rw. Except... yes, / cannot be
> unmounted because it is busy! OK, so I go to use a livecd to
> edit fstab to remove this fscking ro thing

A quick tip for you, since this bit you: you don't want to umount /. That will do things like remove your access to the very program you need to remount it rw: mount. So, that bit actually saved you. The option you needed was:

# mount -o remount,rw /

This will remount it rw in place. I have had to employ it in quite a few similar "misbehaving drive" situations.

Colin Watson (cjwatson)
Changed in debian-installer:
status: New → Triaged
Revision history for this message
Phillip Susi (psusi) wrote :

This is already done by default, and has been at least as far back as breezy AFAIK, so marking as invalid.

Changed in debian-installer:
status: Triaged → Invalid
Revision history for this message
Jonathan Ernst (jonathan.ernst) wrote :

@Philip
This bug (or feature request) is having /tmp and /var/tmp in ram instead of disk. This has never been the default (or an installation option).

Changed in debian-installer:
status: Invalid → New
Revision history for this message
Daniel Hollocher (chogydan) wrote :

per discussion, its been decided that only /tmp should be mounted to tmpfs. /var/tmp should not be, since the data should persist beyond reboots. /tmp on the other hand, can be deleted on boot, since programs shouldn't expect files in /tmp to persist beyond the life of the program's process.

Changed in debian-installer:
status: New → Confirmed
Revision history for this message
Phillip Susi (psusi) wrote :

Hrm... I could have SWORN that /tmp was already mounted as a tmpfs, but it appears it's just /var/run and /var/lock that are. /tmp only gets mounted as a tmpfs by /etc/init.d/mountoverflowtmp if there is less than 1 meg of free space on /.

Revision history for this message
Roman Yepishev (rye) wrote :

As I see in Lucid Lynx current, /tmp is now just a subdir of /.
In Karmic /tmp was actually tmpfs mount, was this an intended change in Lucid?

Revision history for this message
lavinog (lavinog) wrote :

/tmp was a subdir of / in Karmic on my systems.

Revision history for this message
Wesley Schwengle (wesleys) wrote :

/tmp is always a dir of /, you need to have a mount point to mount the filesystem.

toobuntu (toobuntu)
description: updated
summary: - Temporary /tmp and /var/tmp
+ Temporary /tmp on a tmpfs
Colin Watson (cjwatson)
no longer affects: debian-installer
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related blueprints

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.