Cannot make directory '/var/run/screen': Permission denied (convert init to upstart)

[Scott Moser]

Binary package hint: screen

On a freshly booted server install:

$ screen -ls
Cannot make directory '/var/run/screen': Permission denied

This can be fixed with:
$ sudo /etc/init.d/screen-cleanup start

I believe this is because
/etc/rcS.d/S70screen-cleanup is running via upstart much earlier than it expects to have run, and is failing to correctly clean up that directory.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: screen 4.0.3-14ubuntu1
ProcVersionSignature: Ubuntu 2.6.32-21.32-server 2.6.32.11+drm33.2
Uname: Linux 2.6.32-21-server x86_64
Architecture: amd64
Date: Mon May 3 16:45:10 2010
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: screen

============
IMPACT: Sometimes 'screen' is not able to start after a reboot due to a race condition between the mounted-varrun upstart job and the screen init script.
HOW ADDRESSED: This bug is addressed by porting the simple /etc/init.d/screen init script to an upstart job that depends on the mounted-varrun job having run.
HOW TO REPRODUCE: This bug is a race condition, and is difficult to reproduce. I hit it most often on a really fast system -- ie, it's hard for me to reproduce the issue in a virtual machine. To reproduce, reboot until you try to run 'screen' just after boot, and you get the error saying:
  Cannot make directory '/var/run/screen': Permission denied
REGRESSION POTENTIAL: Relatively low, if a veteran upstart job reviewer (hrrrm, Scott/Steve) reviews the proposed upstart job against the original init script and confirms it's identical functionality.
============

[Dustin Kirkland  (kirkland)]

I'll convert this to an upstart script for Maverick.

We'll need some other SRU-able fix, though, for Lucid.

[vedavata (vedavrata)]

i got the same problem on lucid (10.04)... :-(

'sudo /etc/init.d/screen-cleanup start' helped.

thank you.

[Blue Dream (shanxian1987)]

same problem on lucid (10.04)
Waiting for new update

[Launchpad Janitor (janitor)]

This bug was fixed in the package screen - 4.0.3-14ubuntu2

---------------
screen (4.0.3-14ubuntu2) maverick; urgency=low

  * debian/screen-cleanup.upstart, debian/init: convert screen to an
    upstart script to fix some bugs where /var/run is not mounted when
    screen-cleanup runs, LP: #574773
 -- Dustin Kirkland <email address hidden> Mon, 14 Jun 2010 08:13:33 -0500

[Dustin Kirkland  (kirkland)]

This bug is biting *a lot* of people, and biting me quite frequently on Lucid.

I suggest that we SRU this fix.

I'd like to have Scott (or someone) carefully review the upstart script, as converting screen's init script to upstart generally would not be handled in an SRU.

[Dustin Kirkland  (kirkland)]

Minimal debdiff attached.

Scott, can I get your review of this?

Thanks.

[Mike.lifeguard (mikelifeguard)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10-06-16 12:57 AM, Dustin Kirkland wrote:
> This bug is biting *a lot* of people
> ...
> ** Changed in: screen (Ubuntu Lucid)
> Milestone: None => lucid-updates

Can we get this fixed for Karmic too?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkwYUYQACgkQst0AR/DaKHufmgCfdbiB7fmVhgiPsFzRwaJXqNSV
otYAmQHvdpbHkrJwbz+g1v+LaOA6yH5G
=dF3U
-----END PGP SIGNATURE-----

[Scott Moser (smoser)]

On Wed, 16 Jun 2010, Dustin Kirkland wrote:

> Minimal debdiff attached.
>
> Scott, can I get your review of this?

Is the 'Scott' here me ?
I took a quick read. My guess is that this would work fine in most
situtations. However, I think that you would really want 'start on
filesystem' rather than 'local-filesystems'.

see 'man filesystem' for more info. The reasoning is that /var and even
/usr could be on remote filesystems. If var were remote, this would run
before var was mounted, then the mount would go over the top.

All that said, I would suggest asking "the other Scott".

[Dustin Kirkland  (kirkland)]

Sorry for the ambiguity, I did mean upstart-Scott ;-)

[Dustin Kirkland  (kirkland)]

Patch updated accordingly.

[Chuck Frain (chuckfrain-deactivatedaccount)]

I'm not sure if this will help but when I see this issue it is always after a system reboot. I have not seen this occur from a power on boot.

[Dustin Kirkland  (kirkland)]

Attaching debdiff for lucid-proposed, requesting review. Uploading to lucid-proposed now, since this has been working well in Maverick so far.

[Loïc Minier (lool)]

I'm personally not comfortable switching screen to upstart in stable; do we have an option to fix the init script instead?

[Dustin Kirkland  (kirkland)]

Loic, well, a mount/grep/sleep loop, until /var/run is mounted. Is this any better?

[Dustin Kirkland  (kirkland)]

Scott Moser suggested something like this in IRC:
"while ! grep -q /var/run /proc/mounts && sleep 1; :; done

But he also agreed with me -- the upstart job is the *right* solution.

[John Dong (jdong)]

I'm guessing the "Scott" mentioned is Scott James Remnant? Transitioning from a init script to an Upstart job is a fairly invasive change for a LTS SRU, especially for a widely used package like screen. While I agree this is the *right* solution, I'd like a second opinion from another SRU team member, or from Scott before approving this. I'm mainly concerned about the behavior change (e.g. hardcoded calls to the screen init script, and so on) that SRU's typically don't have.

[Scott Moser (smoser)]

On Sun, 8 Aug 2010, John Dong wrote:

> I'm mainly concerned about the behavior change (e.g. hardcoded calls to
> the screen init script, and so on) that SRU's typically don't have.

For this point in particular, debhelper installs a replacement
/etc/init.d/<sysvinit-script> that invokes the upstart job correctly. If
something was parsing output text that may break, but if something is just
calling 'start', then it would be fine. You can see an example just by
looking at your /etc/init.d for symlinks to /lib/init/upstart-job.

reading the changes, i'm not sure that that debhelper code would kick in,
as kirkland has deleted the debian/init script from the package. you may
actually have to leave both in place... Just have to test that.
It can be done though.

[Dustin Kirkland  (kirkland)]

I discussed this upstart script in person this evening with Scott James Remnant and Kees Cook.

Keybuk reviewed the upstart script and deemed it acceptable by his standards. (I have prodded him to state as much here).

Martin (and SRU review team) -- can you please accept this into lucid-proposed for testing now? Note that there have been no complaints in Maverick for several months now, and I have never see it there (and I have upgraded *all* of my Lucid servers to this package and upstart script because this bug sucks so much).

[Martin Pitt (pitti)]

Accepted screen into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Kevin Roy (kiniou)]

Hi,

The screen-cleanup script never starts at launch because it's triggered on "filesytems" signal but the mount-all script emits "filesytem".

[Loïc Minier (lool)]

Grmpf, I fixed this issue in maverick already...

[Dustin Kirkland  (kirkland)]

I just uploaded .2 to lucid-proposed.

[Dustin Kirkland  (kirkland)]

I can confirm that the lucid-proposed package fixes the problem for me.

Can someone else do the same?

[Scott Moser (smoser)]

I've verified that after installing the -proposed deb a reboot results in working screen.
$ dpkg-query --show screen
screen 4.0.3-14ubuntu1.2

[Simon Olofsson (simono)]

Yes, works flawless. Thanks :)

[Mike.lifeguard (mikelifeguard)]

So, will this ever be released?

[Martin Pitt (pitti)]

Ah, this fell off the radar because the changelog doesn't refer to the bug, sorry. Releasing now.

[Bob Ballister (78luphr0rnk2nuqimstywepozxn9kl19tqh0tx66b5dki1xxsh5mkz9gl21a5rlwfnr8jn6ln0m3jxne2k9x1ohg-launchpad)]

I am experiencing this bug on Ubuntu 12.04.1

To reproduce, all you have to do is make a bunch of screen commands in crontab and one of them will randomly fail with this message.

@reboot screen -dmS name1 command1
@reboot screen -dmS name2 command2
@reboot screen -dmS name3 command3
etc...

[airtonix (airtonix-gmail)]

still a problem on Ubuntu 12.04 with :

$ dpkg-query --show screen
screen 4.0.3-14ubuntu8

[Dan Muresan (danmbox)]

Still a problem on Trusty (!), fixed with /etc/init.d/screen-cleanup start.
screen == 4.1.0~20120320gitdb59704-9

It seems screen-cleanup is not activated by default in rc.d

[Rémi MEVAERE (ajmin)]

Same problem with Trusty here. I solved it with adding sleep to my script.

[Roman Slobodchikov (roman-b)]

Same as Remi

Solved by adding sleep 10 (sec) to crontab

@reboot sleep 10 && my_script.sh

[stripwax (dave-beermex)]

Sorry, how come screen 4.1.0-20120320gitdb59704-9 is *not* using upstart ? Was the whole thing reverted again?

[Bob Ballister (78luphr0rnk2nuqimstywepozxn9kl19tqh0tx66b5dki1xxsh5mkz9gl21a5rlwfnr8jn6ln0m3jxne2k9x1ohg-launchpad)]

I am having this problem with Ubuntu 14.04.

There is an /etc/init.d/screen-cleanup script instead of an /etc/init/screen-cleanup.conf script unlike 12.04.

Is this intended?

[Bob Ballister (78luphr0rnk2nuqimstywepozxn9kl19tqh0tx66b5dki1xxsh5mkz9gl21a5rlwfnr8jn6ln0m3jxne2k9x1ohg-launchpad)]

Does anyone know how to add Trusty Tahr to the effects box above? I have a feeling that this bug report won't be looked at since a fix has already been submitted for an old version of Ubuntu.

[Scott Kitterman (kitterman)]

On Friday, October 10, 2014 03:09:08 you wrote:
> Does anyone know how to add Trusty Tahr to the effects box above? I have
> a feeling that this bug report won't be looked at since a fix has
> already been submitted for an old version of Ubuntu.

At this point, you should file a new bug. Even if you're seeing similar
symptoms, you're having a different problem.

[Growlf (growlf)]

In fact, look here: https://bugs.launchpad.net/ubuntu/+source/screen/+bug/1380080 for a possible solution.

[Michael Richardson (rainabba)]

Just got setup with the new Windows 10 Linux subsystem bash environment and am hitting this bug with screen installed there (did apt-get remove and installed again). Am I on a package based on old code then (It's version 4.01.00devel (GNU) 2-May-06 which I hope isn't from 2006) or is this bug still an issue?

[Dave Vree (hdave)]

+1 for Michael Richardson's question

[Nickolas Burr (nburr)]

+1 for #39, also running into this same issue on WSL.