Strange getcwd() behaviour

[Alberto Bertogli]

Hi!

A friend is having a very strange bug, that I think (I'm not sure) it might be
ecryptfs related. She has Ubuntu 9.10 (installed 9.04, then upgraded) and uses
ecryptfs to encrypt her home directory (using the standard Ubuntu setup).

Sometimes (no idea when or why) the following happens: using GNU screen, you
open a new shell (ctrl-a c) bash prompt says it's in '/'; pwd says it's in
'/'.

ls shows the contents of her home directory. You can do cat <file in her home
directory> and it works doing cat <file in /> does not work.

I've done a couple of straces and the behaviour is consistant with the current
directory being her home, but getcwd() returning '/'. I verified this also
using Python's os module, just in case it was a tool issue.

I can also cd <dir inside her home> and pwd shows '/<dir inside her home>',
with the same behaviour as before. While I'm in there, I get the same
behaviour as before (cat works, open works, etc., but getcwd() returns the
wrong directory). However, if I do 'cd /<dir in her home>' I get ENOENT.

The problem goes away if I do 'cd' or 'cd /<her home dir>'

From what I can see, it looks like getcwd() is using '/' instead of $HOME.

At the moment I can reproduce this at will by creating new shells inside an
existing screen. It does not happen in new terminals. She said this has
happened before, but has no idea when or why (although it happened also in
9.04). I'm not sure if after she reboots or closes this screen it will be so
easy to reproduce (it looks like the shells are inheriting this behaviour from
the screen process).

If you need any further information (or want me to test anything), please let
me know.

Thanks a lot,
                Alberto

[sndfnsdfin (qawsnews)]

The same here!
This bug affects me sporadically on an ecryptfs directory (/home/xxxxx) in a screen session.

Those are my findings:
(Note that strace changes the behaviour of pwd!)

xxxxx@yyyy-e53:~/studium_s8/da$ strace git diff 2>> /home/xxxxx/strage_problem_git.txt
xxxxx@yyyy-e53:~/studium_s8/da$ strace pwd 2>> /home/xxxxx/strage_problem_pwd.txt
/studium_s8/da
xxxxx@yyyy-e53:~/studium_s8/da$ pwd
/home/xxxxx/studium_s8/da
xxxxx@yyyy-e53:~/studium_s8/da$ cd
xxxxx@yyyy-e53:~$ cd /var/log
xxxxx@yyyy-e53:/var/log$ pwd
/var/log
xxxxx@yyyy-e53:/var/log$ strace pwd 2>> /home/xxxxx/strage_problem_pwd2.txt
/var/log

[sndfnsdfin (qawsnews)]

Well, the Heisenberg-strace mystery seems to be related to piping in the home directory...

xxxxx@yyyy-e53:~/studium_s8/da$ pwd 2>> /tmp/strage_problem_pwd3.txt
/home/xxxxx/studium_s8/da

I hope somebody can help, because this behaviour is some kind of show-stopper for batch processing usecase.

Thanks in advance

[Dustin Kirkland  (kirkland)]

Hmm, is there anything monkeying with her $HOME variable?

Also, what's her $HOME according to /etc/passwd?

[Dustin Kirkland  (kirkland)]

Also, is the ecryptfs mount active?

mount | grep ecryptfs

And are her keys properly loaded?

keyctl list @u

As for opening new shells in a screen session, screen has a behavior of opening each subsequent shell in the same directory as the original PWD at the initial start of screen. It's possible that was buggered up. Try exiting screen and starting a new screen session.

[Alberto Bertogli (albertito)]

On Thu, Jan 28, 2010 at 12:01:42AM -0000, Dustin Kirkland wrote:
> Hmm, is there anything monkeying with her $HOME variable?

I don't think so.

> Also, what's her $HOME according to /etc/passwd?

The usual one, /home/<her username>.

She can't reproduce the bug at the moment, so I'll answer the other questions
when the bug occurs again.

Thanks,
  Alberto

[Alberto Bertogli (albertito)]

Sorry it took so long to reply, but the bug did not re-appear until today.

On Thu, Jan 28, 2010 at 12:06:43AM -0000, Dustin Kirkland wrote:
> Also, is the ecryptfs mount active?
>
> mount | grep ecryptfs

$ mount | grep ecryptfs
/home/username/.Private on /home/username type ecryptfs
(ecryptfs_sig=466c5178c2a30351,ecryptfs_fnek_sig=add28c0fcbae8831,ecryptfs_cipher=aes,ecryptfs_key_bytes=16)

> And are her keys properly loaded?
>
> keyctl list @u

$ keyctl list @u
2 keys in keyring:
277263190: --alswrv 1000 0 user: add28c0fcbae8831
638313147: --alswrv 1000 0 user: 466c5178c2a30351

> As for opening new shells in a screen session, screen has a behavior of
> opening each subsequent shell in the same directory as the original PWD
> at the initial start of screen. It's possible that was buggered up.
> Try exiting screen and starting a new screen session.

Opening a new terminal results in a new, valid shell.

After discussing it with her, we came up with the following scenario that
seems to cause the problem:

 1. She launches a screen session
 2. She logs off (her X session), leaving the screen detached
 3. She logs back in
 4. She re-attaches the screen, creates a new terminal, and it's broken.

That reliably reproduces the problem.

If the ecryptfs mount is automatically unmounted after 2, the working
directory of screen disappears and causes it to break. A "cd" fixes it because
it changes the working directory to her home, which is the new mount point.

What we are not sure is that if the semantics for the broken shell are
correct (i.e. consistant with the dissapearance of the underlying working
directory).

Thanks a lot,
  Alberto

[Alberto Bertogli (albertito)]

Hi! This bug is still marked as incomplete, but I don't think it's still lacking.

Is there any other information you need? If so, let me know and I'd be
glad to provide it.

Thanks,
  Alberto

[Russell Davies (russelldavies)]

I can also confirm this behavior (as I detailed in bug #622858).

I've tested it on multiple systems, including VMs that have had no additional packages installed post install.

[Serge Hallyn (serge-hallyn)]

It sounds pretty easy to explain (if a bit annoying). Screen isn't
pinning whatever ecryptfs-utils is using to pin the mounts (PAM?).
So (PAM?) umounts $HOME. But your screen session is still using it.
So the dentry which represented the root of your ecryptfs home-dir,
which previously could be named relative to the system's / by being
mounted on top of /home/xyz, has been lazily unmounted and no longer
has a name relative to '/'.

It won't hurt anything, but it'd be good if screen would also
pin the use count.

[Russell Davies (russelldavies)]

So this is a bug with screen?

On Mon, Aug 23, 2010 at 20:45, Serge Hallyn <email address hidden>wrote:

> It sounds pretty easy to explain (if a bit annoying). Screen isn't
> pinning whatever ecryptfs-utils is using to pin the mounts (PAM?).
> So (PAM?) umounts $HOME. But your screen session is still using it.
> So the dentry which represented the root of your ecryptfs home-dir,
> which previously could be named relative to the system's / by being
> mounted on top of /home/xyz, has been lazily unmounted and no longer
> has a name relative to '/'.
>
> It won't hurt anything, but it'd be good if screen would also
> pin the use count.
>
> --
> Strange getcwd() behaviour
> https://bugs.launchpad.net/bugs/479248
> You received this bug notification because you are a direct subscriber
> of a duplicate bug (622858).
>
> Status in eCryptfs - Enterprise Cryptographic Filesystem: Incomplete
>
> Bug description:
> Hi!
>
> A friend is having a very strange bug, that I think (I'm not sure) it might
> be
> ecryptfs related. She has Ubuntu 9.10 (installed 9.04, then upgraded) and
> uses
> ecryptfs to encrypt her home directory (using the standard Ubuntu setup).
>
> Sometimes (no idea when or why) the following happens: using GNU screen,
> you
> open a new shell (ctrl-a c) bash prompt says it's in '/'; pwd says it's in
> '/'.
>
> ls shows the contents of her home directory. You can do cat <file in her
> home
> directory> and it works doing cat <file in /> does not work.
>
> I've done a couple of straces and the behaviour is consistant with the
> current
> directory being her home, but getcwd() returning '/'. I verified this also
> using Python's os module, just in case it was a tool issue.
>
> I can also cd <dir inside her home> and pwd shows '/<dir inside her home>',
> with the same behaviour as before. While I'm in there, I get the same
> behaviour as before (cat works, open works, etc., but getcwd() returns the
> wrong directory). However, if I do 'cd /<dir in her home>' I get ENOENT.
>
> The problem goes away if I do 'cd' or 'cd /<her home dir>'
>
> >From what I can see, it looks like getcwd() is using '/' instead of $HOME.
>
>
> At the moment I can reproduce this at will by creating new shells inside an
> existing screen. It does not happen in new terminals. She said this has
> happened before, but has no idea when or why (although it happened also in
> 9.04). I'm not sure if after she reboots or closes this screen it will be
> so
> easy to reproduce (it looks like the shells are inheriting this behaviour
> from
> the screen process).
>
>
> If you need any further information (or want me to test anything), please
> let
> me know.
>
> Thanks a lot,
> Alberto
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ecryptfs/+bug/479248/+subscribe
>

[Serge Hallyn (serge-hallyn)]

Quoting Russell Davies (<email address hidden>):
> So this is a bug with screen?

No, it has to do with however ecryptfs-utils and pam are
doing the homedir automount. TBH I'm not even sure it's
as clever as I was thinking it was (bc I can't find it
in the code).

[Russell Davies (russelldavies)]

Haven't managed to find anything in ecryptfs-utils relating to this.
Furthermore, I discovered this issue doesn't effect other terminal
multiplexers such as tmux so possibly thinking this is specific to gnu
screen.

On Wed, Aug 25, 2010 at 18:02, Serge Hallyn <email address hidden>wrote:

> Quoting Russell Davies (<email address hidden><russell%<email address hidden>>
> ):
> > So this is a bug with screen?
>
> No, it has to do with however ecryptfs-utils and pam are
> doing the homedir automount. TBH I'm not even sure it's
> as clever as I was thinking it was (bc I can't find it
> in the code).
>
> --
> Strange getcwd() behaviour
> https://bugs.launchpad.net/bugs/479248
> You received this bug notification because you are a direct subscriber
> of a duplicate bug (622858).
>
> Status in eCryptfs - Enterprise Cryptographic Filesystem: Incomplete
>
> Bug description:
> Hi!
>
> A friend is having a very strange bug, that I think (I'm not sure) it might
> be
> ecryptfs related. She has Ubuntu 9.10 (installed 9.04, then upgraded) and
> uses
> ecryptfs to encrypt her home directory (using the standard Ubuntu setup).
>
> Sometimes (no idea when or why) the following happens: using GNU screen,
> you
> open a new shell (ctrl-a c) bash prompt says it's in '/'; pwd says it's in
> '/'.
>
> ls shows the contents of her home directory. You can do cat <file in her
> home
> directory> and it works doing cat <file in /> does not work.
>
> I've done a couple of straces and the behaviour is consistant with the
> current
> directory being her home, but getcwd() returning '/'. I verified this also
> using Python's os module, just in case it was a tool issue.
>
> I can also cd <dir inside her home> and pwd shows '/<dir inside her home>',
> with the same behaviour as before. While I'm in there, I get the same
> behaviour as before (cat works, open works, etc., but getcwd() returns the
> wrong directory). However, if I do 'cd /<dir in her home>' I get ENOENT.
>
> The problem goes away if I do 'cd' or 'cd /<her home dir>'
>
> >From what I can see, it looks like getcwd() is using '/' instead of $HOME.
>
>
> At the moment I can reproduce this at will by creating new shells inside an
> existing screen. It does not happen in new terminals. She said this has
> happened before, but has no idea when or why (although it happened also in
> 9.04). I'm not sure if after she reboots or closes this screen it will be
> so
> easy to reproduce (it looks like the shells are inheriting this behaviour
> from
> the screen process).
>
>
> If you need any further information (or want me to test anything), please
> let
> me know.
>
> Thanks a lot,
> Alberto
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ecryptfs/+bug/479248/+subscribe
>

[Dustin Kirkland  (kirkland)]

I'm marking confirmed, as I, too, have seen this from time to time in Byobu (gnu screen under the covers).

Per discussion on the ecryptfs-users@ mailing list, we believe this has to do with the kernel's lazy unmount. Not sure where or how to fix it yet...

[Serge Hallyn (serge-hallyn)]

Quoting Dustin Kirkland (<email address hidden>):
> I'm marking confirmed, as I, too, have seen this from time to time in
> Byobu (gnu screen under the covers).
>
> Per discussion on the ecryptfs-users@ mailing list, we believe this has
> to do with the kernel's lazy unmount. Not sure where or how to fix it
> yet...

Well, the only way you could prevent this, if i'm thinking right, would
be for screen/byobu to bump the mount count. Screen does appear to
support pam, so you could have perhaps

/etc/pam.d/screen:
session optional pam_ecryptfs.so inc_only

where inc_only would just bump the mount count on $HOME if that is
already mounted.

-serge

[Dustin Kirkland  (kirkland)]

Noting a duplicate bug here for myself, Bug #779318 has very good, concise reproduce instructions.