getpwnam() and parsing /etc/passwd gives wrong value for HOME to snaps

[ILIV]

Hi,

psql, which is part of PostgreSQL, writes to ~/.psql_history on exit. The hidden file is basically equivalent of the .bash_history system file.

However, the home: plug/interface currently prohibits writing to hidden files:

iliv@xenial:~$ postgresql94.psql -h 127.0.0.1 -d postgres -p 5433
psql (9.4.9)
...
postgres=# \q
could not save history to file "/home/iliv/.psql_history": Permission denied

I doubt PostgreSQL is going to change this behavior so I was wondering if snapcraft would be willing to reconsider the way home: plug/interface works.

[Jamie Strandboge (jdstrand)]

The HOME environment variable is set to '$HOME/snap/name/revision' so that snaps don't need to use the transitional 'home' interface and have full access to all hidden directories in HOME. I would expect psql to write its history file write the file to $HOME/snap/name/revision/.psql_history if it is honoring the HOME environment variable (which it should, since HOME could be set to any number of places outside of /home/).

Is HOME not being set correctly for you?

[ILIV (ivan-lezhnjov-iv)]

Thanks for reply Jamie. Where do I check if it is?

[Jamie Strandboge (jdstrand)]

Use 'snap run --shell <snap name>'

Eg:
$ snap run --shell hello-world
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

$ set | grep HOME
HOME=/home/jamie/snap/hello-world/27

In your case, it looks like you would use 'snap run --shell postgresql94'

[ILIV (ivan-lezhnjov-iv)]

Unfortunately, it ends in error:

admin@lab-xenial:~/snaps/postgresql93$ snap run --shell postgresql93
error: cannot find app "postgresql93" in "postgresql93"

Never mind the different version of the app. It's essentially the same package, just a different version.

[Jamie Strandboge (jdstrand)]

I think you need snapd 2.16 installed for this to work. 'snap list' would show the snaps you have installed and what you can give to 'snap run --shell'. If you have snapd 2.16 installed and you are passing the correct snap name and it still isn't working for you, please file a different bug. If this is the case, to workaround that bug you can add something like this to your snap:

...
apps:
  sh:
    command: bin/sh
...

$ cat ./files/bin/sh (or wherever you put 'bin/sh')
#!/bin/bash
/bin/bash --norc -i

then install and run:
$ postgresql93.sh
bash-4.3$ set | grep HOME
...

[ILIV (ivan-lezhnjov-iv)]

The bug report (#1618085) was already filed by someone else.

In short, I have 2.16 (as of today; I had to update) and it doesn't make any difference.

[Michael Hall (mhall119)]

It actually is "snap run --shell <command name>" not <snap name> since every command in your snap might have different access based on it's plugs list. So, for example, you could use: "snap run --shell postgresql93.psql"

[ILIV (ivan-lezhnjov-iv)]

Thank Michael. That worked:

$ snap run --shell postgresql93.psql
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

admin@lab-xenial:/home/admin/snaps$ set |grep HOME
HOME=/home/admin/snap/postgresql93/2

Alright, so it appears that actually $HOME is set correctly (it is, right?) but psql doesn't use it for some reason.

[Jamie Strandboge (jdstrand)]

HOME=/home/admin/snap/postgresql93/2 is correct for this snap, yes. This may suggest a bug in psql: perhaps it is not evaluating the HOME environment variable and instead coming up with some other method to figure out where to write its history file. Assuming this is the case, upstream should most likely be interested in a patch since fixing it would likely fix other non-snappy bugs with home handling.

[ILIV (ivan-lezhnjov-iv)]

Turns out psql and many other parts of PostgreSQL expect that tilde (~) character expansion is happening according to this specification: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01

This doesn't seem to work in a snap environment.

[Jamie Strandboge (jdstrand)]

FYI, ~ expansion works fine with bash:

$ hello-world.sh
$ cd /tmp
$ cd ~
$ pwd
/home/jamie/snap/hello-world/27

and that should be following the guidelines as outlined in http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01: "If the login name is null (that is, the tilde-prefix contains only the tilde), the tilde-prefix is replaced by the value of the variable HOME. If HOME is unset, the results are unspecified. Otherwise, the tilde-prefix shall be replaced by a pathname of the initial working directory associated with the login name obtained using the getpwnam() function as defined in the System Interfaces volume of POSIX.1-2008."

It seems clear that using getpwnam() to look in /etc/passwd for the home directory instead of using HOME would be broken. Looking at postgresql-9.5 I see in src/bin/psql/common.c the expand_tilde() function and get_home_path() in src/port/path.c are not considering the HOME environment variable and instead only looking at /etc/passwd.

[Jamie Strandboge (jdstrand)]

For now, the snap will need to patch psql to consider HOME before falling looking at /etc/passwd. Trying to fix this in snapd would require design, perhaps using the LD_PRELOAD idea from Gustavo in bug #1577514.

[Jamie Strandboge (jdstrand)]

"... before falling looking..." should be "... before falling back to looking..."

[ILIV (ivan-lezhnjov-iv)]

That seems to be correct, Jamie. I must admit I was so certain it couldn't be PostgreSQL because it is a very mature project. But okay, lesson learned. Never assume anything. Thanks a lot for all the help! I was able to implement a workaround and we also started a conversation in PostgreSQL community about this method of determining user's home directory. No idea if and how soon this is going to be fixed, though.

[Jamie Strandboge (jdstrand)]

For now I'm going to mark this as Invalid since snapd is operating as designed. That said, you may want to keep an eye on https://forum.snapcraft.io/t/go-homedir-reads-etc-passwd-not-home/2727 in case the design changes.

[林博仁(Buo-ren Lin) (brlin)]

Refer https://forum.snapcraft.io/t/the-homeishome-stage-snap-fix-snapped-apps-that-wrongly-referring-to-the-users-real-home-directory-instead-of-home/19763 for a possible workaround.

[Zygmunt Krynicki (zyga)]

FYI snapd master now defines SNAP_REAL_HOME which is the vanilla home directory before remapping.