/etc/ssl/private/ssl-cert-snakeoil.key is world readable

[Lukasz]

I am currently using Ubuntu 8.04 upgraded from 4th alpha. Postgresql was running on my machine some time before. Today I noticed an error while starting its process related to file permissions of server.key. I know that such problem occured in 2007 (http://ubuntuforums.org/showthread.php?t=518354) and was also considered by postgresql developers (http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php)

Temporary (?) solution is to change mode to 740:

~: psql mydatabase
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

# ls -l server.key
lrwxrwxrwx 1 root root 38 2008-03-15 23:42 server.key -> /etc/ssl/private/ssl-cert-snakeoil.key
/var/lib/postgresql/8.3/main# /etc/init.d/postgresql-8.3 start
 * Starting PostgreSQL 8.3 database server * The PostgreSQL server failed to start. Please check the log output:
2008-05-01 09:42:04 CEST KATASTROFALNY: unsafe permissions on private key file "server.key"
2008-05-01 09:42:04 CEST SZCZEGÓŁY: File must be owned by the database user or root, must have no write permission for "group", and must have no permissions for "other".

/var/lib/postgresql/8.3/main# chmod 700 server.key
/var/lib/postgresql/8.3/main# /etc/init.d/postgresql-8.3 stop ; /etc/init.d/postgresql-8.3 start
 * Stopping PostgreSQL 8.3 database server [ OK ]
 * Starting PostgreSQL 8.3 database server * The PostgreSQL server failed to start. Please check the log output:
2008-05-01 09:53:33 CEST KATASTROFALNY: could not load private key file "server.key": Permission denied
                                                                                                             [fail]
/var/lib/postgresql/8.3/main# chmod 744 server.key
/var/lib/postgresql/8.3/main# /etc/init.d/postgresql-8.3 stop ; /etc/init.d/postgresql-8.3 start
 * Stopping PostgreSQL 8.3 database server [ OK ]
 * Starting PostgreSQL 8.3 database server * The PostgreSQL server failed to start. Please check the log output:
2008-05-01 09:54:18 CEST KATASTROFALNY: unsafe permissions on private key file "server.key"
2008-05-01 09:54:18 CEST SZCZEGÓŁY: File must be owned by the database user or root, must have no write permission for "group", and must have no permissions for "other".
                                                                                                             [fail]
/var/lib/postgresql/8.3/main# chmod 740 server.key
/var/lib/postgresql/8.3/main# /etc/init.d/postgresql-8.3 stop ; /etc/init.d/postgresql-8.3 start
 * Stopping PostgreSQL 8.3 database server [ OK ]
 * Starting PostgreSQL 8.3 database server [ OK ]

[Martin Pitt (pitti)]

What were the original permissions of the key, i. e. what did

  ls -l /etc/ssl/private/ssl-cert-snakeoil.key

show? Can you please do it now?

I assume some other package broke the default permissions of it (it should be -rw-r----- 1 root ssl-cert).

[Lukasz (lmakowsk)]

It seems that after installing Alpha4 *almost* everything was alright. I am certain that I was not meddling with /etc/ssl/private/ssl-cert-snakeoil.key before the problem occured.

As far as I remember (or just guessing) the problem was due to 644 value of rights set. Owner and group were OK.

Now I have: -rw-r----- 1 root ssl-cert 891 2008-02-27 22:02 ssl-cert-snakeoil.key and psql works.
On another box I installed final version of 8.04, tons of software and ssl*key is OK.

Below is excerpt from postgresql.log on my machine, unfortunately with l10n.
2008-04-20 09:08:11 CEST DZIENNIK: could not load root certificate file "root.crt": no SSL error reported
2008-04-20 09:08:11 CEST SZCZEGÓŁY: Will not verify client certificates.
2008-04-20 09:08:11 CEST DZIENNIK: system bazodanowy został zamknięty o 2008-04-20 01:34:08 CEST /* was shoot down */
2008-04-20 09:08:11 CEST DZIENNIK: autovacuum launcher started
2008-04-20 09:08:11 CEST DZIENNIK: database system is ready to accept connections
2008-04-20 09:08:11 CEST DZIENNIK: incomplete startup packet
2008-04-20 11:39:33 CEST DZIENNIK: incomplete startup packet
2008-04-20 11:39:33 CEST DZIENNIK: received fast shutdown request
2008-04-20 11:39:33 CEST DZIENNIK: aborting any active transactions
2008-04-20 11:39:33 CEST DZIENNIK: autovacuum launcher shutting down
2008-04-20 11:39:33 CEST DZIENNIK: zamykanie /* shutting down */
2008-04-20 11:39:33 CEST DZIENNIK: system bazodanowy jest zamknięty /* database system is closed */
2008-04-20 11:41:36 CEST KATASTROFALNY: unsafe permissions on private key file "server.key"
2008-04-20 11:41:36 CEST SZCZEGÓŁY: File must be owned by the database user or root, must have no write permission for "group", and must have no permissions for "other".
2008-04-20 18:39:06 CEST KATASTROFALNY: unsafe permissions on private key file "server.key"
2008-04-20 18:39:06 CEST SZCZEGÓŁY: File must be owned by the database user or root, must have no write permission for "group", and must have no permissions for "other".

grep '04-20.* installed ' dpkg.log
2008-04-20 10:09:18 status installed libartsc0 1.5.9-0ubuntu2
2008-04-20 10:09:18 status installed libaudio2 1.9.1-1
2008-04-20 10:09:18 status installed libdvdnav4 0.1.10-0.2
2008-04-20 10:09:18 status installed libenca0 1.9-4
2008-04-20 10:09:18 status installed libgii1-target-x 1:1.0.1-3
2008-04-20 10:09:18 status installed libgii1 1:1.0.1-3
2008-04-20 10:09:18 status installed libggi2 1:2.2.1-5ubuntu1
2008-04-20 10:09:18 status installed libggi-target-x 1:2.2.1-5ubuntu1
2008-04-20 10:09:18 status installed libsvga1 1:1.4.3-24
2008-04-20 10:09:18 status installed mplayer-skins 2-7
2008-04-20 10:09:19 status installed mplayer 2:1.0~rc2-0ubuntu13
2008-04-20 10:09:19 status installed libc6 2.7-10ubuntu3

Reboot was about an hour after upgrade. Output from "last" for interesting time:
reboot system boot 2.6.24-16-generi Sun Apr 20 11:41 - 16:47 (05:05)
reboot system boot 2.6.24-16-generi Sun Apr 20 09:08 - 11:39 (02:31)

[Martin Pitt (pitti)]

Indeed, permissions 644 (world-readable) for a private SSL key file is a grave bug. Is there a chance that you accidentally changed this yourself? If not, we need to track down the package which broke it.

[Lukasz (lmakowsk)]

Yes, there is a chance that I accidentally changed file permission :(

I am not so certain anymore that it was not my mistake. I didn't need psql for 10 days and didn't care if it is running either. So after such a long time I could have forgotten what I was doing. On the other hand I don't know how I could have messed with the file - before I noticed the error I didn't even know that there is something like this keyfile. I haven't had acct yet, so I cannot investigate my movements further. My 'history' is too short.

Today I was trying to reproduce the bug but failed. In 8.04-final I installed packages incriminated above (except libc) and nothing malicious happened to keyfile.

It could be false alarm, error on my side. Shame on me :(

[Martin Pitt (pitti)]

> Shame on me :(

No reason for that at all, I'm glad that you reported this. If this is really caused by an Ubuntu package, it's a very serious problem. But with the currently available data I don't know where to look and fix it. :-/ Thus I cannot do much with the current report.

[Mohan (mohan-eai-san)]

Hello

 Am trying to install openbravoERP (it needs postgresql) on Kubuntu 8.4.
Through adept manager, I installed postgresql 8.3.

Initially it would not recognise postgres as a user - following tips on the internet
I had to change the following line in the folder /etc/postgres/8.3/main
and in file pg_hba.conf - replaced
the line 'local all all ident sameuser' with 'local all all md5'

 Though the K Menu ->SystemServices->Advanced->SystemServices I notice
that though postgres is part of the init.d script to start automatically on boot, it is not running.

When I try to restart it, I get the following message:
 * Starting PostgreSQL 8.3 database server
 * The PostgreSQL server failed to start. Please check the log output:
2008-06-09 18:39:40 IST FATAL: could not access private key file "server.key": Permission denied
   ...fail!

I googled on the above and searched postgresforum to come across your post here ... and followed some
instructions on changing the permissions on the server.key file under /etc/ssl_cert/private. However
it did not work for me.

Here is a clip of my window:
======================
root@Ananda:/var/lib/postgresql/8.3/main# chmod 740 server.key
root@Ananda:/var/lib/postgresql/8.3/main# ls -l
total 40
drwx------ 5 postgres postgres 4096 2008-06-07 18:21 base
drwx------ 2 postgres postgres 4096 2008-06-09 15:56 global
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_clog
drwx------ 4 postgres postgres 4096 2008-06-07 18:21 pg_multixact
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_subtrans
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_tblspc
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_twophase
-rw------- 1 postgres postgres 4 2008-06-07 18:21 PG_VERSION
drwx------ 3 postgres postgres 4096 2008-06-07 18:21 pg_xlog
-rw------- 1 postgres postgres 125 2008-06-09 15:15 postmaster.opts
lrwxrwxrwx 1 root root 31 2008-06-07 18:21 root.crt -> /etc/postgresql-common/root.crt
lrwxrwxrwx 1 root root 36 2008-06-07 18:21 server.crt -> /etc/ssl/certs/ssl-cert-snakeoil.pem
lrwxrwxrwx 1 root root 38 2008-06-07 18:21 server.key -> /etc/ssl/private/ssl-cert-snakeoil.key
root@Ananda:/var/lib/postgresql/8.3/main# ls -l /etc/ssl/private/ssl-cert-snakeoil.key
-rwxr----- 1 root ssl-cert 891 2008-05-28 16:19 /etc/ssl/private/ssl-cert-snakeoil.key
root@Ananda:/var/lib/postgresql/8.3/main# /etc/init.d/postgresql-8.3 start
 * Starting PostgreSQL 8.3 database server * The PostgreSQL server failed to start. Please check the log output:
2008-06-09 18:34:12 IST FATAL: could not access private key file "server.key": Permission denied

==================

what should I do to get postgresql 8.3 running on my system?

Thanks
Ddrake

PS: Hardware config (Acer aspire 4710 - core2duo, 2GB RAM, 160 GB hd)

[Lukasz (lmakowsk)]

Hello Mohan!
I have psql happily running. Executable flag on key file is not necessary (that was my fault as well) and/or could even be forbidden in this case - try to set exact permission flags. Check also directory permission and owners.

# ls -ld /etc/ssl/private/
drwx--x--- 2 root ssl-cert 4096 2008-05-08 12:26 /etc/ssl/private/
# ls -l /etc/ssl/private/
-rw-r----- 1 root ssl-cert 887 2008-05-08 12:26 ssl-cert-snakeoil.key

[Martin Pitt (pitti)]

Mohan [2008-06-09 13:19 -0000]:
> root@Ananda:/var/lib/postgresql/8.3/main# ls -l /etc/ssl/private/ssl-cert-snakeoil.key
> -rwxr----- 1 root ssl-cert 891 2008-05-28 16:19 /etc/ssl/private/ssl-cert-snakeoil.key

It should be "640", not "740", but that isn't the cause of your
breakage here. Can you please give the output of

  ls -ld /etc/ssl/private

? Maybe the entire directory is only accessible as root.

[Mohan (mohan-eai-san)]

Hi Lukasz and Martin

 Thanks a lot for your quick response.

Here is the output desired:
--------------------------
root@Ananda:/# ls -ld /etc/ssl/private/
drwxr-x--- 2 root ssl-cert 4096 2008-05-28 16:19 /etc/ssl/private/
root@Ananda:/# ls -l /etc/ssl/private/
total 4
-rwxr----- 1 root ssl-cert 891 2008-05-28 16:19 ssl-cert-snakeoil.key
root@Ananda:/#
-------------------------------

Just FYI, my needs have changed - openbravo 2.35MP1 the current release
works only with Postgresql 8.2 and not 8.3. So I am forced to purge 8.3 installation.
I have not been a postgres user/admin. I have run into an interesting problem pattern -
with postgres on kubuntu 8.04 - thought I should share that as well:
------------------------
root@Ananda:/# psql -U postgres
psql: FATAL: Ident authentication failed for user "postgres"
root@Ananda:/# su postgres
postgres@Ananda:/$ psql -U postgres
Welcome to psql 8.2.7, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit

postgres=# \q
postgres@Ananda:/$ exit
exit
root@Ananda:/# psql -U postgres
psql: FATAL: Ident authentication failed for user "postgres"
=============================
Question: I thought 'psql -U postgres' should work irrespective of who
is invoking it ( I expected it to prompt me for the password).
Is this behaviour odd? Has this got to do anything with
the authentication (or the ssl-cert-snakeoil.key permissions?)

thanks once more
Mohan

[Martin Pitt (pitti)]

Mohan, this has got nothing to do with the ssl cert, but is the default configured in pg_hba.conf. See http://www.postgresql.org/docs/8.2/interactive/client-authentication.html for details. In particular, if you want password based authentication, change "ident" to "md5".

As for your SSL problem, the directory permissions are fine. Can you please give the output of

 id postgres

? Is it in the ssl-cert group?

[Mohan (mohan-eai-san)]

Hi Martin

 Yes, here is the clip:
-------------------
mohan@Ananda:~$ id postgres
uid=110(postgres) gid=108(ssl-cert) groups=108(ssl-cert),120(postgres)
mohan@Ananda:~$
--------------------------

Postgres is in the ssl-cert group.

regards
Mohan

[Martin Pitt (pitti)]

Hm, this is really weird. Just to confirm, if you do this:

 sudo -u postgres head /var/lib/postgresql/8.3/main/server.key

does that work, or do you get an error message? What is the current permission on that file, still 640 root:ssl-cert?

[Stani (stani)]

I run into the same problem when trying to install postgresql 8.3 ... (sorry for the dutch)

$ sudo apt-get dist-upgrade
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
Statusinformatie wordt gelezen... Klaar
Opwaardering wordt doorgerekend... Klaar
0 pakketten opgewaardeerd, 0 pakketten nieuw geïnstalleerd, 0 te verwijderen en 0 niet opgewaardeerd.
2 pakketten niet volledig geïnstalleerd of verwijderd.
Na deze handeling, zal er 0B extra schijfruimte gebruikt worden.
Wilt u doorgaan [J/n]? j
Instellen van postgresql-8.3 (8.3.1-1) ...
 * Starting PostgreSQL 8.3 database server * The PostgreSQL server failed to start. Please check the log output:
2008-06-19 01:15:19 CEST FATAL: unsafe permissions on private key file "server.key"
2008-06-19 01:15:19 CEST DETAIL: File must be owned by the database user or root, must have no write permission for "group", and must have no permissions for "other".
                                                                         [fail]
invoke-rc.d: initscript postgresql-8.3, action "start" failed.
dpkg: fout bij afhandelen van postgresql-8.3 (--configure):
 subproces post-installation script gaf een foutwaarde 1 terug
dpkg: vereistenproblemen verhinderen de configuratie van postgresql-8.3-postgis:
 postgresql-8.3-postgis is afhankelijk van postgresql-8.3; maar:
  Pakket postgresql-8.3 is nog niet geconfigureerd.
dpkg: fout bij afhandelen van postgresql-8.3-postgis (--configure):
 vereistenproblemen - blijft ongeconfigureerd
Fouten gevonden tijdens behandelen van:
 postgresql-8.3
 postgresql-8.3-postgis
E: Sub-process /usr/bin/dpkg returned an error code (1)

$ sudo ls -ld /etc/ssl/private/
drwx--x--- 2 root ssl-cert 4096 2008-04-29 02:46 /etc/ssl/private/
$ sudo ls -l /etc/ssl/private/
totaal 4
-rwxrwxrwx 1 root ssl-cert 887 2008-04-29 02:46 ssl-cert-snakeoil.key
$ id postgres
uid=116(postgres) gid=126(postgres) groepen=126(postgres),108(ssl-cert)
$ sudo -u postgres head /var/lib/postgresql/8.3/main/server.key
-----BEGIN RSA PRIVATE KEY-----
(listing of the private key)

[Stani (stani)]

After changing the permissions, I could dist-upgrade successfully:

$ sudo ls -l /etc/ssl/private/ssl-cert-snakeoil.key
-rw-r----- 1 root ssl-cert 887 2008-04-29 02:46 /etc/ssl/private/ssl-cert-snakeoil.key

[Mohan (mohan-eai-san)]

Hello Martin

 Apologies for being out of loop for a few days....

Here is the output that you requested:
---------------------
mohan@Ananda:~$ sudo -u postgres head /var/lib/postgresql/8.3/main/server.key
-----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQDJYQBvBGn0qar3EXCxgrEXfKrnuUIfGDSIhQSOh5LLjDYubnM7
0uuRykuTtNxMjCvqTDrNZBMlUcJIfEdnhk4oM7Wb67FNncQMqR9Lim3AniASwt/P
QHiFskXsn8800v/Cqm4+e0DfF3tbhqBw3FKnvXjoof4Ki6OpkSThnzjM4wIDAQAB
AoGBAKdDHHsZbXA75LbmE3Y9GR5q/AEQDO2Ky0eBRHGmBbVvNimGr8vGuscTAYcj
t78Uxf0LZ+Pk9UoQCJDwONax5QNhFCWMAUbeCMiBLqVrQcFbwWmw6Ez6avg1BV9q
aqHkLJFYJyC/qZfxBu2eS87QFRnYMTdqU9YEQ9HfWaGpX7uxAkEA9B+Y6kGBKXj+
SmHTXfGL1KyEqx/5uQB/ar35NX6jyBnaU5XRHhA7Vg1WJpZgW9H7uG01kDbw/ZYj
HV7VpJccZwJBANMtDGuRoKUXsozLmoivgMUclNg9qDyISzSvAu008KZMypqVmoIw
7uCiaKOdKGvxF2j4BjNoRJ+8lj+mBj8BfiUCQEIcXHj0DFqC3bqfC3Khe1C496Sw
mohan@Ananda:~$
-------------------------------------------------

root@Ananda:/var/lib/postgresql/8.3/main# ls -al
total 48
drwx------ 10 postgres postgres 4096 2008-06-09 22:26 .
drwxr-xr-x 3 root root 4096 2008-06-07 18:21 ..
drwx------ 5 postgres postgres 4096 2008-06-07 18:21 base
drwx------ 2 postgres postgres 4096 2008-06-09 15:56 global
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_clog
drwx------ 4 postgres postgres 4096 2008-06-07 18:21 pg_multixact
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_subtrans
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_tblspc
drwx------ 2 postgres postgres 4096 2008-06-07 18:21 pg_twophase
-rw------- 1 postgres postgres 4 2008-06-07 18:21 PG_VERSION
drwx------ 3 postgres postgres 4096 2008-06-07 18:21 pg_xlog
-rw------- 1 postgres postgres 125 2008-06-09 15:15 postmaster.opts
lrwxrwxrwx 1 root root 31 2008-06-07 18:21 root.crt -> /etc/postgresql-common/root.crt
lrwxrwxrwx 1 root root 36 2008-06-07 18:21 server.crt -> /etc/ssl/certs/ssl-cert-snakeoil.pem
lrwxrwxrwx 1 root root 38 2008-06-07 18:21 server.key -> /etc/ssl/private/ssl-cert-snakeoil.key
root@Ananda:/var/lib/postgresql/8.3/main# ls -al /etc/ssl/private/ssl-cert-snakeoil.key
-rwxr----- 1 root ssl-cert 891 2008-05-28 16:19 /etc/ssl/private/ssl-cert-snakeoil.key
root@Ananda:/var/lib/postgresql/8.3/main#
mohan@Ananda:~$
----------------------------------------

Hope this helps...

regards
Mohan

[Martin Pitt (pitti)]

stani,

-rwxrwxrwx 1 root ssl-cert 887 2008-04-29 02:46 ssl-cert-snakeoil.key

ugh, a world-readable and writable private SSL key? that's really, really bad; how did that happen, just during a gutsy->hardy upgrade, or did you configure that manually at some point?

Mohan,

actually I just asked whether the command worked, not to post the output here. You just posted your private SSL key to the public, so I advise you to generate a new one by doing:

  sudo make-ssl-cert generate-default-snakeoil --force-overwrite

But anyway it proves that user postgres can read the certificate, so I wonder what's wrong with it.

[Stani (stani)]

@Martin
I always do a fresh install, so I did of Hardy as well. I didn't configure anything manually. I tried to install postgresql on a different machine and no errors occurred there. So it must be my machine. If you want me to post more stuff, just tell me.

[Mohan (mohan-eai-san)]

Hi Martin

 Thanks for pointing out about my private ssl key. In reality, the 'head' command gave only a
part of my private ssl-key file. So in that sense it is useless even if advertised.

Yes, to be more secure, I did replace it with a new one that I generated - using the make-ssl-cert command

thanks and regards
Mohan

[Mathi (psgmathi)]

Hi all,
I am using postgres 7.4 . I tried to enable SSL in it..
I was succesful in creating the files server.key, server.crt,server.crt.der
But when i try to restart my server after that, it says

FATAL: could not load private key file "/var/lib/postgresql/7.4/main/server.key": Permission denied

I read this post but i couldn't solve the problem..
The command "sudo -u postgres head /var/lib/postgresql/8.3/main/server.key" gives error as below

head: cannot open `/var/lib/postgresql/7.4/main/server.key' for reading: Permission denied

what is wrong from my side??

Please help..

Thanks,
mathi

[Martin Pitt (pitti)]

Mathi [2008-09-18 10:48 -0000]:
> The command "sudo -u postgres head /var/lib/postgresql/8.3/main/server.key" gives error as below
>
> head: cannot open `/var/lib/postgresql/7.4/main/server.key' for reading:
> Permission denied

Please give the output of

 sudo ls -ld /var/lib
 sudo ls -ld /var/lib/postgresql
 sudo ls -ld /var/lib/postgresql/7.4
 sudo ls -ld /var/lib/postgresql/7.4/main

[Ludovic Fierville (l-fierville)]

A possible solution to the impossibility to launch postgres : you should check that the postgres user is still a member of the sss-cert group. I botched the group membership by mistake and wasn't able to launch the server with the same error as above.

Restoring the right membership solved the issue (sudo usermod -aG ssl-cert postgres)

As for the permissions on the key, I have this :
root@endor:/var/lib/postgresql/8.3/main# ls -l server*
lrwxrwxrwx 1 root root 36 2008-10-10 15:15 server.crt -> /etc/ssl/certs/ssl-cert-snakeoil.pem
lrwxrwxrwx 1 root root 38 2008-10-10 15:15 server.key -> /etc/ssl/private/ssl-cert-snakeoil.key

and this :
root@endor:/etc/ssl/private# ls -l
total 4
-rw-r----- 1 root ssl-cert 887 2008-10-10 14:53 ssl-cert-snakeoil.key

So it seems that the key is not rw to the world but the symbolic link is. I'm not good enought to know if that is an issue, unfortunately.

[the.jxc (jonathan-spiderfan)]

I encountered essentially the same problem with a Jaunty (9.04) installation.

My problem was seemingly caused by having installed and configured dovecot BEFORE installing the postgres 8.3 server. My snakeoil key had the following permissions:

root@kingpin:/etc/postgresql/8.3/main# ls -l /etc/ssl/private/ssl-cert-snakeoil.key
-rw------- 1 root dovecot 887 2008-12-18 11:56 /etc/ssl/private/ssl-cert-snakeoil.key

I had to both:

# chown root:ssl-cert
AND
# chmod 640

...before I could get postgres to start.

[Launchpad Janitor (janitor)]

[Expired for ssl-cert (Ubuntu) because there has been no activity for 60 days.]