SSL renegotiation fails

[Stuart Bishop]

With PostgreSQL 9.1, SSL renegotiation is enabled by default. This fails under Ubuntu 12.04, most noticeably when using streaming replication as the renegotiation limit is hit quickly.

On the master:

2012-06-25 16:16:26 PDT LOG: SSL renegotiation failure
2012-06-25 16:16:26 PDT LOG: SSL error: unexpected record
2012-06-25 16:16:26 PDT LOG: could not send data to client: Connection reset by peer

On the hot standby:

2012-06-25 11:12:11 PDT FATAL: could not receive data from WAL stream: SSL error: sslv3 alert unexpected message
2012-06-25 11:12:11 PDT LOG: record with zero length at 1C5/95D2FE00

If our SSL libraries do not support SSL renegotiation, the default setting is wrong and perhaps warnings emitted if attempts are made to enable it.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: postgresql-9.1 9.1.4-0ubuntu12.04
ProcVersionSignature: Ubuntu 3.2.0-25.40-generic 3.2.18
Uname: Linux 3.2.0-25-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.0.1-0ubuntu8
Architecture: amd64
Date: Wed Jun 27 16:38:33 2012
ProcEnviron:
 LANGUAGE=en_AU:en
 TERM=xterm
 PATH=(custom, user)
 LANG=en_AU.UTF-8
 SHELL=/bin/bash
SourcePackage: postgresql-9.1
UpgradeStatus: Upgraded to precise on 2012-04-27 (60 days ago)

[Stuart Bishop (stub)]

Added openssl, as perhaps this is supposed to be working now? Probably INVALID or WONTFIX.

[Stuart Bishop (stub)]

Workaround is to set 'ssl_renegotiation_limit=0' in postgresql.conf

[Martin Pitt (pitti)]

The 512 MB is an upstream default, the packages don't change it.

(http://www.postgresql.org/docs/9.1/static/runtime-config-connection.html)

However, it does not seem very bad to set it to 0. I'm mostly wondering if that is a bug in OpenSSL and that should be supplied dynamically (pg_ctlctluster could check the OpenSSL version and add that option unless it's set explicly) or whether it's generally considered better to have it default to 0?

[Marc Deslauriers (mdeslaur)]

Maybe related?

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=675990
http://cvs.openssl.org/chngview?cn=22567

[Marc Deslauriers (mdeslaur)]

There is an openssl package in precise-proposed that may fix this issue. Could you try it and see if it solves it?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Stuart Bishop (stub)]

Still seeing failures with precise and openssl 1.0.1-4ubuntu5.7 at both ends.

The errors are now less informative:

<postgres@[unknown]:19761> 2013-03-15 03:55:12 UTCLOG: SSL renegotiation failure
<postgres@[unknown]:19761> 2013-03-15 03:55:41 UTCLOG: SSL renegotiation failure
<postgres@[unknown]:19761> 2013-03-15 03:56:20 UTCLOG: SSL renegotiation failure

It is interesting I'm getting spam from one procpid rather than it dying and restarting.

[Stuart Bishop (stub)]

Investigations upstream: http://postgresql.1045698.n5.nabble.com/SSL-renegotiation-td5763395.html

[Martin Pitt (pitti)]

Is that still a problem in current releases?

[Dan Fairs (danfairs)]

fwiw, I'm seeing this using PostgreSQL 9.3.2 (installed from postgresql.org's APT repository) using OpenSSL 1.0.1-4ubuntu5.11 on 12.04.4.

[Nelson Hernandez (nelsonh)]

I am also seeing this as of 2014-03-14
Setting 'ssl_renegotiation_limit=0' in postgresql.conf did not work for me.
Are their any other known workarounds (aside from downgrading Ubuntu and other packages as noted below)?

does not work on:
PostgreSQL 9.3.2 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, 64-bit
OpenSSL 1.0.1-4ubuntu5.11
libpq-dev 9.1.12-0ubuntu0.12.04
Ubuntu 12.04.4 LTS

ssl works when using the same database and connecting from a client with
OpenSSL 1.0.1-4ubuntu5.10
libpq-dev 9.1.10-0ubuntu12.04
Ubuntu 12.04.3 LTS

[David Peall (dkpeall)]

2014-04-10 13:19:50 SAST FATAL: could not receive data from WAL stream: SSL error: sslv3 alert unexpected message
2014-04-10 13:19:50 SAST LOG: invalid magic number 0000 in log segment 0000000100000DB400000052, offset 16384000
2014-04-10 13:19:50 SAST LOG: started streaming WAL from primary at DB4/52000000 on timeline 1

2014-04-10 13:21:38 SAST FATAL: could not receive data from WAL stream: server closed the connection unexpectedly
                This probably means the server terminated abnormally
                before or while processing the request.

2014-04-10 13:21:40 SAST LOG: invalid magic number 0000 in log segment 0000000100000DB4000000B0, offset 16384000
2014-04-10 13:21:40 SAST LOG: started streaming WAL from primary at DB4/B0000000 on timeline 1

[Luis Antonio Dias de Sá Junior (luisjunior-sa)]

Hi, I have same problem with ubuntu 14.04 and postgres 9.3.5 :(

But change the 'ssl_renegotiation_limit=0' resolve

[Olivier Dony (Odoo) (odo-openerp)]

FWIW, still happening today on 14.04 with Postgresql 9.3+154 and openssl 1.0.1f-1ubuntu2.7 0 on both master and slave.

This post from PG contributor Laurenz Albe has some extra debugging info that may be relevant, but sadly got no interest on the openssl mailing-list: https://groups.google.com/forum/#!topic/mailing.openssl.users/WAmXHwrExNI

[Adrien Nader (adrien)]

I'm going to mark this bug as Incomplete. If it is encountered again, please try to provide a reproducer: having to reproduce against a multi-tenant postgresql is a lot of work (especially when you're not familiar with pg).