postgres nodes fail install, retry works

[Francis Ginther]

I've seen this with both the ppa-assigner and ticket system django nodes. If I do just a "juju resolved --retry", it usually works again without further intervention. It acts as if there is a race condition which occasionally causes dpkg to fail.

[log]
2014-03-03 21:20:11 DEBUG juju.agent agent.go:253 Reading agent config, format: format 1.16
2014-03-03 21:20:11 INFO juju.jujud unit.go:75 unit agent unit-ts-postgres-0 start (1.17.4-precise-amd64)
2014-03-03 21:20:11 INFO juju runner.go:262 worker: start "api"
2014-03-03 21:20:11 INFO juju apiclient.go:113 state/api: dialing "wss://10.55.60.75:17070/"
2014-03-03 21:20:11 INFO juju apiclient.go:123 state/api: connection established
2014-03-03 21:20:11 DEBUG juju.agent agent.go:394 writing configuration file
2014-03-03 21:20:11 INFO juju runner.go:262 worker: start "upgrader"
2014-03-03 21:20:11 INFO juju runner.go:262 worker: start "logger"
2014-03-03 21:20:11 DEBUG juju.worker.logger logger.go:35 initial log config: "<root>=DEBUG"
2014-03-03 21:20:11 INFO juju runner.go:262 worker: start "uniter"
2014-03-03 21:20:11 DEBUG juju.worker.logger logger.go:60 logger setup
2014-03-03 21:20:11 INFO juju runner.go:262 worker: start "rsyslog"
2014-03-03 21:20:11 DEBUG juju.worker.logger logger.go:45 reconfiguring logging from "<root>=DEBUG" to "<root>=WARNING;unit=DEBUG"
2014-03-03 21:20:26 INFO juju-log Running install hook
2014-03-03 21:20:27 INFO install Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
2014-03-03 21:20:27 INFO install Get:2 http://security.ubuntu.com precise-security Release [49.6 kB]
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise Release.gpg
2014-03-03 21:20:27 INFO install Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools Release.gpg
2014-03-03 21:20:27 INFO install Get:3 http://nova.clouds.archive.ubuntu.com precise-updates Release.gpg [198 B]
2014-03-03 21:20:27 INFO install Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools Release
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise Release
2014-03-03 21:20:27 INFO install Get:4 http://nova.clouds.archive.ubuntu.com precise-updates Release [49.6 kB]
2014-03-03 21:20:27 INFO install Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main amd64 Packages
2014-03-03 21:20:27 INFO install Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main i386 Packages
2014-03-03 21:20:27 INFO install Ign http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/main Sources
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/universe Sources
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/main amd64 Packages
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/universe amd64 Packages
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/main i386 Packages
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/universe i386 Packages
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/main TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/universe TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/main Translation-en
2014-03-03 21:20:27 INFO install Get:5 http://security.ubuntu.com precise-security/main Sources [98.0 kB]
2014-03-03 21:20:27 INFO install Ign http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main Translation-en
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise/universe Translation-en
2014-03-03 21:20:27 INFO install Get:6 http://nova.clouds.archive.ubuntu.com precise-updates/main Sources [452 kB]
2014-03-03 21:20:27 INFO install Get:7 http://security.ubuntu.com precise-security/universe Sources [30.9 kB]
2014-03-03 21:20:27 INFO install Get:8 http://security.ubuntu.com precise-security/main amd64 Packages [364 kB]
2014-03-03 21:20:27 INFO install Get:9 http://security.ubuntu.com precise-security/universe amd64 Packages [91.0 kB]
2014-03-03 21:20:27 INFO install Get:10 http://security.ubuntu.com precise-security/main i386 Packages [388 kB]
2014-03-03 21:20:27 INFO install Get:11 http://security.ubuntu.com precise-security/universe i386 Packages [95.5 kB]
2014-03-03 21:20:27 INFO install Get:12 http://nova.clouds.archive.ubuntu.com precise-updates/universe Sources [105 kB]
2014-03-03 21:20:27 INFO install Get:13 http://nova.clouds.archive.ubuntu.com precise-updates/main amd64 Packages [753 kB]
2014-03-03 21:20:27 INFO install Hit http://security.ubuntu.com precise-security/main TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://security.ubuntu.com precise-security/universe TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://security.ubuntu.com precise-security/main Translation-en
2014-03-03 21:20:27 INFO install Hit http://security.ubuntu.com precise-security/universe Translation-en
2014-03-03 21:20:27 INFO install Get:14 http://nova.clouds.archive.ubuntu.com precise-updates/universe amd64 Packages [236 kB]
2014-03-03 21:20:27 INFO install Get:15 http://nova.clouds.archive.ubuntu.com precise-updates/main i386 Packages [775 kB]
2014-03-03 21:20:27 INFO install Get:16 http://nova.clouds.archive.ubuntu.com precise-updates/universe i386 Packages [241 kB]
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise-updates/main TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise-updates/universe TranslationIndex
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise-updates/main Translation-en
2014-03-03 21:20:27 INFO install Hit http://nova.clouds.archive.ubuntu.com precise-updates/universe Translation-en
2014-03-03 21:20:28 INFO install Fetched 3728 kB in 1s (2563 kB/s)
2014-03-03 21:20:40 INFO install Reading package lists...
2014-03-03 21:20:40 INFO juju-log map version from distro release ...
2014-03-03 21:20:40 INFO juju-log version=9.1 from distro_codename='precise'
Reading package lists... Doneall
Building dependency tree l
Reading state information... Done
2014-03-03 21:20:41 INFO juju-log Installing ['python-psutil', 'postgresql-9.1', 'postgresql-contrib-9.1', 'postgresql-plpython-9.1', 'python-jinja2', 'syslinux', 'python-psycopg2'] with options: []
2014-03-03 21:20:41 INFO install Reading package lists...
2014-03-03 21:20:41 INFO install Building dependency tree...
2014-03-03 21:20:41 INFO install Reading state information...
2014-03-03 21:20:41 INFO install The following extra packages will be installed:
2014-03-03 21:20:41 INFO install libcrypt-passwdmd5-perl libossp-uuid16 libpq5 libxslt1.1 mtools
2014-03-03 21:20:41 INFO install postgresql-client-9.1 postgresql-client-common postgresql-common
2014-03-03 21:20:41 INFO install python-egenix-mxdatetime python-egenix-mxtools python-markupsafe ssl-cert
2014-03-03 21:20:41 INFO install syslinux-common
2014-03-03 21:20:41 INFO install Suggested packages:
2014-03-03 21:20:41 INFO install uuid floppyd oidentd ident-server locales-all postgresql-doc-9.1
2014-03-03 21:20:41 INFO install libdbd-pg-perl python-egenix-mxdatetime-dbg python-egenix-mxdatetime-doc
2014-03-03 21:20:41 INFO install python-egenix-mxtools-dbg python-egenix-mxtools-doc python-jinja2-doc
2014-03-03 21:20:41 INFO install python-psycopg2-doc openssl-blacklist
2014-03-03 21:20:41 INFO install The following NEW packages will be installed:
2014-03-03 21:20:41 INFO install libcrypt-passwdmd5-perl libossp-uuid16 libpq5 libxslt1.1 mtools
2014-03-03 21:20:41 INFO install postgresql-9.1 postgresql-client-9.1 postgresql-client-common
2014-03-03 21:20:41 INFO install postgresql-common postgresql-contrib-9.1 postgresql-plpython-9.1
2014-03-03 21:20:41 INFO install python-egenix-mxdatetime python-egenix-mxtools python-jinja2
2014-03-03 21:20:41 INFO install python-markupsafe python-psutil python-psycopg2 ssl-cert syslinux
2014-03-03 21:20:41 INFO install syslinux-common
2014-03-03 21:20:41 INFO install 0 upgraded, 20 newly installed, 0 to remove and 0 not upgraded.
2014-03-03 21:20:41 INFO install Need to get 8523 kB of archives.
2014-03-03 21:20:41 INFO install After this operation, 23.6 MB of additional disk space will be used.
2014-03-03 21:20:41 INFO install Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main libxslt1.1 amd64 1.1.26-8ubuntu1.3 [167 kB]
2014-03-03 21:20:41 INFO install Get:2 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main libcrypt-passwdmd5-perl all 1.3-10 [10.4 kB]
2014-03-03 21:20:44 INFO install Get:3 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main libossp-uuid16 amd64 1.6.2-1.1build3 [51.4 kB]
2014-03-03 21:20:44 INFO install Get:4 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main libpq5 amd64 9.1.12-0ubuntu0.12.04 [96.0 kB]
2014-03-03 21:20:44 INFO install Get:5 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main mtools amd64 4.0.12-1ubuntu0.12.04.1 [186 kB]
2014-03-03 21:20:44 INFO install Get:6 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-client-common all 129ubuntu1 [25.1 kB]
2014-03-03 21:20:44 INFO install Get:7 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-client-9.1 amd64 9.1.12-0ubuntu0.12.04 [959 kB]
2014-03-03 21:20:44 INFO install Get:8 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main ssl-cert all 1.0.28ubuntu0.1 [12.3 kB]
2014-03-03 21:20:44 INFO install Get:9 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-common all 129ubuntu1 [96.5 kB]
2014-03-03 21:20:44 INFO install Get:10 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-9.1 amd64 9.1.12-0ubuntu0.12.04 [4312 kB]
2014-03-03 21:20:44 INFO install Get:11 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-contrib-9.1 amd64 9.1.12-0ubuntu0.12.04 [464 kB]
2014-03-03 21:20:44 INFO install Get:12 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise-updates/main postgresql-plpython-9.1 amd64 9.1.12-0ubuntu0.12.04 [43.3 kB]
2014-03-03 21:20:44 INFO install Get:13 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-egenix-mxtools amd64 3.2.1-1ubuntu1 [89.1 kB]
2014-03-03 21:20:44 INFO install Get:14 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-egenix-mxdatetime amd64 3.2.1-1ubuntu1 [79.3 kB]
2014-03-03 21:20:44 INFO install Get:15 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-markupsafe amd64 0.15-1 [13.6 kB]
2014-03-03 21:20:44 INFO install Get:16 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-jinja2 amd64 2.6-1 [158 kB]
2014-03-03 21:20:44 INFO install Get:17 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/universe python-psutil amd64 0.4.1-1ubuntu1 [81.7 kB]
2014-03-03 21:20:44 INFO install Get:18 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-psycopg2 amd64 2.4.5-1 [153 kB]
2014-03-03 21:20:44 INFO install Get:19 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main syslinux-common all 2:4.05+dfsg-2 [1465 kB]
2014-03-03 21:20:44 INFO install Get:20 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main syslinux amd64 2:4.05+dfsg-2 [61.1 kB]
2014-03-03 21:20:44 INFO install Preconfiguring packages ...
2014-03-03 21:20:45 INFO install Fetched 8523 kB in 0s (26.0 MB/s)
2014-03-03 21:20:52 INFO install Selecting previously unselected package libxslt1.1.
2014-03-03 21:20:52 INFO install (Reading database ... 48202 files and directories currently installed.)
2014-03-03 21:20:52 INFO install Unpacking libxslt1.1 (from .../libxslt1.1_1.1.26-8ubuntu1.3_amd64.deb) ...
2014-03-03 21:20:55 INFO install Selecting previously unselected package libcrypt-passwdmd5-perl.
2014-03-03 21:20:55 INFO install Unpacking libcrypt-passwdmd5-perl (from .../libcrypt-passwdmd5-perl_1.3-10_all.deb) ...
2014-03-03 21:21:02 INFO install Selecting previously unselected package libossp-uuid16.
2014-03-03 21:21:02 INFO install Unpacking libossp-uuid16 (from .../libossp-uuid16_1.6.2-1.1build3_amd64.deb) ...
2014-03-03 21:21:06 INFO install Selecting previously unselected package libpq5.
2014-03-03 21:21:06 INFO install Unpacking libpq5 (from .../libpq5_9.1.12-0ubuntu0.12.04_amd64.deb) ...
2014-03-03 21:21:09 INFO install Selecting previously unselected package mtools.
2014-03-03 21:21:10 INFO install Unpacking mtools (from .../mtools_4.0.12-1ubuntu0.12.04.1_amd64.deb) ...
2014-03-03 21:21:19 INFO install Selecting previously unselected package postgresql-client-common.
2014-03-03 21:21:19 INFO install Unpacking postgresql-client-common (from .../postgresql-client-common_129ubuntu1_all.deb) ...
2014-03-03 21:21:22 INFO install Selecting previously unselected package postgresql-client-9.1.
2014-03-03 21:21:22 INFO install Unpacking postgresql-client-9.1 (from .../postgresql-client-9.1_9.1.12-0ubuntu0.12.04_amd64.deb) ...
2014-03-03 21:21:31 INFO install Selecting previously unselected package ssl-cert.
2014-03-03 21:21:31 INFO install Unpacking ssl-cert (from .../ssl-cert_1.0.28ubuntu0.1_all.deb) ...
2014-03-03 21:21:40 INFO install Selecting previously unselected package postgresql-common.
2014-03-03 21:21:40 INFO install Unpacking postgresql-common (from .../postgresql-common_129ubuntu1_all.deb) ...
2014-03-03 21:21:40 INFO install Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
2014-03-03 21:21:47 INFO install Selecting previously unselected package postgresql-9.1.
2014-03-03 21:21:47 INFO install Unpacking postgresql-9.1 (from .../postgresql-9.1_9.1.12-0ubuntu0.12.04_amd64.deb) ...
2014-03-03 21:21:56 INFO install Selecting previously unselected package postgresql-contrib-9.1.
2014-03-03 21:21:56 INFO install Unpacking postgresql-contrib-9.1 (from .../postgresql-contrib-9.1_9.1.12-0ubuntu0.12.04_amd64.deb) ...
2014-03-03 21:22:06 INFO install Selecting previously unselected package postgresql-plpython-9.1.
2014-03-03 21:22:06 INFO install Unpacking postgresql-plpython-9.1 (from .../postgresql-plpython-9.1_9.1.12-0ubuntu0.12.04_amd64.deb) ...
2014-03-03 21:22:13 INFO install Selecting previously unselected package python-egenix-mxtools.
2014-03-03 21:22:13 INFO install Unpacking python-egenix-mxtools (from .../python-egenix-mxtools_3.2.1-1ubuntu1_amd64.deb) ...
2014-03-03 21:22:17 INFO install Selecting previously unselected package python-egenix-mxdatetime.
2014-03-03 21:22:17 INFO install Unpacking python-egenix-mxdatetime (from .../python-egenix-mxdatetime_3.2.1-1ubuntu1_amd64.deb) ...
2014-03-03 21:22:21 INFO install Selecting previously unselected package python-markupsafe.
2014-03-03 21:22:21 INFO install Unpacking python-markupsafe (from .../python-markupsafe_0.15-1_amd64.deb) ...
2014-03-03 21:22:24 INFO install Selecting previously unselected package python-jinja2.
2014-03-03 21:22:24 INFO install Unpacking python-jinja2 (from .../python-jinja2_2.6-1_amd64.deb) ...
2014-03-03 21:22:30 INFO install Selecting previously unselected package python-psutil.
2014-03-03 21:22:30 INFO install Unpacking python-psutil (from .../python-psutil_0.4.1-1ubuntu1_amd64.deb) ...
2014-03-03 21:22:34 INFO install Selecting previously unselected package python-psycopg2.
2014-03-03 21:22:34 INFO install Unpacking python-psycopg2 (from .../python-psycopg2_2.4.5-1_amd64.deb) ...
2014-03-03 21:22:37 INFO install Selecting previously unselected package syslinux-common.
2014-03-03 21:22:37 INFO install Unpacking syslinux-common (from .../syslinux-common_2%3a4.05+dfsg-2_all.deb) ...
2014-03-03 21:22:50 INFO install Selecting previously unselected package syslinux.
2014-03-03 21:22:50 INFO install Unpacking syslinux (from .../syslinux_2%3a4.05+dfsg-2_amd64.deb) ...
2014-03-03 21:22:53 INFO install Processing triggers for man-db ...
2014-03-03 21:22:56 INFO install Processing triggers for install-info ...
2014-03-03 21:22:58 INFO install Processing triggers for ureadahead ...
2014-03-03 21:22:59 INFO install Setting up libxslt1.1 (1.1.26-8ubuntu1.3) ...
2014-03-03 21:23:00 INFO install Setting up libcrypt-passwdmd5-perl (1.3-10) ...
2014-03-03 21:23:01 INFO install Setting up libossp-uuid16 (1.6.2-1.1build3) ...
2014-03-03 21:23:02 INFO install Setting up libpq5 (9.1.12-0ubuntu0.12.04) ...
2014-03-03 21:23:03 INFO install Setting up mtools (4.0.12-1ubuntu0.12.04.1) ...
2014-03-03 21:23:04 INFO install Setting up postgresql-client-common (129ubuntu1) ...
2014-03-03 21:23:05 INFO install Setting up postgresql-client-9.1 (9.1.12-0ubuntu0.12.04) ...
2014-03-03 21:23:07 INFO install update-alternatives: using /usr/share/postgresql/9.1/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode.
2014-03-03 21:23:08 INFO install Setting up ssl-cert (1.0.28ubuntu0.1) ...
2014-03-03 21:23:11 INFO install Setting up postgresql-common (129ubuntu1) ...
2014-03-03 21:23:17 INFO install Adding user postgres to group ssl-cert
2014-03-03 21:23:18 INFO install Building PostgreSQL dictionaries from installed myspell/hunspell packages...
2014-03-03 21:23:19 INFO install Setting up postgresql-9.1 (9.1.12-0ubuntu0.12.04) ...
2014-03-03 21:23:20 INFO install Creating new cluster (configuration: /etc/postgresql/9.1/main, data: /var/lib/postgresql/9.1/main)...
2014-03-03 21:23:22 INFO install Moving configuration file /var/lib/postgresql/9.1/main/postgresql.conf to /etc/postgresql/9.1/main...
2014-03-03 21:23:22 INFO install Moving configuration file /var/lib/postgresql/9.1/main/pg_hba.conf to /etc/postgresql/9.1/main...
2014-03-03 21:23:22 INFO install Moving configuration file /var/lib/postgresql/9.1/main/pg_ident.conf to /etc/postgresql/9.1/main...
2014-03-03 21:23:36 INFO install Configuring postgresql.conf to use port 5432...
2014-03-03 21:23:36 INFO install update-alternatives: using /usr/share/postgresql/9.1/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode.
2014-03-03 21:23:36 INFO install * Starting PostgreSQL 9.1 database server
2014-03-03 21:24:07 INFO install * The PostgreSQL server failed to start. Please check the log output.
2014-03-03 21:24:07 INFO install ...fail!
2014-03-03 21:24:07 INFO install invoke-rc.d: initscript postgresql, action "start" failed.
2014-03-03 21:24:07 INFO install dpkg: error processing postgresql-9.1 (--configure):
2014-03-03 21:24:07 INFO install subprocess installed post-installation script returned error exit status 1
2014-03-03 21:24:07 INFO install dpkg: dependency problems prevent configuration of postgresql-contrib-9.1:
2014-03-03 21:24:07 INFO install postgresql-contrib-9.1 depends on postgresql-9.1 (= 9.1.12-0ubuntu0.12.04); however:
2014-03-03 21:24:07 INFO install Package postgresql-9.1 is not configured yet.
2014-03-03 21:24:07 INFO install dpkg: error processing postgresql-contrib-9.1 (--configure):
2014-03-03 21:24:07 INFO install dependency problems - leaving unconfigured
2014-03-03 21:24:07 INFO install dpkg: dependency problems prevent configuration of postgresql-plpython-9.1:
2014-03-03 21:24:07 INFO install postgresql-plpython-9.1 depends on postgresql-9.1 (= 9.1.12-0ubuntu0.12.04); however:
2014-03-03 21:24:07 INFO install Package postgresql-9.1 is not configured yet.
2014-03-03 21:24:07 INFO install dpkg: error processing postgresql-plpython-9.1 (--configure):
2014-03-03 21:24:07 INFO install dependency problems - leaving unconfigured
2014-03-03 21:24:07 INFO install Setting up python-egenix-mxtools (3.2.1-1ubuntu1) ...
2014-03-03 21:24:07 INFO install No apport report written because the error message indicates its a followup error from a previous failure.
2014-03-03 21:24:07 INFO install No apport report written because the error message indicates its a followup error from a previous failure.
2014-03-03 21:24:19 INFO install Setting up python-egenix-mxdatetime (3.2.1-1ubuntu1) ...
2014-03-03 21:24:20 INFO install Setting up python-markupsafe (0.15-1) ...
2014-03-03 21:24:21 INFO install Setting up python-jinja2 (2.6-1) ...
2014-03-03 21:24:22 INFO install Setting up python-psutil (0.4.1-1ubuntu1) ...
2014-03-03 21:24:24 INFO install Setting up python-psycopg2 (2.4.5-1) ...
2014-03-03 21:24:25 INFO install Setting up syslinux-common (2:4.05+dfsg-2) ...
2014-03-03 21:24:25 INFO install Setting up syslinux (2:4.05+dfsg-2) ...
2014-03-03 21:24:27 INFO install Processing triggers for libc-bin ...
2014-03-03 21:24:27 INFO install ldconfig deferred processing now taking place
2014-03-03 21:24:28 INFO install Errors were encountered while processing:
2014-03-03 21:24:28 INFO install postgresql-9.1
2014-03-03 21:24:28 INFO install postgresql-contrib-9.1
2014-03-03 21:24:28 INFO install postgresql-plpython-9.1
2014-03-03 21:24:28 INFO install E: Sub-process /usr/bin/dpkg returned an error code (1)
2014-03-03 21:24:29 INFO install Traceback (most recent call last):
2014-03-03 21:24:29 INFO install File "/var/lib/juju/agents/unit-ts-postgres-0/charm/hooks/install", line 2228, in <module>
2014-03-03 21:24:29 INFO install hooks.execute(sys.argv)
2014-03-03 21:24:29 INFO install File "/var/lib/juju/agents/unit-ts-postgres-0/charm/hooks/charmhelpers/core/hookenv.py", line 322, in execute
2014-03-03 21:24:29 INFO install self._hooks[hook_name]()
2014-03-03 21:24:29 INFO install File "/var/lib/juju/agents/unit-ts-postgres-0/charm/hooks/install", line 1013, in install
2014-03-03 21:24:29 INFO install update_repos_and_packages()
2014-03-03 21:24:29 INFO install File "/var/lib/juju/agents/unit-ts-postgres-0/charm/hooks/install", line 1560, in update_repos_and_packages
2014-03-03 21:24:29 INFO install fetch.apt_install(packages, fatal=True)
2014-03-03 21:24:29 INFO install File "/var/lib/juju/agents/unit-ts-postgres-0/charm/hooks/charmhelpers/fetch/__init__.py", line 54, in apt_install
2014-03-03 21:24:29 INFO install subprocess.check_call(cmd)
2014-03-03 21:24:29 INFO install File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
2014-03-03 21:24:29 INFO install raise CalledProcessError(retcode, cmd)
2014-03-03 21:24:29 INFO install subprocess.CalledProcessError: Command '['apt-get', '-y', 'install', 'python-psutil', 'postgresql-9.1', 'postgresql-contrib-9.1', 'postgresql-plpython-9.1', 'python-jinja2', 'syslinux', 'python-psycopg2']' returned non-zero exit status 100
2014-03-03 21:24:29 ERROR juju.worker.uniter uniter.go:480 hook failed: exit status 1

[Francis Ginther (fginther)]

Also found that when this happens to the ts-postgres node, it can lead to failure to start the lander worker upstart job because the ts-django node isn't responding before the upstart job gives up.

May need to update the lander upstart job to never give up.

[Francis Ginther (fginther)]

*Sigh*

We're using the standard postgresql charm: https://bugs.launchpad.net/ubuntu-ci-services-itself/+bug/1287872

The failure appears to be timing related. When nodes take a long time to go from pending to active, I'm more likely to see this. I don't think we'll see this too much when using a reliable cloud, but will keep monitoring.