New upstream microreleases 9.3.23, 9.5.13, 9.6.9 and 10.4

[Christian Ehrhardt ]

Postgresql stable update

Current versions in supported releases:
 postgresql-9.3 | 9.3.22-0ubuntu0.14.04 trusty
 postgresql-9.5 | 9.5.12-0ubuntu0.16.04 xenial
 postgresql-9.6 | 9.6.8-0ubuntu0.17.10 artful
 postgresql-10 | 10.3-1 bionic
 postgresql-10 | 10.3-2 cosmic

Special cases:
- Cosmic will be synced from Debian which usually releases fast.
  So no Cosmic upload by us.

Last related stable updates: 9.3.23, 9.5.13, 9.6.9, 10.4

So the todo is to pick:
MRE: Trusty 9.3.23 from https://borka.postgresql.org/staging/cb8f0c2ac40d4189a887fbf076c597239afd5264/postgresql-9.3.23.tar.gz
MRE: Xenial 9.5.13 from https://borka.postgresql.org/staging/cb8f0c2ac40d4189a887fbf076c597239afd5264/postgresql-9.5.13.tar.gz
MRE: Artful 9.6.9 from https://borka.postgresql.org/staging/cb8f0c2ac40d4189a887fbf076c597239afd5264/postgresql-9.6.9.tar.gz
MRE: Bionic 10.4 from https://borka.postgresql.org/staging/cb8f0c2ac40d4189a887fbf076c597239afd5264/postgresql-10.4.tar.gz
Sync: Cosmic 10.4 via Debian auto-sync

Standing MRE - Consider last updates as template:
- pad.lv/1637236
- pad.lv/1664478
- pad.lv/1690730
- pad.lv/1713979
- pad.lv/1730661
- pad.lv/1747676
- pad.lv/1752271
New - this bug
- pad.lv/1769888

Test builds and tests took place in PPAs
T: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3254
X: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3255
A: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3256
B: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3257

[Christian Ehrhardt  (paelzer)]

Test builds in the referred PPAs are all good.
Need some more tests as usual on the MRE for confidence, but other than that all looks good atm.

[Christian Ehrhardt  (paelzer)]

Along the ppas bileto tickets are:
https://bileto.ubuntu.com/#/ticket/3254
https://bileto.ubuntu.com/#/ticket/3255
https://bileto.ubuntu.com/#/ticket/3256
https://bileto.ubuntu.com/#/ticket/3257

[Christian Ehrhardt  (paelzer)]

Bileto is no more helping in pre-qualification so much so I ran a few in KVM images.
The most relevant tests are from postgresql-common and postgresql itself running the full testsuite.
I ran those in autopkgtest VMs locally, and results are all good - so I think we can push these for SRU.

Given that this time only "some" are CVE affected I'll ping mdeslaur how he wants to handle this.

[Christian Ehrhardt  (paelzer)]

The CVE Sev is low so we decided to push through the normal SRU process.
ALl pre-tests are ready, so making them available after a final check with my co-maintainer.

[Christian Ehrhardt  (paelzer)]

Also Cosmic already has 10.4-2 uploaded (in -proposed atm).
With all prereqs done sponsoring for SRu review.

[Christian Ehrhardt  (paelzer)]

Hi,
this is waiting in -unapproved for more than a week now.
I explicitly subscribed ~ubuntu-sru to ping for this and ask if there is anything ongoing that blocks this where Andreas or myself can help with (or just busy times in which we can't help a lot, but at leas know that there is nothing missing from our side)?

[Łukasz Zemczak (sil2100)]

Hello , or anyone else affected,

Accepted postgresql-9.5 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postgresql-9.5/9.5.13-0ubuntu0.16.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello , or anyone else affected,

Accepted postgresql-9.6 into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postgresql-9.6/9.6.9-0ubuntu0.17.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello , or anyone else affected,

Accepted postgresql-10 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postgresql-10/10.4-0ubuntu0.18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello , or anyone else affected,

Accepted postgresql-9.3 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/postgresql-9.3/9.3.23-0ubuntu0.14.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Christian Ehrhardt  (paelzer)]

For the reoccurring issues I opened two MPs to skip them unversioned (it won't change in T/X)
- https://code.launchpad.net/~paelzer/britney/hints-ubuntu-trusty-postgresMREII/+merge/346934
- https://code.launchpad.net/~paelzer/britney/hints-ubuntu-xenial-postgresMREII/+merge/346935

There are a few tests failing which seem just flaky/noise like libreoffice. Those woll be retried a few times before investigating.

But this time there are also a few tests which need to be investigated:
- Xenial&Artful: skytools (I think this never popped up in the past in regard to postgres MREs)
- Artful: dbconfig-common
- Artful: pg-repack

[Christian Ehrhardt  (paelzer)]

Bionic really was just flaky armhf, Bionic is fully good now after this retry worked.

[Christian Ehrhardt  (paelzer)]

I identified libreoffice on armhg as failing unrelated as well, made it part of the proposed hint changes.

Started to debug gearmand issue to decide if it shall be hinted as well.
This had issues before "us" but on the last SRU (util-linux by juliank) the test fail was ignored.

[Christian Ehrhardt  (paelzer)]

Geamand is silly,
First of all it passes on armhf when trying to reproduce (as LP armhf lxd)
./run-tests
01.gearman-job-server.t ..
ok 1 - service status
ok 2 - process running
# passed all 2 test(s)
1..2
ok
02.gearman-worker.t ......
ok 1 - start worker
ok 2 - start client
# passed all 2 test(s)
1..2
ok
All tests successful.
Files=2, Tests=4, 0 wallclock secs ( 0.11 usr 0.02 sys + 0.04 cusr 0.11 csys = 0.28 CPU)
Result: PASS

And further this isn't a dependency for postgres anyway, when the test runs it has:
apt-cache policy 'postg*' | grep Installed | uniq -c
     91 Installed: (none)

So while I can't make a strong enough argument to force-badtest gearmand yet this is a test fail that we should ignore on this SRU.

A few more resolved due to retries and lets for now assume all the overrides get accepted.
That leaves the analysis of dbconfig-common, skytools and pg-repack to be evaluated.

[Christian Ehrhardt  (paelzer)]

Test check:
- skytools3
This is reproducible locally with the pre-MRE version just as much.
Something "else" yet unknown must have changed
  test init_ext ... ok
  test londiste_provider ... ok
  test londiste_subscriber ... ok
  test londiste_fkeys ... ok
  test londiste_execute ... ok
  test londiste_seqs ... ok
  test londiste_merge ... ok
  test londiste_leaf ... ok
  test londiste_create_part ... FAILED
...
  select tgenabled, pg_get_triggerdef(oid) from pg_trigger where tgrelid = 'events_2011_01'::regclass::oid;
! tgenabled | pg_get_triggerdef
! -----------+-------------------------------------------------------------------------------------------------------------
! A | CREATE TRIGGER "Fooza" AFTER DELETE ON public.events_2011_01 FOR EACH ROW EXECUTE PROCEDURE "NullTrigger"()
  (1 row)

  -- test weird quoting

Comparing with old logs of the same test on 2017-11-24 doesn't reveal anything obvious at first and it seems to be just whitespaces but then I found the reason for the new indent (almost missed, thanks colordiff).
"events_2011_01" is now called "public.events_2011_01"
It could be that one of the security-pushed CVE changes modified that.

This change seems safe, so with the check adapted to expect the new prefix I retested.
And all is good, all further tests pass without further modification as well.

We can either co-SRU a change to skytools3 for the test or badtest the old version or now (or both).

[Christian Ehrhardt  (paelzer)]

I spawned bug 1773763 for skytools3

Until resolved lets mark the current version as broken as it is.
part of the already mentioned X branch at:
- https://code.launchpad.net/~paelzer/britney/hints-ubuntu-xenial-postgresMREII/+merge/346935
And for Artful a new MP at:
A - https://code.launchpad.net/~paelzer/britney/hints-ubuntu-artful-postgresMREII/+merge/346960

[Christian Ehrhardt  (paelzer)]

Next I checked pg-repack.
Fist of all, this is also reproducible with the current version in Artful (and seems only to affect Artful).

This is fixed in 1.4.2-2 as in Bionic and later (carries way more alternative repack_#.out files).
I'm not entirely sure, but this could be a real issue in pg-repack yet I don't understand it enough yet.
For this scope here:
1. the MRE is not the trigger, so lets go on
2. Lets document in britney that the current pg-repack de-facto is broken in Artful
3. I filed bug 1773772 to take a look and keep separate to this bug here

Next issue to look at is debconfig-common, but that is a bit harder to rerun as it breaks the testbed.

[Christian Ehrhardt  (paelzer)]

dbconfig-common in fact fails without the new postgres as well.
But this is even less related than the others before as it fails on all mariadb related tests but not due to postgresql.

There is 10.1.30-0ubuntu0.17.10.1 that was pushed as security fix, not sure if that is related.
The last test in autopkgtest was against mariadb-10.1/10.1.28-1 3 months before.

I filed bug 1773792 for this, but as it is not related to this MRE here is should not stall it.

[Christian Ehrhardt  (paelzer)]

Summary:
1. some flaky tests resolved by retries
2. a few regular failing tests have MPs to improve Britney hints
  - https://code.launchpad.net/~paelzer/britney/hints-ubuntu-trusty-postgresMREII/+merge/346934
  - https://code.launchpad.net/~paelzer/britney/hints-ubuntu-xenial-postgresMREII/+merge/346935
  - https://code.launchpad.net/~paelzer/britney/hints-ubuntu-artful-postgresMREII/+merge/346960
3. found some broken tests unrelated to this particular MRE upload and opened
  - bug 1773792 dbconfig-common
  - bug 1773772 pg-repack
  - bug 1773763 skytools3

Now waiting on review of the Britney-MPs as next step.

[Christian Ehrhardt  (paelzer)]

FYI - after discussing with sil2100 he fixed Bileto and it seems to work for me again, so next time we should again (as we did in the past) be able to pre-sniff and prep the autopkg findings out of the actual SRU \o/

[Christian Ehrhardt  (paelzer)]

Hints all got merged, lets re-evaluate after the next Britney run how the remaining state looks like.

[Christian Ehrhardt  (paelzer)]

The PDNS test on xenial fails from the archive as is without the postgres upload.
See e.g. the same hickup on openldap. It is not bad enough for a force-badtest yet, but it seems unreliable and unrelated in regard of this update here.

[Christian Ehrhardt  (paelzer)]

After our work on all these things (details above) I summarize the status of the tests:

- Trusty - all good
- Xenial
  - gearmband on armhf - as outlined not related to this update
  - pdns - same issue ocured in the past, not related to this update
- Artful
  - dbconfig-common broken by mariadb see 1773792, not related to this update
  - skytools3 broken by former updates see 1773763, not triggered by this update
- Bionic - all good

With that, I'd ask to ignore the remaining fails of autopkgtests on Xenial and Artful.
The update seems good and not to be the trigger of any of those issues - setting verification tags.

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-10 - 10.4-0ubuntu0.18.04

---------------
postgresql-10 (10.4-0ubuntu0.18.04) bionic; urgency=medium

  * New upstream release (LP: #1769888)
    - A dump/restore is not required for those running 10.X.
      However, if you use the adminpack extension, you should update it as
      per the first changelog entry of the changelog.
      (CVE-2018-1115)
      Also, if the function marking mistakes mentioned in the first
      changelog entry affect you, you will want to take steps to
      correct your database catalogs.
    - Details about changes can be found at
      https://www.postgresql.org/docs/10/static/release-10-4.html

 -- Christian Ehrhardt <email address hidden> Tue, 08 May 2018 15:17:41 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for postgresql-10 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-9.6 - 9.6.9-0ubuntu0.17.10

---------------
postgresql-9.6 (9.6.9-0ubuntu0.17.10) artful; urgency=medium

  * New upstream release (LP: #1769888)
    - A dump/restore is not required for those running 9.6.X.
      However, if you use the adminpack extension, you should update it as
      per the first changelog entry of the changelog.
      (CVE-2018-1115)
      Also, if the function marking mistakes mentioned in the first
      changelog entry affect you, you will want to take steps to
      correct your database catalogs.
    - Details about changes can be found at
      https://www.postgresql.org/docs/9.6/static/release-9-6-9.html

 -- Christian Ehrhardt <email address hidden> Tue, 08 May 2018 15:17:44 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-9.5 - 9.5.13-0ubuntu0.16.04

---------------
postgresql-9.5 (9.5.13-0ubuntu0.16.04) xenial; urgency=medium

  * New upstream release (LP: #1769888)
    - A dump/restore is not required for those running 9.5.X.
      However, if the function marking mistakes mentioned in the first
      changelog entry affect you, you will want to take steps to
      correct your database catalogs.
    - Details about changes can be found at
      https://www.postgresql.org/docs/9.5/static/release-9-5-13.html

 -- Christian Ehrhardt <email address hidden> Tue, 08 May 2018 15:17:49 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-9.3 - 9.3.23-0ubuntu0.14.04

---------------
postgresql-9.3 (9.3.23-0ubuntu0.14.04) trusty; urgency=medium

  * New upstream release (LP: #1769888)
    - A dump/restore is not required for those running 9.3.X.
      However, if the function marking mistakes mentioned in the first
      changelog entry affect you, you will want to take steps to
      correct your database catalogs.
    - Details about changes can be found at
      https://www.postgresql.org/docs/9.3/static/release-9-3-23.html

 -- Christian Ehrhardt <email address hidden> Tue, 08 May 2018 15:17:51 +0200