FTBFS in Focal with PG-12 present

[Christian Ehrhardt ]

self = <asyncpg.cluster.TempCluster object at 0x7f8a019b6610>, binary = 'pg_ctl'

    def _find_pg_binary(self, binary):
        bpath = platform_exe(os.path.join(self._pg_bin_dir, binary))

        if not os.path.isfile(bpath):
> raise ClusterError(
                'could not find {} executable: '.format(binary) +
                '{!r} does not exist or is not a file'.format(bpath))
E asyncpg.cluster.ClusterError: could not find pg_ctl executable: '/usr/lib/postgresql/12/bin/pg_ctl' does not exist or is not a file

(and similar)

Fails on all architectures.

[Christian Ehrhardt  (paelzer)]

I had this running in background, first of all this confirmed that this builds fine for Eoan but fails for Focal.
Therefore we know it is reproducible and dependent on the changes in Focal to trigger.

[Christian Ehrhardt  (paelzer)]

It misses the pg-12 version, but has PG-11 installed:

$ find /usr/ -name pg_ctl
/usr/lib/postgresql/11/bin/pg_ctl

There are a lot of tests, but it is enough to run just e.g.:
$ cd <BUILDDIR>
$ cd .pybuild/cpython3_3.8_asyncpg/build
$ python3.8 -m pytest -k-test_flake8 tests/test__environment.py

There is no PG-12 installed:
postgresql 11+204 11+204ubuntu1
postgresql-11 11.5-1 11.5-3sid2
postgresql-client-11 11.5-1 11.5-3sid2
postgresql-client-common 204 204ubuntu1
postgresql-common 204 204ubuntu1
postgresql-contrib 11+204 11+204ubuntu1

Maybe the newer versions of the common packages make it already look for PG-12?

[Christian Ehrhardt  (paelzer)]

postgresql-11 already has:
   * Stop building lib packages, postgresql-12 is already in unstable.
     (Closes: #941845)

Those come from postgresql-12 now.
This here might already show things are going further towards v12.

The reason it looks there is in attribute self._pg_bin_dir = /usr/lib/postgresql/12/
Which is loaded from config.
1. look for environment config (none)
2. look for /usr/bin/pg_config (exists)
...
This already points to postgresql-12

=> BINDIR = /usr/lib/postgresql/12/bin

This is held in "the old" state by Infinities correct initial fix in postgresql-common
   1 postgresql-common (204ubuntu1) focal; urgency=medium
   2
   3 * Set Ubuntu 20.04 default PostgreSQL version to 11 for now.

But it is now time to follow the migration to PG-12
OTOH the current postgresql-common v207 in Debian still has 11 for them as well.
102 11|11.*) /bin/echo -e "11" ;; # Bullseye

The probing code is actually from "/usr/bin/pg_config.libpq-dev"
(diversion)

And after all it is based on:
ii libpq-dev 12.0-1 amd64 header files for libpq5 (PostgreSQL library)
which is from postgresql-12

[Christian Ehrhardt  (paelzer)]

Ok, after analyzing the case it was clear that it is a mismatch between defaults e.g. postgresql -> postgresql-12 while at the same time e.g. libpq-dev already pointing to 12.

That will be resolved via a coming upload of postgresql-common to Debian (and a matching sync to Ubuntu to drop.

Adding a bug task and revisiting in a few days.
Then we have to re-trigger tests as needed.

[1]: https://git.launchpad.net/ubuntu/+source/postgresql-common/commit/?id=a24ab904d15bd750c1c9139ec67047a87de22eb5

While at it - Big thanks to Myon for all his work on PG!

[Christian Ehrhardt  (paelzer)]

This bug was fixed in the package postgresql-common - 208

---------------
postgresql-common (208) unstable; urgency=medium

  [ Christoph Berg ]
  * gitlab/gitlab-ci.yml: Too many packages fail because reprotest runs the
    build as root, disable for now.
  * README.Debian: Redirect reference to architecture.html to README.md.gz.
    (Closes: #942021)
  * README.Devel: Remove, bzr is long obsolete.
  * Remove dependency on dctrl-tools; rewrite control file updating in perl.
    As a side effect, generated debian/control files will no longer have a
    trailing empty line. Die if no supported versions were found.
  * pg_upgradecluster: Accept -m link and -m clone for simplicity.
  * debian/supported-versions: Add 12 as default version and for Ubuntu 20.04.
  * pg_createcluster: Add note on security implications of using --auth-host
    and --auth-local.
  * postgresql-all: Don't depend on postgresql-plpython-12, it doesn't exist.
  * pg_buildext: New action "installed-versions" reporting the list of
    PostgreSQL major versions used by packages built from a source package.
  * pg_buildext: "installcheck" uses "installed-versions" now.

  [ Tilman Koschnick ]
  * pg_upgradecluster: Support passing --clone to pg_upgrade.

 -- Christoph Berg <email address hidden> Mon, 28 Oct 2019 15:04:12 +0100

postgresql-common (207) unstable; urgency=medium

  * PgCommon.pm: Fix infinite recursion in get_program_path if
    /usr/lib/postgresql/ contains a non-version directory.
    Patch by ITANI Eiichiro, thanks! (Closes: #940220)

 -- Christoph Berg <email address hidden> Fri, 20 Sep 2019 14:24:23 +0200

postgresql-common (206) unstable; urgency=medium

  * pg_ctlcluster, pg_upgradecluster: Always use latest psql.
  * t/135_pg_virtualenv.t: Run pg_virtualenv on all versions.
  * gitlab/gitlab-ci.yml: Include salsa-ci pipeline instead of our own.

 -- Christoph Berg <email address hidden> Thu, 12 Sep 2019 15:49:10 +0200

postgresql-common (205) unstable; urgency=medium

  * Disable plpython2 packages by default. (Closes: #937310)
  * testsuite: Prefer PgCommon.pm from current directory.
  * pgcommon.sh: Remove obsolete Ubuntu code in locale_gen function.
  * Drop lsb-release dependencies, we prefer /etc/os-release anyway.
  * apt.postgresql.org.sh: Parse VERSION_CODENAME in /etc/os-release.

 -- Christoph Berg <email address hidden> Wed, 04 Sep 2019 08:38:08 +0200

[Christian Ehrhardt  (paelzer)]

Synced postgresql-common after Debian got the change for 20.04 that we need.
Lets give this a night to trigger tests and then we might need to re-trigger all kind of hanging proposed migrations with custom triggers to pass.

[Christian Ehrhardt  (paelzer)]

While more of the postgresql migration goes on the former bug is resolved.
Just to hit another FTBFS which effectively means it still fails to build.
One solved, more to go ...

The new error is about:
1529 E asyncpg.cluster.ClusterError: pg_ctl stop exited with status 1: pg_ctl: PID file "/tmp/tmp5jss46lb/postmaster.pid" does not exist
1530 E Is server running?

and
E asyncpg.exceptions._base.InternalClientError: cannot switch to state 11; another operation (19) is in progress

and most of the time:
E AssertionError: unexpected exceptions in asynchronous code:
E 09. An open stream object is being garbage collected; call "stream.close()" explicitly.:
ERROR asyncio:base_events.py:1692 An open stream object is being garbage collected; call "stream.close()" explicitly.

And more ...

Maybe a python 3.8 issue in regard to those tests?

[Christian Ehrhardt  (paelzer)]

It might be incompatible with postgresql-12 python bindings.
There is a asyncpg (upstream) v0.19.0 which includes among other things:
 * Add PostgreSQL 12 support
   (by @elprans in 23261532)

We might either need go to full 0.19.0 [1] or backport the fix [2] it it works as-is.

[1]: https://github.com/MagicStack/asyncpg/releases/tag/v0.19.0
[2]: https://github.com/MagicStack/asyncpg/commit/23261532ad0f2548aef2ec539c7c0162da761b1a

[Christian Ehrhardt  (paelzer)]

It builds further now and might be ok, but it has other build breakage.
I might need someone with more python-foo for this to help me.

I have pushed the interim state to [1]
Breaking buildlog at [2]

[1]: https://salsa.debian.org/paelzer-guest/asyncpg/tree/package-19.0
[2]: http://paste.ubuntu.com/p/VRntTFgrs7/

[Christian Ehrhardt  (paelzer)]

FYI I filed https://github.com/MagicStack/asyncpg/issues/501

[Christian Ehrhardt  (paelzer)]

Yeah, as assumed it seems to by py3.8, here from upstream on my issue:
"Python 3.8 is not supported yet."

I wondered if we should remove it for now, but there are more packages to this.

Recommends: silver-platter -> lintian-brush -> python3-asyncpg

Surely we could reduce this to a suggest and then remove asyncpg, but we could as well wait a bit until upstream looked into this and then sync in the new version from Debian or upstream into Ubuntu.

I need to talk to doko who pinged me initially what is blocked by it and maybe should open a Debian bug as FYI over there.

[Christian Ehrhardt  (paelzer)]

Oh doko was faster and reported the initial (test fail) issue already. I'll chime in there.
=> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=943832

[Christian Ehrhardt  (paelzer)]

Updated the Debian bug and opened Salsa MRs with the work so far in [1] and its siblings.

[1]: https://salsa.debian.org/python-team/modules/asyncpg/merge_requests/2

[Christian Ehrhardt  (paelzer)]

Fixes got merged in https://github.com/MagicStack/asyncpg/pull/504
And upstream tagged 20.0 https://github.com/MagicStack/asyncpg/releases/tag/v0.20.0

TODO: test if packaging that makes it work

[Christian Ehrhardt  (paelzer)]

I did work on 0.20 a bit, but it still fails to build

It can be found at: https://salsa.debian.org/paelzer-guest/asyncpg/commits/debian-update-from-upstream

But I'm afraid we might need to properly split and package pgproto into its own package and then (re-)build 0.20.0 again.

But that is probably too much python-depth for me atm to do it as drive-by solution.
Maybe someone picks it up in Debian I need to ping there again ...

[Christian Ehrhardt  (paelzer)]

Resolved with 0.20.0-1