Tests fail on armel/panda

[Martin Pitt]

All postgresql versions used to build fine on the older babbage buildds, and on Debian's armel builders. However, on Ubuntu's Panda buillds a lot of the tests fail:

parallel group (19 tests): comments polygon lseg timetz macaddr inet date reltime abstime tinterval interval circle point timestamptz box tstypes path time timestamp
     point ... FAILED (test process exited with exit code 2)
     lseg ... ok
     box ... FAILED (test process exited with exit code 2)
     path ... FAILED (test process exited with exit code 2)
     polygon ... ok
     circle ... FAILED (test process exited with exit code 2)
     date ... FAILED (test process exited with exit code 2)
     time ... FAILED (test process exited with exit code 2)
     timetz ... FAILED (test process exited with exit code 2)
     timestamp ... FAILED (test process exited with exit code 2)
     timestamptz ... FAILED (test process exited with exit code 2)
     interval ... FAILED (test process exited with exit code 2)
     abstime ... FAILED (test process exited with exit code 2)
     reltime ... FAILED (test process exited with exit code 2)
     tinterval ... FAILED (test process exited with exit code 2)
     inet ... FAILED (test process exited with exit code 2)
     macaddr ... FAILED (test process exited with exit code 2)
     tstypes ... FAILED (test process exited with exit code 2)
     comments ... ok
[...]

and eventually hang forever.

This sounds like our debian/patches/01-armel-tas.patch needs updating or __sync_lock_test_and_set() is broken on that board.

[Alexander Sack (asac)]

think would be cool if devplatform team could look into this as part of their porting effort. Seems it's a problem t have the tests fail with the new precise policy. pitti said that __sync_lock_test_and_set() seems to not work

[Martin Pitt (pitti)]

It's nothing new in precise. The lucid-security/lucid-updates builds on the panda buildds fail just as well. It also affects all versions of PostgreSQL (8.4, 9.0, 9.1 were tested).

I'm currently working on the porter box to confirm that it's the test-and-set function and will then isolate it to a small test case.

[Martin Pitt (pitti)]

I'm now fairly convinced that the problem is in the defined tas() in src/include/storage/s_lock.h. If I comment out the armel implementation and build with --disable-spinlocks, it works just fine (but will have a poor performance).

[Martin Pitt (pitti)]

Ah, I got it. http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html says that gcc would accept any data type of 1, 2, 4, or 8 bytes for __sync_lock_test_and_set(), but it turns out that "char" (as Alex originally committed for bug 497331) doesn't work on the Pandas. I switched it to int now and also used __sync_lock_release() to comply to the API, and it works fine now.

[Martin Pitt (pitti)]

Committed to bzr.

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-9.1 - 9.1.2-1bzr1

---------------
postgresql-9.1 (9.1.2-1bzr1) precise; urgency=low

  Upload current packaging bzr head to Precise to fix crashes and hangs on ARM
  panda boards.

  * 01-armel-tas.patch: Turn slock_t datatype into an int, and define
    S_UNLOCK() to call __sync_lock_release() instead of using the default
    implementation. This complies to the gcc built-in atomic operations
    specifiction more strictly and now also works on the Panda boards.
    (LP: #904828)
 -- Martin Pitt <email address hidden> Fri, 16 Dec 2011 09:06:13 +0100

[Martin Pitt (pitti)]

I forced above fix to build on a panda board, worked fine:

https://launchpad.net/ubuntu/+source/postgresql-9.1/9.1.2-1bzr1/+build/3014820

Uploading fix to oneiric now, so that the SRU can build.

[Martin Pitt (pitti)]

Hello Martin, or anyone else affected,

Accepted postgresql-9.1 into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

This package has the same patch applied and got built on a Babbage board. This demonstrates that the new patch does not regress on the original platform:

https://launchpad.net/ubuntu/+source/postgresql-9.1/9.1.2-0ubuntu0.11.10.1/+build/3014897

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-9.1 - 9.1.2-0ubuntu0.11.10.2

---------------
postgresql-9.1 (9.1.2-0ubuntu0.11.10.2) oneiric-proposed; urgency=low

  * Add 00git_inet_cidr_unpack.patch: Revert the behavior of inet/cidr
    functions to not unpack the arguments. This fixes the memory leak when
    sorting inet values. Patch taken from upstream git HEAD. Spotted during
    testing in LP #904631.

postgresql-9.1 (9.1.2-0ubuntu0.11.10.1) oneiric-proposed; urgency=low

  * 01-armel-tas.patch: Turn slock_t datatype into an int, and define
    S_UNLOCK() to call __sync_lock_release() instead of using the default
    implementation. This complies to the gcc built-in atomic operations
    specifiction more strictly and now also works on the Panda boards.
    (LP: #904828)

postgresql-9.1 (9.1.2-0ubuntu0.11.10) oneiric-proposed; urgency=low

  * New upstream release (LP: #904631):
    - Fix bugs in information_schema.referential_constraints view.
      This view was being insufficiently careful about matching the
      foreign-key constraint to the depended-on primary or unique key
      constraint. That could result in failure to show a foreign key
      constraint at all, or showing it multiple times, or claiming that
      it depends on a different constraint than the one it really does.
      Since the view definition is installed by initdb, merely upgrading
      will not fix the problem. If you need to fix this in an existing
      installation, you can (as a superuser) drop the information_schema
      schema then re-create it by sourcing
      "SHAREDIR/information_schema.sql". (Run pg_config --sharedir if
      you're uncertain where "SHAREDIR" is.) This must be repeated in
      each database to be fixed.
    - Make "contrib/citext"'s upgrade script fix collations of citext
      columns and indexes.
      Existing citext columns and indexes aren't correctly marked as
      being of a collatable data type during pg_upgrade from a pre-9.1
      server. That leads to operations on them failing with errors such
      as "could not determine which collation to use for string
      comparison". This change allows them to be fixed by the same script
      that upgrades the citext module into a proper 9.1 extension during
      CREATE EXTENSION citext FROM unpackaged.
      If you have a previously-upgraded database that is suffering from
      this problem, and you already ran the "CREATE EXTENSION" command,
      you can manually run (as superuser) the "UPDATE" commands found at
      the end of "SHAREDIR/extension/citext--unpackaged--1.0.sql". (Run
      pg_config --sharedir if you're uncertain where "SHAREDIR" is.)
    - Fix possible crash during "UPDATE" or "DELETE" that joins to the
      output of a scalar-returning function.
    - Fix incorrect replay of WAL records for GIN index updates.
    - Fix TOAST-related data corruption during CREATE TABLE dest AS
      SELECT - FROM src or INSERT INTO dest SELECT * FROM src.
    - Fix possible failures during hot standby startup.
    - Start hot standby faster when initial snapshot is incomplete.
    - Fix race condition during toast table access from stale syscache
      entries. The ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-8.4 - 8.4.10-0ubuntu0.10.04.1

---------------
postgresql-8.4 (8.4.10-0ubuntu0.10.04.1) lucid-proposed; urgency=low

  * Add 00git_inet_cidr_unpack.patch: Revert the behavior of inet/cidr
    functions to not unpack the arguments. This fixes the memory leak when
    sorting inet values. Patch taken from upstream git HEAD. Spotted during
    testing in LP #904631.
  * 01-armel-tas.patch: Turn slock_t datatype into an int, and define
    S_UNLOCK() to call __sync_lock_release() instead of using the default
    implementation. This complies to the gcc built-in atomic operations
    specifiction more strictly and now also works on the Panda boards.
    (LP: #904828)

postgresql-8.4 (8.4.10-0ubuntu0.10.04) lucid-proposed; urgency=low

  * New upstream release: (LP: #904631)
    - Fix bugs in information_schema.referential_constraints view.
      This view was being insufficiently careful about matching the
      foreign-key constraint to the depended-on primary or unique key
      constraint. That could result in failure to show a foreign key
      constraint at all, or showing it multiple times, or claiming that
      it depends on a different constraint than the one it really does.
      Since the view definition is installed by initdb, merely upgrading
      will not fix the problem. If you need to fix this in an existing
      installation, you can (as a superuser) drop the information_schema
      schema then re-create it by sourcing
      "SHAREDIR/information_schema.sql". (Run pg_config --sharedir if
      you're uncertain where "SHAREDIR" is.) This must be repeated in
      each database to be fixed.
    - Fix incorrect replay of WAL records for GIN index updates.
      This could result in transiently failing to find index entries
      after a crash, or on a hot-standby server. The problem would be
      repaired by the next "VACUUM" of the index, however.
    - Fix TOAST-related data corruption during CREATE TABLE dest AS
      SELECT - FROM src or INSERT INTO dest SELECT * FROM src.
      If a table has been modified by "ALTER TABLE ADD COLUMN", attempts
      to copy its data verbatim to another table could produce corrupt
      results in certain corner cases. The problem can only manifest in
      this precise form in 8.4 and later, but we patched earlier versions
      as well in case there are other code paths that could trigger the
      same bug.
    - Fix race condition during toast table access from stale syscache
      entries.
    - Track dependencies of functions on items used in parameter default
      expressions. Previously, a referenced object could be dropped without
      having dropped or modified the function, leading to misbehavior when the
      function was used. Note that merely installing this update will not fix
      the missing dependency entries; to do that, you'd need to "CREATE OR
      REPLACE" each such function afterwards. If you have functions whose
      defaults depend on non-built-in objects, doing so is recommended.
    - Allow inlining of set-returning SQL functions with multiple OUT
      parameters.
    - Make DatumGetInetP() unpack inet datums that ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-8.4 - 8.4.10-0ubuntu0.10.10.1

---------------
postgresql-8.4 (8.4.10-0ubuntu0.10.10.1) maverick-proposed; urgency=low

  * Add 00git_inet_cidr_unpack.patch: Revert the behavior of inet/cidr
    functions to not unpack the arguments. This fixes the memory leak when
    sorting inet values. Patch taken from upstream git HEAD. Spotted during
    testing in LP #904631.
  * 01-armel-tas.patch: Turn slock_t datatype into an int, and define
    S_UNLOCK() to call __sync_lock_release() instead of using the default
    implementation. This complies to the gcc built-in atomic operations
    specifiction more strictly and now also works on the Panda boards.
    (LP: #904828)

postgresql-8.4 (8.4.10-0ubuntu0.10.10) maverick-proposed; urgency=low

  * New upstream release (LP: #904631)
    - Fix bugs in information_schema.referential_constraints view.
      This view was being insufficiently careful about matching the
      foreign-key constraint to the depended-on primary or unique key
      constraint. That could result in failure to show a foreign key
      constraint at all, or showing it multiple times, or claiming that
      it depends on a different constraint than the one it really does.
      Since the view definition is installed by initdb, merely upgrading
      will not fix the problem. If you need to fix this in an existing
      installation, you can (as a superuser) drop the information_schema
      schema then re-create it by sourcing
      "SHAREDIR/information_schema.sql". (Run pg_config --sharedir if
      you're uncertain where "SHAREDIR" is.) This must be repeated in
      each database to be fixed.
    - Fix incorrect replay of WAL records for GIN index updates.
      This could result in transiently failing to find index entries
      after a crash, or on a hot-standby server. The problem would be
      repaired by the next "VACUUM" of the index, however.
    - Fix TOAST-related data corruption during CREATE TABLE dest AS
      SELECT - FROM src or INSERT INTO dest SELECT * FROM src.
      If a table has been modified by "ALTER TABLE ADD COLUMN", attempts
      to copy its data verbatim to another table could produce corrupt
      results in certain corner cases. The problem can only manifest in
      this precise form in 8.4 and later, but we patched earlier versions
      as well in case there are other code paths that could trigger the
      same bug.
    - Fix race condition during toast table access from stale syscache
      entries.
    - Track dependencies of functions on items used in parameter default
      expressions. Previously, a referenced object could be dropped without
      having dropped or modified the function, leading to misbehavior when the
      function was used. Note that merely installing this update will not fix
      the missing dependency entries; to do that, you'd need to "CREATE OR
      REPLACE" each such function afterwards. If you have functions whose
      defaults depend on non-built-in objects, doing so is recommended.
    - Allow inlining of set-returning SQL functions with multiple OUT
      parameters.
    - Make DatumGetInetP() unpack inet datums ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package postgresql-8.4 - 8.4.10-0ubuntu0.11.04.1

---------------
postgresql-8.4 (8.4.10-0ubuntu0.11.04.1) natty-proposed; urgency=low

  * Add 00git_inet_cidr_unpack.patch: Revert the behavior of inet/cidr
    functions to not unpack the arguments. This fixes the memory leak when
    sorting inet values. Patch taken from upstream git HEAD. Spotted during
    testing in LP #904631.
  * 01-armel-tas.patch: Turn slock_t datatype into an int, and define
    S_UNLOCK() to call __sync_lock_release() instead of using the default
    implementation. This complies to the gcc built-in atomic operations
    specifiction more strictly and now also works on the Panda boards.
    (LP: #904828)

postgresql-8.4 (8.4.10-0ubuntu0.11.04) natty-proposed; urgency=low

  * New upstream release (LP: #904631):
    - Fix bugs in information_schema.referential_constraints view.
      This view was being insufficiently careful about matching the
      foreign-key constraint to the depended-on primary or unique key
      constraint. That could result in failure to show a foreign key
      constraint at all, or showing it multiple times, or claiming that
      it depends on a different constraint than the one it really does.
      Since the view definition is installed by initdb, merely upgrading
      will not fix the problem. If you need to fix this in an existing
      installation, you can (as a superuser) drop the information_schema
      schema then re-create it by sourcing
      "SHAREDIR/information_schema.sql". (Run pg_config --sharedir if
      you're uncertain where "SHAREDIR" is.) This must be repeated in
      each database to be fixed.
    - Fix incorrect replay of WAL records for GIN index updates.
      This could result in transiently failing to find index entries
      after a crash, or on a hot-standby server. The problem would be
      repaired by the next "VACUUM" of the index, however.
    - Fix TOAST-related data corruption during CREATE TABLE dest AS
      SELECT - FROM src or INSERT INTO dest SELECT * FROM src.
      If a table has been modified by "ALTER TABLE ADD COLUMN", attempts
      to copy its data verbatim to another table could produce corrupt
      results in certain corner cases. The problem can only manifest in
      this precise form in 8.4 and later, but we patched earlier versions
      as well in case there are other code paths that could trigger the
      same bug.
    - Fix race condition during toast table access from stale syscache
      entries.
    - Track dependencies of functions on items used in parameter default
      expressions. Previously, a referenced object could be dropped without
      having dropped or modified the function, leading to misbehavior when the
      function was used. Note that merely installing this update will not fix
      the missing dependency entries; to do that, you'd need to "CREATE OR
      REPLACE" each such function afterwards. If you have functions whose
      defaults depend on non-built-in objects, doing so is recommended.
    - Allow inlining of set-returning SQL functions with multiple OUT
      parameters.
    - Make DatumGetInetP() unpack inet datums that ...