aptdaemon frontend locking, locking fixes

[Julian Andres Klode]

[Impact]
aptdaemon needs to implement frontend locking to avoid it hijacking dpkg status lock from clients that do implement frontend locking; and in order to be protected from apt clients that acquire the status lock (which they won't do if aptdaemon holds the frontend lock).

This also change the locking order to lock in APT order, and the unlock order becomes the reverse of that so that the behavior is sane wrt other clients, as it was somewhat broken.

[Test case]
Lock /var/lib/dpkg/lock-frontend and then try do so something with aptdaemon, it should not be possible until you release lock-frontend.

[Regression potential]
There may be other race conditions appearing now due to the order of (un)locking changing.

[Launchpad Janitor (janitor)]

This bug was fixed in the package aptdaemon - 1.1.1+bzr982-0ubuntu23

---------------
aptdaemon (1.1.1+bzr982-0ubuntu23) eoan; urgency=medium

  * Various locking fixes (LP: #1831981)
    - Implement frontend locking
    - Adjust locking order to match APT
    - Use reverse order for unlocking

 -- Julian Andres Klode <email address hidden> Fri, 07 Jun 2019 12:04:55 +0200

[Brian Murray (brian-murray)]

Hello Julian, or anyone else affected,

Accepted aptdaemon into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/aptdaemon/1.1.1+bzr982-0ubuntu21.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Julian, or anyone else affected,

Accepted aptdaemon into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/aptdaemon/1.1.1+bzr982-0ubuntu20.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Julian, or anyone else affected,

Accepted aptdaemon into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/aptdaemon/1.1.1+bzr982-0ubuntu19.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Julian, or anyone else affected,

Accepted aptdaemon into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/aptdaemon/1.1.1+bzr982-0ubuntu14.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

All autopkgtests for the newly accepted aptdaemon (1.1.1+bzr982-0ubuntu14.1) for xenial have finished running.
There have been regressions in tests triggered by the package. Please visit the sru report page and investigate the failures.

https://people.canonical.com/~ubuntu-archive/pending-sru.html#xenial

[Julian Andres Klode (juliank)]

xenial verified.

1. Installed aptdaemon 1.1.1+bzr982-0ubuntu14.1)
2. Ran python3 -c "import apt_pkg, time; apt_pkg.FileLock('/var/lib/dpkg/lock-frontend').__enter__(); time.sleep(1000)"
3. Ran aptdcon --install xterm in other window

Result:
1. aptdcon waited while the Python 3 process was running
2. After exiting the process, aptdcon executed dpkg

[Julian Andres Klode (juliank)]

Same procedure on bionic, with 1.1.1+bzr982-0ubuntu19.1 - same result :)

[Julian Andres Klode (juliank)]

disco - 1.1.1+bzr982-0ubuntu21.1 - was fine too

Skipping cosmic, as releasing this 9 days before EOL is pointless if not potentially harmful.

[Julian Andres Klode (juliank)]

Regarding the regression report on aptdaemon/armhf/xenial: aptdaemon test suite hangs on arm* across all releases, but that's not new (it hang occasionally before, or it just failed earlier, or in xenial's case had the wrong depends and never ran).

It's working on all other releases, so it's unlikely to be caused by the locking fix; and running the test suite and failing due to hanging on one arch and passing on the others is better than failing everywhere :)

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for aptdaemon has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package aptdaemon - 1.1.1+bzr982-0ubuntu19.1

---------------
aptdaemon (1.1.1+bzr982-0ubuntu19.1) bionic; urgency=medium

  * Various locking fixes (LP: #1831981)
    - Implement frontend locking (incl. fixup from 0ubuntu24)
    - Adjust locking order to match APT
    - Use reverse order for unlocking
  * Test suite fixes:
    - Fix failing tests and disable ones we do not care about anymore (PK/PEP8)
    - Make failing tests fail the build again

 -- Julian Andres Klode <email address hidden> Fri, 07 Jun 2019 12:42:20 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package aptdaemon - 1.1.1+bzr982-0ubuntu21.1

---------------
aptdaemon (1.1.1+bzr982-0ubuntu21.1) disco; urgency=medium

  * Various locking fixes (LP: #1831981)
    - Implement frontend locking (incl. fixup from 0ubuntu24)
    - Adjust locking order to match APT
    - Use reverse order for unlocking
  * Test suite fixes:
    - Fix failing tests and disable ones we do not care about anymore (PK/PEP8)
    - Make failing tests fail the build again

 -- Julian Andres Klode <email address hidden> Fri, 07 Jun 2019 12:42:20 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package aptdaemon - 1.1.1+bzr982-0ubuntu14.1

---------------
aptdaemon (1.1.1+bzr982-0ubuntu14.1) xenial; urgency=medium

  * Various locking fixes (LP: #1831981)
    - Implement frontend locking (incl. fixup from 0ubuntu24)
    - Adjust locking order to match APT
    - Use reverse order for unlocking
  * Test suite fixes:
    - Fix failing tests
    - Make failing tests fail the build again
    - debian/tests/control: Depend on gir1.2-vte-2.91, not gir1.2-vte-2.90

 -- Julian Andres Klode <email address hidden> Wed, 03 Jul 2019 12:39:38 +0200

[Steve Langasek (vorlon)]

The version of aptdaemon in the proposed pocket of Cosmic that was purported to fix this bug report has been removed because one or more bugs that were to be fixed by the upload have failed verification and been in this state for more than 10 days.