sbuild focal python3 import errors

[Chad Smith]

[Impact]
The command fails to run due to an undeclared library dependency (python-ubuntutools)
While this dependency used to be satisfied transitively through a indirect dependency on ubuntu-dev-tools, ubuntu-dev-tools has since dropped this dependency due to a migration to python3.

This is currently only an issue in bionic because in newer releases s-l-c has been ported to python3, so ubuntu-dev-tools happens to transitively provide the necessary dependency. I therefore consider this a low-impact SRU for focal and groovy.

[Test Case]
$ sudo sbuild-launchpad-chroot -h
Traceback (most recent call last):
  File "/usr/bin/sbuild-launchpad-chroot", line 30, in <module>
    from ubuntutools.misc import host_architecture
ImportError: No module named ubuntutools.misc
ubuntu@dannf-bionic:~$

[Regression Risk]
Here we are just adding a new/obvious package dependency. The worst regression I can think of is that pulling in the new dependency could result in an out-of-disk scenario. This would almost certainly imply the user was not actually using this package, as any usage of it would certainly require more disk space then the new dependency (Installed-Size: 57.3 kB).

Note that the ubuntu-dev-tools dependency appears to no longer be necessary with a direct dependency on python[3]-ubuntutools. While I dropped it for the hirsute upload, I left it there for the SRUs, as I could imagine some scenarios in which that could break automation that assumes ubuntu-dev-tools will get installed as a dependency of this package.

[Paride Legovini (paride)]

Hi,

I think this is now fixed in Focal, but still an issue on Bionic.

sbuild-launchpad-chroot doesn't declare an explicit dependency on python-ubuntutools (or python3-ubuntutools): it depends on ubuntu-dev-tools and relies on it to pull in the ubuntutools dependency.

However ubuntu-dev-tools from bionic-updates switched to python3, and stopped pulling it python-ubuntutools, leaving sbuild-launchpad-chroot broken.

I think sbuild-launchpad-chroot should declare an explicit dependency on python/python3-ubuntutools across all the releases.

Paride

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[dann frazier (dannf)]

While the symptom has gone away in focal, the underlying issue that @paride describes is technically still there, so reopening.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sbuild-launchpad-chroot - 0.18

---------------
sbuild-launchpad-chroot (0.18) hirsute; urgency=medium

  * (Build-)Depend directly on python3-ubuntutools instead of indirectly
    on ubuntu-dev-tools. (LP: #1852677)
  * debian/copyright: Replace http URLs with https equivalents.
  * debian/control: Use https for Homepage URL.
  * Bump Standards-Version to 4.5.0.
  * Bump debhelper compatibility level to 13.

 -- dann frazier <email address hidden> Fri, 06 Nov 2020 14:34:26 -0700

[Brian Murray (brian-murray)]

Hello Chad, or anyone else affected,

Accepted sbuild-launchpad-chroot into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.17ubuntu0.20.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Chad, or anyone else affected,

Accepted sbuild-launchpad-chroot into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.16ubuntu0.20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Chad, or anyone else affected,

Accepted sbuild-launchpad-chroot into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.14ubuntu0.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[dann frazier (dannf)]

The bionic-proposed build has failed due to a regression in ubuntu-dev-tools (bug 1892879).

[dann frazier (dannf)]

= groovy verification =
(EPERMs are due to running under lxd)
root@groovy:~# sbuild-launchpad-chroot create -n $(lsb_release -cs) -s $(lsb_release -cs) -a $(dpkg --print-architecture)
[groovy] Creating config
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binfmt-support
The following NEW packages will be installed:
  binfmt-support qemu-user-static
0 upgraded, 2 newly installed, 0 to remove and 12 not upgraded.
Need to get 18.5 MB of archives.
After this operation, 197 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://ports.ubuntu.com/ubuntu-ports groovy/universe arm64 binfmt-support arm64 2.2.1-1 [50.9 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports groovy-updates/universe arm64 qemu-user-static arm64 1:5.0-5ubuntu9.1 [18.4 MB]
Fetched 18.5 MB in 1s (13.9 MB/s)
Selecting previously unselected package binfmt-support.
(Reading database ... 49601 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.2.1-1_arm64.deb ...
Unpacking binfmt-support (2.2.1-1) ...
Selecting previously unselected package qemu-user-static.
Preparing to unpack .../qemu-user-static_1%3a5.0-5ubuntu9.1_arm64.deb ...
Unpacking qemu-user-static (1:5.0-5ubuntu9.1) ...
Setting up qemu-user-static (1:5.0-5ubuntu9.1) ...
Setting up binfmt-support (2.2.1-1) ...
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/register for writing: Permission denied
update-binfmts: warning: unable to enable binary format python3.8
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-alpha already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-armeb already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-cris already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/register for writing: Permission denied
update-binfmts: warning: unable to enable binary format qemu-hppa
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-i386 already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-m68k already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-microblaze already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-mips already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc...

[dann frazier (dannf)]

= focal verification =
root@focal:~# sbuild-launchpad-chroot create -n $(lsb_release -cs) -s $(lsb_release -cs) -a $(dpkg --print-architecture)
[focal] Creating config
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binfmt-support
The following NEW packages will be installed:
  binfmt-support qemu-user-static
0 upgraded, 2 newly installed, 0 to remove and 17 not upgraded.
Need to get 17.2 MB of archives.
After this operation, 191 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ports.ubuntu.com/ubuntu-ports focal/universe arm64 binfmt-support arm64 2.2.0-2 [55.5 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates/universe arm64 qemu-user-static arm64 1:4.2-3ubuntu6.9 [17.2 MB]
Fetched 17.2 MB in 1s (12.7 MB/s)
Selecting previously unselected package binfmt-support.
(Reading database ... 48951 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.2.0-2_arm64.deb ...
Unpacking binfmt-support (2.2.0-2) ...
Selecting previously unselected package qemu-user-static.
Preparing to unpack .../qemu-user-static_1%3a4.2-3ubuntu6.9_arm64.deb ...
Unpacking qemu-user-static (1:4.2-3ubuntu6.9) ...
Setting up qemu-user-static (1:4.2-3ubuntu6.9) ...
Setting up binfmt-support (2.2.0-2) ...
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/register for writing: Permission denied
update-binfmts: warning: unable to enable binary format python3.8
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-alpha already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-armeb already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-cris already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/register for writing: Permission denied
update-binfmts: warning: unable to enable binary format qemu-hppa
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-i386 already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-m68k already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-microblaze already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
update-binfmts: warning: qemu-mips already enabled in kernel.
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/status for writing: Permission denied
upd...

[Brian Murray (brian-murray)]

Hello Chad, or anyone else affected,

Accepted sbuild-launchpad-chroot into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.17ubuntu0.20.10.1~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Chad, or anyone else affected,

Accepted sbuild-launchpad-chroot into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.17ubuntu0.20.10.1~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[dann frazier (dannf)]

Verification for groovy/focal/bionic below. TLDR; the versions in -proposed all pass the test case.

$ for release in groovy focal bionic; do lxc launch ubuntu:${release} dannf-slc-${release}; sleep 5; lxc exec dannf-slc-${release} -- apt-add-repository ppa:dannf/dannf -y; lxc exec dannf-slc-${release} -- apt install -y dannf; lxc exec dannf-slc-${release} -- dannf-enable-proposed.sh; lxc exec dannf-slc-${release} -- apt install -y sbuild-launchpad-chroot; lxc exec dannf-slc-${release} -- sbuild-launchpad-chroot -h; lxc stop dannf-slc-${release}; lxc delete dannf-slc-${release}; done
Creating dannf-slc-groovy
Starting dannf-slc-groovy
Repository: 'deb http://ppa.launchpad.net/dannf/dannf/ubuntu/ groovy main'
More info: https://launchpad.net/~dannf/+archive/ubuntu/dannf
Adding repository.
Adding deb entry to /etc/apt/sources.list.d/dannf-ubuntu-dannf-groovy.list
Adding disabled deb-src entry to /etc/apt/sources.list.d/dannf-ubuntu-dannf-groovy.list
Adding key to /etc/apt/trusted.gpg.d/dannf-ubuntu-dannf.gpg with fingerprint 1C4582E5CDE936B785D7C969D255A4056A8519FA
Get:1 http://security.ubuntu.com/ubuntu groovy-security InRelease [110 kB]
Get:2 http://ppa.launchpad.net/dannf/dannf/ubuntu groovy InRelease [17.5 kB]
Hit:3 http://archive.ubuntu.com/ubuntu groovy InRelease
Get:4 http://security.ubuntu.com/ubuntu groovy-security/main amd64 Packages [166 kB]
Get:5 http://archive.ubuntu.com/ubuntu groovy-updates InRelease [110 kB]
Get:6 http://security.ubuntu.com/ubuntu groovy-security/main amd64 c-n-f Metadata [2724 B]
Get:7 http://security.ubuntu.com/ubuntu groovy-security/restricted amd64 Packages [64.6 kB]
Get:8 http://security.ubuntu.com/ubuntu groovy-security/restricted Translation-en [10.1 kB]
Get:9 http://security.ubuntu.com/ubuntu groovy-security/restricted amd64 c-n-f Metadata [380 B]
Get:10 http://security.ubuntu.com/ubuntu groovy-security/universe amd64 Packages [44.2 kB]
Get:11 http://security.ubuntu.com/ubuntu groovy-security/universe Translation-en [19.2 kB]
Get:12 http://security.ubuntu.com/ubuntu groovy-security/universe amd64 c-n-f Metadata [2092 B]
Get:13 http://security.ubuntu.com/ubuntu groovy-security/multiverse amd64 Packages [1272 B]
Get:14 http://security.ubuntu.com/ubuntu groovy-security/multiverse Translation-en [352 B]
Get:15 http://security.ubuntu.com/ubuntu groovy-security/multiverse amd64 c-n-f Metadata [116 B]
Get:16 http://archive.ubuntu.com/ubuntu groovy-backports InRelease [101 kB]
Get:17 http://ppa.launchpad.net/dannf/dannf/ubuntu groovy/main amd64 Packages [444 B]
Get:18 http://archive.ubuntu.com/ubuntu groovy/universe amd64 Packages [12.6 MB]
Get:19 http://ppa.launchpad.net/dannf/dannf/ubuntu groovy/main Translation-en [168 B]
Get:20 http://archive.ubuntu.com/ubuntu groovy/universe Translation-en [5262 kB]
Get:21 http://archive.ubuntu.com/ubuntu groovy/universe amd64 c-n-f Metadata [270 kB]
Get:22 http://archive.ubuntu.com/ubuntu groovy/multiverse amd64 Packages [202 kB]
Get:23 http://archive.ubuntu.com/ubuntu groovy/multiverse Translation-en [106 kB]
Get:24 http://archive.ubuntu.com/ubuntu groovy/multiverse amd64 c-n-f Metadata [9260 B]
Get:25 http://archive.ubuntu.com/ubuntu groovy-upd...

[Chris Halse Rogers (raof)]

Why is this marked “blocked-proposed-groovy”? It looks like this is ready to release for groovy, focal & bionic.

[dann frazier (dannf)]

@raof The blocked-proposed-* tags were originally added because I was considering them low-impact SRUs for focal & groovy. It fixes no issues for them, but they required version bumps to remain upgradeable from the bionic SRU version. Perhaps that's enough reason to push them out at the same time as bionic? I don't have any concerns with releasing them.

[Łukasz Zemczak (sil2100)]

Let's maybe release this in this case. Thanks for clearing it up!

[Launchpad Janitor (janitor)]

This bug was fixed in the package sbuild-launchpad-chroot - 0.17ubuntu0.20.10.1

---------------
sbuild-launchpad-chroot (0.17ubuntu0.20.10.1) groovy; urgency=medium

  * Add direct (Build-)Dependency on python3-ubuntutools instead of
    relying on the indirect dependency via ubuntu-dev-tools.
    (LP: #1852677)

 -- dann frazier <email address hidden> Fri, 06 Nov 2020 14:44:18 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for sbuild-launchpad-chroot has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sbuild-launchpad-chroot - 0.17ubuntu0.20.10.1~20.04.1

---------------
sbuild-launchpad-chroot (0.17ubuntu0.20.10.1~20.04.1) focal; urgency=medium

  * Backport to focal. (LP: #1905634)

sbuild-launchpad-chroot (0.17ubuntu0.20.10.1) groovy; urgency=medium

  * Add direct (Build-)Dependency on python3-ubuntutools instead of
    relying on the indirect dependency via ubuntu-dev-tools.
    (LP: #1852677)

 -- dann frazier <email address hidden> Wed, 25 Nov 2020 17:06:49 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package sbuild-launchpad-chroot - 0.17ubuntu0.20.10.1~18.04.1

---------------
sbuild-launchpad-chroot (0.17ubuntu0.20.10.1~18.04.1) bionic; urgency=medium

  * Backport to bionic. (LP: #1905634)

sbuild-launchpad-chroot (0.17ubuntu0.20.10.1) groovy; urgency=medium

  * Add direct (Build-)Dependency on python3-ubuntutools instead of
    relying on the indirect dependency via ubuntu-dev-tools.
    (LP: #1852677)

 -- dann frazier <email address hidden> Wed, 25 Nov 2020 17:07:26 -0700