27.0 package build failures focal riscv64 architecture

[Chad Smith]

[Impact]

 * Local package build attempts on riscv64 platform as well as cloud image builds for riscv64 on Hirsute were affected by inability to resolve correct golang dependencies. Allowing debian/control and package Makefiles to ignore building and packaging ubuntu-advantage-tools' golang hook binary allows for package builds to succeed with minimal impact to usability and user-messaging.

[Test Plan]

1. Run sbuild --resolve-alternatives --dist=focal --arch=riscv64 to
confirm package build success and does not include golang dependencies or
/usr/lib/ubuntu-advantage/apt-esm-json-hook
dpkg-deb -c *riscv64.deb | grep hook # expect only apt-esm-hook

2. Upload to ppa:ua-client/staging which is set to build both powerpc and riscv64 platforms by default and ensure no package build failures.

3. Ensure go APT hook is still built and included in binary package as
/usr/lib/ubuntu-advantage/apt-esm-json-hook and adds golang depends
dpkg-deb -c *amd64.deb | grep hook # expect both apt-esm-hook & apt-esm-json-hook

4. Test functionality of go json hook on Focal amd64:

4a. get token from ubuntu.com/advantage
4b. ua attach <token_from_4a>
4c ua disable esm-infra
4d apt upgrade # check output for esm-infra package updates available.

[Where problems could occur]

* This is a low risk update as most problems will occur at package build time.

* Accidental exclusion of /usr/lib/ubuntu-advantage/apt-esm-json-hook only results in the absence of a single informative message during `apt upgrade` or `apt dist-upgrade` which would describe detailed source for where package updates are originating:

"1 standard security update, 1 esm-infra update and 2 esm-apps updates"

* Without a packaged json hook, the message will be absent and no tracebacks or non-zero exits from APT

 * [racb] Mutating the Build-Depends line could cause alternatives to be resolved differently on the buildds, resulting in toolchain changes that might result in different sets of build failures or different build output causing functional issues.

[Other Info]

[Original Description]
Unable to sbuild focal 27.0~20.04.1 package on riscv64 cpu arch

sbuild errors on build dependency errors w/ golang package:

....
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 sbuild-build-depends-main-dummy : Depends: golang but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
apt-get failed.

E: Package installation failed
Not removing build depends: cloned chroot in use
Keeping session: focal-riscv64-484fa0a7-506e-4b64-b7b8-9c5b454e8f77
E: Failed to explain bd-uninstallable

+------------------------------------------------------------------------------+
| Summary |
+------------------------------------------------------------------------------+

Build Architecture: riscv64
Build Type: binary
Build-Space: n/a
Build-Time: 0
Distribution: focal
Fail-Stage: explain-bd-uninstallable
Host Architecture: riscv64
Install-Time: 0
Job: ../out/ubuntu-advantage-tools_27.0~20.04.1.dsc
Machine Architecture: amd64
Package: ubuntu-advantage-tools
Package-Time: 0
Source-Version: 27.0~20.04.1
Space: n/a
Status: given-back
Version: 27.0~20.04.1

[Chad Smith (chad.smith)]

Currently queued for upload into impish, hirsute, groovy, focal and bionic.

impish: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402459

hirsute: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402465

groovy: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402466

focal: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402467

bionic:https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402469

[Christian Ehrhardt  (paelzer)]

Also Xenial:
https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402474

[Robie Basak (racb)]

Hello Chad, or anyone else affected,

Accepted ubuntu-advantage-tools into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.0.2~21.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello Chad, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.0.2~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~20.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chad Smith (chad.smith)]

Focal Groovy and Hirsute: (bionic n/a because not applicable build arch)

1. Running the following suceeds locally with sbuild emulating hirsute riscv64 builds
[attached sbuild-hirsute-riscv64.log]

2. all ppa builds succeeded for riscv64 Focal, Groovy, Hisute and Impishin ppa:ua-client/staging
https://launchpad.net/~ua-client/+archive/ubuntu/staging/+packages
Bionic is inapplicable here as riscv64 is not a supported build platform.

3. Ensure ast-esm-json-hook is packaged in amd64 and not in riscv64

csmith@downtown:/tmp$ grep apt-esm-hook ubuntu-advantage-tools_27.0.2~20.04.1_riscv64.build
csmith@downtown:/tmp$ grep apt-esm-hook ubuntu-advantage-tools_27.0.2~20.04.1_amd64.build
install -D -m 644 20apt-esm-hook.conf /<<PKGBUILDDIR>>/debian/ubuntu-advantage-tools/etc/apt/apt.conf.d/20apt-esm-hook.conf
install -D -m 755 hook /<<PKGBUILDDIR>>/debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-hook
 objcopy --only-keep-debug --compress-debug-sections debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-hook debian/.debhelper/ubuntu-advantage-tools/dbgsym-root/usr/lib/debug/.build-id/65/aca9731638f907818fb804060431f78eab4b12.debug
 strip --remove-section=.comment --remove-section=.note debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-hook
 objcopy --add-gnu-debuglink debian/.debhelper/ubuntu-advantage-tools/dbgsym-root/usr/lib/debug/.build-id/65/aca9731638f907818fb804060431f78eab4b12.debug debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-hook
 dpkg-shlibdeps -Tdebian/ubuntu-advantage-tools.substvars debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-hook debian/ubuntu-advantage-tools/usr/lib/ubuntu-advantage/apt-esm-json-hook
-rw-r--r-- root/root 602 2021-05-12 16:10 ./etc/apt/apt.conf.d/20apt-esm-hook.conf
-rwxr-xr-x root/root 43328 2021-05-12 16:10 ./usr/lib/ubuntu-advantage/apt-esm-hook

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~21.04.1) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~21.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chad Smith (chad.smith)]

$ TOKEN=<REDACTED> ./sru-1927795.sh
Creating test-1927795-bionic
Starting test-1927795-bionic
......................................................................
status: done
time: Wed, 12 May 2021 21:10:06 +0000
detail:
DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]
27.0.2~18.04.1
Enabling default service esm-apps
Updating package lists
UA Apps: ESM enabled
Enabling default service esm-infra
Updating package lists
UA Infra: ESM enabled
This machine is now attached to 'UA Applications - Essential (Physical)'

SERVICE ENTITLED STATUS DESCRIPTION
esm-apps yes enabled UA Apps: Extended Security Maintenance (ESM)
esm-infra yes enabled UA Infra: Extended Security Maintenance (ESM)
fips yes n/a NIST-certified FIPS modules
fips-updates yes n/a Uncertified security updates to FIPS modules
livepatch yes n/a Canonical Livepatch service

NOTICES
Operation in progress: ua attach

Enable services with: ua enable <service>

                Account:
           Subscription: UA Applications - Essential (Physical)
            Valid until: 3999-12-31 00:00:00
Technical support level: essential
/etc/apt/apt.conf.d/20apt-esm-hook.conf
/usr/lib/ubuntu-advantage/apt-esm-hook
/usr/lib/ubuntu-advantage/apt-esm-json-hook
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following packages will be upgraded:
  initramfs-tools initramfs-tools-bin initramfs-tools-core krb5-locales
  libgssapi-krb5-2 libk5crypto3 libkrb5-3 libkrb5support0 libparted2
  linux-base login openssh-client openssh-server openssh-sftp-server parted
  passwd python-apt-common python3-apt snapd uidmap
20 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 25.8 MB of archives.
After this operation, 5932 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
Creating test-1927795-focal
Starting test-1927795-focal
...................................................................................................
status: done
time: Wed, 12 May 2021 21:10:56 +0000
detail:
DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]
27.0.2~20.04.1
Enabling default service esm-apps
Updating package lists
UA Apps: ESM enabled
Enabling default service esm-infra
Updating package lists
UA Infra: ESM enabled
This machine is now attached to 'UA Applications - Essential (Physical)'

SERVICE ENTITLED STATUS DESCRIPTION
esm-apps yes enabled UA Apps: Extended Security Maintenance (ESM)
esm-infra yes enabled UA Infra: Extended Security Maintenance (ESM)
fips yes n/a NIST-certified FIPS modules
fips-updates yes n/a Uncertified security updates to FIPS modules
livepatch yes n/a Canonical Livepatch service

NOTICES
Operation in progress: ua attach

Enable services with: ua enable <service>

                Account:
           Subscription: UA Applications - Essential (P...

[Chad Smith (chad.smith)]

hirsute riscv64 sbuild log success

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~20.10.1) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~20.10.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chad Smith (chad.smith)]

As mentioned above in comment #7. Bionic is n/a for this changeset because riscv64 is not a valid CPU arch supported on that release. Since we sync ubuntu-advantage-tools to all applicable supported Series, this bug "fix" gets pulled in, but there is nothing to test as all viable cpu arches are built successfully https://launchpad.net/~ua-client/+archive/ubuntu/staging/+packages

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~21.04.1

---------------
ubuntu-advantage-tools (27.0.2~21.04.1) hirsute; urgency=medium

  * Backport to Hirsute

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~20.10.1

---------------
ubuntu-advantage-tools (27.0.2~20.10.1) groovy; urgency=medium

  * Backport to Groovy

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~20.04.1

---------------
ubuntu-advantage-tools (27.0.2~20.04.1) focal; urgency=medium

  * Backport to Focal

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~18.04.1

---------------
ubuntu-advantage-tools (27.0.2~18.04.1) bionic; urgency=medium

  * Backport to Bionic

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2

---------------
ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - ammend changelog: add ommitted apt-hook message for 27.0.1 stanza

 -- Chad Smith <email address hidden> Fri, 07 May 2021 11:58:03 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.1~16.04.1

---------------
ubuntu-advantage-tools (27.1~16.04.1) xenial; urgency=medium

  * Backport new upstream release: (LP: #1929597) to xenial

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 15:40:20 -0300