xenial powerpc build failures golang dependency issues

[Chad Smith]

[Impact]

 * Local package build attempts on powerpc platform as well as cloud image builds for powerpc on Xenial were affected by inability to resolve correct golang dependencies. Allowing debian/control and package Makefiles to ignore building and packaging ubuntu-advantage-tools' golang hook binary allows for package builds to succeed with minimal impact to usability and user-messaging.

[Test Plan]

1. Run sbuild --resolve-alternatives --dist=xenial --arch=powerpc to confirm package build success and does not include golang dependencies or
/usr/lib/ubuntu-advantage/apt-esm-json-hook
dpkg-deb -c *powerpc.deb | grep hook # expect only apt-esm-hook

2. Upload to ppa:ua-client/staging which is set to build both powerpc and riscv64 platforms by default and ensure no package build failures.

3. Ensure go APT hook is still built and included in binary package as
/usr/lib/ubuntu-advantage/apt-esm-json-hook
dpkg-deb -c *amd64.deb | grep hook # expect both apt-esm-hook & apt-esm-json-hook

4. Test functionality of go json hook on Xenial:

4a. get token from ubuntu.com/advantage
4b. ua attach <token_from_4a>
4c ua disable esm-infra
4d apt upgrade # check output for esm-infra package updates available.

[Where problems could occur]

* This is a low risk update as most problems will occur at package build time.

* Accidental exclusion of /usr/lib/ubuntu-advantage/apt-esm-json-hook only results in the absence of a single informative message during `apt upgrade` or `apt dist-upgrade` which would describe detailed source for where package updates are originating:

"1 standard security update, 1 esm-infra update and 2 esm-apps updates"

* Without a packaged json hook, the message will be absent and no tracebacks or non-zero exits from APT

 * [racb] Mutating the Build-Depends line could cause alternatives to be resolved differently on the buildds, resulting in toolchain changes that might result in different sets of build failures or different build output causing functional issues.

[Other Info]

[Original Description]

 * Anything else you think is useful to include
 * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
 * and address these questions in advance

ubuntu-advantage-tools 27.0~16.04.1 fails to build from source in PPAs or locally via sbuild on powerpc platform due to golang package dependency issues:

$ sbuild --resolve-alternatives --dist=xenial --arch=powerpc --arch-all ../out/ubuntu-advantage-tools_27.0~16.04.1.dsc
....

Merged Build-Depends: bash-completion, debhelper (>= 9), debianutils, dh-python, dh-systemd | debhelper (>= 13.3), gettext, git, golang, libapt-pkg-dev, po-debconf, python3 (>= 3.4), distro-info, python3-flake8, python3-mock, python3-pytest, python3-setuptools, python3-yaml, build-essential, fakeroot
Filtered Build-Depends: bash-completion, debhelper (>= 9), debianutils, dh-python, dh-systemd | debhelper (>= 13.3), gettext, git, golang, libapt-pkg-dev, po-debconf, python3 (>= 3.4), distro-info, python3-flake8, python3-mock, python3-pytest, python3-setuptools, python3-yaml, build-essential, fakeroot
dpkg-deb: building package 'sbuild-build-depends-main-dummy' in '/<<RESOLVERDIR>>/apt_archive/sbuild-build-depends-main-dummy.deb'.
Ign:1 copy:/<<RESOLVERDIR>>/apt_archive ./ InRelease
Get:2 copy:/<<RESOLVERDIR>>/apt_archive ./ Release [957 B]
Ign:3 copy:/<<RESOLVERDIR>>/apt_archive ./ Release.gpg
Get:4 copy:/<<RESOLVERDIR>>/apt_archive ./ Sources [481 B]
Get:5 copy:/<<RESOLVERDIR>>/apt_archive ./ Packages [565 B]
Fetched 2003 B in 0s (9192 B/s)
Reading package lists...
Reading package lists...

Install main build dependencies (apt-based resolver)
----------------------------------------------------

Installing build dependencies
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 sbuild-build-depends-main-dummy : Depends: golang but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
apt-get failed.
E: Package installation failed

...
   missing:
     pkg:
      package: golang:powerpc
      version: 2:1.6-1ubuntu4
      architecture: all
      essential: false
      unsat-dependency: golang-src:powerpc (>= 2:1.6-1ubuntu4)
     depchains:
      -
       depchain:
        -
         package: sbuild-build-depends-main-dummy:powerpc
         version: 0.invalid.0
         architecture: powerpc
         essential: false
         depends: golang:powerpc

[Christian Ehrhardt  (paelzer)]

FYI:
an update on the latest set of MPs
X: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402510
B: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402508
F: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402513
G: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402505
H: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402502

^^ all of the above are just changelog stanza updates to the impish MP

I: https://code.launchpad.net/~chad.smith/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/402459

[Robie Basak (racb)]

Hello Chad, or anyone else affected,

Accepted ubuntu-advantage-tools into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.0.2~21.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~20.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~21.04.1) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~21.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chad Smith (chad.smith)]

Bionic, Focal, Groovy and Hirsute do not have powerpc arch build support, so we need only to ensure that packages build and contain the apt-esm-json-hook. Successful package build indicates proper conditional build logic for this SRU to include/exclude both go build of apt-esm-json-hook and go test runs of that hook

1. Running the following suceedslocally with sbuild emulating powerpc on Xenial
$ sbuild --resolve-alternatives --dist=xenial --arch=powerpc
[ATTACHED sbuild-xenial-powerpc.log]

2. Upload to ppa:ua-client/staging succeeded with Xenial builds at
https://launchpad.net/~ua-client/+archive/ubuntu/staging/+packages

3 & 4 ensure esm-apt-json-hook is included in amd64 arch builds and doesn't break `apt upgrade`:

$ cat sru-1927795.sh
#!/bin/bash
# Only LTS releases have ESM package ability so ignore Groovy/Hirsute
# Exercise apt JSON hook messaging to make sure hook messages show up.

cat > proposed-bionic.yaml <<EOF
#cloud-config
write_files:
- content: ZGViIGh0dHA6Ly9hcmNoaXZlLnVidW50dS5jb20vdWJ1bnR1LyBiaW9uaWMtcHJvcG9zZWQgcmVzdHJpY3RlZCBtYWluIG11bHRpdmVyc2UgdW5pdmVyc2UK
  encoding: b64
  owner: root:root
  path: /etc/apt/sources.list.d/proposed-ua.list
  permissions: '0644'

package_update: true
packages: [ubuntu-advantage-tools]
EOF

cat > proposed-focal.yaml <<EOF
#cloud-config
write_files:
- content: ZGViIGh0dHA6Ly9hcmNoaXZlLnVidW50dS5jb20vdWJ1bnR1LyBmb2NhbC1wcm9wb3NlZCByZXN0cmljdGVkIG1haW4gbXVsdGl2ZXJzZSB1bml2ZXJzZQo=
  encoding: b64
  owner: root:root
  path: /etc/apt/sources.list.d/proposed-ua.list
  permissions: '0644'
package_update: true
packages: [ubuntu-advantage-tools]
EOF

for release in bionic focal; do
  vm=test-1927795-${release}
  lxc launch ubuntu-daily:${release} -c user.user-data="$(cat proposed-${release}.yaml)" ${vm}
  sleep 4
  lxc exec ${vm} -- cloud-init status --wait --long
  lxc exec ${vm} -- ua version
  lxc exec ${vm} -- ua attach ${TOKEN}
  lxc exec ${vm} -- dpkg -L ubuntu-advantage-tools | grep hook
  lxc exec ${vm} -- python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py
  lxc exec ${vm} -- apt upgrade # confirm no exit 1 failures due to broken apt JSON hook
done

$ TOKEN=<REDACTED> ./sru-1927795.sh
Creating test-1927795-bionic
Starting test-1927795-bionic
......................................................................
status: done
time: Wed, 12 May 2021 21:10:06 +0000
detail:
DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]
27.0.2~18.04.1
Enabling default service esm-apps
Updating package lists
UA Apps: ESM enabled
Enabling default service esm-infra
Updating package lists
UA Infra: ESM enabled
This machine is now attached to 'UA Applications - Essential (Physical)'

SERVICE ENTITLED STATUS DESCRIPTION
esm-apps yes enabled UA Apps: Extended Security Maintenance (ESM)
esm-infra yes enabled UA Infra: Extended Security Maintenance (ESM)
fips yes n/a NIST-certified FIPS modules
fips-updates yes n/a Uncertified security updates to FIPS modules
livepatch yes n/a Canonical Livepatch service

NOTICES
Operation in progress: ua attach

Enable services with: ua enable <service>

                Ac...

[Chad Smith (chad.smith)]

xenial sbuild success emulating powerpc with current branch:

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.0.2~20.10.1) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.0.2~20.10.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~21.04.1

---------------
ubuntu-advantage-tools (27.0.2~21.04.1) hirsute; urgency=medium

  * Backport to Hirsute

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~20.10.1

---------------
ubuntu-advantage-tools (27.0.2~20.10.1) groovy; urgency=medium

  * Backport to Groovy

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~20.04.1

---------------
ubuntu-advantage-tools (27.0.2~20.04.1) focal; urgency=medium

  * Backport to Focal

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2~18.04.1

---------------
ubuntu-advantage-tools (27.0.2~18.04.1) bionic; urgency=medium

  * Backport to Bionic

ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - amend changelog: add omitted apt-hook message for 27.0.1 stanza

ubuntu-advantage-tools (27.0.1) impish; urgency=medium

  * Add .gitignore and cleanup ignored directory .pytest_cache
  * apt-hook: mitigate failures with true

 -- Chad Smith <email address hidden> Wed, 12 May 2021 10:10:45 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.0.2

---------------
ubuntu-advantage-tools (27.0.2) impish; urgency=medium

  * d/control:
    - order build-depends alternatives newer first (LP: #1926949)
    - apt-hook: do not attempt to package go APT JSON hook on some
      architectures (GH: #1603) (LP: #1927886, LP: #1927795)
  * Bug-fix release 27.0.2: build failures on riscv64 and powerpc
    - apt-hook: refactor json hook messaging to be dry
    - tests: fix subp ls error case for powerpc builds
    - jenkinsfile: add --resolve-alternatives for trusty builds
    - ammend changelog: add ommitted apt-hook message for 27.0.1 stanza

 -- Chad Smith <email address hidden> Fri, 07 May 2021 11:58:03 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.1~16.04.1

---------------
ubuntu-advantage-tools (27.1~16.04.1) xenial; urgency=medium

  * Backport new upstream release: (LP: #1929597) to xenial

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 15:40:20 -0300