subiquity package-hook is outdated

[Chris Peterson]

[Impact]

The Subiquity package hook in Apport has not been updated to collect the type of information we are typically interested in (e.g. /var/log/installer/\*). This means users who run `ubuntu-bug subiquity` from the terminal will not automatically collect relevant information to attach to their bug reports. This upload fixes this by updating Subiquity's package hook to collect logs and information typically used for debugging Subiquity bugs. This fix needs to be SRUd to Noble and Jammy so that users performing these installs can run `ubuntu-bug subiquity` and collect relevant information.

[Test Plan]

The test plan is focused on testing the behavior of apport within the live installation environment, where we primarily expect users to submit bug reports from. Thus the test plan is split into three different test plans: Two for Desktop images and one for Server. For Noble, all three test scenarios are relevant. For Jammy, it is only necessary to test Server since the Desktop installer for Jammy is not Subiquity based.

Desktop - Manual:
1) Download the latest Noble Desktop ISO [1].
2) Begin an installation using the media you downloaded. You may progress the install as far as you like, with the following restrictions: (1) At minimum you need wait for the first screen to show up and (2) at most you may allow the install to finish but do not allow the system to restart (you need to be in the live-environment to proceed).
3) Open a terminal (ctl + alt + t)
4) Enable proposed[4] and update apport
5) Run the following command "APPORT_LAUNCHPAD_INSTANCE=qastaging ubuntu-bug subiquity"
6) 4 authorization prompts will display, requesting access to collect information from the system. Allow all of them.
7) Inspect the fields of the report and ensure that none report "permission denied" or similar.
8) Click "Send" on the resulting dialog to send the report. Firefox will open up to finish filling out the bug information on launchpad.
9) Log in and enter anything as the title and hit continue. Click "No, I need to report a new bug" if necessary
10) Enter anything in the "further information section". Ensure that the field below this section, titled "This bug contains information that is", is marked "Private" by default.
11) You can now submit the bug and inspect the automatically filled description for further errors in the report. (Errors messages are typically short enough to show up in the body of the bug description while larger attachments with real information end up as attachments)

Desktop - Automatic:
1) Download the latest Noble Desktop ISO [1].
2) Begin an installation using the media you downloaded. You may progress the install as far as you like, with the following restrictions: (1) At minimum you need wait for the first screen to show up and (2) at most you may allow the install to finish but do not allow the system to restart (you need to be in the live-environment to proceed).
3) Open a terminal (ctl + alt + t)
4) Enable proposed[4] and update apport
5) Create a fake subiquity crash file: sudo printf "Package: subiquity\nProblemType: crash\n" > /var/crash/test.crash
6) authorize the collection of information
7) Inspect the generated data. Ensure:
 - The key LaunchpadPrivate exists
 - There are no "permission denied" errors
- Do NOT submit the report

Server (Jammy or Noble):
1) Download the latest Server ISO for the release you have chosen to verify (Noble[2] or Jammy[3])
2) Begin an installation using the media you downloaded. You may progress the install as far as you like, with the following restrictions: (1) At minimum you need wait for the first screen to show up and (2) at most you may allow the install to finish but do not allow the system to restart (you need to be in the live-environment to proceed).
3) Enter the debug shell (F2 or ctrl+z)
4) Enable proposed[4] and update apport
5) Run the following command "sudo APPORT_LAUNCHPAD_INSTANCE=qastaging ubuntu-bug subiquity" (sudo is required)
6) Apport will ask for which matching application you want to report against. Select option 2, which should read "subiquity (1234 of snap /snap/subiquity/1234 of version $release_ver) deb package"
7) Now apport should ask how you want to proceed. Select "S: Send report"
8) Once the upload is complete, open the URL in a browser on another machine to complete the installation.
9) Log in and enter anything as the title and hit continue. Click "No, I need to report a new bug" if necessary
10) Enter anything in the "further information section". Ensure that the field below this section, titled "This bug contains information that is", is marked "Private" by default.
11) You can now submit the bug and inspect the automatically filled description for further errors in the report. (Errors messages are typically short enough to show up in the body of the bug description while larger attachments with real information end up as attachments)

[1] https://releases.ubuntu.com/noble/ubuntu-24.04-desktop-amd64.iso
[2] https://releases.ubuntu.com/noble/ubuntu-24.04-live-server-amd64.iso
[3] https://releases.ubuntu.com/jammy/ubuntu-22.04.4-live-server-amd64.iso
[4] https://wiki.ubuntu.com/Testing/EnableProposed

[Where Problems Could Occur]

This update only makes changes to the file: /usr/share/apport/package-hooks/subiquity.py
thus I only expect regressions in the form of Subiquity bug reporting failures. This code in this file is only ever executed as part of Subiquity's "package hook" and so should only regress behavior as part of data collection for subiquity bug reporting.

Potential regressions:
- The data collection for subiquity bugs could now crash, preventing users from submitting bugs manually via ubuntu-bug at all.
- For Desktop, subiquity relies on the Whoopsie service picking up the automatic crash report written to /var/crash/ to get users to submit bug reports. Even though the report is populated, Whoopsie still runs the apport package hook and updates the report with missing information. Thus it's possible crashes in the package hook will prevent reporting of automatically generated crash reports which otherwise already had sufficient information.

[Other Info]

- The fields starting with "lsusb" may report an error with exit code 1 when run on a QEMU VM. This is acceptable.

Potential questions:

- Q: Jammy and Noble have already released, how will users get this update?
  A: There is still one more point release for Jammy and the upcoming .1 release for Noble, wherein the ISO builds will pick up the newly SRU'd version. (It's possible a user could manually update apport, but I don't expect users to know or need to do this)

- Q: Why is sudo necessary on Server?
- A: Due to a bug in Subiquity's debug shell, it is required to run ubuntu-bug with sudo to pick up the correct hook (outside of the snap).

[ Original Description ]
The Subiquity package hook in Apport has not been updated to collect the type of information we are typically interested in (e.g. /var/log/installer/subiquity-*).

[Chris Peterson (cpete)]

In preparing the SRU uploads, I found that the pkexec permissions elevation can fail which caused the hook to crash. I will open a separate bug with the details, but I opened MP:#468177 to address the problem in the subiquity hook and included the relevant change in the SRU MPs.

Edit: I believe the "bug" was in the way I was adding a user to the machine with cloud-config. Switching to the pre-existing user on the machine allowed the permission elevation just fine. Although I still think the code change to handle this case is worthwhile.

[Benjamin Drung (bdrung)]

We should combine this SRU with the fixes for:
* bug #2067120 (apport-retrace fix retracing crashes on Ubuntu)
* bug #2066995 (apport: create /var/crash with setgid permission)
* bug #2069360 (apport-autoreport.service fails if autoreporting is disabled)

I'll plan to work on it next week.

[Timo Aaltonen (tjaalton)]

Hello Chris, or anyone else affected,

Accepted apport into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.28.1-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Chris, or anyone else affected,

Accepted apport into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu82.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apport (2.20.11-0ubuntu82.6) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-drivers-common/unknown (armhf)
ubuntu-release-upgrader/1:22.04.19 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#apport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apport (2.28.1-0ubuntu3.1) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.28.1-0ubuntu3.1 (arm64)
apport/unknown (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#apport

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.29.0-0ubuntu1

---------------
apport (2.29.0-0ubuntu1) oracular; urgency=medium

  [ Benjamin Drung ]
  * New upstream release:
    - Ignore container crash reports from systemd-coredump (LP: #2063349)
    - apport: do not modify permission of existing /var/crash or
      create /var/crash with setgid permission (mode 3777) (LP: #2066995)
    - Fix retracing crashes on Ubuntu 24.04 (noble) (LP: #2067120)
  * ubuntu: fix possibly using dpkg_log_without_error before assignment
  * Drop python3-requests dependency
  * autopkgtest: add chaos-marmosets for system-tests
  * apport.postinst: Create /var/crash with mode 3777 (LP: #2066995)
  * Build-depend on mypy and python3-typeshed for type checking
  * Build-depend on systemd-dev to determine systemd/udev paths

  [ Chris Peterson ]
  * d/package-hooks/subiquity.py: update information collection (LP: #2067775)
    - Add information about snap update/refresh state
    - Attach logs and change key names to align with the information attached
      by subiquity's self-collection
    - Attach hardware information
    - Ensure bug reports are always private since there may be private
      information in the logs
    - Don't require sudo for logs collection
    - Add journal logs
    - Ensure snap revision is included in both deb and snap collection style

 -- Benjamin Drung <email address hidden> Fri, 14 Jun 2024 10:18:31 +0200

[Olivier Gayot (ogayot)]

While investigating another issue (see bug 2076233), I stumbled upon this change. The subiquity.py hook (currently in jammy-proposed / 2.20.11-0ubuntu82.6) calls report.add_tags but Report.add_tags() seems to only be available in more recent versions of apport.

[Chris Peterson (cpete)]

Noble looks good:

Desktop manual reporting: https://bugs.qastaging.launchpad.net/subiquity/+bug/2043879

Server manual reporting: https://bugs.qastaging.launchpad.net/subiquity/+bug/2043881

Desktop: automatic reporting worked as expected

I'm not immediately concerned with the `report.add_tags` call introduced in the Subiquity hook on jammy since: (1) On jammy, our hook is never called since it only executes on Desktop where subiquity isn't the backend for Jammy Desktop; (2) The error is in a pre-existing code path. It's unclear to me how exactly we we run into that error today, but at least the changes here couldn't introduce that regression.

[Chris Peterson (cpete)]

Jammy also looks good. I installed the latest version of apport and was able to run the updated hook. I was unable to connect to qastaging, so here's the bug I made on the production tracker:

https://bugs.launchpad.net/subiquity/+bug/2076371

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.20.11-0ubuntu82.6

---------------
apport (2.20.11-0ubuntu82.6) jammy; urgency=medium

  [ Chris Peterson ]
  * d/package-hooks/subiquity.py: update information collection (LP: #2067775)
    - Collect the ubuntu-desktop-bootstrap details if available
    - Add information about snap update/refresh state
    - Attach logs and change key names to align with the information
      attached by subiquity's self-collection
    - Attach hardware information
    - Ensure bug reports are always private since there may be private
      information in the logs
    - Don't require sudo for logs collection
    - Add journal logs
    - Ensure snap revision is included in both deb and snap collection
      style
    - Handle version information when permission elevation fails.

  [ Benjamin Drung ]
  * Switch source format from 1.0 native to 3.0 (native)
    for betting handling in git

 -- Chris Peterson <email address hidden> Wed, 10 Jul 2024 16:48:53 +0200

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for apport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Andreas Hasenack (ahasenack)]

I released noble before, but I don't see a comment here. Maybe LP timed out (it did for another release I did earlier). I'll wait a bit, and if nothing shows up here, I'll just release noble again.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.28.1-0ubuntu3.1

---------------
apport (2.28.1-0ubuntu3.1) noble; urgency=medium

  [ Chris Peterson ]
  * d/package-hooks/subiquity.py: update information collection (LP: #2067775)
    - Add information about snap update/refresh state
    - Attach logs and change key names to align with the information
      attached by subiquity's self-collection
    - Attach hardware information
    - Ensure bug reports are always private since there may be private
      information in the logs
    - Don't require sudo for logs collection
    - Add journal logs
    - Ensure snap revision is included in both deb and snap collection
      style
    - Handle version information when permission elevation fails.

  [ Benjamin Drung ]
  * apport: do not modify permission of existing /var/crash or
    create /var/crash with setgid permission (mode 3777) (LP: #2066995)
  * apport.postinst: Create /var/crash with mode 3777 (LP: #2066995)
  * whoopsie-upload-all: exit with 0 if whoopsie is disabled (LP: #2069360)
  * Fix retracing crashes on Ubuntu 24.04 (noble) and add chaos-marmosets as
    dependency for the newly added system-tests (LP: #2067120)

  [ Sebastien Bacher ]
  * Fix some issues in the hook for ubuntu-desktop-bootstrap (LP: #2072751)

 -- Benjamin Drung <email address hidden> Mon, 22 Jul 2024 16:59:07 +0200

[Andreas Hasenack (ahasenack)]

Ah, I forgot the --override-freeze flag