Disabling apt spam results in a conffile prompt on upgrade

[Dan Watkins]

[Impact]
Users who have disabled apt-news will get prompted regarding their uaclient configuration file, even if they didn't touch the file for anything else.

Given this command may be executed by people who are not even aware of the internals of the configuration and how it was implemented, this prompt is undesired.

[Test Case]
Run a ubuntu machine with ubuntu-advantage-tools > 27.11 and < 27.13

After every scenario below, ensure that:

1. /etc/ubuntu-advantage/uaclient.conf.preinst-remove is not present

2. /var/lib/ubuntu-advantage/preinst-detected-apt-news-disabled is not
present

Scenarios:

A: perform the upgrade. There should be no conffile prompt. apt_news
should be enabled. The conffile should be on the new package default.

B: pro config set apt_news=false, then perform the upgrade. There should
be no conffile prompt. apt_news should remain disabled.

C: pro config set apt_news=false then pro config set apt_news=true
again. Then perform the upgrade. There should be no conffile prompt.
apt_news should remain enabled. The conffile should be on the new
package default.

D: make an unrelated change to the conffile (eg. add a comment). Perform
the upgrade. There should be a conffile prompt.

E (error unwind check): pro config set apt_news=false. Note the md5 of
the conffile. Arrange for the preinst to fail *at the end* (I hacked it
to exit 1). Perform the upgrade. The conffile hash should not have
changed.

[Regression Potential]
If the fix with the checksums is flawed, then users will still see the prompt for no reason. There is little to no regression potential - the worse that can happen is that the regression already introduced is still present. Even in this case, the tests show us that the expected majority of use cases is covered, and there is no clear sign of corner cases.

[Discussion]
The next release of ubuntu-advantage-tools MUST have an update to those checksums to take into consideration the scenario where people already refused the prompt triggered by the current version.
The team will make sure to not change anything else in this conffile in future releases until this issue is completely sorted out and tested.

comment from racb in the txt below:
> the restore_previous_conffile function will need updating to
> restore the correct version of the conffile based on the previous
> version supplied by dpkg in $2. It will work fine right now but not on
> the subsequent upload. At that point, both previous conffile versions
> will need to remain indefinitely, since users might upgrade from any
> previous version we shipped.

[Original Description]
Due to "product interest", ubuntu-advantage-tools started polluting apt output: https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1992026. Executing `pro config set apt_news=false` is given as the way of disabling this unnecessary output.

On upgrading to the most recent version of ubuntu-advantage-tools in focal, having executed this command results in a conffile prompt:

--- /etc/ubuntu-advantage/uaclient.conf 2022-11-28 16:15:17.609674342 +0000 [0/1857]
+++ /etc/ubuntu-advantage/uaclient.conf.dpkg-new 2023-01-23 15:28:43.000000000 +0000
@@ -1,24 +1,18 @@
-# Ubuntu Pro client config file.
+# Ubuntu Pro Client config file.
 # If you modify this file, run "pro refresh config" to ensure changes are
-# picked up by Ubuntu Pro client.
+# picked up by Ubuntu Pro Client.

 contract_url: https://contracts.canonical.com
-daemon_log_file: /var/log/ubuntu-advantage-daemon.log
 data_dir: /var/lib/ubuntu-advantage
 log_file: /var/log/ubuntu-advantage.log
 log_level: debug
 security_url: https://ubuntu.com/security
 timer_log_file: /var/log/ubuntu-advantage-timer.log
+daemon_log_file: /var/log/ubuntu-advantage-daemon.log
 ua_config:
   apt_http_proxy: null
   apt_https_proxy: null
- apt_news: false
- global_apt_http_proxy: null
- global_apt_https_proxy: null
   http_proxy: null
   https_proxy: null
- metering_timer: 14400
- ua_apt_http_proxy: null
- ua_apt_https_proxy: null
   update_messaging_timer: 21600
- update_status_timer: 43200
+ metering_timer: 14400

Please stop wasting Ubuntu users' time: either disable this output by default, or make ubuntu-advantage-tools a Recommends of the seed(s) which include it (so users can remove it entirely and not have to worry about any future disruption caused by "product interest").

[Dan Watkins (oddbloke)]

It's worth noting: I, and any other users who have followed the advice of ubuntu-advantage-tools' own messaging, will now have to deal with this conffile prompt every time uaclient.conf changes.

[Robie Basak (racb)]

Thank you for the report. I don't think a conffile prompt was expected to happen in this case, so I'm flagging it as a regression caused by the SRU.

[Steve Langasek (vorlon)]

Phasing of this SRU set to 0 due to regression. A resolution is anticipated tomorrow.

[Robie Basak (racb)]

Confirmed. Steps to reproduce:

lxc launch ubuntu:jammy pro # this gave me ubuntu-advantage-tools 27.12~22.04.1
lxc exec pro bash
pro config set apt_news=false
apt update
apt install -y ubuntu-advantage-tools

This gave me the unexpected conffile prompt.

[Robie Basak (racb)]

Here's a fix that I think should work, but this is the sort of thing that needs very careful review from a highly experienced core dev type person.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted ubuntu-advantage-tools into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.13.2~22.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.13.2~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.13.2~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.13.2~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Dan, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.13.2~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Renan Rodrigo (renanrodrigo)]

We have ran tests against the proposed package, and verified it works for all releases.
Attached is a file with 4 folders, one for each scenario described in the bug (except E).
For each scenario, there is a script which was used to test it and the results for each specific release.

For scenario E, we agreed with vorlon that it would not be tested, as it involves hacking preinst, and it was tested during development time (believe us)

I am marking this bug verification-done.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~22.10.1

---------------
ubuntu-advantage-tools (27.13.2~22.10.1) kinetic; urgency=medium

  * Backport new upstream release: (LP: #2003977) to kinetic

ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 20:14:16 -0300

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~22.04.1

---------------
ubuntu-advantage-tools (27.13.2~22.04.1) jammy; urgency=medium

  * Backport new upstream release: (LP: #2003977) to jammy

ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 20:14:10 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~20.04.1

---------------
ubuntu-advantage-tools (27.13.2~20.04.1) focal; urgency=medium

  * Backport new upstream release: (LP: #2003977) to focal

ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 20:14:06 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~18.04.1

---------------
ubuntu-advantage-tools (27.13.2~18.04.1) bionic; urgency=medium

  * Backport new upstream release: (LP: #2003977) to bionic

ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 20:13:59 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~16.04.1

---------------
ubuntu-advantage-tools (27.13.2~16.04.1) xenial; urgency=medium

  * Backport new upstream release: (LP: #2003977) to xenial

ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 20:13:50 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-advantage-tools - 27.13.2~23.04.1

---------------
ubuntu-advantage-tools (27.13.2~23.04.1) lunar; urgency=medium

  * d/ubuntu-advantage-tools.{postinst,postrm,preinst}:
    - avoid a dpkg conf prompt if the only change to the original config file
      was to the apt_news flag (LP: #2003977)
  * apt-hook:
    - only run the pro client pre-update hook services when the apt update is
      executed as root user (LP: #2004057)

 -- Renan Rodrigo <email address hidden> Fri, 27 Jan 2023 18:46:42 -0300