Ubuntu Pro APT integration is a bit much

Bug #1992026 reported by Julian Andres Klode
318
This bug affects 63 people
Affects Status Importance Assigned to Milestone
ubuntu-advantage-tools (Ubuntu)
In Progress
Critical
Grant Orndorff

Bug Description

[Impact]
This bug impacts any LTS system which is not attached to a Pro subscription. Since 27.11.2, they started seeing the Ubuntu Pro beta message in the output of apt upgrade.

This was considered regression from the point of view of users which were relying on that output. Although this is not recommended, it is what was happening there. Other users are just annoyed that the message is there at all.

The mitigation strategy decided with Product Management for this problem is composed of two steps:
1. Move the Ubuntu Pro message to the very bottom of the apt output, and
2. Provide a `pro` command to disable the pro-client related messaging in APT completely.

[Test Case]
There is an integration test which now covers this functionality. A manual test script is also provided for the postinst behavior.

As soon as the package with the fixes lands in -proposed, output for those can be executed using it for verification.

[Regression Potential]
People may argue that the message in the bottom of the output is still regression. For those, running the command to opt-out makes the message disappear. This is the best alternative our team can provide so far.

[Discussion]
It is important to notice that:
- This message is temporary and will disappear as soon as Ubuntu Pro hits GA.
- There is product interest in keeping the message there.
- The command to disable APT messages will stay, so it can be used for any future message that the Pro Client inserts.
- We cannot, for technical reasons, omit the message when the 'quiet' option is provided for apt (-qq, -qqq). For users who want APT to be quiet all times, we recommend using the command to disable the messaging from the pro client. In the future, if the APT hooks let us, we can comply with that functionality.

[Original Description]

When we designed the whole hook mechanism for what was ua esm back in the day, the intent was for this to be an interface to tastefully display information about additional updates you are missing out on without a subscription.

The new Ubuntu Pro message crosses the line of a tasteful additional information and is negatively impacting apt user experience. The top message I saw today on /r/Ubuntu was

https://www.reddit.com/r/Ubuntu/comments/xxafiu/how_to_remove_advertisements_from_apt/

I'd vastly prefer hooks not to use their freedom to display information irrelevant at the specific point. At this hook, the user should be notified about updates, like updates in Ubuntu pro they are missing out on, or I guess pending blocked snap refreshes are valid options.

For messages like the Ubuntu Pro beta announcement, we already have the motd mechanism.

Related branches

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu-advantage-tools (Ubuntu):
status: New → Confirmed
Revision history for this message
Aaron Rainbolt (arraybolt3) wrote :

Reproduced on Kubuntu 22.04. I heartily agree that this needs fixed (may I add "quickly please"). I'll dig into it some hopefully today and see if I can patch this out.

Revision history for this message
Aaron Rainbolt (arraybolt3) wrote :

Actually, me attempting to patch it out might not be a great idea. Anyway, I've put my support behind removing this notice. We can see it's making users upset and it doesn't seem to be all that useful IMO (there's better places to put a notice like that, like at the Livepatch screen immediately after installation). Hopefully it will be removed. If not, I guess we'll just roll with it.

Revision history for this message
Renan Rodrigo (renanrodrigo) wrote :

The team is looking into this.
We have contacted Lech, which is the requester for this change, to get his input on how to proceed from here.

Changed in ubuntu-advantage-tools (Ubuntu):
status: Confirmed → Triaged
importance: Undecided → Critical
Revision history for this message
Lukas Metzger (loewexy) wrote :

This change broke our monitoring for new available updates in our complete infrastructure, since the monitoring tool can not recognize the output of apt-get anymore. This makes this change ultra critical to be fixed.

Removing the file /etc/apt/apt.conf.d/20apt-esm-hook.conf seems to help. One could also remove the package ubuntu-advantage-tools but since packages like ubuntu-minimal depend on them they would also be removed which is not what one wants here.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Lukas, to be fair this is not machine readable output, you should not be parsing it. Please use proper tools.

Revision history for this message
Renan Rodrigo (renanrodrigo) wrote (last edit ):

After aligning with Product Management, our team is working in the following fixes:

1. We will move the phrase to the end of the APT output, separated by a newline, so it does not get in the middle of anything expected now

2. We will provide a command to remove any messages that the Pro Client is inserting into the APT output.

Keep in mind that while 2 is permanent and will be there as an opt-out mechanism, 1 is temporary, as when Ubuntu Pro goes GA the regular package counts for ESM will be back and this message will be gone.

Changed in ubuntu-advantage-tools (Ubuntu):
status: Triaged → In Progress
assignee: nobody → Grant Orndorff (orndorffgrant)
Revision history for this message
Gionatan Danti (shodanshok) wrote (last edit ):

In addition to what already said please note that the Ubuntu Pro banner does not honor apt -qq option, meaning that "apt-get -qq upgrade" will only show that banner:

root@ubuntu:/tmp# apt-get upgrade -qq
Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
Learn more at https://ubuntu.com/pro

In addition to the opt-out mechanism, please honor apt quiet option.
Thanks.

@juliank: while I understand your point, this is not a reason to let such regressions unfixed.

Revision history for this message
David Tauriainen (david-tauriainen) wrote :

@juliank:
a simple diff of the output compared to a stored "no updates" output is what I use to let me know which machines need manual intervention. No advanced parsing at all. On a rare occasion when the apt-get output changes (and every machine suddenly is listed in the aggregation), I can redo the stored "no updates" output. But the addition of these lines which can change with potentially random intervals means such a simple setup will result in false positives far more often than once or twice per machine lifetime.
"Proper tools" were mentioned as a class. What is recommended as proper if not "apt-get -s dist-upgrade"?

@renanrodrigo:
regarding 2: this is happening on machines which do not have Ubuntu Pro client installed (ubuntu-advantage-pro correct?)
regarding 1: Moving these messages to the bottom of the output still keeps the variable messages. Are they guaranteed to always be exactly two lines tall going forward?

Revision history for this message
Renan Rodrigo (renanrodrigo) wrote :

@shodanshok:
Unfortunately, at the time there is no mechanism for us to leverage so APT will hide the lines when the quiet option is there. I would ask you to please run the command we will provide to hide the message when it is there (and then you'll have all quiet again).

@david-tauriainen:
on 1: Yes, guaranteed this message will not change. This message will be removed once Ubuntu Pro is released as GA. Only the security updates count message (which is already there for quite some time) will persist.

on 2: ubuntu-advantage-pro is a cloud-specific package; the package which delivers the Pro Client is ubuntu-advantage-tools (and yes, it is confusing, we are very sorry for that). ubuntu-advantage-tools is an ubuntu-minimal dependency, so it should be in every Ubuntu machine. We are working with the team to consolidate the packaging but this will take time (and it's a separate thing anyway).

Revision history for this message
Robie Basak (racb) wrote :

Running /usr/lib/update-notifier/apt-check will give you machine readable "how many updates are available" numbers for security and non-security updates. It's called by /usr/lib/update-notifier/update-motd-updates-available which is called from /etc/apt/apt.conf.d/99update-notifier in a APT::Update::Post-Invoke-Success hook, and then presented to the user through /etc/update-motd.d/90-updates-available. This may be an internal implementation detail of update-notifier. I'm not sure to what extent it's a public API, but in practice calling /usr/lib/update-notifier/apt-check for machine readable information has been consistently stable over many releases. If you want to use something that isn't officially stable but works in practice and this gives you the information you need, using this is probably the best option.

On -qq not being honoured by the hook, this is a good point, but I suggest that you file a separate bug for it - the fix would need to be a separate effort, so tracking it separately makes sense.

Revision history for this message
Ken MacInnis (kcm) wrote : Re: [Bug 1992026] Re: Ubuntu Pro APT integration is a bit much

I would suggest that this not be implemented until the mechanism to comply
with the utility's stated functionality is there (-qq), and not vice versa,
as this is quite antithetical to the philosophy of a toolchain in general.

On Mon, Oct 10, 2022 at 8:55 AM Robie Basak <email address hidden>
wrote:

> Running /usr/lib/update-notifier/apt-check will give you machine
> readable "how many updates are available" numbers for security and non-
> security updates. It's called by /usr/lib/update-notifier/update-motd-
> updates-available which is called from /etc/apt/apt.conf.d/99update-
> notifier in a APT::Update::Post-Invoke-Success hook, and then presented
> to the user through /etc/update-motd.d/90-updates-available. This may be
> an internal implementation detail of update-notifier. I'm not sure to
> what extent it's a public API, but in practice calling /usr/lib/update-
> notifier/apt-check for machine readable information has been
> consistently stable over many releases. If you want to use something
> that isn't officially stable but works in practice and this gives you
> the information you need, using this is probably the best option.
>
> On -qq not being honoured by the hook, this is a good point, but I
> suggest that you file a separate bug for it - the fix would need to be a
> separate effort, so tracking it separately makes sense.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1992026
>
> Title:
> Ubuntu Pro APT integration is a bit much
>
> Status in ubuntu-advantage-tools package in Ubuntu:
> In Progress
>
> Bug description:
> When we designed the whole hook mechanism for what was ua esm back in
> the day, the intent was for this to be an interface to tastefully
> display information about additional updates you are missing out on
> without a subscription.
>
> The new Ubuntu Pro message crosses the line of a tasteful additional
> information and is negatively impacting apt user experience. The top
> message I saw today on /r/Ubuntu was
>
>
> https://www.reddit.com/r/Ubuntu/comments/xxafiu/how_to_remove_advertisements_from_apt/
>
> I'd vastly prefer hooks not to use their freedom to display
> information irrelevant at the specific point. At this hook, the user
> should be notified about updates, like updates in Ubuntu pro they are
> missing out on, or I guess pending blocked snap refreshes are valid
> options.
>
> For messages like the Ubuntu Pro beta announcement, we already have
> the motd mechanism.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1992026/+subscriptions
>
>

--
Ken MacInnis - kcm at clueful dot org - http://www.clueful.org/

description: updated
Revision history for this message
Renan Rodrigo (renanrodrigo) wrote :

@shodanshok, @racb, @kcm :
For what is worth it, a bug was filed against APT to improve on the support of the -q flag on hooks.

https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1992472

Once this is implemented, we can adapt any further messaging in the UA side. This is the best we can do so far.

Revision history for this message
Mark Fraser (launchpad-mfraz) wrote :

Can't even remove the package ubuntu-advantage-tools as it removes
ttf-mscorefonts-installer* ubuntu-advantage-tools* ubuntu-minimal* update-manager-core* update-notifier-common*

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

> Can't even remove the package ubuntu-advantage-tools ...

Hi Mark Fraser,
this aspect is not what is discussed in the bug here.
The discussion about your statement has happened a while ago in bug 1950692 and been closed in [1].
Feel free to chime in there as it is waiting for a technical reason to become a recommends instead.

[1]: https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1950692/comments/12

Revision history for this message
Robie Basak (racb) wrote :

I've been asked to review the changes present in 27.11.3 for inclusion in Ubuntu.

This comment relates to the proposed "APT news" feature in 27.11.3, which I think also partly existed in 27.11.2 but had not been individually called out in the review for that until after it was released, when this bug was filed.

My understanding is that the apt hook was added so that the Ubuntu Pro tooling would be able to provide system-specific information about updates available were the user to subscribe. For example, something to the effect of "5 packages have updates available in ESM were you to subscribe; here's how to do that" (but better phrased). It does seem appropriate to me to provide this information to users at the time of running apt.

But I'm not sure about extending this to make it a general messaging mechanism as suggested by "APT news". Given the concerns raised in this bug and the wide community feedback received, and speaking as a member of the Ubuntu Technical Board, I'm not sure I would approve.

In Ubuntu we value having a clear path to leadership to make a decision when it is needed, and then that's something we can all stand behind. I think it's appropriate to take that path here. In this case this kind of decision seems like a matter for the Ubuntu Technical Board rather than individual Ubuntu developers and the SRU team alone.

Can we escalate to the Ubuntu Technical Board to determine if this is appropriate please, or directly to sabdfl if you think an immediate decision is required?

Revision history for this message
Mark Shuttleworth (sabdfl) wrote :

The output of apt (and apt-get) now need to reflect differential availability of updates, so a certain level of disruption is unavoidable. We did have that in -proposed for a while to shake out any issues.

It was slightly cheeky to use that change to also give people a heads-up of new capabilities, but I think it better to have a format change land everywhere rather than scatter-shot based on specific package update patterns (i.e. it would be worse to have a different layout appear on a specific machine only when there is a differentially available update for the first time). On reflection, based on feedback, we'd like to generalise that to provide contextual information about updates to CLI users, much as we have in the GUI. That's the 'apt news' mechanism, which will give us an out-of-band ability to raise awareness of update-related matters without a full apt package update.

For example, if we have another heartbleed, we will use this mechanism to make CLI users aware of it.

The proposal does include a way to suppress news, which will survive upgrades, and also the ability for apt plugins to learn about a 'quiet' mode, which will be valuable for many plugins other than the news one.

I do think it would be reasonable to expect the news to be related to updates and changes in the archive that are interesting for someone using apt, rather than general news, so can happily agree to set that policy and expectation amongst the folk who have write access to the news.

Revision history for this message
mkoniecz (matkoniecz) wrote (last edit ):

This kind of spam is unwelcome, unwanted and unacceptable to me.

I switched away from Windows over (among other things) ads in start menu.

Ads in CLI logs are also not at all OK to me.

> It was slightly cheeky to use that change to also give people a heads-up of new capabilities

No, it was a boring spam and marketing.

> For example, if we have another heartbleed, we will use this mechanism to make CLI users aware of it.

The complain is not about mechanism, it is about abusing it. Though this outcome was quite obvious risk of adding this new ad vector and new ad attack surface.

Revision history for this message
Sky (sky-lake) wrote :

PLEASE no advertising by default.

This is the kind of incredibly annoying thing that gets people to dump Ubuntu and switch distros. Your users have no interest in playing whack-a-mole with ads. Please check some of the popular forums if you are in doubt on how unpopular this decision is (I was linked here from one).

Constructively, I would like to suggest instead that the message is ONLY shown to relevant systems and include why. IE, instead of sending everyone "Try our commercial offering for free!" spam, instead only send to EOL systems: "Your system is currently not receiving security updates as the end of life for (Version) has been reached -- security updates are available via (url). This message can be dismissed with (command)." I don't think people would be nearly so annoyed with that approach.

Revision history for this message
drink (martin-espinoza) wrote :

Advertising belongs in motd where it doesn't matter, changing the output of basic system commands for advertising purposes affects people in real ways when they are scripting them. Making excuses for putting it into apt is just that.

Revision history for this message
Daniel Moerner (dmoerner) wrote :

The original bug report mentioned the following:

"- There is product interest in keeping the message there."

"When we designed the whole hook mechanism for what was ua esm back in the day, the intent was for this to be an interface to **tastefully** display information about additional updates you are missing out on without a subscription." (emphasis added)

The patch which was just pushed switches the message to the following:

#
# News about significant security updates, features and services will
# appear here to raise awareness and perhaps tease /r/Linux ;)
# Use 'pro config set apt_news=false' to hide this and future APT news.
#

Is there product interest in advertising Reddit? Also, I don't think this is a "tasteful" way to display such information, instead it seems juvenile.

Revision history for this message
Wolfgang (wviechtb) wrote :

I appreciate being able to switch off such messages entirely with 'pro config set apt_news=false'. But say I have now read the message above for the 20th time (each time I run 'apt upgrade'). I would really like a way to dismiss the *latest* message, so I don't have to keep seeing it, but if there is a new message, I would like to see it.

Revision history for this message
Kelly Kane (kellyjkane) wrote :

Please make this default off. It was worrying to see a command you run under sudo having unexpected outputs like this.

Revision history for this message
Dan Watkins (oddbloke) wrote :

For reference, disabling this output is now responsible for generating conffile prompts on upgrade: https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2003977

Revision history for this message
Thomas Ward (teward) wrote :

A number of users have been complaining lately about how Universe updates are showing in apt updates with the assumption that updates are not available without an ESM / Pro subscription, independent of the shell login MOTD prompts.

Unfortunately, this causes problems with users who are not familiar with the whole Ubuntu OS structure, and causes concerns that Ubuntu isn't actually free anymore. It is *strongly* suggested by me as a Community Council member that the default be changed to disable this kind of output on apt and MOTD because of the level of "over-integrated" that Pro is becoming vs. the updates/notifications, which impacts User Experience.

Revision history for this message
Erich Eickmeyer (eeickmeyer) wrote (last edit ):

I have to agree with Thomas above. Sadly, I'm pretty sure I do know who is responsible for the heavy-handed scare-tactic messaging lately that is extremely off-putting to the community, and this person needs to start making sure such messaging at the very least gets approval from the Community Team prior to commit, because it's clear this person lacks tact in their messaging and has lost sight of what it means to be truly "Ubuntu" in the sense of the word. In fact, I'd almost consider it to a Code of Conduct violation in that it has attempted manipulate and coerce people into doing something they wouldn't otherwise do.

I understand this is targeted at enterprise, but individuals in the community are getting scared that their systems are now insecure when, in reality, they're not. This advertisement looks like a typical "packages being held back" response when it's not, and that's scaring people that something is wrong with their system and that they need to pay to fix it. That is heavy-handed and a scare tactic, and that is something that I cannot say "I am who I am because of who we all are". It is not in the giving spirit or philosophy of "Ubuntu".

Revision history for this message
Julian Andres Klode (juliank) wrote (last edit ):

Thomas, Erich, there seems to be some misunderstanding in the comments. The systems are (partially) insecure. Before Pro, patches for universe were occasionally made available by the community. Systems were (partially) insecure then, but nobody told you.

With Pro, universe gets security support. If these users do not enable pro, and the community does not provide security updates, then these users are on insecure systems.

Revision history for this message
Renan Rodrigo (renanrodrigo) wrote :

Also, the messaging and documentation is clear that the individuals do not need to pay for it.

Revision history for this message
Aaron Rainbolt (arraybolt3) wrote :

juliank, Thomas is a Core Dev and Erich is a MOTU. Both of them (as well as myself, a Lubuntu developer) are very well aware of the fact that there are insecurities in Universe. But there's a difference between some insecure packages being on a system, and a system being insecure in general. People have been using Ubuntu with these Universe packages being left in a somewhat insecure state for years and years (ever since Ubuntu has been an OS, AFAIK). And people's systems have been fine. The chances of someone loading malicious gunk into a Universe package are slim.

The fact that users think that we're trying to extort them into paying indicates that something has been done wrong. We absolutely should give users the option of using Ubuntu Pro. It should be clear that it's free on a small scale. And we should even help users be able to see what packages could get extra updates if they were using Ubuntu Pro. But what we're doing now almost looks like malware, warning that there are unpatched security holes that one must sign up for a special service in order to get patched. That's actually true in this particular instance, and for a good reason, but this is what malware usually looks like. We don't want to look like malware.

I personally ran into this notification earlier yesterday. I actually didn't mind the "ads" for Ubuntu Pro and apt_news in apt - I found them helpful and beneficial. But even though I knew exactly what the "missing security updates" notification meant, it even made me a tiny bit uneasy when updating a 22.04 system.

Revision history for this message
Julian Andres Klode (juliank) wrote (last edit ):

Hi Aaron,

the difference really depend on how you use. If you installed a web browser from universe for example (I'm not sure this exists), this might matter more to you than a security update for grep in main.

This is not about malicious gunk in the package, security issues are usually accidents.

I'm just trying to make clear what the situation for universe updates is, because I do see a lot of people saying "now they want to make us pay for universe updates when they were free before" and then you clarify it and they say "I did not know universe did not receive security support".

When I built my VPS years ago I was like "which packages can I afford to install from universe - which can I provide security updates for and which are not security critical". I had to be very careful in selecting my packages. I'm glad that Pro solves that issue.

I generally agree though that we need to be careful with the integration and not over-integrate too much, personally, as an apt developer you can imagine I don't particularly enjoy large intrusions into the APT UX because it might not fit with the language or style apt normally uses.

Revision history for this message
mkoniecz (matkoniecz) wrote :

> and that's scaring people that something is wrong with their system
> and that they need to pay to fix it.

Having an ads emitted by system commands means that something IS wrong
with their system.

Nowadays Ubuntu is an adware ( https://en.wikipedia.org/wiki/Adware ).

Revision history for this message
mkoniecz (matkoniecz) wrote :

To be more specific

```
sudo apt dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
Get more security updates through Ubuntu Pro with 'esm-apps' enabled:
  libgraphicsmagick-q16-3 librpmsign8 python2.7-dev rpm2cpio liburiparser1
  libmaven3-core-java libopenexr-dev php-symfony-expression-language
  libgegl-0.4-0 librpmbuild8 lynx-common libzmq5 python2.7-minimal libhdf5-103
  libgegl-common libpython2.7 debugedit python2.7 libpython2.7-dev lynx
  libzmq3-dev phpmyadmin librpmio8 rpm-common maven rpm librpm8
  libhdf5-cpp-103 cpanminus libjs-jquery-ui libopenexr24 libsdl2-2.0-0
  libmysofa1 libpython2.7-minimal php-symfony-cache libgraphicsmagick++-q16-12
  php-symfony-var-exporter libpython2.7-stdlib
Learn more about Ubuntu Pro at https://ubuntu.com/pro
#
# "Phased updates" allow careful rollout of deb package changes at scale.
# You may see these as packages that have been "kept back". Learn more at
# https://ubuntu.com/server/docs/about-apt-upgrade-and-phased-updates
#
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
```

that I just got is a clear ad for Ubuntu Pro, that claims that updates
were deliberately withheld from being available because I am not subscribed
to some product by Canonical.

Revision history for this message
mkoniecz (matkoniecz) wrote :

For now `sudo apt purge ubuntu-advantage-tools` seems to work to get rid of ads.

Though it appears that long term solution is migrating to OS which is not adware.

Revision history for this message
teknopaul (teknopaul) wrote :
Revision history for this message
helpdeskdan (helpdeskdan-gmail) wrote :

I get the need for revenue. A tip for livepatch is one thing. But, literally advertising that your distro won't install security updates unless you pay for something.... that's just poor marketing and an open invitation to move to the vast array of distros that don't!

Revision history for this message
reetp (jcrisp) wrote :

> For now `sudo apt purge ubuntu-advantage-tools` seems to work to get rid of ads.

This does not work because it also removes core packages.

eg ubuntu-minimal* gnome-software* xubuntu-desktop*

Likewise `sudo apt purge ubuntu-pro-client`

Trying to squeeze cash by utilising FOMO or lack of security and and worrying users who then bombard me with questions is not a great look.

There should be a simple switch to disable this marketing nonsense.

Ironically I am a target for your sales pitch, but bully boy tactics have the reverse effect. It's one of the reasons I dumped the Windows world years ago. Sad to say you have steadily become more and more like M$.

You have clearly CHOSEN to monetise 'security' updates. It's by design, not accident.

(Lets not mention the debacle that is snaps that you spent years ignoring issues with, despite choosing to foist them on users when they broke systems and prevented timely upgrades - I have little more confidence in your 'security' than I do snaps)

It therefore gives me intense pleasure to disable it all and steadfastly refuse to play your stupid games.

I can only assume it is making you lots more money because there is no other reason to keep foisting this annoying junk on users otherwise.

I wonder which will come first. Retirement or changing OS again? The latter seems to be approaching much more rapidly than the former.

If your push is 'these packages are insecure but we provide secure ones' (haha - see above) then users may as well just install Debian and be done with it (I have already started moving in this direction for some installs)

All very sad.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.