'snap wait' fails when enabling livepatch

Bug #1927329 reported by Domas Monkus
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
ubuntu-advantage-tools (Ubuntu)
Fix Released
Undecided
Lucas Albuquerque Medeiros de Moura

Bug Description

[Impact]

This bug results in ua failing to enable livepatch with no guidance to the user on how to resolve the issue. This only happens when the user has an older version of snapd installed.

The fix prints a new helpful message when this situation is detected. The message instructs the user to update snapd, which resolves the issue. Without this fix, users may give up while trying to enable livepatch and miss out on its benefits.

[Test Plan]

Reproduce the bug by trying to attach ua on a vm with an old version of snap. For example:

```
multipass launch -n test-xenial-old-snap xenial
multipass shell test-xenial-old-snap
# inside the vm now
sudo apt update
sudo apt install snapd=2.0.2 ubuntu-core-launcher=1.0.27
sudo apt install ubuntu-advantage-tools
sudo ua attach YOUR_UA_TOKEN_HERE
```

You should see the error described in this bug below.

To see the fix in action: detach, install the new version of ua from our staging ppa and try again.

```
sudo ua detach
sudo add-apt-repository ppa:ua-client/staging
sudo apt update
sudo apt install ubuntu-advantage-tools
sudo ua attach YOUR_UA_TOKEN_HERE
```

Enabling livepatch will still likely fail, but ua will output a suggestion to update snapd, which will resolve the issue.

[Where problems could occur]

The fix works by catching an exception when calling `snap wait` and printing the helpful message. It is possible that, if a mistake was made, we may catch a different error and print this message even though it is unrelated.

The fix also relies on the structure of the error message from snapd. If the error message structure were to change, this message won't be triggered properly.

[Other Info]

This issue is really a manifestation of ua's implied dependency on a recent version of snapd when enabling livepatch. ua doesn't require snapd for all features though, which is why it is not an explicit dependency. For example, a common use of ua is to enable a fips certified kernel (which is actually incompatible with livepatch). For those situations, there is no need for snapd to be installed or up to date.

[Original Description]

While trying to reproduce https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1927029, when running `ua attach <token>` on a xenial ec2 instance, I got the following error:
```
Enabling default service esm-infra
Updating package lists
UA Infra: ESM enabled
Enabling default service livepatch
Unexpected error(s) occurred.
For more details, see the log: /var/log/ubuntu-advantage.log
To file a bug run: ubuntu-bug ubuntu-advantage-tools
```

`/var/log/ubuntu-advantage.log` contains the following output (attaching full log):
```
2021-05-06 08:31:30,867 - contract.py:(274) [ERROR]: Unexpected error processing contract delta for livepatch: {'entitlement': {'affordances': {'kernelFlavors': ['generic', 'lowlatency', 'oem', 'aws'], 'series': ['trusty', 'xenial', 'bionic', 'focal'], 'minKernelVersion': '4.4', 'tier': 'updates', 'architectures': ['amd64', 'x86_64']}, 'entitled': True, 'obligations': {'enableByDefault': True}, 'directives': {'caCerts': '', 'remoteServer': 'https://livepatch.canonical.com'}, 'type': 'livepatch'}, 'resourceToken': 'mAgJOEWNBR0pkLVA1NG15WEhwalIxWDd5akREbTNaek1fOU54RGtXSFNDZXFreF9nOmE5NzJkODcwNzA3OTRlNmVhNDMyMDE3YWNkZDRmMzU2AAI4aXMtY29udHJhY3QgY0FHSmQtUDU0bXlYSHBqUjFYN3lqRERtM1p6TV85TnhEa1dIU0NlcWt4X2cAAhVpcy1yZXNvdXJjZSBsaXZlcGF0Y2gAAAYgQSFlXM5eOgCSzpsjCEyAGMxYG_CaoyJ62D8JGCXVoqw'}
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/uaclient/contract.py", line 260, in process_entitlements_delta
    series_overrides=series_overrides,
  File "/usr/lib/python3/dist-packages/uaclient/contract.py", line 330, in process_entitlement_delta
    orig_access, deltas, allow_enable=allow_enable
  File "/usr/lib/python3/dist-packages/uaclient/entitlements/livepatch.py", line 231, in process_contract_deltas
    if super().process_contract_deltas(orig_access, deltas, allow_enable):
  File "/usr/lib/python3/dist-packages/uaclient/entitlements/base.py", line 457, in process_contract_deltas
    self.enable()
  File "/usr/lib/python3/dist-packages/uaclient/entitlements/livepatch.py", line 104, in enable
    [SNAP_CMD, "wait", "system", "seed.loaded"], capture=True
  File "/usr/lib/python3/dist-packages/uaclient/util.py", line 608, in subp
    out, err = _subp(args, rcs, capture, timeout, env=env)
  File "/usr/lib/python3/dist-packages/uaclient/util.py", line 565, in _subp
    stderr=err.decode("utf-8"),
uaclient.util.ProcessExecutionError: Failed running command '/usr/bin/snap wait system seed.loaded' [exit(1)]. Message: error: Unknown command `wait'. Please specify one command of: abort, ack, buy, change, changes, connect, create-user, disable, disconnect, download, enable, find, help, install, interfaces, known, list, login, logout, refresh, remove, run or try
```

Additional information
----------------------
```
ubuntu@ip-172-31-29-198:~$ lsb_release -rd
Description: Ubuntu 16.04.1 LTS
Release: 16.04
```

```
ubuntu@ip-172-31-29-198:~$ apt-cache policy ubuntu-advantage-tools
ubuntu-advantage-tools:
  Installed: 27.0~16.04.1
  Candidate: 27.0~16.04.1
  Version table:
 *** 27.0~16.04.1 500
        500 http://us-west-2.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     27.0~16.04.1 500
        500 http://ppa.launchpad.net/ua-client/stable/ubuntu xenial/main amd64 Packages
```

```
ubuntu@ip-172-31-29-198:~$ snap --version
snap 2.14.2~16.04
snapd 2.14.2~16.04
series 16
ubuntu 16.04
```

Revision history for this message
Domas Monkus (tasdomas) wrote :
Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

Hi Domas, thanks for reporting this issue. Can I ask you which AMI id did you use to launch the image and get this traceback ? I am unable to reproduce it using this AMI:

ami-05b0f906c5f1814e7

Revision history for this message
Domas Monkus (tasdomas) wrote :

Hello, Lucas,
the AMI I used is ami-e1fe2281. This is basically a problem with an older version of snap. Running `apt update; apt upgrade` resolved the issue.

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

We have landed a PR on main that address this issue:
https://github.com/canonical/ubuntu-advantage-client/pull/1621

We will now better informing the user about this situation and we also not block enabling livepatch because of it

Changed in ubuntu-advantage-tools (Ubuntu):
status: New → Fix Committed
assignee: nobody → Lucas Albuquerque Medeiros de Moura (lamoura)
description: updated
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~21.10.1

---------------
ubuntu-advantage-tools (27.1~21.10.1) impish; urgency=medium

  * d/control:
    - specify debianutils min version
  * d/changelog:
    - fix lintian typos amend and redact incorrect 27.0 entry (GH: #1624)
  * lintian:
    - override ubuntu-advantage-pro wanted-by-target cloud-init
    - override xenial specific errors
    - rename package-specific overrides for pro vs tools
  * New upstream release 27.1:
    - apt-hook:
      + avoid segfault when comparing null Apt file origin to esm
        (LP: #1929123)
      + avoid wrapping static message formats at 80 chars
      + update go build flags based on lintian warnings (GH: #1626)
      + only add newlines for MOTD if message file length is non-zero
    - attach: do not print contract name if empty
    - autocomplete: Do not show beta services in autocomplete (GH: #1594)
    - cis:
      + make service non-beta
      + post enable message pointing to docs
      + update cis help url
    - docs: update releases.md per SRU review feedback on branch structuring
    - enable: correct messaging for beta service (GH: #1588)
    - errors: print a more helpful message when ssl fails (GH: #1618)
    - fips:
      + Block enabling fips if fips-updates once enabled (GH: #1600)
      + Update output of fips commands (GH: #1631)
    - livepatch: alert when snapd does not have wait cmd (LP: #1927329)
    - logging: remove tracebacks for UserFacingErrors (GH: #1586)
    - messaging:
      + Infra and Apps messaging is mutually exclusive (GH: #1573)
      + point to u.com/16-04 instead of u.com/advantage on ESM (GH: #1584)
      + separate _remove_msg_template. emit no warranty on infra disabled
    - pro: obtain AWS IMDSv2 API token before trying to grab pkcs7 doc
      (GH: #1608)
    - status: do not show info if not on contract (GH: #1592)
    - tests:
      + drop trusty specific tests
      + fix mock for handle_message_operations
      + fix motd message for bionic (GH: #1615)
      + integration tests for hirsute and groovy
      + manual test for trusty upgrade to xenial
      + reboot after dist-upgrade for upgrade test
      + test enabling CIS on focal (GH: #1582)
      + update messages in integration tests (GH: #1635)
      + use proposed pocket on xenial upgrade test
    - jenkins:
      + add pytest runs for xenial and bionic
      + run focal lxd integration tests

 -- Grant Orndorff <email address hidden> Mon, 24 May 2021 14:50:47 -0400

Changed in ubuntu-advantage-tools (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.