pro sometimes runs before cloud-config.service

Bug #2059952 reported by Lucas Albuquerque Medeiros de Moura
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-advantage-tools (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

[ Impact ]
Currently, the Pro client support a daemon named ubuntu-advantage.service that
performs two actions:

* Actively look for Pro licenses on Azure and GCP images to perform an auto-attach
* Retry auto-attach on Pro images if that command fails on boot

Therefore, this daemon is only being activated on generic Azure and GCP images and all Pro cloud images.

This daemon was originally setup to run after the cloud-config.service. However,
due to a race condition, this is no longer happening. Right now, we manually
check in the daemon code to see if the cloud-config service has finished.

Unfortunately, this new logic now breaks the current Pro setup through cloud-init userdata in both GCP and Azure Pro cloud images. That is because our daemon is now running before cloud-init has even started running. This means that the daemon will perform the attach and not cloud-init itself. This will be clearer, in the following example:

Let's imagine this situation where a user is launching a Pro GCP image:

1) User provides the following cloud-init userdata to the cloud image before booting it:

#cloud-config

ubuntu_advantage:
  enable: []

This means that the user wants no services to be enabled, but still want to attach to the Pro license.

2) Our daemon starts running before cloud-config.service has even started
3) Our daemon see the cloud-config.service as inactive and proceeds normally
4) Our daemon identifies that the user is running on a GCP instance and there is a valid Pro license for it.
5) Due to that, our daemon auto-attach the machine completely ignoring the cloud-init directives.

Therefore, to fix that issue we need to guarantee that we will only execute the daemon, if and only if, cloud-init has already started. That is because, on this situation, the cloud-config.service will already perform the attach operation following the user directives. When the daemon starts running, it will see that the image is already attached and do nothing.

Finally, given this scenario, this bug is only affecting GCP/Azure Pro images, as these are the only ones that will be able to reach the flow described here.

[Discussion]

To address that issue, we are now also checking if the cloud-init service
has already started if we detect that cloud-config service is inactive. If it isn't, the daemon will sleep for an specific amount of time before trying again.

[ Test Plan ]
Since this is a first boot issue, we will need to create a custom image with the package in proposed. Then, we need to guarantee that Pro configuration delivered
through cloud-init is being honored when we launch the image.

Additionally, it is worth noting that we cannot reproduce this issue on a VM easily. That is because, we would need "mock" the VM to pass as one of the affected clouds and also add a valid Pro license to it.

However, CPC is already aware of this issue and will help us creating the test plan here.

[ Where problems could occur ]
We are updating the cloud-init wait logic on the daemon. This could potentially make our daemon to not start. However, since we are just now waiting on the base cloud-init.service to start and we have already tested this solution in a custom image, we believe this is a low risk for this fix.

[ Original Description ]
We have recently updated the Pro to not strictly run after cloud-config.service. If cloud-config.service has not been started when pro runs, it can complete before cloud-config.service begins and thus the user-specificed pro configuration will be ignored since the instance is already attached.

When cloud-config.service has yet to run, ubuntu-advantage.service should wait until it's finished before running.

Related branches

description: updated
description: updated
description: updated
description: updated
Changed in ubuntu-advantage-tools (Ubuntu):
status: New → In Progress
description: updated
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 31.2.3

---------------
ubuntu-advantage-tools (31.2.3) noble; urgency=medium

  * daemon: wait for cloud-init.service to fully activate (LP: #2059952)

 -- Lucas Moura <email address hidden> Tue, 02 Apr 2024 10:13:32 -0300

Changed in ubuntu-advantage-tools (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.