google-startup-scripts service no longer waits for cloud-init/snapd.seeded

[Robert C Jennings]

[Impact]

* We have a functional regression from gce-compute-image-packages to the new google-guest-agent. Without waiting for cloud-final and snapd.seeded we won't present a consistent system for users to run scripts that have archive mirrors set up, GCE's google-cloud-sdk snap installed, users in the proper groups, or other customizations owned by those services.

[Test Case]

* Observe google-startup-scripts waiting for snapd.seeded and cloud-final:

root@rbalint-gg-1:~# systemd-analyze dot| egrep 'google-startup-scripts.*(snapd\.seeded|cloud-final)'
 "google-startup-scripts.service"->"snapd.seeded.service" [color="green"];
 "google-startup-scripts.service"->"cloud-final.service" [color="green"];
 "google-startup-scripts.service"->"cloud-final.service" [color="grey66"];
 "google-startup-scripts.service"->"snapd.seeded.service" [color="grey66"];
   Color legend: black = Requires
                 dark blue = Requisite
                 dark grey = Wants
                 red = Conflicts
                 green = After

[Regression Potential]

* The fix is adding missing dependencies for the services shipped in google-guest-agent. The same dependencies were added when gce-compute-image-packages shipped those services thus restoring them would cause issues, but in theory the added dependencies could introduce cycles making the service ordering unreliable. I have not seen any sign of such issue.

[Original Bug Text]
We have a cloud-images qualification test for google startup scripts to ensure that cloud-init customizations are available before the user startup script is run. That test is failing and investigation shows that we have a functional regression from gce-compute-image-packages to the new google-guest-agent. Without waiting for cloud-final and snapd.seeded we won't present a consistent system for users to run scripts that have archive mirrors set up, GCE's google-cloud-sdk snap installed, users in the proper groups, or other customizations owned by those services. We are not holding groovy release on this bug, but we would like this treated with priority and it is a blocker for SRU.

/usr/lib/systemd/system/google-startup-scripts.service in focal from gce-compute-image-packages:

[Unit]
Description=Google Compute Engine Startup Scripts
After=network-online.target network.target rsyslog.service
After=google-instance-setup.service google-network-daemon.service
After=cloud-final.service multi-user.target
Wants=cloud-final.service
After=snapd.seeded.service
Wants=snapd.seeded.service

/usr/lib/systemd/system/google-startup-scripts.service in groovy from google-guest-agent:

[Unit]
Description=Google Compute Engine Startup Scripts
Wants=network-online.target rsyslog.service
After=network-online.target rsyslog.service google-guest-agent.service
Before=apt-daily.service

===
Workaround:

As a work-around users can add cloud-init status --wait to the beginning of their startup script (as cloud-init does wait for snap seeding to complete).

[Balint Reczey (rbalint)]

While splitting the packages I've lost the patches changing the unit dependences. :-(
Fixing it in the next upload.

[Timo Aaltonen (tjaalton)]

Hello Robert, or anyone else affected,

Accepted google-guest-agent into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/google-guest-agent/20200617.00-0ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Balint Reczey (rbalint)]

Verified 20200617.00-0ubuntu6 on Groovy.

root@rbalint-gg-1:~# dpkg -l google-guest-agent | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==================-====================-============-=================================
ii google-guest-agent 20200617.00-0ubuntu5 amd64 Google Compute Engine Guest Agent
root@rbalint-gg-1:~# systemd-analyze dot| egrep 'google-startup-scripts.*(snapd\.seeded|cloud-final)'
   Color legend: black = Requires
                 dark blue = Requisite
                 dark grey = Wants
                 red = Conflicts
                 green = After
root@rbalint-gg-1:~# sed -i s/backports/proposed/ /etc/apt/sources.list
root@rbalint-gg-1:~# apt update -qq
8 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@rbalint-gg-1:~# apt install -y -qq google-guest-agent
The following package was automatically installed and is no longer required:
  libnuma1
Use 'apt autoremove' to remove it.
The following packages will be upgraded:
  google-guest-agent
1 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Need to get 6780 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 64167 files and directories currently installed.)
Preparing to unpack .../google-guest-agent_20200617.00-0ubuntu6_amd64.deb ...
Unpacking google-guest-agent (20200617.00-0ubuntu6) over (20200617.00-0ubuntu5) ...
Setting up google-guest-agent (20200617.00-0ubuntu6) ...
root@rbalint-gg-1:~# systemd-analyze dot| egrep 'google-startup-scripts.*(snapd\.seeded|cloud-final)'
 "google-startup-scripts.service"->"cloud-final.service" [color="green"];
 "google-startup-scripts.service"->"snapd.seeded.service" [color="green"];
 "google-startup-scripts.service"->"snapd.seeded.service" [color="grey66"];
 "google-startup-scripts.service"->"cloud-final.service" [color="grey66"];
   Color legend: black = Requires
                 dark blue = Requisite
                 dark grey = Wants
                 red = Conflicts
                 green = After
root@rbalint-gg-1:~# dpkg -l google-guest-agent | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==================-====================-============-=================================
ii google-guest-agent 20200617.00-0ubuntu6 amd64 Google Compute Engine Guest Agent
root@rbalint-gg-1:~#

[Launchpad Janitor (janitor)]

This bug was fixed in the package google-guest-agent - 20200617.00-0ubuntu6

---------------
google-guest-agent (20200617.00-0ubuntu6) groovy; urgency=medium

  * Import relevant patches from earlier gce-compute-image-packages package
    (LP: #1901033)

 -- Balint Reczey <email address hidden> Thu, 22 Oct 2020 17:29:37 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for google-guest-agent has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.