[needs-packaging] google-guest-agent

[Balint Reczey]

Part of gce-compute-image-packages is split to a new upstream repository and has been reimplemented in Go:

https://github.com/GoogleCloudPlatform/guest-agent

The binary package will be a dependency of gce-compute-image-packages, which will cease to ship parts shipped in google-guest-agent.

[Brian Murray (brian-murray)]

*** This is an automated message ***

This bug is tagged needs-packaging which identifies it as a request for a new package in Ubuntu. As a part of the managing needs-packaging bug reports specification, https://wiki.ubuntu.com/QATeam/Specs/NeedsPackagingBugs, all needs-packaging bug reports have Wishlist importance. Subsequently, I'm setting this bug's status to Wishlist.

[Zach Marano (zmarano)]

Hi Balint, just checking in with you if you have questions or issues. Also, I wanted to make sure you had a good understanding of where everything lives now.

GCE guest agent: https://github.com/GoogleCloudPlatform/guest-agent
* Replaces the python agents and instance setup.
* Package name we are using: google-guest-agent

GCE OS Config agent: https://github.com/GoogleCloudPlatform/osconfig
* Net new.
* Package name we are using: google-osconfig-agent

GCE OS Login: https://github.com/GoogleCloudPlatform/guest-oslogin
* Same as before
* Package name we are using: google-compute-engine-oslogin

GCE configuration: https://github.com/GoogleCloudPlatform/guest-configs
* Replaces the top level google-compute-engine pkg w/systemd and other configurations (some of this is in cloud-init I believe).
* Package name we are using: google-compute-engine

[Balint Reczey (rbalint)]

All the packages split off from gce-compute-image-packages will be updated together. I'm staging the update here:

https://launchpad.net/~rbalint/+archive/ubuntu/gce-staging/+packages

google-osconfig-agent is still in progress due to required additional updates in the dependency chain.
The packaging itself is in the following temporary repository:
https://code.launchpad.net/~rbalint/+git/google-osconfig-agent

The packages still need testing but are close to the final state.

[Balint Reczey (rbalint)]

There is an updated set of packages which are close to be ready for uploading to groovy:
https://bileto.ubuntu.com/#/ticket/4148

[Balint Reczey (rbalint)]

I've updated the packages in the Bileto ticket's PPA and I've also uploaded them to Groovy. Some of them need Archive Admin approval to show up in -proposed with all binaries.
My testing of the packages went well on a GCE instance.

[Hans Joachim Desserud (hjd)]

Looks like this has built in Groovy now https://launchpad.net/ubuntu/+source/google-guest-agent/20200617.00-0ubuntu1. (Not sure why it wasn't closed automatically when the bug number was mentioned in the change log)

[Liam Hopkins (liamh-google)]

I have looked at this package on a testing image in GCE. The instance configs file has been shipped differently in this package vs ours - here you are shipping it as /etc/defaults/instance_configs.cfg, we ship to /usr/share/google-guest-agent/instance_configs.cfg

There are two problems with this change. First, the directory is incorrect and should be /etc/default not /etc/defaults. This will be a breaking change for new installs and new images. Second, we ship the file to the /usr/share directory and copy it during new install ONLY if it doesn't exist. This file may already exist from legacy software and must not be modified if so.

[Balint Reczey (rbalint)]

Thank you for the review. I'm fixing the config file location typo in the next upload.

Ubuntu's package installation ensures that modified local configuration files are not overwritten without consent.
Having an old file in /etc/... triggers the following question which is better than silently ignoring the newly shipped file, like in upstream's postinst:

Configuration file '/etc/default/instance_configs.cfg'
 ==> File on system created by you or by a script.
 ==> File also in package provided by package maintainer.
   What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : start a shell to examine the situation
 The default action is to keep your current version.
*** instance_configs.cfg (Y/I/N/O/D/Z) [default=N] ?

For installing config files conditionally there is ucf, but it is overkill here.

[Liam Hopkins (liamh-google)]

The way that this file is managed has changed as part of this replacement, and many customers have automatic updates enabled. We chose not to mark this file as a config file, as we don't want that dialog to appear. We only ever copy the file into place if it doesn't already exist, and after that, it's up to the user to edit. This is similar to how the SSHD package handles its configuration file.

[Balint Reczey (rbalint)]

The standard self-descriptive way of doing that is using ucf. Do you mean openssh-server by SSHD? It uses ucf.

[Balint Reczey (rbalint)]

The other well established method is the one used extensively with systemd units, where file in /lib/systemd/system can be overridden by a file with the same name in /etc/systemd/systemd .

[Liam Hopkins (liamh-google)]

Systemd provides that functionality itself, internally. We don't want to use UCF or mark this as a config file. We want to copy the file once on installation iff it doesn't exist. It is otherwise an 'example' file.

[Balint Reczey (rbalint)]

Is there a particular reason for not wanting to mark this configuration file as a configuration file? UCF_FORCE_CONFFOLD and UCF_FORCE_CONFFMISS provide the exact behaviour you described and if used only in non interactive mode the users would even have a chance to review changes.
Without marking the file as a config file the file is not associated with the package and this is usually considered confusing.

[Balint Reczey (rbalint)]

Regarding the splitting of the packages GCE OS Config needs more time to be ready for releasing it because the required version of the Go packages was higher than the version they existed in the archive with and the updates started a chain reaction requiring upgrading several other packages.

The packages which replaced previous packages are in good shape, but the rewrite in Go pulled in new dependencies again and they need to go through the Main Inclusion Process to be installable on official Ubuntu images, such as the ones we provide in GCE.

Also the Go rewrite made a new security review required.

Please follow the MIR at LP: #1891929.

[Liam Hopkins (liamh-google)]

It's a complicated situation, but I'll try to highlight some of the reasons.

First, there is the complexity of existing files. We will only copy the file if no file already exists because it may exist from the previous, python guest which automatically generated this file. There are also the .template and .distro files which may exist, but which we never ship, so which will never be package owned.

Now, we want to support only ever having the user create or edit this file (rather than generating it), so we will never attempt to modify the file again after the first install case. So there is no value in marking it as a config file. Also, since we support many distributions and not all distributions support such upgrade paths with user-editable files, we can't perform such upgrades even if we wanted to.

I think it's somewhat normal to have unowned files for certain cases like this. I found modern 20.04 image has 7 unowned files in /etc/default already. However, if you really think it's against Ubuntu policy to do this, we would prefer the Ubuntu variance be to not copy the file at all. The code is written with defaults built in, the file does not even need to exist. We copy it purely for customer convenience.

[Balint Reczey (rbalint)]

Thank you for the explaining the reasons. I understand the complexity of having to support several distros with less advanced ability to handle configuration files. This was the reason for me to switch to Debian, way before Ubuntu was born. :-)

Since as you described the configuration file does not change behaviour by its presence because of the built in defaults and you are OK with not shipping it I plan not shipping it in the Ubuntu package. This aligns with the effort of "empty /etc" where the goal is having good defaults in the packages across the archives thus /etc can be deleted to reset local configuration.