LXD 2.0.10 doesn't properly auto-update images

[Stéphane Graber]

A number of issues interfere with LXD 2.0.10's ability to update images:
 - The auto_update flag doesn't properly get set on newly downloaded images
 - The cached flag doesn't get properly copied when an image gets refreshed

The combination of those means that LXD effectively only updates images when a user requests a new container. This at least means that there is no security impact from this, but this also slows things down quite a bit and is certainly not the expected behavior.

This fix cherry-picks two upstream fixes that resolve the two highlighted issues and applies an extra change which will automatically restore the auto_update flag for any image that is marked as "cached" in the store.

== Rationale
LXD regressed in its background update code, leading to most LXD hosts storing stale images and only refreshing them when a user asks for a new container to be created.

This is pretty different from the expected behavior of LXD refreshing all its images every 6 hours.

The fix restores the old behavior and attempts to reset the update flag on images which are supposed to auto-update.

== Test case
1) Setup two LXD hosts on LXD 2.0.10
2) On the first LXD host, copy an image ("lxc image copy ubuntu:16.04 local: --alias blah")
3) Add teh first LXD host as a remote to the second LXD host
4) Create a container on the second LXD host using ("lxc init <remote>:blah c1")
5) On the first host, change the target of the alias ("lxc image copy ubuntu:14.04 local: --alias blah")
6) On the second host, look at the content of the image store ("lxc image list")
7) On the second host, restart LXD ("systemctl restart lxd")
8) On the second host, confirm that LXD is done starting ("tail -f /var/log/lxd/lxd.log")
9) On the second host, look at the content of the image store again ("lxc image list")

On a properly functionaling LXD, at step 9) you'll see the new image in the image store with the old one gone. On a broken LXD, the initial image is there and there's no sign of the new one.

== Regression potential
The changes being pushed here are coming from existing LXD releases (2.16) so have seen significant use already. The main risk I can think of is in the DB patch that's included in this change.

That code is very restrictive and will only set the auto-update flag on an image which is marked as cached (was automatically downloaded) and has a known source (as required for auto-update).

This will not fix images which have been already updated by LXD through container launch as those will have lost their cached flag due to the other bug we're fixing here, but attempting to fix those would just be guesswork as any information on how the image ended up in the store is lost at that point.

[Chris Halse Rogers (raof)]

Hello Stéphane, or anyone else affected,

Accepted lxd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxd/2.0.10-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Stéphane Graber (stgraber)]

I've confirmed that the updated package does cause stale images to get updated (those that were marked as cached=true prior to the upgrade) and I've confirmed that LXD now updates images again as it used to.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxd - 2.0.10-0ubuntu1~16.04.2

---------------
lxd (2.0.10-0ubuntu1~16.04.2) xenial; urgency=medium

  * Fix regression in image update logic (LP: #1712455):
    - 0005-Fix-regression-in-image-auto-update-logic.patch
    - 0006-lxd-images-Carry-old-cached-value-on-refresh.patch
    - 0007-Attempt-to-restore-the-auto_update-property.patch

  * Ship a sysctl.d file that bumps inotify watches count. (LP: #1602192)
  * Update debian/watch to look only at LTS releases.

 -- Stéphane Graber <email address hidden> Tue, 22 Aug 2017 20:39:36 -0400

[Stéphane Graber (stgraber)]

The verification of the Stable Release Update for lxd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.