base core16->core18 migration

[Ondrej Kubik]

snapd: 2.37.2
Ubuntu 18.04
core snap: 6405
kernel: 4.15.0-29-generic

Preconditions:
 Snap updated base from core16 to core18
 Snap has one or more services

Expected result:
 snap refreshes to new revision using new core18 base

Actual result:
 snap refresh to new revision fails as it is using still core16 base
 post-refresh hook fails as executables from snap package fail to run against wrong glibc
 if there is no post-refresh hook snap refreshes to new revision, but services do not start for same reason of glibs mismatch

Closer look revealed that service forked additional long running process which was dangling around and was preventing successful namespace unmount.
While best is to define additional process as service to provide better way to control its lifecycle, we probably still need more aggressive process control/kill when moving between snap revisions so we are not leaking dangling processes from previous revisions

[Zygmunt Krynicki (zyga)]

I would add that any snap refresh that changes base snap name must be handled with dedicated logic that we currently don't have a design for. It is unacceptable to run a snap against different base snap than a snap has declared to use.

[Ondrej Kubik (ondrak)]

Related bug https://bugs.launchpad.net/snappy/+bug/1819887

[Zygmunt Krynicki (zyga)]

I've implemented a fix in https://github.com/snapcore/snapd/pull/6671

[Zygmunt Krynicki (zyga)]

I've closed 6671 and re-opened as https://github.com/snapcore/snapd/pull/6673

[Stéphane Graber (stgraber)]

This is preventing LXD from moving to core16 to core18 (roadmap item).

[Zygmunt Krynicki (zyga)]

This is close to landing on https://github.com/snapcore/snapd/pull/7026

[Zygmunt Krynicki (zyga)]

This has just landed into master. The 2.40 backport will be released separately by mvo as a part of the 2.40 cycle.