Asserted and sideloaded snap refresh to incorrect channel

[Simon Fels]

We have a snap named "anbox-cloud-tests" on the snap store which has a set of versioned tracks (1.9 - 1.21) and nothing in the latest/{stable,candidate,beta,edge} channels. Latest is also set as the default track.

We now download a snap from a channel (rev 592 from 1.17/edge in this case) and install it together with its assertion:

$ snap download --revision=592 anbox-cloud-tests
$ snap ack anbox-cloud-tests_592.assert
$ snap install anbox-cloud-tests_592.snap
anbox-cloud-tests 1.17.1-alpha.9+gitd9a1e82 from Canonical✓ installed
$ snap list
Name Version Rev Tracking Publisher Notes
anbox-cloud-tests 1.17.1-alpha.9+gitd9a1e82 592 - canonical✓ -

As a next step we run a manual refresh (but can also be an automatic refresh, does not matter):

$ snap refresh anbox-cloud-tests
anbox-cloud-tests 1.14.1-fix1 from Canonical✓ refreshed
Channel for anbox-cloud-tests is closed; temporarily forwarding to stable

The snap is now incorrectly refreshed to the 1.14/stable track (see `snap info anbox-cloud-tests`) and snapd also fails to indicate what channel it has the snap refreshed to. Looking at the list of installed snaps there is still no tracking channel:

$ snap list
Name Version Rev Tracking Publisher Notes
anbox-cloud-tests 1.14.1-fix1 338 - canonical✓ -

I have not expected the refresh to be successful as I installed the snap without an associated channel. A `snap refresh anbox-cloud` should have failed as no channel is associated and should have required a --channel argument to succeed. The info message from refresh also is confused about what it is actually doing.

I am not sure if this is expected behavior, a bug or both.

[Michael Vogt (mvo)]

Note that Simon says that there is no "default-track" set in the store and there is no "latest" so the choice of the 1.14 track by the store seems to be arbitrary.

[Michael Vogt (mvo)]

I'm adding a bugtask for the snapstore here as well because it seems like the store should have predicable behavior if no "default-track" is set and no "latest" track is defined. Maybe error on refreshes in this case that do not specify a track (out snapd default behavior is iirc to just set the channel to "stable" if we haveno further information)?

[Guillermo Gonzalez (verterok)]

Michael,
thanks for creating the bug for the store.

Will try to collect logs from production and try to reproduce with a test snap in staging or locally to simplify debugging.

[Guillermo Gonzalez (verterok)]

Michael, Simon,

After some testing, and inspecting the channelmap for anbox-cloud-test our findings are:
 - there was a latest/stable channel, which was closed on 2022-11-10
 - the store will try to honor what snapd requests, in this case snapd is sending `tracking-channel: stable` in the refresh context (see attached refresh-context.json). This implicitly refers to latest/stable.
 - As there might be clients tracking latest/stable, the store response is for the last released revision in that channel: 338 (version 1.14.1-fix1) (see attached refresh-response.json) this is because we don't want to force a channel switch without the user explicitly requesting it to snapd.

Following the steps in the bug description, there is no tracking channel change:
$ snap list
Name Version Rev Tracking Publisher Notes
anbox-cloud-tests 1.14.1-fix1 338 - canonical✓ -

Related to this, Snapd seems to assume stable and latest/stable as tracking-channel for side-loaded and installed by revision snaps respectively. Is this the intended behavior? As illustrated by this bug it seems potentially unsafe if multiple tracks with incompatible versions are involved.

For the case of installed by revision, here is the example:
$ sudo snap install --revision=592 anbox-cloud-tests
anbox-cloud-tests 1.17.1-alpha.9+gitd9a1e82 from Canonical✓ installed

$ snap list
Name Version Rev Tracking Publisher Notes
anbox-cloud-tests 1.17.1-alpha.9+gitd9a1e82 592 latest/stable canonical✓ -

[William Grant (wgrant)]

The intended behaviour of the system is that a client that is already tracking a channel gets whatever was most recently in that channel, even if that channel is closed. A client that tries to install from or switch to that channel gets an error.

This is controlled by two fields in the Store refresh API's request body: context's tracking-channel specifies the old channel, and action's channel specifies the new one. The API correctly returns the most recent revision when snapd sends tracking-channel=stable, but erroneously returns an error when tracking-channel=latest/stable. I'm fixing that inconsistency, so both will return the most recent revision, matching the existing behaviour for channel=stable.

Now, this doesn't fix the originally reported issue. I don't actually know what the right behaviour is -- for dangerously sideloaded snaps I think we require an explicit `snap refresh --amend` to allow any upgrade at all, but I guess for asserted sideloads we don't do that. It doesn't seem like snapd should allow a channelless `snap refresh` for a sideloaded snap if it doesn't already have a known channel. And we should probably adjust the Store's refresh API to allow omitting tracking-channel, so refreshing onto a channel from no channel gets the same "can't refresh onto a closed channel" behaviour.

[Simon Fels (morphis)]

For reference: https://github.com/snapcore/snapd/blob/master/store/store_action.go#L334 is the relevant part in the snapd source which seems to set the channel of the refresh request to "stable" in case there is no tracking channel for the installed snap.

[Michael Vogt (mvo)]

Thank you @William and @Guillermo - it sounds like the prefered behavior is indeed that we can omit the tracking-channel in the API. AIUI this requires a store tweak(?). Once that is done we can do a snapd PR. If it works today already I'm happy to get the PR for snapd ready sooner of course :)