Progressive release stops showing metrics when set to 100% target

[Bret Barker]

When using a progressive release [1], if you have been moving your release along 20%, 50%, 80% and watching the real-time metrics as users are refreshed with `snapcraft status`, all is well. However, as soon as you punch it to 100% it becomes non-progressive and you lose the ability to see the metrics.

Expected behavior: continue to show the metrics until 100% has been reached.

1. https://forum.snapcraft.io/t/progressive-releases/20913

[Sergio Schvezov (sergiusens)]

We agreed in the past the 100% would be equivalent to release. We can look at this.

[Maximiliano Bertacchini (maxiberta)]

Some background from the online docs: "Releasing a revision with a progressive percentage of 100 and releasing it without a progressive percentage are not equivalent. This behaviour is slightly surprising, but it’s a consequence of the fact that not all devices can ever get a progressive revision, since a device may only participate in a progressive release if we have a way to consistently identify it." [0]

Based on that, releases with progressive percentage of 100 are being implicitly transformed non-progressive (ie. nullified) to avoid that unintuitive situation.

Maybe we could make the internal logic smarter in that case by storing `100` explicitly instead of `null`, treating both `null` and `100` as non-progressive releases, but with the addition that those with `100` would still return progress stats.

[0] https://dashboard.snapcraft.io/docs/api/snap.html#progressive-releases

[Przemysław Suliga (suligap)]

Hi Sergio.

If we started to return "progressive"."current-percentage" for all returned channel-maps in the API responses (progressive or not) would snapcraft be backwards compatible with that change?

JSON schema wise it's legal now. Both scenarios would be possible:

1) Progressive.current-percentage being non-null, and progressive.percentage and progressive.paused being null -- this would indicate a non-progressive or 100% progressive release.

2) All progressive.current-percentage, progressive.percentage and progressive.paused being non null -- this is already the case for active progressive releases.

[Przemysław Suliga (suligap)]

Change that adds progressive.current-percentage (the field is non null now if there are any devices that reported this revision in a relevant channel) to all releases returned from https://dashboard.staging.snapcraft.io/docs/v2/en/snaps.html#snap-channel-map is deployed to staging.

[Przemysław Suliga (suligap)]

The above change is now deployed to production.

[Przemysław Suliga (suligap)]

SCA change released to production on 2021-06-18

[Callahan Kovacs (mr-cal)]

Handling for @suligap's API changes was added here: https://github.com/canonical/snapcraft/pull/3306.

However, progress data is not returned for 100% releases or non-progressive releases.

If the store implements the changes described in Maxi's comment and Snapcraft drops this conditional (https://github.com/canonical/snapcraft/blob/c0e3aaac7925aeea5dcd4a139db3475022de533d/snapcraft/store/client.py#L374), then metrics for 100% releases would be visible.

[Przemysław Suliga (suligap)]

> Handling for @suligap's API changes was added here:
> https://github.com/canonical/snapcraft/pull/3306.

Hi Callahan. Maxi's snapcraft PR is from October 2020 and predates the changes that were the outcome (API changes) of this bug being raised -- it adds initial support for progress reporting IIRC. We landed my changes "Add release progress metrics for all releases in api-snap-channel-map" in June 2021.

The API change from 2021 means that progress stats are returned in the api-snap-channel-map response for all currently open channels for which we were able to calculate them (this is regardless of the release being progressive or not), in each channelmap entry's progressive.current-percentage field.

"Releasing a revision with a progressive percentage of 100 and releasing it without a progressive percentage are not equivalent..." part of the docs is outdated. Releasing with a percentage of 100 is equivalent to non-progressively releasing the revision and results in the revision being offered for refresh to all clients tracking the channel. There's no need to change anything in the `snapcraft release` logic as far as I can tell.

It is possible for snapcraft to show progress metrics for all open channels if the API response contains a non null progressive.current-percentage value.

[Callahan Kovacs (mr-cal)]

So if I understand correctly, the store returns a progressive.current-percentage field for every channel and Snapcraft attempts to display the progressive.current-percentage for every channel.

Where do you think the problem is? Is it that the store is not able to calculate release data for 100%/non-progressive releases?

[Przemysław Suliga (suligap)]

The store returns a non-null progressive.current-percentage for every channel that it can calculate it for given the metrics it has (it might be null if we cannot no longer make out if there are any clients tracking a given channel for example).

But I think that `snapcraft status <snap>` only tries to display that value if the release is a progressive one and currently in progress, ie. progressive.percentage is not null.