Use OTA terminology in system settings

[Merlijn Sebrechts]

Describing the updates in "OTA x" terms is very common.

- Bugs: https://bugs.launchpad.net/ubuntu/+source/address-book-app/+bug/1437925
- Status updates: https://unity.ubuntu.com/2015/07/15/whats-new-in-ota-5/

However, looking at the system settings' about section on my phone, it is not clear what "OTA" update I have currently installed.
* Is it the version number? I guess not since that number is in its twenties and the current OTA is smaller than ten.
* is it the number after the dot of the image number?

 It would be very handy if the OTA version number was clearly marked somewhere in the system settings.

USE CASE:
I want to check to see if a bug is really fixed in an update. I see that the bug is not fixed on my current device and should be fixed in OTA 4. I do not know what OTA my phone is running, so I cannot know if the bug is falsely marked as fixed. (https://bugs.launchpad.net/ubuntu/+source/address-book-app/+bug/1437925)

[Matthew Paul Thomas (mpt)]

Thanks for this suggestion.

Ubuntu has never been competent at giving software version numbers: the Year.Month scheme is routinely misunderstood <https://goo.gl/IVxKuA>, LTS vs. non-LTS versions are similarly confusing, and hundreds of Ubuntu packages are simultaneously incomplete enough to have pre-1.0 version numbers but somehow complete enough to ship by default (dpkg -l | grep " 0.").

Unfortunately Ubuntu Touch makes this even worse, with at least seven different data used for identifying an OS version. I'm currently trying to distill these into a single brief string, that we can display on the "About This Phone" screen, that tells you most of what you need to know to understand things like whether a particular bug is fixed on your phone. <https://lists.launchpad.net/ubuntu-phone/msg13955.html>

One thing I won't do, though, is add the OTA number as an eighth datum. Not just because adding an eighth thing would be making the problem worse rather than better. But also because numbering OTA updates is meaningless. It made sense while there was only one Ubuntu Touch device widely available, but that is no longer the case. It is nonsense to describe, for example, an update as "OTA-5" when it is, in fact, only the first OTA update ever for the Meizu MX4. "Update" and "version" are words with different meanings.

This is similar to the period before the first Ubuntu phone was released, where engineers commonly referred to a particular version as "Ubuntu RTM". <https://bugs.launchpad.net/ubuntu-rtm> RTM for what, exactly? Only for the BQ Aquaris E4.5. That's silly in retrospect, because we've had and will have RTM for multiple other devices.

If you see any engineer in future refer to an Ubuntu version using an OTA number, please mock them gently.

[Pat McGowan (pat-mcgowan)]

Chuckle

We can stop using the ota terminology as soon as you reach your unifying version string conclusion

[Merlijn Sebrechts (merlijn-sebrechts)]

I do not know if you have that power, but if you want OTA to die you will
have to get everyone on board and present a good alternative that is both
technical and catchy.

From what I've seen, everyone is using the OTA terminology (bug reports,
reddit, community blogs, Canonical blogs) and this is unlikely to change
without a coordinated effort.

I would hate it for the system settings to show one thing and for everyone
to use something else... (which is sadly already the case)

Op vrijdag 24 juli 2015 heeft Matthew Paul Thomas <email address hidden> het
volgende geschreven:
> Thanks for this suggestion.
>
> Ubuntu has never been competent at giving software version numbers: the
> Year.Month scheme is routinely misunderstood <https://goo.gl/IVxKuA>,
> LTS vs. non-LTS versions are similarly confusing, and hundreds of Ubuntu
> packages are simultaneously incomplete enough to have pre-1.0 version
> numbers but somehow complete enough to ship by default (dpkg -l | grep "
> 0.").
>
> Unfortunately Ubuntu Touch makes this even worse, with at least seven
> different data used for identifying an OS version. I'm currently trying
> to distill these into a single brief string, that we can display on the
> "About This Phone" screen, that tells you most of what you need to know
> to understand things like whether a particular bug is fixed on your
> phone. <https://lists.launchpad.net/ubuntu-phone/msg13955.html>
>
> One thing I won't do, though, is add the OTA number as an eighth datum.
> Not just because adding an eighth thing would be making the problem
> worse rather than better. But also because numbering OTA updates is
> meaningless. It made sense while there was only one Ubuntu Touch device
> widely available, but that is no longer the case. It is nonsense to
> describe, for example, an update as "OTA-5" when it is, in fact, only
> the first OTA update ever for the Meizu MX4. "Update" and "version" are
> words with different meanings.
>
> This is similar to the period before the first Ubuntu phone was
> released, where engineers commonly referred to a particular version as
> "Ubuntu RTM". <https://bugs.launchpad.net/ubuntu-rtm> RTM for what,
> exactly? Only for the BQ Aquaris E4.5. That's silly in retrospect,
> because we've had and will have RTM for multiple other devices.
>
> If you see any engineer in future refer to an Ubuntu version using an
> OTA number, please mock them gently.
>
> ** Changed in: ubuntu-system-settings (Ubuntu)
> Status: New => Won't Fix
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1475568
>
> Title:
> Use OTA terminology in system settings
>
> Status in ubuntu-system-settings package in Ubuntu:
> Won't Fix
>
> Bug description:
> Describing the updates in "OTA x" terms is very common.
>
> - Bugs:
https://bugs.launchpad.net/ubuntu/+source/address-book-app/+bug/1437925
> - Status updates:
https://unity.ubuntu.com/2015/07/15/whats-new-in-ota-5/
>
> However, looking at the system settings' about section on my phone, it
is not clear what "OTA" update I have currently installed.
> * Is it the version number? I guess not since that num...

[Pat McGowan (pat-mcgowan)]

After further reflection...

The OTAs are applied to all devices
The OTA number really reflects a minor release of the software
The OTAs have corresponding release notes and bug lists for the milestone
When a new device is released it is based on one of these OTA/minor release versions
We occasionally provide what we call a "hotfix" which is a between OTA critical fix.
Build numbers (i.e. r24) are only meaningful on the proposed and devel channels, not stable

For the stable channel an OS version of the form "Ubuntu 15.04.6" would very nicely reflect the minor release of software running on any device.

The details information could continue to provide build numbers and underlying component levels which would be sufficient for both hotfixes and proposed/devel channels.

[Merlijn Sebrechts (merlijn-sebrechts)]

Great! Will the "milestones" in launchpad be changed to reflect the
corresponding OTA release? I'm not sure this is possible since not all
milestones will be an OTA release?

2015-08-27 17:13 GMT+02:00 Pat McGowan <email address hidden>:

> After further reflection...
>
> The OTAs are applied to all devices
> The OTA number really reflects a minor release of the software
> The OTAs have corresponding release notes and bug lists for the milestone
> When a new device is released it is based on one of these OTA/minor
> release versions
> We occasionally provide what we call a "hotfix" which is a between OTA
> critical fix.
> Build numbers (i.e. r24) are only meaningful on the proposed and devel
> channels, not stable
>
> For the stable channel an OS version of the form "Ubuntu 15.04.6" would
> very nicely reflect the minor release of software running on any
> device.
>
> The details information could continue to provide build numbers and
> underlying component levels which would be sufficient for both hotfixes
> and proposed/devel channels.
>
>
> ** Also affects: canonical-devices-system-image
> Importance: Undecided
> Status: New
>
> ** Changed in: canonical-devices-system-image
> Importance: Undecided => High
>
> ** Changed in: canonical-devices-system-image
> Status: New => Confirmed
>
> ** Changed in: canonical-devices-system-image
> Milestone: None => ww40-2015
>
> ** Changed in: canonical-devices-system-image
> Assignee: (unassigned) => Pat McGowan (pat-mcgowan)
>
> ** Changed in: ubuntu-system-settings (Ubuntu)
> Status: Won't Fix => New
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1475568
>
> Title:
> Use OTA terminology in system settings
>
> Status in Canonical System Image:
> Confirmed
> Status in ubuntu-system-settings package in Ubuntu:
> New
>
> Bug description:
> Describing the updates in "OTA x" terms is very common.
>
> - Bugs:
> https://bugs.launchpad.net/ubuntu/+source/address-book-app/+bug/1437925
> - Status updates:
> https://unity.ubuntu.com/2015/07/15/whats-new-in-ota-5/
>
> However, looking at the system settings' about section on my phone, it
> is not clear what "OTA" update I have currently installed.
> * Is it the version number? I guess not since that number is in its
> twenties and the current OTA is smaller than ten.
> * is it the number after the dot of the image number?
>
> It would be very handy if the OTA version number was clearly marked
> somewhere in the system settings.
>
>
> USE CASE:
> I want to check to see if a bug is really fixed in an update. I see that
> the bug is not fixed on my current device and should be fixed in OTA 4. I
> do not know what OTA my phone is running, so I cannot know if the bug is
> falsely marked as fixed. (
> https://bugs.launchpad.net/ubuntu/+source/address-book-app/+bug/1437925)
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1475568/+subscriptions
>

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jonas G. Drange (jonas-drange)]

For System Settings, this doesn't necessarily mean we have to change anything, since we use the ubuntu system image client dbus API, but that API needs to change. I've sat that project as affected.

Hopefully this could be done as an addition to the Service.Information endpoint.

[Pat McGowan (pat-mcgowan)]

For stable channels we should show something of the form Ubuntu 15.04 (OTA 6) or perhaps (r 6)
For proposed channels what it currently shows: Ubuntu 15.04 (r 252)

I am not convinced the build details page is at all interesting ,although it could be enhanced to include the channel which has been requested several times.

[Barry Warsaw (barry)]

system-image won't need to change at all. Use the .Information() API. Of course this piece of information must be included in the config file that the server writes for each new version. Whatever identifying information you want to add would be added to the [service]version_detail string.

[Barry Warsaw (barry)]

There's a lot of discussion on IRC going on about this, but I feel like this feature is currently too underspecified for us to tackle it. Here are some questions:

When are "OTA numbers" assigned? Is it before or after the image is published? Before or after the image is copied to stable?

Who assigns OTA numbers?

Which images will have an OTA #?

Will an OTA # ever be assigned or changed after an image is published?

Is it only required to know the OTA # that a device is currently on, or so you want to know the OTA # for a target upgrade? E.g. "The device is on OTA 5 but an upgrade will leave you on OTA 7"?

I think it should be a hard requirement that there are no additional downloads from sites other than system-image.ubuntu.com. There should also be no new required download machinery than what's already there, because of the battle-hardened nature of the existing code.

If a system-image client change is required, can we use this as the driving factor for finally adopting system-image 3 on the phone? Or will this have to be backported to 2.5.x? We've wanted to get everything on 3.0 for a long while now and it doesn't make sense absent security requirements to continue the 2.5.x series.

[Łukasz Zemczak (sil2100)]

Hey Barry! So, some answers as we already discussed on IRC:

When are "OTA numbers" assigned?
 - Always after an image is built and before it's published to the stable channel.

Who assigns OTA numbers?
 - The Product Team decides on the name of the current OTA number for the ongoing release.

Will an OTA # ever be assigned or changed after an image is published?
 - Some cases, yes. We have multiple existing official OTA releases that are 'untagged'. When this bug is fixed, we would like all old OTAs (like OTA-5, OTA-6) to be tagged and recognized by the device - so this has to work backwards too.

Is it only required to know the OTA # that a device is currently on, or so you want to know the OTA # for a target upgrade?
 - I suppose we would like to have both best

I suppose if we want to do everything on system-image, we can easily include some tagging mechanism to images in a channel and export that information in the json config files. This way it would be trivially parseable by external tools. In this case I wouldn't be concerned about the server side of things - the harder part is coming up with a good client-side solution. Since I think two cases need to be supported: first, when a new update is pulled in, the updater needs to somehow get the information about the current release tag (if present). Second case is: if the user is online, there needs to be a mechanism that would allow system-settings (or any other app that wants to know) to fetch the information about what release tag we have if such information is not already present.

Anyway, this is a bigger story if we want to get it right. It would be really easy to do as a hack, but I suppose it would be best if we just implement a proper solution. Something for OTA-8 I suppose then.

[Jonas G. Drange (jonas-drange)]

On 7 October 2015 at 11:44, Łukasz Zemczak <email address hidden>
wrote:

> we can easily
> ​ ​
> include some tagging mechanism to images in a channel and export that
> ​
> information in the json config files. This way it would be trivially
> ​
> parseable by external tools.

If the next version of [1] would get an additional key value pair in
version_detail, like so:

"version_detail​": "version=5,ota=6"

​​then the only work we need to do is in System Settings, which is already
done [3]

For older OTA releases, we can use the OTA map [2] until a better solution
is in place.

BR, Jonas

​[1]
http://system-image.ubuntu.com/ubuntu-touch/stable/meizu.en/arale/version-4.json
[2] http://people.canonical.com/~platform/touch/ota-bindings.json​
​[3]
https://code.launchpad.net/~jonas-drange/ubuntu-system-settings/use-information-endpoint/+merge/273555
​

[Jonas G. Drange (jonas-drange)]

On 7 October 2015 at 14:50, Jonas Drange <email address hidden> wrote:

> ​​
> "version_detail​": "version=5,ota=6"
>

Make that

​
​
"version_detail​": "version=5,nickname=OTA6"

or anything, really.

[Jonas G. Drange (jonas-drange)]

On 7 October 2015 at 14:50, Jonas Drange <email address hidden> wrote:

> For older OTA releases, we can use the OTA map [2] until a better solution
> is in place.
>

Ugh, that doesn't make sense, ignore it please.​

[Łukasz Zemczak (sil2100)]

I was more thinking of something like 'tag=OTA-6' or something, so that this can be used for other purposes than just OTA's - any image could then be tagged for various reasons. The map idea would be a good hacky solution, but I think we should design it in a complete way. And we need to make sure that all the edge cases are handled, meaning:

 * Image tagged during image copy - the standard scenario for OTAs
 * Image tagged after image copy

The old, existing images won't need to be handled as anyway a newer u-s-s from a newer OTA will be required for things to work. So images like OTA-4 etc. don't need to be detected (since there's no path that could lead to that being required). But still, we need to handle this rare case when someone makes a mistake and doesn't tag an image before copying it to a proper channel.

[Barry Warsaw (barry)]

On Oct 07, 2015, at 03:00 PM, Łukasz Zemczak wrote:

>I was more thinking of something like 'tag=OTA-6' or something, so that
>this can be used for other purposes than just OTA's - any image could
>then be tagged for various reasons. The map idea would be a good hacky
>solution, but I think we should design it in a complete way. And we need
>to make sure that all the edge cases are handled, meaning:
>
> * Image tagged during image copy - the standard scenario for OTAs
> * Image tagged after image copy
>
>The old, existing images won't need to be handled as anyway a newer
>u-s-s from a newer OTA will be required for things to work. So images
>like OTA-4 etc. don't need to be detected (since there's no path that
>could lead to that being required). But still, we need to handle this
>rare case when someone makes a mistake and doesn't tag an image before
>copying it to a proper channel.

Okay, here's a concrete design strawman then.

We use the 'tag' key in version_detail as the way to label the OTA number.

When the device/channel's index.json is generated, the image specification
already has a version_detail key. We can add "tag=OTA-6" to that
version_detail value.

When the device checks for an update available, it calculates a winning update
path through a number of images. If the end-point image of the update
candidate has a version_detail key, it is already available in the
target_version_detail key of .Information(). Clients (e.g. system-settings)
will have to split that value on commas and parse it themselves, but that
should be easy.

Similarly, when the image is generated if the OTA number is known at that
time, you would add the tag to the version_detail in the .ini file,
e.g. /etc/system-image/config.d/01_channel.ini

This version_detail string is already available in the `version_detail` key of
.Information(). Again, clients would have to split and parse that but the
logic is the same.

As it turns out system-image client doesn't need to change at all to support
this. It already exposes both the current device's version_detail and the
target update's version_detail.

If an OTA number changes, it would require a regeneration of the index.json
file and potentially a new image generated. But only the data file containing
the /etc/system-image/config.d ini files would have to be regenerated.

What this doesn't handle is if an OTA number changes after a device has
already updated to that image. There's no way to get that new data onto the
device without doing an update.

If that's a use case we'd need to handle, then we have to be very careful that
we don't crush a network connection polling for new OTA numbers. I don't know
whether there's a push mechanism in place to handle this. OTOH, I think it
would be pretty confusing if people saw that their device was on OTA-6 one day
and then without ever updating their device, they saw it was on OTA-7.

[Łukasz Zemczak (sil2100)]

@Barry: I think your design seems very feasible, so +1 on it. So, thinking about the case of untagged images that get tagged after users upgrade - I guess the usefulness of this particular case is very very low. The only logical moment (at least in case of Ubuntu Touch) where this could happen is when we make an error and copy the image over to the stable channel without tagging the image. In that particular case we can simply re-publish the same image with the tag appended. Not beautiful, but works and provides a safe and easy way to fix our errors. That being said, I want to add the possibility of tagging during image copy in s-i server. Anyway such accidents will not happen likely for touch stable promotions as now I always do the original image copy with phased percentage 0%, allowing correction of the tag in case of issues.

I will take care of the bits on the server side then and will pass them over to you for review. I suspect this will end up with creating an additional stand-alone script, something like tag-image, which will take the given image, append the tag and create a new image with the version_detail updated. I will also add an argument to the copy-image script to enable modifying/appending the tag during copy.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-system-settings - 0.3+16.04.20151029-0ubuntu1

---------------
ubuntu-system-settings (0.3+16.04.20151029-0ubuntu1) xenial; urgency=medium

  [ CI Train Bot ]
  * New rebuild forced.

  [ jonas-drange ]
  * Deprecated the use of Info endpoint in system_update.cpp and
    hotspot. Exposed detailed version details on UpdateManager. Used
    this to get to a 'tag' key which includes the ota string. Because of
    more advanced system-image testing, about tests now use the
    systemimage dbusmock template. debian/control, s-i-dbus changed to
    3.0.2 because that's where the ota string is. (LP: #1475568)

 -- <email address hidden> (Jonas G. Drange) Thu, 29 Oct 2015 15:48:09 +0000