openstack-upgrade action states "no upgrade available"

[Xav Paice]

Deployed with 18.11 charms, 3 units, xenial, with openstack-origin set to default.

Updated action-managed-upgrade=True, waited for completion.
Updated openstack-origin to 'cloud:xenial-queens', waited for completion.

I expected to be able to run the openstack-upgrade action to kick off each unit's upgrade (starting with the leader unit), but instead I got:

~$ juju run-action --debug --wait keystone/0 openstack-upgrade
02:45:33 INFO juju.cmd supercommand.go:56 running juju [2.5-beta1 gc go1.10.4]
02:45:33 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/5875/bin/juju", "run-action", "--debug", "--wait", "keystone/0", "openstack-upgrade"}
02:45:33 INFO juju.juju api.go:67 connecting to API addresses: [10.245.88.40:17070]
02:45:33 DEBUG juju.api apiclient.go:883 successfully dialed "wss://10.245.88.40:17070/model/2dbab777-e92e-4258-86ea-af1028a96716/api"
02:45:33 INFO juju.api apiclient.go:603 connection established to "wss://10.245.88.40:17070/model/2dbab777-e92e-4258-86ea-af1028a96716/api"
unit-keystone-0:
  id: f4d7bc99-f4e8-4706-8474-25f2789a0dba
  results:
    outcome: no upgrade available.
  status: completed
  timing:
    completed: 2018-11-27 02:46:20 +0000 UTC
    enqueued: 2018-11-27 02:45:45 +0000 UTC
    started: 2018-11-27 02:46:16 +0000 UTC
  unit: keystone/0
02:46:10 DEBUG juju.api monitor.go:35 RPC connection died
02:46:10 INFO cmd supercommand.go:465 command finished

If I try to run the install hook on that unit, the packages did get upgraded, but keystone fall over with:

ProgrammingError: (pymysql.err.ProgrammingError) (1146, u"Table 'keystone.project_tag' doesn't exist") [SQL: u'SELECT project_tag.project_id AS project_tag_project_id, project_tag.name AS project_tag_name, anon_1.project_id AS anon_1_project_id \nFROM (SELECT project.id AS project_id \nFROM project \nWHERE project.id = %(param_1)s) AS anon_1 INNER JOIN project_tag ON project_tag.project_id = anon_1.project_id ORDER BY anon_1.project_id'] [parameters: {u'param_1': u'b44b1750102a4403aba782947f13c408'}]

I ran a `keystone-manage db_sync to fix that.

[Xav Paice (xavpaice)]

I note in the unit logs:

2018-11-27 03:42:00 INFO juju-log DEPRECATION WARNING: Function configure_installation_source is being removed on/around 2017-07 : use charmhelpers.fetch.add_source() instead.

Also, it appears the config-changed hook isn't adding the fresh source even if I set action-managed-upgrades to false. It's added when I run the install hook.

FWIW, if I set the source to cloud:xenial-newton, everything's fine. I guess we need to go one release at a time.

[James Page (james-page)]

In the case of an action managed upgrade, the action that performs the upgrade reconfigures the source, not the config-changed hook (so that's expected behaviour).

You're correct in that you do need to step through:

   xenial/mitaka -> xenial/newton -> xenial/ocata -> xenial/pike -> xenial/queens

A direct jump from mitaka -> queens won't work.

[James Page (james-page)]

That said the version comparison code does appear to be wonky:

 apt.version_compare(avail_vers, cur_vers) == 1

where *_vers are codenames.

[James Page (james-page)]

Using apt for the comparison and checking for a value of '1' ensures the stepped upgrade approach thus enforcing the upgrade policy:

>>> apt.version_compare('queens', 'mitaka')
4
>>> apt.version_compare('pike', 'mitaka')
3
>>> apt.version_compare('ocata', 'mitaka')
2
>>> apt.version_compare('newton', 'mitaka')
1

This may have been by design but it feels a little brittle.

[James Page (james-page)]

I've validated I can step a keystone deployment through each series from xenial release (mitaka) to xenial queens.