Gnocchi Charm

stein: assess-status hook triggers upgrade with is a no-op

Bug #1824119 reported by James Page on 2019-04-10

This bug affects 3 people

Affects		Status	Importance	Assigned to	Milestone
	Gnocchi Charm	Fix Released	High	Alex Kavanagh	Gnocchi Charm 20.10

Bug Description

During execution of the assess-status hook, the charm incorrectly identifies it needs to upgrade; this results in a no-op package upgrade, db sync etc...:

tracer: -- dequeue handler reactive/layer_openstack.py:61:run_default_update_status
unit-gnocchi-0: 10:01:58 INFO unit.gnocchi/0.juju-log Invoking reactive handler: reactive/gnocchi_handlers.py:46:render_config
unit-gnocchi-0: 10:02:00 INFO unit.gnocchi/0.juju-log Performing OpenStack upgrade to stein.
unit-gnocchi-0: 10:02:00 INFO unit.gnocchi/0.juju-log DEPRECATION WARNING: Function configure_installation_source is being removed on/around 2017-07 : use charmhelpers.fetch.add_source() instead.
unit-gnocchi-0: 10:02:00 DEBUG unit.gnocchi/0.update-status Hit:1 http://nova.clouds.archive.ubuntu.com/ubuntu disco InRelease
unit-gnocchi-0: 10:02:00 DEBUG unit.gnocchi/0.update-status Hit:2 http://security.ubuntu.com/ubuntu disco-security InRelease
unit-gnocchi-0: 10:02:01 DEBUG unit.gnocchi/0.update-status Hit:3 http://nova.clouds.archive.ubuntu.com/ubuntu disco-updates InRelease
unit-gnocchi-0: 10:02:01 DEBUG unit.gnocchi/0.update-status Hit:4 http://nova.clouds.archive.ubuntu.com/ubuntu disco-backports InRelease
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Reading package lists...
unit-gnocchi-0: 10:02:02 INFO unit.gnocchi/0.juju-log Upgrading with options: ['--option', 'Dpkg::Options::=--force-confnew', '--option', 'Dpkg::Options::=--force-confdef']
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Reading package lists...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Building dependency tree...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Reading state information...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Calculating upgrade...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
unit-gnocchi-0: 10:02:02 INFO unit.gnocchi/0.juju-log Installing ['gnocchi-api', 'gnocchi-metricd', 'python3-apt', 'ceph-common', 'python3-rados', 'python3-keystonemiddleware', 'python3-memcache', 'memcached', 'python-memcache', 'haproxy'] with options: ['--option', 'Dpkg::Options::=--force-confnew', '--option', 'Dpkg::Options::=--force-confdef']
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Reading package lists...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Building dependency tree...
unit-gnocchi-0: 10:02:02 DEBUG unit.gnocchi/0.update-status Reading state information...
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status ceph-common is already the newest version (13.2.4+dfsg1-0ubuntu1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status haproxy is already the newest version (1.8.19-1ubuntu1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status memcached is already the newest version (1.5.10-0ubuntu1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status python-memcache is already the newest version (1.59-1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status python3-apt is already the newest version (1.8.4).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status python3-keystonemiddleware is already the newest version (6.0.0-0ubuntu1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status python3-memcache is already the newest version (1.59-1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status python3-rados is already the newest version (13.2.4+dfsg1-0ubuntu1).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status gnocchi-api is already the newest version (4.3.2-0ubuntu2).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status gnocchi-metricd is already the newest version (4.3.2-0ubuntu2).
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
unit-gnocchi-0: 10:02:03 DEBUG unit.gnocchi/0.update-status none
unit-gnocchi-0: 10:02:04 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:04 DEBUG unit.gnocchi/0.update-status none
unit-gnocchi-0: 10:02:04 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:05 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:05 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:05 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:05 WARNING unit.gnocchi/0.juju-log Not adding haproxy listen stanza for gnocchi-api_int port is already in use
unit-gnocchi-0: 10:02:05 WARNING unit.gnocchi/0.juju-log Not adding haproxy listen stanza for gnocchi-api_public port is already in use
unit-gnocchi-0: 10:02:05 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:06 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:06,596 [30053] INFO gnocchi.service: Gnocchi version 4.3.2
unit-gnocchi-0: 10:02:06 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:06,953 [30053] INFO gnocchi.cli.manage: Upgrading indexer SQLAlchemyIndexer: mysql+pymysql://gnocchi:VV9RBSFsfPbhxywSBWy73PCN2msPhJrw@10.5.0.95/gnocchi
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:07,053 [30053] INFO gnocchi.common.ceph: Ceph storage backend use 'rados' python library
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:07,074 [30053] INFO gnocchi.cli.manage: Upgrading storage CephStorage: b'5fff1714-5b70-11e9-aa30-fa163e5148f4'
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:07,076 [30053] INFO gnocchi.common.ceph: Ceph storage backend use 'rados' python library
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status 2019-04-10 10:02:07,092 [30053] INFO gnocchi.cli.manage: Upgrading incoming storage CephStorage: b'5fff1714-5b70-11e9-aa30-fa163e5148f4'
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status Synchronizing state of gnocchi-metricd.service with SysV service script with /lib/systemd/systemd-sysv-install.
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status Executing: /lib/systemd/systemd-sysv-install enable gnocchi-metricd
unit-gnocchi-0: 10:02:07 DEBUG unit.gnocchi/0.update-status active
unit-gnocchi-0: 10:02:08 DEBUG unit.gnocchi/0.update-status Synchronizing state of apache2.service with SysV service script with /lib/systemd/systemd-sysv-install.
unit-gnocchi-0: 10:02:08 DEBUG unit.gnocchi/0.update-status Executing: /lib/systemd/systemd-sysv-install enable apache2
unit-gnocchi-0: 10:02:08 DEBUG unit.gnocchi/0.update-status active
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.update-status none
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.update-status none
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:09 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:09 WARNING unit.gnocchi/0.juju-log Not adding haproxy listen stanza for gnocchi-api_int port is already in use
unit-gnocchi-0: 10:02:09 WARNING unit.gnocchi/0.juju-log Not adding haproxy listen stanza for gnocchi-api_public port is already in use
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.juju-log Changing permissions on existing content: 33184 -> 416
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.update-status gnocchi-api (enabled by maintainer script)
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.juju-log Configuration rendered
unit-gnocchi-0: 10:02:10 INFO unit.gnocchi/0.juju-log Invoking reactive handler: reactive/gnocchi_handlers.py:81:storage_ceph_connected
unit-gnocchi-0: 10:02:10 INFO unit.gnocchi/0.juju-log Json request: {"api-version": 1, "ops": [{"op": "create-pool", "name": "gnocchi", "replicas": 3, "pg_num": null, "weight": null, "group": null, "group-namespace": null, "app-name": null, "max-bytes": null, "max-objects": null}], "request-id": "a78948ca-5b70-11e9-a616-fa163ed98813"}
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.juju-log Request already sent but not complete, not sending new request
unit-gnocchi-0: 10:02:10 INFO unit.gnocchi/0.juju-log Invoking reactive handler: reactive/gnocchi_handlers.py:86:configure_ceph
unit-gnocchi-0: 10:02:10 INFO unit.gnocchi/0.juju-log Making dir /etc/ceph root:root 555
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.juju-log Ceph keyring exists at /etc/ceph/ceph.client.gnocchi.keyring and has not changed.
unit-gnocchi-0: 10:02:10 INFO unit.gnocchi/0.juju-log Invoking reactive handler: reactive/gnocchi_handlers.py:106:provide_gnocchi_url
unit-gnocchi-0: 10:02:10 DEBUG unit.gnocchi/0.juju-log Providing gnocchi URL: http://10.5.0.72:8041
unit-gnocchi-0: 10:02:11 INFO unit.gnocchi/0.juju-log Invoking reactive handler: hooks/relations/tls-certificates/requires.py:103:broken:certificates
unit-gnocchi-0: 10:02:11 DEBUG unit.gnocchi/0.juju-log Running _assess_status()
unit-gnocchi-0: 10:02:13 DEBUG unit.gnocchi/0.update-status active
unit-gnocchi-0: 10:02:13 DEBUG unit.gnocchi/0.update-status active

James Page (james-page) on 2019-04-10

summary:	- stein: upgrade hook triggers upgrade with is a no-op + stein: assess-status hook triggers upgrade with is a no-op
Changed in charm-gnocchi:
status:	New → Triaged
importance:	Undecided → High

Revision history for this message

Alex Kavanagh (ajkavanagh) wrote on 2020-06-25:

Also, does it in ussuri, and I expect train. (just hit it today on a mojo run with ussuri).

Alex Kavanagh (ajkavanagh) on 2020-08-20

Changed in charm-gnocchi:
assignee:	nobody → Alex Kavanagh (ajkavanagh)
milestone:	none → 20.10

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-08-20: Fix proposed to charm-gnocchi (master)

Fix proposed to branch: master
Review: https://review.opendev.org/747159

Changed in charm-gnocchi:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-08-21: Fix merged to charm-gnocchi (master)

Reviewed: https://review.opendev.org/747159
Committed: https://git.openstack.org/cgit/openstack/charm-gnocchi/commit/?id=88e450a6bbe5a484b50aee7e41f916d6bf926414
Submitter: Zuul
Branch: master

commit 88e450a6bbe5a484b50aee7e41f916d6bf926414
Author: Alex Kavanagh <email address hidden>
Date: Thu Aug 20 14:19:22 2020 +0100

Stop charm running handlers during update-status

    The update-status hook is supposed to be lightweight; prior to this
    change the charm was running all of the handlers including a no-op
    checking for upgrades to openstack. This patchset builds on the
    depends-on patchset and gates the update-status hook against those
    handlers. Note that update-status is handled using the default handler.

    Depends-On: I7ca97db646ada95d7f5541ca4e2cf14013c0d7a6
    Change-Id: I1e14e609567d00bed6e8bb755acabd08c1113469
    Closes-Bug: #1824119

Changed in charm-gnocchi:
status:	In Progress → Fix Committed

Alex Kavanagh (ajkavanagh) on 2020-11-02

Changed in charm-gnocchi:
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.