Status set multiple times in OpenStack charms with optional relations
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
ceph-radosgw (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
cinder (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
glance (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
keystone (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
neutron-api (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
neutron-gateway (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh | ||
nova-cloud-controller (Juju Charms Collection) |
Fix Released
|
Undecided
|
Unassigned | ||
rabbitmq-server (Juju Charms Collection) |
Fix Released
|
Undecided
|
Alex Kavanagh |
Bug Description
In charms that call assess_status() on every hook execution and that have
optional relations there are two places where status_set() is called in
set_os_
leading to a race condition in which a status may be overwritten.
For charms like nova-cloud-
set_os_
only see this during pause and resume actions which do call assess_status().
Regardless of __if__ you hit the race condition you can see the problem simply
using print statements and an update-status hook run:
'STATUS SET IN set_os_
'STATUS SET IN _assess_
If the first happened to be different the second would clobber it.
In the set_os_
status before setting status. The assess_status() stack needs to do the same
possibly by calling set_os_
The goal should be to have a single canonical source for status_set.
This will affect all OpenStack charms that have optional relations.
Changed in glance (Juju Charms Collection): | |
status: | New → Fix Committed |
Changed in ceph-radosgw (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
assignee: | Alex Kavanagh (ajkavanagh) → nobody |
status: | New → In Progress |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
Changed in nova-cloud-controller (Juju Charms Collection): | |
status: | New → Fix Committed |
Changed in cinder (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
Changed in cinder (Juju Charms Collection): | |
status: | New → In Progress |
Changed in glance (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
Changed in keystone (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
summary: |
- Status set race condition in OpenStack charms with optional relations + Status set multiple times in OpenStack charms with optional relations |
Changed in keystone (Juju Charms Collection): | |
status: | New → In Progress |
Changed in keystone (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in cinder (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in neutron-api (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
status: | New → In Progress |
Changed in neutron-gateway (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
status: | New → In Progress |
Changed in neutron-gateway (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in neutron-api (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in ceph-radosgw (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
no longer affects: | rabbitmq (Juju Charms Collection) |
Changed in rabbitmq-server (Juju Charms Collection): | |
assignee: | nobody → Alex Kavanagh (ajkavanagh) |
status: | New → In Progress |
Changed in rabbitmq-server (Juju Charms Collection): | |
status: | In Progress → Fix Committed |
Changed in ceph-radosgw (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in neutron-gateway (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in rabbitmq-server (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in neutron-api (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in keystone (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in glance (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in nova-cloud-controller (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
Changed in cinder (Juju Charms Collection): | |
status: | Fix Committed → Fix Released |
I guess we're trying to minimise the changes to the full set of charms, but I think that this might not be avoidable.
The main issue, as David has investigated, is that the optional charm_func= function passed to assess_status call 'set_os_ workload_ status( )' which sets the status, followed by 'assess_status()' doing the same thing after calling the charm_func.
The reason the optional charm_func sets the status is to discover what it will be prior to returning it to assess_status() which then sets it.
Obviously, this is a design failure. In charm_helpers, the consequence of the optional relations charm_func function, is that _determine_ os_workload_ status( ) is called TWICE, once before the optional relations, and then afterwards as part of the optional relations.
The fix is to change optional_relations such that it returns a dictionary of the possible optional interfaces that can be dynamically added to the required_ interfaces, which is then passed to assess_status() so that _determine_ os_workload_ status( ) [in charmhelpers] is only called once. I'll do this on glance and propose it for review.