restart race when modifying config
Bug #1450543 reported by
Edward Hope-Morley
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| cinder (Juju Charms Collection) |
Fix Released
|
High
|
Edward Hope-Morley | ||
| cinder-ceph (Juju Charms Collection) |
Fix Released
|
High
|
Edward Hope-Morley | ||
| glance (Juju Charms Collection) |
Fix Released
|
High
|
Edward Hope-Morley | ||
| nova-compute (Juju Charms Collection) |
Fix Released
|
High
|
Edward Hope-Morley | ||
Bug Description
We have 3 cinder units all related to ceph. All three cinder-volume services were seemingly not restarted after being configured to work with ceph since they were reporting that they were uninitialised which would imply that the cinder-volume service was not properly restarted after being configured despite the fact that it was already configured for use with RBD. Possibly ceph.conf was not ready? Restarting cinder-volume service resolved the issue.
Related branches
lp:~hopem/charms/trusty/nova-compute/fix-config-restart-svc-race
- Billy Olsen: Approve
-
Diff: 14 lines (+4/-0)1 file modifiedhooks/nova_compute_hooks.py (+4/-0)
lp:~hopem/charms/trusty/glance/fix-config-restart-svc-race
- Billy Olsen: Approve
-
Diff: 21 lines (+4/-0)1 file modifiedhooks/glance_relations.py (+4/-0)
lp:~hopem/charms/trusty/cinder/fix-config-restart-svc-race
- Billy Olsen: Approve
-
Diff: 21 lines (+4/-0)1 file modifiedhooks/cinder_hooks.py (+4/-0)
lp:~hopem/charms/trusty/nova-compute/service-restart-race-stable-backport
- OpenStack Charmers: Pending requested
- Diff: 0 lines
lp:~hopem/charms/trusty/cinder/service-restart-race-stable-backport
- OpenStack Charmers: Pending requested
- Diff: 0 lines
lp:~hopem/charms/trusty/glance/service-restart-race-stable-backport
- OpenStack Charmers: Pending requested
- Diff: 0 lines
lp:~hopem/charms/trusty/cinder-ceph/fix-config-restart-svc-race
- Corey Bryant (community): Approve
-
Diff: 26 lines (+8/-1)1 file modifiedhooks/cinder_hooks.py (+8/-1)
Revision history for this message
| Edward Hope-Morley (hopem) wrote | #1 |
| Changed in cinder (Juju Charms Collection): | |
| importance: | Undecided → High |
| milestone: | none → 15.07 |
Revision history for this message
| Edward Hope-Morley (hopem) wrote | #2 |
| Changed in glance (Juju Charms Collection): | |
| status: | New → In Progress |
| Changed in nova-compute (Juju Charms Collection): | |
| status: | New → In Progress |
| Changed in glance (Juju Charms Collection): | |
| importance: | Undecided → High |
| Changed in nova-compute (Juju Charms Collection): | |
| importance: | Undecided → High |
| Changed in glance (Juju Charms Collection): | |
| assignee: | nobody → Edward Hope-Morley (hopem) |
| Changed in nova-compute (Juju Charms Collection): | |
| assignee: | nobody → Edward Hope-Morley (hopem) |
| Changed in glance (Juju Charms Collection): | |
| milestone: | none → 15.07 |
| Changed in nova-compute (Juju Charms Collection): | |
| milestone: | none → 15.07 |
| summary: |
- cinder-volume restart race when modifying config + restart race when modifying config |
| tags: | added: backport-potential openstack |
| Changed in cinder (Juju Charms Collection): | |
| status: | In Progress → Fix Released |
| Changed in glance (Juju Charms Collection): | |
| status: | In Progress → Fix Released |
| Changed in nova-compute (Juju Charms Collection): | |
| status: | In Progress → Fix Released |
| Changed in cinder-ceph (Juju Charms Collection): | |
| status: | New → In Progress |
| importance: | Undecided → High |
| assignee: | nobody → Edward Hope-Morley (hopem) |
| milestone: | none → 15.07 |
| Changed in cinder-ceph (Juju Charms Collection): | |
| status: | In Progress → Fix Committed |
| Changed in cinder-ceph (Juju Charms Collection): | |
| status: | Fix Committed → Fix Released |
To post a comment you must log in.
We've analysed the given logs and there is a race between the ceph pool being ready and cinder-volume being restarted. Unfortunately when cinder-volume starts it does not to try to re-connect if it fails and it also does not stop itself.