during upgrade to luminous charm updated the kv store that the upgrade was finished before all OSDs were up/in
Bug #1821028 reported by
dongdong tao
This bug affects 4 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceph OSD Charm |
Fix Released
|
High
|
Corey Bryant |
Bug Description
When we do upgrade from Jewel to Luminous.
The charm would create the mon-kv entry saying the upgrade was complete prior to the OSDs being up/in and the next would fire off.
So, this could cause multiple osd nodes to be down at a very short time which causes some of the PG not available and causes many slow request.
I think we should have a mechanism to ensure that only when all the osd on one node is completely "up/in" then we can insert the "upgrade_done" key to the mon kv.
Changed in charm-ceph-osd: | |
assignee: | nobody → dongdong tao (taodd) |
description: | updated |
Changed in charm-ceph-osd: | |
assignee: | dongdong tao (taodd) → nobody |
Changed in charm-ceph-osd: | |
milestone: | none → 20.01 |
tags: | added: ceph-upgrade |
Changed in charm-ceph-osd: | |
milestone: | 20.01 → 20.05 |
Changed in charm-ceph-osd: | |
assignee: | nobody → Corey Bryant (corey.bryant) |
Changed in charm-ceph-osd: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.
Hello. How did you do the upgrade from Jewel to Luminous? I'm assuming that you are on Ubuntu Xenial and upgraded to OpenStack xenial-pike or later? Or did you upgrade Xenial to Bionic?
Please could you state the Ubuntu version (and version changes)
Please could you state the OpenStack version (if using Ceph with OpenStack)
Please could you include the Juju bundle (or fragment).
Please could you add any relevant Juju debug-log files (for the osd(s) in question).
Thanks.