This issue is caused if there were previous OSD's on the host. Example:
I removed and deployed 2 osd's, so I removed ceph-1 and ceph-2, then added ceph-3 ceph-4.
In /var/lib/ceph/osd, folder for ceph-1 and ceph-2 are still there even after removal. When attaching the iscsi charm to the mon, when the osd changes kick in, it errors because a specific osd is no longer there but it thinks it is.
Workaround:
delete offending OSD folders from /var/lib/ceph/osd directory (assuming that osd is no longer being used and is no longer in crush map)
ISSUE FOUND!
This issue is caused if there were previous OSD's on the host. Example:
I removed and deployed 2 osd's, so I removed ceph-1 and ceph-2, then added ceph-3 ceph-4.
In /var/lib/ceph/osd, folder for ceph-1 and ceph-2 are still there even after removal. When attaching the iscsi charm to the mon, when the osd changes kick in, it errors because a specific osd is no longer there but it thinks it is.
Workaround:
delete offending OSD folders from /var/lib/ceph/osd directory (assuming that osd is no longer being used and is no longer in crush map)