Ceph mon IP addresses in export location is never re-validated

Bug #1996793 reported by Enrico Bocchi
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Medium
Carlos Eduardo

Bug Description

Ceph mon IP addresses in export_location is never re-validated

Description
===========
The Manila driver for CephFS computes the share export locations when started. Export locations contain the IP address(es) of the Ceph monitors, which are retrieved via a Ceph rados "mon dump" command and the path to be mounted (`{mon ip addr:port}[,{mon ip addr:port}]:{path to be mounted}`). The IP addresses of the monitors are never re-validated. If Ceph monitors IP addresses change, the export location will contain a list of stale IP addresses, making clients unable to connect to the Ceph monitors and to mount the shared filesystem.

Possible solutions and/or mitigations include:
- Support an alias to Ceph monitors instead of enumerating their IP addresses
- Re-evaluate periodically the list of Ceph monitors and update the export locations
- Provide an API to trigger the re-evaluation of export locations without the need of restarting the Manila controller

Steps to reproduce
==================
* Configure one CephFS cluster to be used with Manila for the provisioning of the shared file system
* Use an alias in `ceph.conf` to reach the mons of the Ceph cluster
* Start the Manila controller
* Replace one existing mons with another one (e.g., due to HW failure) that has a different IP address
* Update the alias members to remove the old mon and add the new one

Expected result
===============
Export locations managed by Manila follow the alias update.

Actual result
=============
The export locations contain stale IP addresses, which are not replaced by the new alias members.

Further comments
================
- It would be convenient to have a callable API to trigger the re-evaluation of the export location without the need of restarting the Manila service
- This bug for Manila shares and CephFS matches this other bug (https://bugs.launchpad.net/nova/+bug/1452641) for Cinder volumes and Ceph RBD.

Enrico Bocchi (ebocchi)
summary: - Ceph mon IP addresses in export_location is never re-validated
+ Ceph mon IP addresses in export location is never re-validated
Revision history for this message
Vida Haririan (vhariria) wrote :
Changed in manila:
assignee: nobody → Carlos Eduardo (silvacarlose)
milestone: none → antelope-3
importance: Undecided → Medium
Changed in manila:
milestone: antelope-3 → antelope-rc1
Changed in manila:
milestone: antelope-rc1 → bobcat-1
Vida Haririan (vhariria)
Changed in manila:
status: New → Triaged
Changed in manila:
milestone: bobcat-1 → bobcat-2
Changed in manila:
milestone: bobcat-2 → bobcat-3
Changed in manila:
milestone: bobcat-3 → bobcat-rc1
Changed in manila:
milestone: bobcat-rc1 → caracal-1
Changed in manila:
milestone: caracal-1 → caracal-3
Changed in manila:
milestone: caracal-3 → none
Changed in manila:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/c/openstack/manila/+/924279
Committed: https://opendev.org/openstack/manila/commit/101becf9b4e7326d1079bdba43c7d5bc537f5737
Submitter: "Zuul (22348)"
Branch: master

commit 101becf9b4e7326d1079bdba43c7d5bc537f5737
Author: Carlos Eduardo <email address hidden>
Date: Tue Jul 16 16:01:34 2024 -0300

    API for ensure shares

    Introduces a new API to allow OpenStack administrators to start
    the share manager's ensure shares operation.

    Also, introduced a new configuration option named
    `update_shares_status_on_ensure`, so administrators can define
    whether the shares status should be updated during the ensure
    shares operation or not.

    A new column was added to the services table, named `ensuring`.
    Through this field, we can identify whether there is an undergoing
    ensure shares operation or not.

    Closes-Bug: #1996793
    Partially-Implements: bp ensure-shares-api
    Change-Id: If7bf059eb8581f20a3ceb7c1af93558774f4ef5e

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 19.0.0.0rc1

This issue was fixed in the openstack/manila 19.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.