Nautilus cluster goes into warn because of msgr2

[Chris MacNaughton]

A newly deployed Ceph cluster on Nautilus goes into a warning state because the monitors haven't enabled msgr2:

cluster:
    id: 6d13cfea-c29e-11e9-abc2-fa163ea48793
    health: HEALTH_WARN
            3 monitors have not enabled msgr2

[Chris MacNaughton (chris.macnaughton)]

https://docs.ceph.com/docs/master/rados/configuration/msgr2/#transitioning-from-v1-only-to-v2-plus-v1

[OpenStack Infra (hudson-openstack)]

Change abandoned by Chris MacNaughton (icey) (<email address hidden>) on branch: master
Review: https://review.opendev.org/677279
Reason: Abandoned in favor of previous work

[Chris MacNaughton (chris.macnaughton)]

This is resolved in https://review.opendev.org/#/c/662248/5

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/677280
Committed: https://git.openstack.org/cgit/openstack/charms.ceph/commit/?id=64c885438203d7cfa5fb8d9656c2ede3707b49bf
Submitter: Zuul
Branch: master

commit 64c885438203d7cfa5fb8d9656c2ede3707b49bf
Author: Chris MacNaughton <email address hidden>
Date: Mon Aug 19 15:18:48 2019 -0400

    Messanger v2 should be enabled in Nautilus

    Change-Id: Id1eef4676ac7bdae76c0b8975c3e9ae57e0c9e2d
    Closes-Bug: #1840701

[Chris MacNaughton (chris.macnaughton)]

The fix above only helps new deploys on Nautilus. Upgrades from Mimic -> Nautilus still run into this.

[Vladimir Grevtsev (vlgrevtsev)]

fwiw, this issue is still actual - just got this with the latest charm versions upgrading from Stein to Train (and Ceph Mimic->Nautilus).

[Ponnuvel Palaniyappan (pponnuvel)]

I have observed this problem when upgrading from Train to Ussuri as well.

[James Vaughn (jmcvaughn)]

Have a customer that has bumped into this issue going from Stein to Train. Note that this issue prevents additional ceph-mon units from joining the quorum after an upgrade; in other words, this is more problematic than just a warning.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charms.ceph/+/820455

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-ceph-mon/+/821707

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charms.ceph/+/820455
Committed: https://opendev.org/openstack/charms.ceph/commit/ff752de0f06d8a06a4da722211931905fd4a827f
Submitter: "Zuul (22348)"
Branch: master

commit ff752de0f06d8a06a4da722211931905fd4a827f
Author: James Vaughn <email address hidden>
Date: Sun Dec 5 13:07:53 2021 +0000

    Explicitly enable msgr2 for Nautilus and later

    When updating to Ceph Nautilus (OpenStack Train) or later releases from
    a pre-Nautilus release, msgr2 will not be enabled without manual
    intervention (e.g. running `juju ssh ceph-mon/leader 'sudo ceph mon
    enable-msgr2'`).

    Leaving msgr2 disabled (i.e. using msgr1 only) isn't an issue for normal
    operations; Ceph will simply show a warning in `sudo ceph status` that
    this is the case. However, Ceph Nautilus and later releases default to
    msgr2, meaning that new ceph-mon units will attempt to communicate with
    the existing cluster using msgr2.

    This commit enables msgr2 on package upgrades on all Ceph releases
    including and after Nautilus.

    Closes-Bug: #1840701
    Change-Id: Ib90cd03b16f2062fa6c6a43d242d51306ae8ca95

[OpenStack Infra (hudson-openstack)]

Change abandoned by "James Page <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-ceph-mon/+/821707
Reason: This review is > 12 weeks without comment, and failed testing the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Chris MacNaughton (chris.macnaughton)]

ceph-mon no longer gets charms.ceph vendored in but rather imported at build time, so this fix is included in the latest build of the ceph-mon charm in latest/edge