mgr: failed dependency - no module named distutils

[James Page]

[ Impact ]
dashboard and volume ceph mgr modules fail to activate under Python 3.12 due to use of distutils.

[ Test Plan ]
sudo snap install --channel latest/edge/core24 microceph
sudo microceph cluster bootstrap
sudo microceph status

for proposed testing we'll bake a core24-proposed snap to test with.

[ Where problems could occur ]
The proposed patch switches to using the vendored distutils in setuptools for the two imports in the ceph mgr modules that exhibit this issue - this is a minimal fix; codebase really needs refactoring to drop all use of distutils but that's outside of the scope on an SRU update.

Other distutils usage gets caught by the distutils_hack that setuptools uses to inject its vendored copy into the distutils module location.

[ Original Bug Report ]
When running microceph on a core24 base, the ceph-mgr has errors on enabling specific modules - volume and dashboard.

$ sudo microceph.ceph status
  cluster:
    id: 4e3ff87c-5320-4494-9d3c-42e69cc11398
    health: HEALTH_WARN
            Module 'volumes' has failed dependency: No module named 'distutils'
            OSD count 0 < osd_pool_default_size 3

  services:
    mon: 1 daemons, quorum joplin.glenview.com (age 5s)
    mgr: joplin.glenview.com(active, starting, since 0.942931s)
    osd: 0 osds: 0 up, 0 in

  data:
    pools: 0 pools, 0 pgs
    objects: 0 objects, 0 B
    usage: 0 B used, 0 B / 0 B avail
    pgs:

distutils as a standalone package was removed from noble - the ceph codebase makes quite a bit of use of distuils still which gets picked up by the distutils_hack in setuptools but not in the context of the mgr daemon.

[James Page (james-page)]

Note that ceph in oracular FTBFS due to a broken API in the snappy package which is pending a transition across Debian and Ubuntu.

[Steve Langasek (vorlon)]

The proposed test case here involves a snap, not the deb which is being SRUed. The test plan needs to provide a way for testing the .deb from the noble-proposed pocket.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 19.2.0~git20240301.4c76c50-0ubuntu7

---------------
ceph (19.2.0~git20240301.4c76c50-0ubuntu7) oracular; urgency=medium

  [ Luciano Lo Giudice]
  * d/control: Add python3-{packaging,ceph-common} to (Build-)Depends
    as these are undocumented/detected runtime dependencies in
    ceph-volume (LP: #2064717).

  [ James Page ]
  * d/cephadm.install: Install cephadmlib Python module which the
    cephadm script uses (LP: #2063456).
  * d/control: cephadm linux-any -> all.
  * d/gbp.conf: Update for ubuntu/oracular packaging branch.
  * d/control: Update Vcs-* to point to Launchpad for Ubuntu packaging.
  * d/p/mgr-distutils.patch: Directly use vendored distutils from
    setuptools for Python that runs in the mgr daemon (LP: #2065867).

 -- James Page <email address hidden> Thu, 23 May 2024 16:30:19 +0100

[James Page (james-page)]

@vorlon I'd gone with the snap approach to testing as the snap is built from the debs and provides the smallest footprint/simplest test case for this particular bug.

[Timo Aaltonen (tjaalton)]

An upload of ceph to noble-proposed has been rejected from the upload queue for the following reason: "will be amended".

[Timo Aaltonen (tjaalton)]

Hello James, or anyone else affected,

Accepted ceph into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/19.2.0~git20240301.4c76c50-0ubuntu6.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Page (james-page)]

Verification completed using a core24 based microceph built with proposed enabled:

# microceph.ceph -s
  cluster:
    id: 75f7e089-684c-43ad-9c32-1bbfd3d50f9a
    health: HEALTH_OK

  services:
    mon: 1 daemons, quorum sunbeam-allinone (age 2m)
    mgr: sunbeam-allinone(active, since 2m)
    osd: 3 osds: 3 up (since 8s), 3 in (since 10s)

  data:
    pools: 1 pools, 1 pgs
    objects: 2 objects, 577 KiB
    usage: 83 MiB used, 150 GiB / 150 GiB avail
    pgs: 1 active+clean