Comment 10 for bug 1832105

Here's some help for others facing this:

If the ceph dashboard was working in before upgrading to disco (which
killed it in a regression), then your hope to get it working via upgrade
to nautilus (owing to 'fixed-released' advertising in the bug report)
was to move to ceph v14/nautlius available in ubuntu-eoan.

After 'do-release-upgrade --devel' to eoan / ceph nautilus on every
system running ceph do:

systemctl status ceph<esc>  and make sure there is only one entry there
for every osd/mon/mgr/mds.  On my system there were entries there with
the hostname and with the hostname.domainname as well.

There are a number of other instructions involved in getting nautilus
running, see them here:

http://docs.ceph.com/docs/nautilus/releases/nautilus/

Also one of the osd's that was not managed by LVM was ignored and not
started.  I 'replaced it' with itself and it started backfilling normally.

On the systems meant to run the dashboard, this is now necessary:

apt install ceph-mgr-dashboard

    The following will tell you 'the module is already enabled'.

And when you think you're done and ready to log in ... the screen
accepts your password then does nothing further other than redisplay the
login screen.  If you put in the wrong password, it tells you. The
correct password does nothing.  So,  __on every instance of ceph mgr
even the ones you are not using __ you have to

ceph mgr module disable dashboard

then edit

/usr/share/ceph/mgr/dashboard/services/access_control.py

and change line 186 from self.lastUpdate = int(time.mktime(time.gmtime()))

to

self.lastUpdate = int(time.time())

Be sure to use spaces and not tabs.

Then

ceph mgr module enable dashboard

ceph dashboard ac-user-set-password admin <new password>

And then, you get to where you were before the update to disco with a
working dashboard.  Hopefully this saved you a day or two.

I'm no longer able/interested to test whether mimic's dashboard works in
disco, sorry.   If I'd somehow known an official release would break a
major user facing function on something as central to operations as ceph
I would have skipped disco entirely and waited for eoan.

On 7/12/19 8:33 AM, James Page wrote:
> Sorry wrong PPA:
>
> https://launchpad.net/~ci-train-ppa-
> service/+archive/ubuntu/3534/+packages
>
> ** Description changed:
>
> - If Ubuntu is really committed to ceph as I think I've been reading:
> - Notice the ceph dashboard went entirely broken in a major regression of
> - the disco upgrade. It won't load at all in 13.2.4+dfsg1-0ubuntu2.
> + [Impact]
> + The ceph-mgr daemon is unable to load additional module due to a new check in cython >= 0.29. This limits the function of the manager.
> +
> +
> + [Test Case]
> + Deploy ceph
> + Check /var/log/ceph/ceph-mgr.`hostname`.log
> + Errors about loading rados module in subprocesses will be seen.
> +
> + [Regression Potential]
> + The fix from upstream actually just works around this issue by overriding the check that cython does; the code works in a subprocess when loaded multiple times. Regression potential low; cython may produce a longer term fix which means we can drop this patch.
> +
> + [Original Bug Report]
> + If Ubuntu is really committed to ceph as I think I've been reading: Notice the ceph dashboard went entirely broken in a major regression of the disco upgrade. It won't load at all in 13.2.4+dfsg1-0ubuntu2.
>
> The detail is ceph-mgr (and lots of ceph) relied on a non-feature in
> cython that went away in cython v29, to do with sub-interpreters. The
> ceph folks responded with a hack/workaround to avoid the bug being
> noticed, and a requirement of the package for an earlier version of
> cython. This was done some weeks and months ago. Actually fixing the
> problem is a major project the ceph maintainers are struggling to
> engage, perhaps waiting for later versions of cython to provide a
> different way forward.
>
> However, as of today, on disco ths error message remains:
>
> Module 'dashboard' has failed dependency: Interpreter change detected -
> this module can only be loaded into one interpreter per process.
>
> The ceph primary development platform is Debian, on which the workaround
> has been available for some time.
>
> However in our ubuntu case, a major feature of a core packge (web
> health/monitoring/config interface of a distributed file system), was
> allowed to both ship broken and remain so for a long time, even through
> today.
>
> I urge quick attention to the necessary backports.
> https://github.com/ceph/ceph/pull/25585
> http://tracker.ceph.com/issues/38788
> http://tracker.ceph.com/issues/37472
>