Grafana Charm

[CEPH] Grafana misses performance-related info from OSDs

Bug #1899908 reported by Pedro Guimarães on 2020-10-15

This bug report is a duplicate of: Bug #1867100: Some metrics are not collected with Ceph Nautilus. Edit Remove

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Grafana Charm	New	Undecided	Unassigned

Bug Description

Using bionic-ussuri and Ceph Octopus.
My deployment runs with ceph_exporter snap v2.0.0

Also saw that problem on Focal distro.

Deployed ceph with the following bundle: https://pastebin.ubuntu.com/p/NxbTDR3ncf/

After I restart prometheus because of LP #1891942,grafana comes up with only Ceph Pools dashboard.
On grafana charm logs, I can see:
2020-10-15 07:51:19 DEBUG juju-log Skipping Dashboard Template: CephCluster.json.j2 missing 2 metrics.Missing: ceph_osd_perf_commit_latency_seconds, ceph_osd_perf_apply_latency_seconds

Likewise for CephOSD dashboard:
2020-10-15 07:51:18 DEBUG juju-log Skipping Dashboard Template: CephOSD.json.j2 missing 2 metrics.Missing: ceph_osd_perf_commit_latency_seconds, ceph_osd_perf_apply_latency_seconds

I can see there was another bug reported with similar issue:
https://bugs.launchpad.net/charm-grafana/+bug/1796628
Although the bug is outdated, comments are pretty recent, from a couple of months ago.

So, looking into the missing metrics a little closer, I am not sure where this is supposed to be built, either collected directly from Ceph or calculated with existing metrics.

I cannot find them on ceph_exporter source code.
Only reference on upstream ceph_exporter I can see is on: https://github.com/digitalocean/ceph_exporter/search?q=0327480c545f006a546c3279c4d7605d2f90a0bf&type=commits
Which only adds some tests to check for those metrics but no logic to collect or calculate them.

The commit that added those metrics to grafana charm is:
https://git.launchpad.net/charm-grafana/commit/?id=37d881043d25588f2f6541aea66db1ecb003d6b4

See original description