Comment 1 for bug 1899706

Thanks for the detailed report and reproduction steps!

I think this will need to be fixed in the prometheus charm. The code that injects the UUID[1] is part of the to_json function that's invoked by prometheus[2]. This is supposed to "ensure uniqueness" i.e. prevent multiple requests from clashing. A side effect of this is that when different units request the same job, the prometheus charm will generate multiple copies of that job.

The request ID that's used is part of the request-response pattern from charms.reactive. It is stored in relation data, and relation data is unit-scoped. Updating that pattern to allow re-use of a request ID across units would be doable but challenging. I strongly believe that, instead, this is a case where the request ID is being misused by the receiving side. Perhaps the relation ID should be used instead, which seems to be unique per application?

I am adding prometheus to this issue, but also leaving it open against kubernetes-master, since I would like another engineer from our team to look at this.

[1]: https://github.com/juju-solutions/interface-prometheus-manual/blob/3f775242c16d53243c993d7ba0c896169ad1639e/common.py#L36
[2]: https://git.launchpad.net/charm-prometheus2/tree/src/reactive/prometheus.py#n748