Add a cronjob deployed by the charm to cleanup metrics for deleted resources
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Gnocchi Charm |
Triaged
|
Wishlist
|
Unassigned |
Bug Description
We are in the situation where the metric db has more than 40 million records, a lot of those for deleted resources. This [0] states that nothing will be done to automatically cleanup those, so we need to use the scripts from [1] to do so. Running the instance related cleanup script takes about 3 seconds per record to compare and mark as delete to metrics, so this is about 4 years of sql execution for the current situation.
We need a cronjob to be implemented in the charm and deployed by the charm (also config option to enable it on specific interval or disable it) that runs the queries from [1] on regular basis to not allow this situation to recur.
[0] https:/
[1] https:/
I agree that something along these lines is useful, but I was curious about this bit:
> Running the instance related cleanup script takes about 3 seconds per record to compare and mark as delete to metrics, so this is about 4 years of sql execution for the current situation.
That seems a little slow. Looking at the sql, I'm wondering if joins might perform better that sub-queries (as the mysql optimiser is a bit more mature for joins)? However, it's been a while since I brushed up on my sql!