Implement database stats collection

[Paul Everitt]

We need to collect time series data on queries in production (I/O, CPU, etc.) We can go back to using DataDog, or we can integrate with whatever FC might offer.

Evaluate the choices based on the measurements you're looking for, and let me know if there's anything I need to do.

[Jim Fulton (jim-zope)]

FWIW, the stats I suggested the other day were more low-level that this, I was interested in looking at IO statistics, to understand whether SSD would help.

Trying to view this as a time series is awkward, because:

- different queries
- mixing relstorage queries with regular search.

To understand pain points, I think it would be better to do snapshot analysis (iow, look at request stats over some period of time) to identify problem requests and to look at postgres query stats, which we may need to turn on, to understand expensive sql queries.

[Paul Everitt (paul-agendaless)]

I'll move this to April but make it a low priority.

[Paul Everitt (paul-agendaless)]

Still interested in this? Or do you feel like we know enough that this isn't worth using the hours on?

[Jim Fulton (jim-zope)]

I don't think we need an issue for this.