The service_statuses table in Designate database is running full of records in our deployment:
MariaDB [designate]> select count(*) from service_statuses;
+----------+
| count(*) |
+----------+
| 24474342 |
+----------+
1 row in set (7 min 19.09 sec)
We got millions of rows in just couple of month. The problem is that the same services running on the same hosts create new record (instead of updating existing) during status report to Designate.
This is how it looks in DB:
It is not exactly clear to me why the id is needed to identify the service. The 'hostname' and 'service_name' seem to be enough - it is rather unlikely that there are two api services are running on the same host..
So, I've removed the id part:
if service_status.obj_attr_is_set('id'): criterion["id"] = service_status.id
and cleaned up the service_statuses table, now it looks fine:
Hi,
The service_statuses table in Designate database is running full of records in our deployment:
MariaDB [designate]> select count(*) from service_statuses;
+----------+
| count(*) |
+----------+
| 24474342 |
+----------+
1 row in set (7 min 19.09 sec)
We got millions of rows in just couple of month. The problem is that the same services running on the same hosts create new record (instead of updating existing) during status report to Designate.
This is how it looks in DB:
MariaDB [designate]> select * from service_statuses; ------- ------- ------- ------- +------ ------- ------- -+----- ------- ------- --+---- ------- ---+--- ------- ------- ------- ------- -----+- ------- ------- ------+ ------- -+----- --+---- ------- ---+ ------- ------- ------- ------- +------ ------- ------- -+----- ------- ------- --+---- ------- ---+--- ------- ------- ------- ------- -----+- ------- ------- ------+ ------- -+----- --+---- ------- ---+ 5995cb0338841bd 50 | 2018-05-02 12:06:03 | NULL | producer | designate- producer- 855855776- cr8d9 | 2018-05-02 12:06:03 | UP | {} | {} | d97066eba619490 a3 | 2018-05-02 12:05:14 | NULL | api | designate- api-2042646259- 6090v | 2018-05-02 12:05:13 | UP | {} | {} | ea1931824357048 2c | 2018-05-02 12:05:48 | NULL | producer | designate- producer- 855855776- cr8d9 | 2018-05-02 12:05:48 | UP | {} | {} | 5b052680e5e2f40 ae | 2018-05-02 12:05:59 | NULL | api | designate- api-2042646259- 6090v | 2018-05-02 12:05:58 | UP | {} | {} | 091b981b0c32d04 f7 | 2018-05-02 12:05:41 | NULL | worker | designate- worker- 3446544- 7fzqx | 2018-05-02 12:05:35 | UP | {} | {} | 096b54005675e80 11 | 2018-05-02 12:05:25 | NULL | mdns | designate- mdns-4198843580 -lw6s2 | 2018-05-02 12:05:25 | UP | {} | {} | 8bc191e49923d13 ba | 2018-05-02 12:05:58 | NULL | producer | designate- producer- 855855776- cr8d9 | 2018-05-02 12:05:58 | UP | {} | {} | 58fa14750d0c2a1 81 | 2018-05-02 12:05:18 | NULL | producer | designate- producer- 855855776- cr8d9 | 2018-05-02 12:05:18 | UP | {} | {} | 09f65f4b8cf0a72 b5 | 2018-05-02 12:05:25 | NULL | api | designate- api-2042646259- drzws | 2018-05-02 12:05:25 | UP | {} | {} | 98159f104b13a06 6b | 2018-05-02 12:06:00 | NULL | mdns | designate- mdns-4198843580 -lw6s2 | 2018-05-02 12:06:00 | UP | {} | {} |
+------
| id | created_at | updated_at | service_name | hostname | heartbeated_at | status | stats | capabilities |
+------
| 0dde2b5f228549d
| 0e311d3000d8403
| 168448cd97cd428
| 1685d7f80d8c4f7
| 192275eb33854b4
| 1e465011f21f47f
| 22e0ab87b3cd422
| 284d6b6bf53f473
| 3617ac740b9945c
| 3880157829f9463
The respective part of the code is: /github. com/openstack/ designate/ blob/master/ designate/ central/ service. py#L2879- L2898
https:/
It is not exactly clear to me why the id is needed to identify the service. The 'hostname' and 'service_name' seem to be enough - it is rather unlikely that there are two api services are running on the same host..
So, I've removed the id part:
if service_ status. obj_attr_ is_set( 'id'):
criterion[ "id"] = service_status.id
and cleaned up the service_statuses table, now it looks fine:
MariaDB [designate]> select * from service_statuses; ------- ------- ------- ------- +------ ------- ------- -+----- ------- ------- --+---- ------- ---+--- ------- ------- ------- ------- ------+ ------- ------- ------- +------ --+---- ---+--- ------- ----+ ------- ------- ------- ------- +------ ------- ------- -+----- ------- ------- --+---- ------- ---+--- ------- ------- ------- ------- ------+ ------- ------- ------- +------ --+---- ---+--- ------- ----+ f8a9522ebd62fa6 29 | 2018-05-02 13:03:21 | 2018-05-02 13:03:37 | worker | designate- worker- 2851226431- 9274c | 2018-05-02 13:03:36 | UP | {} | {} | 99b78e90c1b88ac f8 | 2018-05-02 13:03:23 | 2018-05-02 13:03:38 | api | designate- api-1699569558- 9rvg6 | 2018-05-02 13:03:38 | UP | {} | {} | b86277e21af3d73 9f | 2018-05-02 13:03:18 | 2018-05-02 13:03:38 | producer | designate- producer- 3892623035- 6vwt0 | 2018-05-02 13:03:38 | UP | {} | {} | dbfa83b9ed0c2b4 38 | 2018-05-02 13:03:19 | 2018-05-02 13:03:39 | api | designate- api-1699569558- r018m | 2018-05-02 13:03:39 | UP | {} | {} | 2949ced39d76e29 58 | 2018-05-02 13:03:20 | 2018-05-02 13:03:40 | central | designate- central- 2632925199- wwj7j | 2018-05-02 13:03:40 | UP | {} | {} | 090dc74a1dc6869 21 | 2018-05-02 13:03:21 | 2018-05-02 13:03:41 | mdns | designate- mdns-4162010419 -pvclv | 2018-05-02 13:03:41 | UP | {} | {} | ------- ------- ------- ------- +------ ------- ------- -+----- ------- ------- --+---- ------- ---+--- ------- ------- ------- ------- ------+ ------- ------- ------- +------ --+---- ---+--- ------- ----+
+------
| id | created_at | updated_at | service_name | hostname | heartbeated_at | status | stats | capabilities |
+------
| 1e89d10b0ddb459
| 8d29d50270ee489
| 9602ccf55d204c7
| cec705e082c542c
| ddd48f13b67448d
| ef0ae921a78d4ac
+------
6 rows in set (0.00 sec)
Our Designate version is Pike, but the code seems to be the same in master branch.