[2.4] Serialization errors in single region/rack, due to each worker update every 1 minute
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MAAS |
Fix Released
|
High
|
Blake Rouse |
Bug Description
A lot of serialization errors on a single region/rack (It is node with ID=1) (see below).
Also, while MAAS is deploying/
2018-03-16 13:04:51.661 UTC [9139] LOG: listening on IPv4 address "127.0.0.1", port 5432
2018-03-16 13:04:51.662 UTC [9139] LOG: listening on Unix socket "/var/run/
2018-03-16 13:04:51.674 UTC [9140] LOG: database system was shut down at 2018-03-16 13:04:50 UTC
2018-03-16 13:04:51.677 UTC [9139] LOG: database system is ready to accept connections
2018-03-16 13:04:52.252 UTC [9147] [unknown]@[unknown] LOG: incomplete startup packet
2018-03-16 13:06:01.782 UTC [9139] LOG: received fast shutdown request
2018-03-16 13:06:01.783 UTC [9139] LOG: aborting any active transactions
2018-03-16 13:06:01.786 UTC [9139] LOG: worker process: logical replication launcher (PID 9146) exited with exit code 1
2018-03-16 13:06:01.789 UTC [9141] LOG: shutting down
2018-03-16 13:06:01.796 UTC [9139] LOG: database system is shut down
2018-03-16 13:06:01.947 UTC [10575] LOG: listening on IPv4 address "127.0.0.1", port 5432
2018-03-16 13:06:01.948 UTC [10575] LOG: listening on Unix socket "/var/run/
2018-03-16 13:06:01.957 UTC [10576] LOG: database system was shut down at 2018-03-16 13:06:01 UTC
2018-03-16 13:06:01.960 UTC [10575] LOG: database system is ready to accept connections
2018-03-16 13:06:02.541 UTC [10583] [unknown]@[unknown] LOG: incomplete startup packet
2018-03-16 13:06:04.333 UTC [10760] maas@template1 FATAL: password authentication failed for user "maas"
2018-03-16 13:06:04.333 UTC [10760] maas@template1 DETAIL: Role "maas" does not exist.
Connection matched pg_hba.conf line 92: "host all all 127.0.0.1/32 md5"
2018-03-16 13:06:04.335 UTC [10761] maas@template1 FATAL: password authentication failed for user "maas"
2018-03-16 13:06:04.335 UTC [10761] maas@template1 DETAIL: Role "maas" does not exist.
Connection matched pg_hba.conf line 92: "host all all 127.0.0.1/32 md5"
2018-03-16 13:06:04.552 UTC [10838] postgres@template1 ERROR: role "maas" does not exist
2018-03-16 13:06:04.552 UTC [10838] postgres@template1 STATEMENT: ALTER USER "maas"
2018-03-16 13:06:04.983 UTC [10995] postgres@maasdb FATAL: database "maasdb" does not exist
2018-03-16 13:06:25.868 UTC [11089] maas@maasdb ERROR: relation "maasserver_
2018-03-16 13:06:25.868 UTC [11089] maas@maasdb STATEMENT: CREATE SEQUENCE maasserver_
MINVALUE 1 MAXVALUE 4294967295 CYCLE OWNED BY maasserver_
2018-03-16 13:06:45.048 UTC [11436] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.048 UTC [11436] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:06:45.062 UTC [11444] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.062 UTC [11444] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:06:45.204 UTC [11444] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.204 UTC [11444] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:06:45.229 UTC [11468] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.229 UTC [11468] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:06:45.844 UTC [11494] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.844 UTC [11494] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:06:45.995 UTC [11494] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:45.995 UTC [11494] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:06:46.402 UTC [11488] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:46.402 UTC [11488] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:06:49.124 UTC [11504] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:49.124 UTC [11504] maas@maasdb CONTEXT: SQL statement "UPDATE maasserver_node SET managing_process_id = region_process.id
WHERE maasserver_node.id = rack.id"
PL/pgSQL function sys_core_
SQL statement "SELECT sys_core_
PL/pgSQL function sys_core_
2018-03-16 13:06:49.124 UTC [11504] maas@maasdb STATEMENT: INSERT INTO "maasserver_
2018-03-16 13:06:49.264 UTC [11436] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:49.264 UTC [11436] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:06:49.265 UTC [11482] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:49.265 UTC [11482] maas@maasdb CONTEXT: SQL statement "UPDATE maasserver_node SET managing_process_id = region_process.id
WHERE maasserver_node.id = rack.id"
PL/pgSQL function sys_core_
SQL statement "SELECT sys_core_
PL/pgSQL function sys_core_
2018-03-16 13:06:49.265 UTC [11482] maas@maasdb STATEMENT: INSERT INTO "maasserver_
2018-03-16 13:06:52.416 UTC [11441] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:06:52.416 UTC [11441] maas@maasdb STATEMENT: DELETE FROM "maasserver_
2018-03-16 13:07:06.079 UTC [11534] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.085 UTC [11314] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.090 UTC [11468] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.092 UTC [11494] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.096 UTC [11310] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.097 UTC [11308] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.098 UTC [11306] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.100 UTC [11304] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.101 UTC [11302] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.101 UTC [11302] maas@maasdb LOG: unexpected EOF on client connection with an open transaction
2018-03-16 13:07:06.102 UTC [11300] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.103 UTC [11298] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.105 UTC [11296] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.106 UTC [11278] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.108 UTC [11513] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.109 UTC [11504] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.111 UTC [11488] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.113 UTC [11482] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.115 UTC [11479] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.116 UTC [11473] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.117 UTC [11453] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.119 UTC [11441] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.120 UTC [11439] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.121 UTC [11436] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.122 UTC [11524] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.124 UTC [11458] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:06.125 UTC [11444] maas@maasdb LOG: could not receive data from client: Connection reset by peer
2018-03-16 13:07:21.978 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:21.978 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:07:22.353 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:22.353 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:07:22.379 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:22.379 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:07:22.432 UTC [11888] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:22.432 UTC [11888] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:07:22.716 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:22.716 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:07:23.171 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:23.171 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:07:23.172 UTC [11896] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:23.172 UTC [11896] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:07:23.172 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:23.172 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:07:23.242 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:23.242 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:07:23.352 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:23.352 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:07:24.819 UTC [11852] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:24.819 UTC [11852] maas@maasdb CONTEXT: SQL statement "UPDATE maasserver_node SET managing_process_id = region_process.id
WHERE maasserver_node.id = rack.id"
PL/pgSQL function sys_core_
SQL statement "SELECT sys_core_
PL/pgSQL function sys_core_
2018-03-16 13:07:24.819 UTC [11852] maas@maasdb STATEMENT: INSERT INTO "maasserver_
2018-03-16 13:07:24.875 UTC [11877] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:24.875 UTC [11877] maas@maasdb CONTEXT: SQL statement "UPDATE maasserver_node SET managing_process_id = region_process.id
WHERE maasserver_node.id = rack.id"
PL/pgSQL function sys_core_
SQL statement "SELECT sys_core_
PL/pgSQL function sys_core_
2018-03-16 13:07:24.875 UTC [11877] maas@maasdb STATEMENT: INSERT INTO "maasserver_
2018-03-16 13:07:24.875 UTC [11863] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:07:24.875 UTC [11863] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:08:53.193 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:08:53.193 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:09:22.556 UTC [12129] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.556 UTC [12129] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:09:22.606 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.606 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:09:22.622 UTC [11877] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.622 UTC [11877] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:09:22.895 UTC [11877] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.895 UTC [11877] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:09:22.895 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.895 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:09:22.895 UTC [11887] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:22.895 UTC [11887] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:09:23.143 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:23.143 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:09:23.143 UTC [11877] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:23.143 UTC [11877] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:09:23.417 UTC [11875] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:09:23.417 UTC [11875] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:10:22.082 UTC [11873] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.082 UTC [11873] maas@maasdb STATEMENT: UPDATE "maasserver_
2018-03-16 13:10:22.377 UTC [11863] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.377 UTC [11863] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:10:22.377 UTC [11850] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.377 UTC [11850] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:10:22.574 UTC [11863] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.574 UTC [11863] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:10:22.574 UTC [11887] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.574 UTC [11887] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:10:22.660 UTC [11863] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:10:22.660 UTC [11863] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:19:22.015 UTC [14002] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:19:22.015 UTC [14002] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:22:22.085 UTC [14002] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:22:22.085 UTC [14002] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:22:22.387 UTC [14599] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:22:22.387 UTC [14599] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:22.462 UTC [14243] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:23:22.462 UTC [14243] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:22.501 UTC [14599] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:23:22.501 UTC [14599] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:22.957 UTC [14599] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:23:22.957 UTC [14599] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:22.957 UTC [15010] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:23:22.957 UTC [15010] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:23.451 UTC [14599] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:23:23.451 UTC [14599] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:23:27.889 UTC [14507] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:23:27.889 UTC [14507] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:24:22.212 UTC [15012] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:24:22.212 UTC [15012] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:24:22.502 UTC [15122] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:24:22.502 UTC [15122] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:24:22.798 UTC [15033] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:24:22.798 UTC [15033] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:25:22.654 UTC [15012] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:25:22.654 UTC [15012] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:25:22.655 UTC [15525] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:25:22.655 UTC [15525] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:25:23.256 UTC [15525] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:25:23.256 UTC [15525] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:25:23.256 UTC [15010] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:25:23.256 UTC [15010] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:25:23.687 UTC [15525] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:25:23.687 UTC [15525] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:26:01.325 UTC [15028] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:01.325 UTC [15028] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:26:01.326 UTC [15572] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:01.326 UTC [15572] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:26:02.450 UTC [15061] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:02.450 UTC [15061] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:26:21.719 UTC [15012] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:26:21.719 UTC [15012] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:26:22.139 UTC [15007] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:22.139 UTC [15007] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:26:22.523 UTC [14993] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:22.523 UTC [14993] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:26:22.876 UTC [15061] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:22.876 UTC [15061] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:26:32.026 UTC [14982] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:32.026 UTC [14982] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:26:33.151 UTC [15562] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:33.151 UTC [15562] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:26:34.251 UTC [14993] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:26:34.251 UTC [14993] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:27:22.015 UTC [15530] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:27:22.015 UTC [15530] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:27:22.159 UTC [15011] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:27:22.159 UTC [15011] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:27:22.499 UTC [14993] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:27:22.499 UTC [14993] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:27:22.761 UTC [15029] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:27:22.761 UTC [15029] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:27:28.345 UTC [15272] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:27:28.345 UTC [15272] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:27:28.364 UTC [15272] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:27:28.364 UTC [15272] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:27:28.416 UTC [15272] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:27:28.416 UTC [15272] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:27:28.874 UTC [15060] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:27:28.874 UTC [15060] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:27:59.667 UTC [15825] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:27:59.667 UTC [15825] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:12.196 UTC [15797] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:12.196 UTC [15797] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:13.433 UTC [15833] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:13.433 UTC [15833] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:14.597 UTC [15853] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:14.597 UTC [15853] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:14.725 UTC [15562] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:14.725 UTC [15562] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:15.913 UTC [15623] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:15.913 UTC [15623] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:16.992 UTC [15826] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:16.992 UTC [15826] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:20.302 UTC [15825] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:20.302 UTC [15825] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:20.303 UTC [15263] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:20.303 UTC [15263] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:21.471 UTC [15825] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:21.471 UTC [15825] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:21.491 UTC [15122] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:21.491 UTC [15122] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:22.724 UTC [15776] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:22.724 UTC [15776] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:22.821 UTC [15544] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:22.821 UTC [15544] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:30.061 UTC [15798] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:30.061 UTC [15798] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:41.207 UTC [15544] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:41.207 UTC [15544] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:50.259 UTC [15825] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:50.259 UTC [15825] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:28:50.703 UTC [15718] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:28:50.703 UTC [15718] maas@maasdb STATEMENT: UPDATE "metadataserver
2018-03-16 13:33:22.154 UTC [16086] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:33:22.154 UTC [16086] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:33:22.524 UTC [18522] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:33:22.524 UTC [18522] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:33:22.856 UTC [19747] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:33:22.856 UTC [19747] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:34:22.128 UTC [16086] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:34:22.128 UTC [16086] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:34:22.933 UTC [18522] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:34:22.933 UTC [18522] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:34:24.066 UTC [19747] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:34:24.066 UTC [19747] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:35:22.115 UTC [19651] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:35:22.115 UTC [19651] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:35:22.422 UTC [18522] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:35:22.422 UTC [18522] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:35:22.745 UTC [19749] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:35:22.745 UTC [19749] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:22.860 UTC [19427] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:22.860 UTC [19427] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:22.860 UTC [21867] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:22.860 UTC [21867] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:22.961 UTC [19673] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:22.961 UTC [19673] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:23.314 UTC [19427] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:23.314 UTC [19427] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:23.314 UTC [19673] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:23.314 UTC [19673] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:23.834 UTC [19673] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:36:23.834 UTC [19673] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:36:24.917 UTC [21235] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:36:24.917 UTC [21235] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:37:22.229 UTC [19428] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:37:22.229 UTC [19428] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:37:22.529 UTC [21867] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:37:22.529 UTC [21867] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:37:22.788 UTC [19673] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:37:22.788 UTC [19673] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:22.772 UTC [21866] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:22.772 UTC [21866] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:22.772 UTC [21867] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:22.772 UTC [21867] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:22.812 UTC [19749] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:22.812 UTC [19749] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:23.229 UTC [21867] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:23.229 UTC [21867] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:23.229 UTC [19749] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:23.229 UTC [19749] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:38:23.637 UTC [21867] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:38:23.637 UTC [21867] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:39:22.439 UTC [21235] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:39:22.439 UTC [21235] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:39:22.636 UTC [19762] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:39:22.636 UTC [19762] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:40:21.564 UTC [19671] maas@maasdb ERROR: could not obtain lock on row in relation "metadataserver
2018-03-16 13:40:21.564 UTC [19671] maas@maasdb STATEMENT: SELECT "metadataserver
2018-03-16 13:40:22.772 UTC [25313] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:40:22.772 UTC [25313] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:40:22.772 UTC [24477] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:40:22.772 UTC [24477] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:40:23.323 UTC [25313] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:40:23.323 UTC [25313] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:40:23.323 UTC [27318] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:40:23.323 UTC [27318] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:40:23.711 UTC [25313] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:40:23.711 UTC [25313] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:41:22.245 UTC [22482] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:41:22.245 UTC [22482] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:41:22.464 UTC [26117] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:41:22.464 UTC [26117] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:41:22.740 UTC [27396] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:41:22.740 UTC [27396] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:42:22.529 UTC [25313] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:42:22.529 UTC [25313] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
2018-03-16 13:42:22.808 UTC [25238] maas@maasdb ERROR: could not serialize access due to concurrent update
2018-03-16 13:42:22.808 UTC [25238] maas@maasdb STATEMENT: UPDATE "maasserver_node" SET "updated" = '2018-03-
Related branches
- MAAS Lander: Approve
- Alberto Donato (community): Approve
-
Diff: 4220 lines (+1137/-1712)16 files modifiedsrc/maasserver/eventloop.py (+12/-23)
src/maasserver/ipc.py (+521/-21)
src/maasserver/models/node.py (+4/-3)
src/maasserver/rack_controller.py (+5/-7)
src/maasserver/regiondservices/service_monitor_service.py (+6/-7)
src/maasserver/regiondservices/tests/test_service_monitor_service.py (+12/-24)
src/maasserver/rpc/regionservice.py (+37/-602)
src/maasserver/rpc/tests/test_regionservice.py (+74/-830)
src/maasserver/server.py (+4/-1)
src/maasserver/tests/test_eventloop.py (+7/-21)
src/maasserver/tests/test_ipc.py (+352/-21)
src/maasserver/tests/test_plugin.py (+1/-3)
src/maasserver/tests/test_rack_controller.py (+39/-44)
src/maasserver/triggers/tests/test_websocket_listener.py (+2/-1)
src/maasserver/views/rpc.py (+41/-40)
src/maasserver/views/tests/test_rpc.py (+20/-64)
Changed in maas: | |
milestone: | none → 2.4.0beta1 |
tags: | added: performance |
Changed in maas: | |
importance: | Undecided → High |
status: | New → Triaged |
assignee: | nobody → Blake Rouse (blake-rouse) |
description: | updated |
summary: |
- [2.4] Serialization errors in single region/rack + [2.4] Serialization errors in single region/rack, every 1 minute |
summary: |
- [2.4] Serialization errors in single region/rack, every 1 minute + [2.4] Serialization errors in single region/rack, due to each worker + update every 1 minute |
Changed in maas: | |
status: | Triaged → In Progress |
Changed in maas: | |
milestone: | 2.4.0beta1 → 2.4.0beta2 |
Changed in maas: | |
status: | In Progress → Fix Committed |
milestone: | 2.4.0beta2 → 2.4.0beta1 |
Changed in maas: | |
status: | Fix Committed → Fix Released |
Other errors happen every 1 minute: https:/ /pastebin. ubuntu. com/p/6zB53kWXJ k/