Fuel for OpenStack

Bug #1352945
Comment #2

Comment 2 for bug 1352945

Revision history for this message

Sergii Golovatiuk (sgolovatiuk) wrote on 2014-08-05:

According to logs,

MySQL with Galera was not able to Initiate transfer on node-2

Here is the log from node-1
2014-08-03T17:11:17.120625+01:00 info: INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left
2014-08-03T17:11:25.101678+01:00 info: INFO: MySQL started
2014-08-03T17:11:25.245771+01:00 info: INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:968
2014-08-03T17:12:25.302240+01:00 info: INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:998
2014-08-03T17:13:25.363481+01:00 info: INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1028
2014-08-03T17:14:27.217568+01:00 info: INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1059

It seems to be good

Logs from node-4

2014-08-03T17:25:22.330226+01:00 info: INFO: MySQL started
2014-08-03T17:25:22.448313+01:00 err: ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:25:22.449267+01:00 err: ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:27:09.164154+01:00 err: ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:27:09.164154+01:00 err: ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:27:23.411150+01:00 info: INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1430

Logs from node-2
014-08-03T17:25:14.335806+01:00 info: INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left
2014-08-03T17:25:19.629571+01:00 info: INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 1 retries left
2014-08-03T17:25:27.391246+01:00 info: INFO: MySQL started
2014-08-03T17:25:32.156082+01:00 err: ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:25:32.157074+01:00 err: ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:26:32.187558+01:00 err: ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:26:32.198079+01:00 err: ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:28:48.345912+01:00 err: ERROR: GTID have wrong format: :
2014-08-03T17:28:48.345912+01:00 err: ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:28:48.345912+01:00 info: INFO: MySQL failed to stop after 15s using SIGTERM. Trying SIGKILL...
2014-08-03T17:28:48.345912+01:00 info: INFO: MySQL stopped

Since then there were attempts to transfer data but all of them were not successful.

I deleted everything from /var/log/mysql/*, generated a new structure and tried to start mysqld manually. It was not successful. It looks like it's a sporadic issue on Ubuntu and could be related to MySQL or galera packages themselves.

According to logs,

MySQL with Galera was not able to Initiate transfer on node-2

Here is the log from node-1
2014-08-03T17:11:17.120625+01:00 info:  INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left
2014-08-03T17:11:25.101678+01:00 info:  INFO: MySQL started
2014-08-03T17:11:25.245771+01:00 info:  INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:968
2014-08-03T17:12:25.302240+01:00 info:  INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:998
2014-08-03T17:13:25.363481+01:00 info:  INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1028
2014-08-03T17:14:27.217568+01:00 info:  INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1059

It seems to be good

Logs from node-4

2014-08-03T17:25:22.330226+01:00 info:  INFO: MySQL started
2014-08-03T17:25:22.448313+01:00 err:  ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:25:22.449267+01:00 err:  ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:27:09.164154+01:00 err:  ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:27:09.164154+01:00 err:  ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:27:23.411150+01:00 info:  INFO: GTID OK: ef86fc0c-1b26-11e4-8e83-eb36c97e0a94:1430

Logs from node-2
014-08-03T17:25:14.335806+01:00 info:  INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 2 retries left
2014-08-03T17:25:19.629571+01:00 info:  INFO: PIDFile /var/run/mysqld/mysqld.pid of MySQL server not found. Sleeping for 5 seconds. 1 retries left
2014-08-03T17:25:27.391246+01:00 info:  INFO: MySQL started
2014-08-03T17:25:32.156082+01:00 err:  ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:25:32.157074+01:00 err:  ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:26:32.187558+01:00 err:  ERROR: GTID have wrong format: wsrep_local_state_uuid:18446744073709551615
2014-08-03T17:26:32.198079+01:00 err:  ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:28:48.345912+01:00 err:  ERROR: GTID have wrong format: :
2014-08-03T17:28:48.345912+01:00 err:  ERROR: Wrong GTID, not updating gtid attribute
2014-08-03T17:28:48.345912+01:00 info:  INFO: MySQL failed to stop after 15s using SIGTERM. Trying SIGKILL...
2014-08-03T17:28:48.345912+01:00 info:  INFO: MySQL stopped

Since then there were  attempts to transfer data but all of them were not successful.

I deleted everything from /var/log/mysql/*, generated a new structure and tried to start mysqld manually. It was not successful.  It looks like it's a sporadic issue on Ubuntu and could be related to MySQL or galera packages themselves.