The way how Trove is spawning Galera cluster with MariaDB 10.1 and systemd is not working anymore.
In MariaDB 10.1 the <email address hidden>[1] systemd unit file is deprecated.
MariaDB recommends to use "galera_new_cluster"[2] command to bootstrap a Galera cluster.
/var/log/trove/guestagent.log
(trove.guestagent.datastore.galera_common.service): 2018-02-05 17:22:32,519 DEBUG bootstrap the instance? : True
(trove.guestagent.datastore.galera_common.service): 2018-02-05 17:22:32,520 INFO Bootstraping cluster.
(oslo_concurrency.processutils): 2018-02-05 17:22:32,521 DEBUG Running cmd (subprocess): sudo systemctl start <email address hidden>
(oslo_concurrency.processutils): 2018-02-05 17:22:32,646 DEBUG CMD "sudo systemctl start <email address hidden>" returned: 1 in 0.125s
(oslo_concurrency.processutils): 2018-02-05 17:22:32,647 DEBUG u'sudo systemctl start <email address hidden>' failed. Not Retrying.
(trove.guestagent.datastore.galera_common.manager): 2018-02-05 17:22:32,648 ERROR Cluster installation failed.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/trove/guestagent/datastore/galera_common/manager.py", line 55, in install_cluster
replication_user, cluster_configuration, bootstrap)
File "/usr/local/lib/python2.7/dist-packages/trove/guestagent/datastore/galera_common/service.py", line 72, in install_cluster
self._bootstrap_cluster(timeout=CONF.restore_usage_timeout)
File "/usr/local/lib/python2.7/dist-packages/trove/guestagent/datastore/galera_common/service.py", line 53, in _bootstrap_cluster
shell=True, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/trove/common/utils.py", line 237, in execute_with_timeout
return execute(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
cmd=sanitized_cmd)
ProcessExecutionError: Unexpected error while running command.
Command: sudo systemctl start <email address hidden>
Exit code: 1
Stdout: u''
Stderr: u'Job for <email address hidden> failed because the control process exited with error code. See "systemctl status <email address hidden>" and "journalctl -xe" for details.\n'
(trove.guestagent.datastore.service): 2018-02-05 17:22:32,652 DEBUG Prepare has not completed yet, skipping heartbeat.
# systemctl status mariadb\@bootstrap.service
● <email address hidden> - MariaDB 10.1.30 database server (multi-instance)
Loaded: loaded (/lib/systemd/system/mariadb@.service; disabled; vendor preset: enabled)
Drop-In: /<email address hidden>
└─use_galera_new_cluster.conf
Active: failed (Result: exit-code) since Mon 2018-02-05 17:22:32 UTC; 3min 13s ago
Docs: man:mysqld(8)
https://mariadb.com/kb/en/library/systemd/
Process: 1955 ExecStart=/usr/bin/false (code=exited, status=1/FAILURE)
Process: 1951 ExecStart=/usr/bin/echo Please use galera_new_cluster to start the mariadb service with --wsrep-new-cluster (code=exited, status=0/SUCCESS)
Process: 1938 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery --defaults-file=/etc/mysql/conf.d/my%I.cnf`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION%I=$VAR ||
Process: 1932 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION%I (code=exited, status=0/SUCCESS)
Main PID: 1955 (code=exited, status=1/FAILURE)
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: Starting MariaDB 10.1.30 database server (multi-instance)...
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: Could not open required defaults file: /etc/mysql/conf.d/mybootstrap.cnf
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: Fatal error in defaults handling. Program aborted
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: WARNING: Defaults file '/etc/mysql/conf.d/mybootstrap.cnf' not found!
Feb 05 17:22:32 mariadb-member-1517851079 echo[1951]: Please use galera_new_cluster to start the mariadb service with --wsrep-new-cluster
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Main process exited, code=exited, status=1/FAILURE
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: Failed to start MariaDB 10.1.30 database server (multi-instance).
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Unit entered failed state.
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Failed with result 'exit-code'.
...skipping...
● <email address hidden> - MariaDB 10.1.30 database server (multi-instance)
Loaded: loaded (/lib/systemd/system/mariadb@.service; disabled; vendor preset: enabled)
Drop-In: /<email address hidden>
└─use_galera_new_cluster.conf
Active: failed (Result: exit-code) since Mon 2018-02-05 17:22:32 UTC; 3min 13s ago
Docs: man:mysqld(8)
https://mariadb.com/kb/en/library/systemd/
Process: 1955 ExecStart=/usr/bin/false (code=exited, status=1/FAILURE)
Process: 1951 ExecStart=/usr/bin/echo Please use galera_new_cluster to start the mariadb service with --wsrep-new-cluster (code=exited, status=0/SUCCESS)
Process: 1938 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery --defaults-file=/etc/mysql/conf.d/my%I.cnf`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION%I=$VAR ||
Process: 1932 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION%I (code=exited, status=0/SUCCESS)
Main PID: 1955 (code=exited, status=1/FAILURE)
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: Starting MariaDB 10.1.30 database server (multi-instance)...
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: Could not open required defaults file: /etc/mysql/conf.d/mybootstrap.cnf
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: Fatal error in defaults handling. Program aborted
Feb 05 17:22:32 mariadb-member-1517851079 sh[1938]: WARNING: Defaults file '/etc/mysql/conf.d/mybootstrap.cnf' not found!
Feb 05 17:22:32 mariadb-member-1517851079 echo[1951]: Please use galera_new_cluster to start the mariadb service with --wsrep-new-cluster
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Main process exited, code=exited, status=1/FAILURE
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: Failed to start MariaDB 10.1.30 database server (multi-instance).
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Unit entered failed state.
Feb 05 17:22:32 mariadb-member-1517851079 systemd[1]: <email address hidden>: Failed with result 'exit-code'.
[1]https://github.com/MariaDB/server/blob/10.1/support-files/use_galera_new_cluster.conf
[2]https://mariadb.com/kb/en/library/getting-started-with-mariadb-galera-cluster/#bootstrapping-a-new-cluster
I don't think zuul test the clustering.