[ERROR] WSREP: galerautils/src/gu_init.c:gu_init():20: GU_PAGE_SIZE(4096) does not maptch real system page size(65536) on PPC64EL

[Larry Michel]

Running into config-changed hook failures on a PowerNV system.

  mysql:
    charm: cs:xenial/percona-cluster-2
    series: xenial
    os: ubuntu
    charm-origin: jujucharms
    charm-name: percona-cluster
    charm-rev: 2
    exposed: false
    application-status:
      current: error
      message: 'hook failed: "config-changed"'
      since: 13 Oct 2016 18:35:40Z
    relations:
      cluster:
      - mysql
      shared-db:
      - cinder
      - glance
      - keystone
      - neutron-api
      - nova-cloud-controller

From Unit logs:

2016-10-13 17:01:11 DEBUG worker.uniter.jujuc server.go:178 hook context id "mysql/0-config-changed-9017096756540705596"; dir "/var/lib/juju/agents/unit-mysql-0/charm"
2016-10-13 17:01:11 INFO juju-log Writing file /etc/mysql/percona-xtradb-cluster.conf.d/mysqld.cnf root:root 444
2016-10-13 17:01:13 INFO config-changed Unknown operation bootstrap-pxc.
2016-10-13 17:01:13 INFO config-changed * Bootstrapping Percona XtraDB Cluster database server mysqld
2016-10-13 17:01:23 INFO config-changed * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
2016-10-13 17:01:23 INFO config-changed ...fail!
2016-10-13 17:01:23 DEBUG worker.uniter.jujuc server.go:178 hook context id "mysql/0-config-changed-9017096756540705596"; dir "/var/lib/juju/agents/unit-mysql-0/charm"
2016-10-13 17:01:23 DEBUG worker.uniter.jujuc server.go:178 hook context id "mysql/0-config-changed-9017096756540705596"; dir "/var/lib/juju/agents/unit-mysql-0/charm"
2016-10-13 17:01:23 ERROR juju-log Bootstrap PXC failed: Command '['service', 'mysql', 'bootstrap-pxc']' returned non-zero exit status 1
2016-10-13 17:01:23 INFO config-changed Traceback (most recent call last):
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/config-changed", line 722, in <module>
2016-10-13 17:01:23 INFO config-changed main()
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/config-changed", line 715, in main
2016-10-13 17:01:23 INFO config-changed hooks.execute(sys.argv)
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/charmhelpers/core/hookenv.py", line 715, in execute
2016-10-13 17:01:23 INFO config-changed self._hooks[hook_name]()
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
2016-10-13 17:01:23 INFO config-changed return f(*args, **kwargs)
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/config-changed", line 279, in config_changed
2016-10-13 17:01:23 INFO config-changed bootstrap=not bootstrapped)
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/config-changed", line 184, in render_config_restart_on_changed
2016-10-13 17:01:23 INFO config-changed bootstrap_pxc()
2016-10-13 17:01:23 INFO config-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/percona_utils.py", line 384, in bootstrap_pxc
2016-10-13 17:01:23 INFO config-changed raise Exception(error_msg)
2016-10-13 17:01:23 INFO config-changed Exception: Bootstrap PXC failed: Command '['service', 'mysql', 'bootstrap-pxc']' returned non-zero exit status 1
2016-10-13 17:01:23 ERROR juju.worker.uniter.operation runhook.go:107 hook "config-changed" failed: exit status 1
2016-10-13 17:01:23 DEBUG juju.worker.uniter.operation executor.go:84 lock released
2016-10-13 17:01:23 INFO juju.worker.uniter resolver.go:100 awaiting error resolution for "config-changed" hook
2016-10-13 17:01:23 DEBUG juju.worker.uniter agent.go:17 [AGENT-STATUS] error: hook failed: "config-changed"
2016-10-13 17:01:23 INFO juju.worker.uniter resolver.go:100 awaiting error resolution for "config-changed" hook

And from Mysql logs:

161013 17:01:13 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
161013 17:01:13 mysqld_safe Starting mysqld daemon with databases from /var/lib/percona-xtradb-cluster
161013 17:01:13 mysqld_safe Skipping wsrep-recover for empty datadir: /var/lib/percona-xtradb-cluster
161013 17:01:13 mysqld_safe Assigning 00000000-0000-0000-0000-000000000000:-1 to wsrep_start_position
2016-10-13 17:01:14 0 [Note] WSREP: wsrep_start_position var submitted: '00000000-0000-0000-0000-000000000000:-1'
2016-10-13 17:01:14 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2016-10-13 17:01:14 13852 [Warning] Buffered warning: Changed limits: max_open_files: 65536 (requested 100000)

2016-10-13 17:01:14 13852 [Note] WSREP: Read nil XID from storage engines, skipping position init
2016-10-13 17:01:14 13852 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
2016-10-13 17:01:14 13852 [Note] WSREP: wsrep_load(): Galera 3.8(rXXXX) by Codership Oy <email address hidden> loaded successfully.
2016-10-13 17:01:14 13852 [ERROR] WSREP: galerautils/src/gu_init.c:gu_init():20: GU_PAGE_SIZE(4096) does not maptch real system page size(65536)
2016-10-13 17:01:14 13852 [Note] WSREP: /usr/sbin/mysqld: Terminated.
Aborted (core dumped)
161013 17:01:15 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
161013 17:01:31 mysqld_safe Starting mysqld daemon with databases from /var/lib/percona-xtradb-cluster
161013 17:01:31 mysqld_safe Skipping wsrep-recover for empty datadir: /var/lib/percona-xtradb-cluster
161013 17:01:31 mysqld_safe Assigning 00000000-0000-0000-0000-000000000000:-1 to wsrep_start_position
2016-10-13 17:01:31 0 [Note] WSREP: wsrep_start_position var submitted: '00000000-0000-0000-0000-000000000000:-1'
2016-10-13 17:01:31 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2016-10-13 17:01:31 14948 [Warning] Buffered warning: Changed limits: max_open_files: 65536 (requested 100000)

2016-10-13 17:01:31 14948 [Note] WSREP: Read nil XID from storage engines, skipping position init
2016-10-13 17:01:31 14948 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
2016-10-13 17:01:31 14948 [Note] WSREP: wsrep_load(): Galera 3.8(rXXXX) by Codership Oy <email address hidden> loaded successfully.
2016-10-13 17:01:31 14948 [ERROR] WSREP: galerautils/src/gu_init.c:gu_init():20: GU_PAGE_SIZE(4096) does not maptch real system page size(65536)
2016-10-13 17:01:31 14948 [Note] WSREP: /usr/sbin/mysqld: Terminated.
Aborted (core dumped)

<repeats multiple time>

[Krunal Bauskar (krunal-bauskar)]

This seems to be an upstream limitation or bug.
Can you file it upstream (Codership lp). Would be easier to fix such bugs in upstream release.

[Larry Michel (lmic)]

Krunal, I see another bug, bug 1606102, that's opened for the same issue.

[Krunal Bauskar (krunal-bauskar)]

@larry,

Great. We can track that issue for the fix then.

[Krunal Bauskar (krunal-bauskar)]

I assume galera-3.19 has fixed this problem

"Galera would refuse to start on PPC64EL (lp1633226, GAL-418)"

PXC will inherit it.