Percona-cluster should show in status when mysql is down

Bug #1673555 reported by David Ames
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack HA Cluster Charm
Fix Released
High
David Ames
OpenStack Percona Cluster Charm
Fix Released
High
David Ames

Bug Description

There are times when mysql (percona-cluster) has crashed but the juju status still shows Unit Ready.

Neither the percona-cluster charm nor the hacluster charm seem to recognize this.

Percona seems completely unaware if the service is actually running. asses_status should validate the functionality of the service.

The hacluster charm has the mysql monitor feature but it is not presenting that back to the end user via juju status or being in a blocked state.

David Ames (thedac)
Changed in charm-hacluster:
status: New → Triaged
Changed in charm-percona-cluster:
status: New → Triaged
Changed in charm-hacluster:
importance: Undecided → High
Changed in charm-percona-cluster:
importance: Undecided → High
Changed in charm-hacluster:
milestone: none → 17.05
Changed in charm-percona-cluster:
milestone: none → 17.05
Revision history for this message
David Ames (thedac) wrote :

Hacluster msyql monitor:

mysql_monitor(res_mysql_monitor)[178679]: ERROR: Not enough arguments [1] to ocf_log.
mysql_monitor(res_mysql_monitor)[178813]: MYSQL IS NOT RUNNING:
mysql_monitor(res_mysql_monitor)[178923]: MYSQL IS NOT RUNNING:

James Page (james-page)
Changed in charm-percona-cluster:
milestone: 17.05 → 17.08
Changed in charm-hacluster:
milestone: 17.05 → 17.08
James Page (james-page)
Changed in charm-percona-cluster:
milestone: 17.08 → 17.11
Changed in charm-hacluster:
milestone: 17.08 → 17.11
Revision history for this message
Liam Young (gnuoy) wrote :

The percona-cluster charm does seem to validate that mysql is running:

Unit Workload Agent Machine Public address Ports Message
percona-cluster/0* active idle 0 10.5.0.31 3306/tcp Unit is ready
  hacluster/0* active idle 10.5.0.31 Unit is ready and clustered
percona-cluster/1 blocked idle 1 10.5.0.3 3306/tcp Services not running that should be: mysql
  hacluster/1 active idle 10.5.0.3 Unit is ready and clustered
percona-cluster/2 active idle 2 10.5.0.72 3306/tcp Unit is ready
  hacluster/2 active idle 10.5.0.72 Unit is ready and clustered

I don't see an obvious commit since this bug was raised that would account for status being fixed but I don't seem to be seeing the issue. Am I missing something here?

James Page (james-page)
Changed in charm-percona-cluster:
milestone: 17.11 → 18.02
Changed in charm-hacluster:
milestone: 17.11 → 18.02
Ryan Beisner (1chb1n)
Changed in charm-percona-cluster:
milestone: 18.02 → 18.05
Changed in charm-hacluster:
milestone: 18.02 → 18.05
David Ames (thedac)
Changed in charm-percona-cluster:
milestone: 18.05 → 18.08
Changed in charm-hacluster:
milestone: 18.05 → 18.08
James Page (james-page)
Changed in charm-percona-cluster:
milestone: 18.08 → 18.11
Changed in charm-hacluster:
milestone: 18.08 → 18.11
David Ames (thedac)
Changed in charm-percona-cluster:
milestone: 18.11 → 19.04
Changed in charm-hacluster:
milestone: 18.11 → 19.04
David Ames (thedac)
Changed in charm-percona-cluster:
milestone: 19.04 → 19.07
Changed in charm-hacluster:
milestone: 19.04 → 19.07
Revision history for this message
David Ames (thedac) wrote :

This hacluster change added more robust status:
https://github.com/openstack/charm-hacluster/commit/93644400756170bb8e37d5a5f995a54a4a3bd8dd

This percona-cluster charm fix will also close this bug:
https://review.opendev.org/#/c/667753/4

Changed in charm-hacluster:
status: Triaged → Fix Committed
assignee: nobody → David Ames (thedac)
Changed in charm-percona-cluster:
assignee: nobody → David Ames (thedac)
status: Triaged → In Progress
David Ames (thedac)
Changed in charm-hacluster:
status: Fix Committed → Fix Released
David Ames (thedac)
Changed in charm-percona-cluster:
milestone: 19.07 → 19.10
David Ames (thedac)
Changed in charm-percona-cluster:
milestone: 19.10 → 20.01
Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

According to comment #3 this is now fixed as the associated patchset has been merged.

Changed in charm-percona-cluster:
milestone: 20.01 → 19.10
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.