crm cluster health does not work: python3-parallax and cluster-glue (hb_report) dependencies

[Jorge Niedbalski]

[Environment]

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

[Description]

Running 'crm cluster health' raises the following exception:

root@juju-niedbalski-xenial-machine-15:/home/ubuntu# crm cluster health
INFO: Check the health of the cluster
INFO: Nodes: juju-niedbalski-xenial-machine-13, juju-niedbalski-xenial-machine-14, juju-niedbalski-xenial-machine-15
Traceback (most recent call last):
  File "/usr/sbin/crm", line 54, in <module>
    rc = main.run()
  File "/usr/lib/python2.7/dist-packages/crmsh/main.py", line 351, in run
    return main_input_loop(context, user_args)
  File "/usr/lib/python2.7/dist-packages/crmsh/main.py", line 240, in main_input_loop
    rc = handle_noninteractive_use(context, user_args)
  File "/usr/lib/python2.7/dist-packages/crmsh/main.py", line 196, in handle_noninteractive_use
    if context.run(' '.join(l)):
  File "/usr/lib/python2.7/dist-packages/crmsh/ui_context.py", line 75, in run
    rv = self.execute_command() is not False
  File "/usr/lib/python2.7/dist-packages/crmsh/ui_context.py", line 245, in execute_command
    rv = self.command_info.function(*arglist)
  File "/usr/lib/python2.7/dist-packages/crmsh/ui_cluster.py", line 158, in do_health
    return scripts.run(script, script_args(params), script_printer())
  File "/usr/lib/python2.7/dist-packages/crmsh/scripts.py", line 2045, in run
    opts = _make_options(params)
  File "/usr/lib/python2.7/dist-packages/crmsh/scripts.py", line 383, in _make_options
    opts = parallax.Options()
NameError: global name 'parallax' is not defined

[Solution]

- Depends on python-parallax package.

[Christian Ehrhardt  (paelzer)]

There are multiple levels of "not so easy" here.

1. You nominated for X/Y but IIRC python-parallax is only in >=zesty right?
2. crmsh is in main but python parallax is not, so this should be a component mismatch and needs an MIR I'd think?

Actually I wonder how this is in zesty, there the dependency exists but should be a component mismatch. In any case this might need more work than expected.

[Christian Ehrhardt  (paelzer)]

Also added the FYI to the Debian bug that handled this issue.

[Casey & Gina (caseyandgina)]

Is any fix to this available?

[Casey & Gina (caseyandgina)]

Ugh, this has been open for 14 months, so I guess not.

[Andreas Hasenack (ahasenack)]

The change from pssh to parallax was done by upstream in Jan 2015:
* Fri Jan 9 2015 Kristoffer Grönlund <email address hidden> and many others
...
- high: crm_pssh: Switch to python-parallax over pssh (bnc#905116)

This wasn't caught by the packager, nor by the automatic dependency generator when building the package.

Starting with bionic, crmsh was demoted to Universe, and that version of ubuntu has parallax, so it works there.

Even after parallax is installed (for example, via pip install parallax), it fails because it calls /bin/rpm (!):
root@xenial-crm:~# crm cluster health
INFO: Check the health of the cluster
INFO: Nodes: xenial-crm
Collect cluster information...ERROR: [xenial-crm]: Error (1): Traceback (most recent call last):
  File "./crm_rpmcheck.py", line 42, in <module>
    main()
  File "./crm_rpmcheck.py", line 39, in main
    data = [package_data(pkg) for pkg in sys.argv[1:]]
  File "./crm_rpmcheck.py", line 26, in package_data
    rc, out, err = run(['/bin/rpm', '-q', '--queryformat=' + _qfmt, pkg])
  File "./crm_rpmcheck.py", line 15, in run
    stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
ERROR: Collect cluster information (rc=False)

This package is very broken in xenial indeed, at least where clustering support is concerned :(

[Rafael David Tinoco (rafaeldtinoco)]

In Eoan:

$ sudo crm cluster health
/usr/lib/python3/dist-packages/crmsh/scripts.py:431: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(f)
INFO: Verify health and configuration
INFO: Nodes: cluster01, cluster02, cluster03
OK: Collect information
Ctrl-C, leavinglth check...

<hangs>

It looks like, despite parallax, crm cluster health might need *a real* fix in Eoan as well.

After the command is issued, it hangs and one of the nodes has:

root 14476 0.0 0.0 2864 2116 ? Ss 21:46 0:00 /bin/sh /usr/sbin/hb_report __slave DEST=health-report FROM_TIME=1567460760.0 TO_TIME=0 USER_NODES= NODES=cluster01 cluster02 cluster03 HA_LOG=/var/log/syslog SANITIZE=passw.* DO_SANITIZE=0 SKIP_LVL=False EXTRA_LOGS=/var/log/syslog /var/log/pacemaker/pacemaker.log /var/log/pacemaker.log /var/log/ha-cluster-bootstrap.log PCMK_LOG=/var/log/pacemaker/pacemaker.log /var/log/pacemaker.log VERBOSITY=0

And it is basically waiting on a read() - likely from "cat".

When issuing the hb_report by hand:

hb_report -n "cluster01 cluster02 cluster03" -f 10:00 -t 21:00

I'm able to get the report (important for a remote debug, for example).

For me, this is likely an issue in between crmsh <-> cluster-glue compatibility due changes in one or another over time.

I'll review this later as crmsh is in -universe and this is not one of its core functions to be cleaned up right now.

[Rafael David Tinoco (rafaeldtinoco)]

I'm dropping this from server-next because this fits the regular ubuntu-server queue purposes instead. Its importance is low in that queue (due to pcs being chosen as the platform to go when configuring crm and this crmsh sub command not being so commonly used).

[Athos Ribeiro (athos-ribeiro)]

This bug has been fixed since bionic. However, from focal on, there is a new issue due to usage of a deprecated (and removed) function in python's standard library. I have reported a new bug at https://bugs.launchpad.net/ubuntu/+source/crmsh/+bug/1947022 to track that.

Since this bug does not affect Ubuntu >= 18.04, I am closing it as wontfix, since Xenial is no longer under standard support.