Juju does not show relation status

[Eduardo Damato]

Juju doesn't show the status of a relation. If the relation hook script never exits, it doesn't report it.

I did:

$ juju add-relation mysql ceph

which returned quickly. However, on the mysql box the relation hook never finished, and is still lingering due to ceph issues:

$ date
Thu Nov 21 09:39:53 UTC 2013

root 3063 0.0 1.4 481004 14904 ? Ssl Nov20 0:03 /var/lib/juju/tools/unit-mysql-0/jujud unit --data-dir /var/lib/juju --unit-name mysql/0 --debug
root 6456 0.0 1.0 48044 10208 ? S Nov20 0:00 \_ python /var/lib/juju/agents/unit-mysql-0/charm/hooks/ceph-relation-changed
root 6566 0.0 0.0 4404 604 ? S Nov20 0:00 \_ sh -c { rbd list --id mysql --pool mysql; } 2>&1
root 6567 0.0 0.5 437556 5056 ? Sl Nov20 0:05 \_ rbd list --id mysql --pool mysql

At the same time, in juju status, the relation seems to be ok:

  mysql:
    charm: local:precise/mysql-309
    exposed: false
    relations:
      ceph:
      - ceph
      cluster:
      - mysql
    units:
      mysql/0:
        agent-state: started
        agent-version: 1.16.4.1
        machine: "3"
        public-address: x3.master

I understand that this could be a fault of the charm, at the same time juju should inform me that the relation hook never completed. This is juju 1.16.3.

$ cat precise/mysql/revision
309
$ cat precise/ceph/revision
103

[William Reade (fwereade)]

We don't time hooks out -- some of them legitimately take a very long time to run. If the hook is unable to complete, it's responsible for failing out on its own.

 https://bugs.launchpad.net/juju-core/+bug/1254766 (which we intend to fix by 14.04) may be somewhat relevant -- it would allow you to distinguish between a ready and an unready mysql.

[Marco Ceppi (marcoceppi)]

Can you please paste the output of /var/log/juju/unit-mysql-0.log? This isn't a problem with juju per se, but rather an issue with the charm not handling errors properly.

[Eduardo Damato (edamato)]

@fwereade:

I do understand what you mean, but if we have hundreds of relations, we can't really login to every single instance to check if it's all fine because charms have problems. We should have a way of telling that the charms never finished, after all juju is the one deploying them.

@Marco:

Adding the file, but I think the issue is that the following in ./hooks/lib/ceph_utils.py doesn't seem to timeout:

def rbd_exists(service, pool, rbd_img):
    (rc, out) = commands.getstatusoutput('rbd list --id %s --pool %s' %\
                                         (service, pool))
    return rbd_img in out

I can still the the rbd list process in my box.

Cheers,
Eduardo.