Can't remove a CMR from a defunct model

[Barry Price]

I've successfully set up a CMR to consume an offer from a remote controller.

The model providing that offer at the remote end has now been destroyed, but I can't remove-relation at the local/consuming end. Or rather the command runs without complaint, but fails silently - and the offer still hangs around in 'juju status'.

This is on Juju 2.4.6 at the local/consuming end - but was on 2.4.3 at the time the CMR was set up.

The remote/offering end was 2.4.3 and no longer exists.

We need some way of forcing a clean-up of the local/consuming end of these relations when the remote/offering end goes away.

[Ian Booth (wallyworld)]

https://pastebin.canonical.com/p/r3chFGk2Rd/

[Barry Price (barryprice)]

Ian and I discussed this online, the above pastebin shows the offending relation at line 90 (we related a local juju-deployed openstack cloud to a remote nagios app).

The defunct nagios still shows up under SAAS:

$ juju status | grep -A 2 ^SAAS
SAAS Status Store URL
nagios waiting mycloud admin/mycloud.nagios

$

But all attempts to remove it from the consuming end fail (and the offering end has gone away, so we can't do anything more with that):

$ juju remove-application nagios
removing application nagios failed: application "nagios" not found
$ juju remove-relation nagios:monitors nrpe
$

(Still shows up under SAAS, still shows up in 'juju status --relations', see pastebin above)

[Ian Booth (wallyworld)]

https://github.com/juju/juju/pull/9509

On the consuming side, if the offer has been removed from the offering model and the consumer didn't get notified, next time there's connectivity between the controllers the SAAS entry in status will be marked as "terminated". The relation will still be in tact, but when remove-saas is used to remove the SAAS app, the relation will be removed as well, and the departed/broken hooks run etc.

On both the consuming and offer sides, we are now more aggressive in removing from scope units which may have disappeared from the other side, also helping to fix the issue with stuck relations.