Zombie relations remain after a charm was removed with --no-wait

[Claudiu Belu]

If there are 2 related charms, and one of them is removed with ``juju remove-application --force --no-wait``, the second charm will still have a relation set to the now removed charm. If the first charm gets redeployed and related to the second charm, the second charm now has 2 relations.

This can become a problem if the second charm expects to only have one relation set.

Example:

# Add the model, initial charms, and relate them.
juju add-model relations
juju deploy hello-kubecon
juju deploy nginx-ingress-integrator ingress
juju relate hello-kubecon ingress
juju status

# Forcefully remove the hello-kubecon charm.
juju remove-application --force --no-wait hello-kubecon
juju status

# Redeploy the hello-kubecon charm and relate it to the nginx-ingress-integrator.
juju deploy hello-kubecon
juju relate hello-kubecon ingress

# The nginx-ingress-integrator will be in an error state because it sees 2 relations, both requiring the same route.
juju status

Having a pdb open into the ``nginx-ingress-integrator`` charm will also reveal that it actually sees 2 relations set, even though there should only be one:

> /var/lib/juju/agents/unit-ingress-0/charm/lib/charms/nginx_ingress_integrator/v0/ingress.py(191)_on_relation_changed()
-> if not self.model.unit.is_leader():
(Pdb) self.model.relations["ingress"]
[<ops.model.Relation ingress:1>, <ops.model.Relation ingress:2>]

Sample output from the commands above: https://paste.ubuntu.com/p/JyWTgYtP8G/

Note that this is not an issue with the hello-kubecon and nginx-ingress-integrator charms; they are just 2 charms that can be related and they are used purely to demonstrate this issue. This issue can be observed with other charms as well.

juju remove-relation won't clear the zombie relation.

Juju version used: 2.9.27
OS: Ubuntu 20.04
Cloud: Kubernetes (microk8s, EKS)