Comment 3 for bug 1943786

Revision history for this message
Connor Chamberlain (lcvcode) wrote : Re: Connections fail when using a vip with a port in Charmed Operator Framework

It looks like Kubernetes may be setting service target ports to 65535 by default. This breaks networking when two Charmed Operator charms attempt to communicate using cluster IPs.

In my case, I'm deploying:
* Pulsar https://code.launchpad.net/~lcvcode/+git/pulsar
* Bookie https://code.launchpad.net/~lcvcode/+git/bookie

After deploying Pulsar, I can see:
```
kubectl describe service -n pulsar pulsar
Name: pulsar
Namespace: pulsar
...
IP: 10.152.183.12
IPs: 10.152.183.12
Port: placeholder 65535/TCP
TargetPort: 65535/TCP
Endpoints: 10.1.195.244:65535,10.1.195.245:65535,10.1.195.247:65535
...
```
TargetPort should be configurable from within a charm, otherwise other applications cannot communicate using cluster IP.

A workaround for this is to use each unit's IP address (ops.model.network.bind_address [0]) rather than the application's ingress address (ops.model.network.ingress_address [1]) when providing addresses in relation data. In this way, units can communicate directly using the configured ports.

[0] - https://operator-framework.readthedocs.io/en/latest/index.html?highlight=ops.model#ops.model.Network.bind_address
[1] - https://operator-framework.readthedocs.io/en/latest/index.html?highlight=ops.model#ops.model.Network.ingress_address