obtain advertise-address in kubernetes master is unexpected

Bug #1956291 reported by AnonSaber
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Kubernetes Common Layer
New
Undecided
Unassigned

Bug Description

The advertise-address of kube-api in some nodes has obtained the cni address.

For this case:

$ juju run --unit kubernetes-master/0 "network-get --format yaml kube-control"
bind-addresses:
- mac-address: 16:fa:03:ce:ef:83
  interface-name: flannel.1
  addresses:
  - hostname: ""
    address: 172.31.42.0
    cidr: 172.31.42.0/32
  macaddress: 16:fa:03:ce:ef:83
  interfacename: flannel.1
- mac-address: 52:54:00:d4:84:a7
  interface-name: eth0
  addresses:
  - hostname: ""
    address: 172.31.72.18
    cidr: 172.31.72.0/23
  - hostname: ""
    address: 172.31.72.4
    cidr: 172.31.72.4/32
  macaddress: 52:54:00:d4:84:a7
  interfacename: eth0
egress-subnets:
- 172.31.42.0/32
ingress-addresses:
- 172.31.42.0
- 172.31.72.18
- 172.31.72.4

So these code in kubernetes_common.py will obtain the kube-control address to 172.31.42.0 (flannel.1)

```python
def get_ingress_address(endpoint_name, ignore_addresses=None):
    try:
        network_info = hookenv.network_get(endpoint_name)
    except NotImplementedError:
        network_info = {}

    if not network_info or "ingress-addresses" not in network_info:
        # if they don't have ingress-addresses they are running a juju that
        # doesn't support spaces, so just return the private address
        return hookenv.unit_get("private-address")

    addresses = network_info["ingress-addresses"]

    if ignore_addresses:
        hookenv.log("ingress-addresses before filtering: {}".format(addresses))
        iter_filter = filter(lambda item: item not in ignore_addresses, addresses)
        addresses = list(iter_filter)
        hookenv.log("ingress-addresses after filtering: {}".format(addresses))

    # Need to prefer non-fan IP addresses due to various issues, e.g.
    # https://bugs.launchpad.net/charm-gcp-integrator/+bug/1822997
    # Fan typically likes to use IPs in the 240.0.0.0/4 block, so we'll
    # prioritize those last. Not technically correct, but good enough.
    try:
        sort_key = lambda a: int(a.partition(".")[0]) >= 240 # noqa: E731
        addresses = sorted(addresses, key=sort_key)
    except Exception:
        hookenv.log(traceback.format_exc())

    return addresses[0]
```

Obviously this is not the physical network interface (eth0) that we expected,Maybe we need to ignore these network interfaces:

lxdbr
flannel
cni
virbr
docker
cali
kube-ipvs

AnonSaber (anonsaber)
description: updated
AnonSaber (anonsaber)
Changed in charm-kubernetes-master:
status: New → Opinion
status: Opinion → New
Revision history for this message
Vuk Vasic (vasicvuk) wrote :

We have this issue when using kube-ipvs. In general all masters, workers takes 10.152.0.1. So we ended up editing every charm to use fixed IP addresses in each node instead of calling network-get which is very hard to maintain.

We encountered issue with master, worker, calico-node.

AnonSaber (anonsaber)
affects: charm-kubernetes-master → layer-kubernetes-common
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.