rfe: Support build multiple fully network isolated regions at one k8s cluster

[yangjianfeng]

Before, generally the k8s cluster was deployed in virtual machine based on a IaaS cloud, so the user can get a exclusive k8s cluster easily. But now, more and more k8s cluster be deployed at baremetal machine directly, this means all tenants share a same infrastructure. This require the platform provider provide more strict isolated method to build a fully isolated region at one k8s cluster for some security issues. K8s already provide namespace object to isolate resource at visibleness. But at network aspect, there are no CNI plugin that can build multiple fully isolated network region at on k8s cluster, I think the kuryr has a enormous advantage to do that, because it's network ability base on neutron. I know this maybe teensy violate the k8s's principle about the all pods is interconnected at one cluster, but I think it's worth it. It just makes up for the lack of k8s on the network

[Michal Dulko (michal-dulko-f)]

I think I understand the use case. I bet it could be as simple as allowing routerID or subnetID to be set in the namespace annotations, do I get it right?

Had you tried looking at Network Policies? Can they provide such isolation you're looking for?

[yangjianfeng (yangjianfeng)]

Yep, you are right, but I want to replenish some details yet:
Kuryr should support that the below annotations:
openstack.org/kuryr-subnet-pool
  used to auto create pod subnet, if not specified means user want to use the communal subnet pool(configured by pod_subnet_pool).
openstack.org/kuryr-subnet
  if not specified, kuryr will auto create a subnet for the namespace
openstack.org/kuryr-router
  support three types value: router id, auto, ""
  if the value is auto, the kuryr will auto create a router for the namespace
  if the value is "", kuryr will do not connect the pod subnet to router
  if the namespace has no the annotation, means user want to connect the pod subnet to the communal router.
openstack.org/kuryr-project
  For details: https://review.opendev.org/c/openstack/kuryr-kubernetes/+/832768

For Network Policies, I don't test it yet. But, I think that a independent tenant will expect a independent IPAM (The cidr will not be affected by other tenant). In my opinion, only use Network Policy can not meet this require.