Multicloud :: Azure OnPrem :: Adding Vnet to existing cloud cluster fails.

[Ritam Gangopadhyay]

Adding new vnet to existing cloud on vnet under the same region is failing. The failure is that the gateways in the original vnet become unreachable. The public IPs of the original vnets are non-pingable and gateway ansible play fails with the unreachability error in the task below.

2018-10-23 08:26:11,323 p=3017 u=root | PLAY [Wait for connection] ********************************************************************************************************************************
2018-10-23 08:26:11,336 p=3017 u=root | TASK [Wait up to 600 seconds for the gateways to be reachable via SSH] ************************************************************************************
2018-10-23 08:26:11,750 p=3017 u=root | ok: [192.168.2.1 -> localhost]
2018-10-23 08:26:11,775 p=3017 u=root | ok: [172.16.2.7 -> localhost]
2018-10-23 08:26:11,777 p=3017 u=root | ok: [172.16.2.5 -> localhost]
2018-10-23 08:36:12,340 p=3017 u=root | fatal: [172.16.1.5 -> localhost]: FAILED! => {"changed": false, "elapsed": 600, "failed": true, "msg": "Timeout when waiting for 52.247.199.188:22"}
2018-10-23 08:36:12,377 p=3017 u=root | fatal: [172.16.1.4 -> localhost]: FAILED! => {"changed": false, "elapsed": 600, "failed": true, "msg": "Timeout when waiting for 52.247.192.95:22"}
2018-10-23 08:36:12,394 p=3017 u=root | PLAY [gateways] *******************************************************************************************************************************************

Topology file with added vnet *******
****************************************************
****************************************************
****************************************************

- provider: OnPrem
  organization: Juniper
  project: multicloud
  instances:
    - name: 5c3s1-node4
      roles:
        - gateway
      provision: true
      username: root
      password: c0ntrail123
      public_ip: 10.87.74.132
      private_ip: 192.168.2.1
      private_subnet:
        - 192.168.2.0/24
        - 192.168.1.0/24
      protocols_mode:
        - ssl_client
      interface: bond0
      gateway: 192.168.2.254
    - name: 5c3s1-node1
      roles:
        - controller: false
        - k8s_master
      provision: true
      username: root
      password: c0ntrail123
      public_ip: 10.87.74.129
      private_ip: 192.168.1.1
      private_subnet: 192.168.1.0/24
      interface: bond0
    - name: 5c3s1-node2
      roles:
        - compute_node: false
        - k8s_node
      provision: true
      username: root
      password: c0ntrail123
      public_ip: 10.87.74.130
      private_ip: 192.168.1.2
      private_subnet: 192.168.1.0/24
      interface: bond0
      gateway: 192.168.1.254
- provider: azure
  organization: Juniper
  project: multicloud
  regions:
    - name: WestUS2
      resource_group: contrail-test-west-us-2
      vnet:
        - name: rg-vpc-1
          cidr_block: 172.16.1.0/24
          subnets:
            - name: rg-subnet-1
              cidr_block: 172.16.1.0/25
              security_group: rg-sg-1
          security_groups:
            - name: rg-sg-1
              rules:
                - name: rg-all_in
                  direction: inbound
                - name: rg-all_out
                  direction: outbound
          instances:
            - name: rg-gw-1
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-1
              interface: eth1
              roles:
               - gateway
              protocols_mode:
                - ssl_server
                - ipsec_server
                - ipsec_client
            - name: rg-gw-2
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-1
              interface: eth1
              roles:
               - gateway
              protocols_mode:
                - ssl_server
                - ipsec_server
                - ipsec_client
            - name: rg-compute-1
              provision: false
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-1
              interface: eth0
              roles:
               - compute_node
            - name: rg-compute-2
              provision: false
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-1
              interface: eth0
              roles:
               - compute_node
        - name: rg-vpc-2
          cidr_block: 172.16.2.0/24
          subnets:
            - name: rg-subnet-2
              cidr_block: 172.16.2.0/25
              security_group: rg-sg-2
          security_groups:
            - name: rg-sg-2
              rules:
                - name: rg-all_in
                  direction: inbound
                - name: rg-all_out
                  direction: outbound
          instances:
            - name: rg-gw-21
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-2
              interface: eth1
              roles:
                - gateway
              protocols_mode:
                - ssl_server
                - ipsec_server
                - ipsec_client
            - name: rg-gw-22
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-2
              interface: eth1
              roles:
                - gateway
              protocols_mode:
                - ssl_server
                - ipsec_server
                - ipsec_client
            - name: rg-compute-21
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-2
              interface: eth0
              roles:
               - compute_node
            - name: rg-compute-22
              provision: true
              username: ubuntu
              os: ubuntu16
              os_version: 16.04.201705080
              instance_type: Standard_F2
              subnets: rg-subnet-2
              interface: eth0
              roles:
               - compute_node

[Paweł Kopka (pkopka)]

The problem was in naming, we have the same names rule in security groups in both vnet. So this is not a bug, but we will provide validation for unique names in 5.1.

[Ritam Gangopadhyay (ritam)]

The fix for this will go in 5.1. 5.0.2 the priority is being reduced because it still works when we specify unique rules in the subnet of the new vnet being added.

[Jeba Paulaiyan (jebap)]

Notes:

Adding new vnet to existing cloud on vnet under the same region is failing. The failure is that the gateways in the original vnet become unreachable. The public IPs of the original vnets are non-pingable and gateway ansible play fails with the unreachability error.

Workaround: Specify unique rules in the subnet of the new vnet being added.