R5.0.1 - multicloud :: Azure :: multiple region Vnet bring-up fails.

[Ritam Gangopadhyay]

topology file:-
**********************************
**********************************

- provider: azure
  organization: contrail-test
  project: contrail-ritam-test-1-multicloud
  regions:
    - name: WestIndia
      resource_group: contrail-test-west-india
      clouds:
        - name: ritam-test-1-a
          cidr_block: 192.169.0.0/16
          subnets:
            - name: subnet-ritam-test-1-a
              cidr_block: 192.169.0.0/24
              security_group: allow_all_protocols
          security_groups:
            - name: allow_all_protocols-ritam-test-1-a
              rules:
                - name: all_in-ritam-test-1-a
                  direction: inbound
                - name: all_out-ritam-test-1-a
                  direction: outbound
          instances:
            - name: gw-ritam-test-1-a
              roles:
                - gateway
              provision: true
              username: ubuntu
              os: ubuntu16
              instance_type: Standard_F1
              subnets: subnet-ritam-test-1-a
              interface: eth1

            - name: controller-ritam-test-1-a
              provision: true
              username: ubuntu
              roles:
                - controller
                - k8s_master
              os: ubuntu16
              instance_type: Standard_F1
              subnets: subnet-ritam-test-1-a
              interface: eth0

            - name: compute-ritam-test-1-a
              provision: true
              username: ubuntu
              roles:
                - compute_node
              os: ubuntu16
              instance_type: Standard_F1
              subnets: subnet-ritam-test-1-a
              interface: eth0

    - name: SouthIndia
      resource_group: contrail-test-south-india
      clouds:
        - name: ritam-test-1-b
          cidr_block: 192.173.0.0/16
          subnets:
            - name: subnet-ritam-test-1-b
              cidr_block: 192.173.0.0/24
              security_group: allow_all_protocols-ritam-test-1-b
          security_groups:
            - name: allow_all_protocols-ritam-test-1-b
              rules:
                - name: all_in-ritam-test-1-b
                  direction: inbound
                - name: all_out-ritam-test-1-b
                  direction: outbound
          instances:
            - name: gw-ritam-test-1-b
              roles:
                - gateway
              provision: true
              username: ubuntu
              os: ubuntu16
              instance_type: Standard_F1
              subnets: subnet-ritam-test-1-b
              interface: eth1

            - name: compute-ritam-test-1-b
              provision: true
              username: ubuntu
              roles:
                - compute_node
              os: ubuntu16
              instance_type: Standard_F1
              subnets: subnet-ritam-test-1-b
              interface: eth0

**********************************
**********************************

FAILURE

**********************************
**********************************

Error: Error applying plan:

1 error(s) occurred:

* azurerm_subnet.subnet-ritam-test-1-a-private: 1 error(s) occurred:

* azurerm_subnet.subnet-ritam-test-1-a-private: Error Creating/Updating Subnet "subnet-ritam-test-1-a-private" (VN "ritam-test-1-a" / Resource Group "contrail-test-west-india"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidResourceReference" Message="Resource /subscriptions/0e42336f-d930-41f9-9661-582c24337897/resourceGroups/CONTRAIL-TEST-SOUTH-INDIA/providers/Microsoft.Network/networkSecurityGroups/ALLOW_ALL referenced by resource /subscriptions/0e42336f-d930-41f9-9661-582c24337897/resourceGroups/contrail-test-west-india/providers/Microsoft.Network/virtualNetworks/ritam-test-1-a/subnets/subnet-ritam-test-1-a-private was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[{"code":"NotFound","message":"Resource /subscriptions/0e42336f-d930-41f9-9661-582c24337897/resourceGroups/CONTRAIL-TEST-SOUTH-INDIA/providers/Microsoft.Network/networkSecurityGroups/ALLOW_ALL not found."}]

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

**********************************
**********************************

ATTACHED TERRAFORM LOGS FOR TOPOLOGY APPLY

**********************************
**********************************

[Sanju Abraham (asanju)]

This is fixed in commit - https://github.com/Juniper/contrail-multi-cloud/pull/167

When a ResourceGroup is created in Azure, it is linked to a Region. All Quota provisioning happens in that Region. There was a miscommunication from Azure stating that resources cannot be created across the regions that are not linked to the ResourceGroup.

After subsequence call with Azure, we made the change in the code to not derive the region from the associated ResourceGroup coz, it would always give only the linked Region.

We will not read valid regions from topology and generate corresponding terraform templates.