Comment 4 for bug 1773425

From: Ritam Gangopadhyay
Sent: Thursday, May 31, 2018 12:09 AM
To: Ramprakash Ram Mohan <email address hidden>
Cc: Michael Henkel <email address hidden>; Andrey Pavlov <email address hidden>; Sudheendra Rao <email address hidden>; Vimal Appachan <email address hidden>; Abhay Joshi <email address hidden>
Subject: Re: Multi Interface Setup with Openstack HA.

Hi Ram,

       The configuration mentioned in your email is for which the bug was filled.

Open stack nodes on mgmt/api n/w
Internal VIP on ctl-data n/w
External VIP on mgmt/api n/w

With this we see haproxy failing to come up while waiting for vip:3306

In the bug you asked me to configure open stack nodes on 10.10.10.X i.e. the ctl-data n/w. And with this I see the problem illustrated in my first mail.

Apart from these configurations we have also tried setting network interface, omitting open stack nodes in the instances.yaml and few others, without any success.

Regards,
Ritam.

On May 30, 2018 23:11, Ramprakash Ram Mohan <email address hidden> wrote:
+Abhay

Hi Ritam,

In your setup, if you want openstack to run its services on the mgmt. network, then you need to change the following in your config like I’ve mentioned in the bug:

contrail_configuration:
    OPENSTACK_NODES: 10.204.216.103,10.204.216.95,10.204.216.96
kolla_config:
   kolla_internal_vip_address: 10.204.216.140
   kolla_external_vip_address: 10.10.10.20

Basic theory is this:
Openstack runs services on the “kolla_internal_vip_address” which should correspond to an IP address on the “network_interface” parameter.
kolla_external_vip_address is the address you want access from. In “kolla” terminology, this should be your management network, but in contrail, we want to run the services themselves on the management network hence the swapped configs. (If you do not want access to openstack services on the 10.10.10.20 via haproxy, then you can avoid that altogether - I have not tested this though)
“network_interface” can be derived from the OPENSTACK_NODES parameter to avoid having to specify network names, so make sure kolla_internal_vip_address and OPENSTACK_NODES are on the same network.

Let me know if this helps.

Thanks,
Ram

From: Ritam Gangopadhyay <email address hidden>
Date: Wednesday, May 30, 2018 at 9:55 AM
To: Michael Henkel <email address hidden>, Ramprakash Ram Mohan <email address hidden>, Andrey Pavlov <email address hidden>
Cc: Sudheendra Rao <email address hidden>, Vimal Appachan <email address hidden>
Subject: Multi Interface Setup with Openstack HA.

instances.yaml_with_openstack_on_mgmt-api_subnet.yaml instances.yaml_with_opnstack_nodes_set_to_ctl_data_subnet.yaml

Hi,

       We are trying to get clarity on how services(contrail and openstack) are supposed to come up on a multi interface openstack HA setup and what would be the right deployment scenario.

       Ideally we should have mgmt, API and CTL/DATA as three separate networks on contrail, but as of now we have mgmt/API in one subnet and CTL/DATA in the other subnet. With that in mind, as of today what we understand from a multi interface setup is:-

1. Openstack services should open up north bound connections on the Mgmt/API network subnet-A and east-west communication on a separate subnet-B
2. Contrail controller services (config, analytics, collector) should come up on the Mgmt/API network subnet-A
3. Contrail control services and compute services should communicate on the CTL/DATA subnet-C

       With above assumption we have tried many combination and each one has lead us to haproxy service failing to provision because it doesn’t see the vip come up properly as mentioned in bug - https://bugs.launchpad.net/juniperopenstack/+bug/1773425

       The most recent configuration that we have tried is as below:-

1. Openstack on mgmt./API network 10.204.216.0/24 – eno1
2. Openstack east-west communication on 9.9.9.0/24 subnet – eno2
3. Contrail controller on mgmt./api network 10.204.216.0/24 – eno1
4. Contrail control data n/w on 10.10.10.0/24 – ens2f1

         Even with the above config we see haproxy fails with the mentioned error. Attached the instances.yaml file for this to the mail

         On Ram’s suggestion in the bug to use 10.10.10.0/24 n/w (i.e. the CTL/DATA subnet for that particular setup) for openstack nodes, we were able to bring up a setup. Though I am not fully convinced about this configuration as it beats the purpose of having openstack on the mgmt./API network.
         The instances.yaml file for this setup is also attached to this mail. The main issue we see here is all the openstack services run on 192.168.100.0/24 n/w (i.e. the CTL/DATA n/w) whereas contrail controller runs on 10.204.217.0/24 n/w which is the mgmt./API subnet.

[root@nodec33 ~]# netstat -anp | grep 8775
tcp 0 0 192.168.100.17:8775 0.0.0.0:* LISTEN 32280/python2
tcp 0 0 192.168.100.17:8775 192.168.100.17:43540 SYN_RECV -
tcp 0 0 192.168.100.17:8775 192.168.100.12:57480 SYN_RECV -
tcp 0 0 192.168.100.17:8775 192.168.100.15:37638 SYN_RECV -
tcp 0 0 192.168.100.17:8775 192.168.100.12:57506 SYN_RECV -
tcp 0 0 10.204.217.184:8775 0.0.0.0:* LISTEN 20874/haproxy
tcp 0 0 192.168.100.20:8775 0.0.0.0:* LISTEN 20874/haproxy
[root@nodec33 ~]# netstat -anp | grep 35357
tcp 0 0 192.168.100.17:35357 0.0.0.0:* LISTEN 17391/httpd
tcp 0 0 192.168.100.17:35357 192.168.100.12:60476 SYN_RECV -
tcp 0 0 192.168.100.17:35357 192.168.100.15:34448 SYN_RECV -
tcp 0 0 192.168.100.20:35357 0.0.0.0:* LISTEN 20874/haproxy
[root@nodec33 ~]# netstat -anp | grep 8082
tcp 0 0 10.204.217.168:8082 0.0.0.0:* LISTEN 28966/python
tcp 0 0 10.204.217.168:50238 10.204.217.168:8082 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:8082 10.204.217.13:38660 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:50240 10.204.217.168:8082 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:54176 10.204.217.13:8082 ESTABLISHED 5591/python
tcp 0 0 10.204.217.168:8082 10.204.217.176:59740 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:8082 10.204.217.176:59872 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:56088 10.204.217.168:8082 ESTABLISHED 5591/python
tcp 0 0 10.204.217.168:45208 10.204.217.176:8082 ESTABLISHED 5591/python
tcp 0 0 10.204.217.168:8082 10.204.217.176:60284 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:8082 10.204.217.168:56088 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:8082 10.204.217.168:50240 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:8082 10.204.217.168:50238 ESTABLISHED 28966/python
[root@nodec33 ~]# netstat -anp | grep 8086
tcp 0 0 0.0.0.0:8086 0.0.0.0:* LISTEN 6077/contrail-colle
tcp 0 0 10.204.217.168:51194 10.204.217.13:8086 ESTABLISHED 28756/python
tcp 0 0 10.204.217.168:56640 10.204.217.13:8086 ESTABLISHED 31452/python
tcp 0 0 10.204.217.168:50742 10.204.217.13:8086 ESTABLISHED 5447/contrail-query
tcp 0 0 10.204.217.168:54064 10.204.217.13:8086 ESTABLISHED 28966/python
tcp 0 0 10.204.217.168:50604 10.204.217.13:8086 ESTABLISHED 3731/python

          We need to get clarity on the deployment of the services and how we can bring openstack and contrail controllers to run on same subnet.

Regards,
Ritam