Summarizing to avoid confusion:

Bug 1617034 (Reported by Shaju) -> Decided to be duplicate of 1602618. However, the same topology is there in Sanity and it passed in 3.0.2.0-51. So bug 1617034 is wrongly stated as duplicate of 1602618 (OR) it is a breakage after 3.0.2.0-51.
Bug 1616178 (Reported by Vivek) -> This is there in both 3.0.2.0-51 and now. In future, this case will be taken care when we add separate openstack and contrail HA nodes in Sanity.


Sanity testbed: (No Contrail-HA; No Openstack-HA; Multiple cfgm nodes and one of them is openstack node) - Bug 1602618

Config Nodes : [u'nodea35', u'nodea34']
Control Nodes : [u'nodea35', u'nodea34', u'nodec53']
Compute Nodes : [u'nodec54', u'nodec55', u'nodec56']
Openstack Node : nodea34
WebUI Node : nodec53
Analytics Nodes : [u'nodea35', u'nodec53’]


Shaju’s testbed: (No Contrail-HA; No Openstack-HA; cfgm and openstack on different nodes) - Bug 1617034

env.roledefs = {
    'all': [host1, host2, host3],
    'cfgm': [host1],
    'openstack': [host2],
    'control': [host1],
    'compute': [host3],
    'collector': [host1],
    'webui': [host2],
    'database': [host1],
    'build': [host_build],
}

Vivek’s testbed: (Contrail-HA and Openstack node separate ) - Bug 1616178

env.roledefs = {
    'all': [host1, host2, host3, host4, host5, host6 ],
    'cfgm': [host1, host2, host3],
    'openstack': [host1],
    'control': [host1, host2, host3],
    'compute': [host4, host5, host6],
    'collector': [host1, host2, host3],
    'webui': [host1, host2, host3],
    'database': [host1, host2, host3],
    'build': [host_build],
}




Thanks,
Jeba


From: Vivek Garg <email address hidden>
Date: Friday, August 26, 2016 at 2:30 PM
To: Jeba Paulaiyan <email address hidden>, Ashish Ranjan <email address hidden>, Amudha R <email address hidden>, "Shaju V.K" <email address hidden>
Cc: Sachin Bansal <email address hidden>, Rudra Rugge <email address hidden>, Deepinder Singh Setia <email address hidden>, Sudheendra Rao <email address hidden>, Ignatious Johnson <email address hidden>
Subject: Re: Provision router failed- R3.0-Build-54-Liberty

Hi Jeba,

I see the following bug : https://bugs.launchpad.net/juniperopenstack/+bug/1616178

Version: R3.0-51 Liberty
Roles:
    'all': [host1, host2, host3, host4, host6 ],
    'cfgm': [host1, host2, host3],
    'openstack': [host4],
    'control': [host1, host2, host3],
    'compute': [ host6],
    'collector': [host1, host2, host3],
    'webui': [host1, host2, host3],
    'database': [host1, host2, host3],
    'build': [host_build],

Thanks,
Vivek
 
From: Jeba Paulaiyan
Date: Friday, August 26, 2016 at 9:25 AM
To: Ashish Ranjan, Amudha R, "Shaju V.K", Vivek Garg
Cc: Sachin Bansal, Rudra Rugge, Deepinder Singh Setia, Sudheendra Rao
Subject: Re: Provision router failed- R3.0-Build-54-Liberty

Reviewed the Sanity results. All jobs has openstack and config role on same node. We have separated openstack and control, database, but config and openstack are shared.

Taking these two AIs:

Will confirm if the FCS build had this issue. (Vivek is trying it out)
Will change the Ubuntu HA testbed from 3 x (All in one role)+Computes —to—> 3 x (Openstack HA) + 3 x (Rest of roles and contrail HA) + Computes.
Will change the Centos HA testbed from 1 x (All in one role) + 2 x (Rest of roles and contrail HA) —to—> 1 x (Openstack) + 3 x (Rest of roles and contrail HA) + Computes.
Thanks,
Jeba

From: Ashish Ranjan <email address hidden>
Date: Friday, August 26, 2016 at 7:35 AM
To: Amudha R <email address hidden>, "Shaju V.K" <email address hidden>
Cc: Sachin Bansal <email address hidden>, Rudra Rugge <email address hidden>, Deepinder Singh Setia <email address hidden>, Jeba Paulaiyan <email address hidden>
Subject: Re: Provision router failed- R3.0-Build-54-Liberty

Is this broken in fcs build as well?


From: Amudha R
Sent: Friday, August 26, 2016 7:07:08 AM
To: Ashish Ranjan; Shaju V.K
Cc: Rudra Rugge; Deepinder Singh Setia; Sachin Bansal; Jeba Paulaiyan
Subject: Re: Provision router failed- R3.0-Build-54-Liberty
 
Raised https://review.opencontrail.org/#/c/23636/ for R3.0

-Amudha

From: Ashish Ranjan <email address hidden>
Date: Friday, August 26, 2016 at 8:33 AM
To: Amudha R <email address hidden>, "Shaju V.K" <email address hidden>
Cc: Rudra Rugge <email address hidden>, Deepinder Singh Setia <email address hidden>, Sachin Bansal <email address hidden>, Jeba Paulaiyan <email address hidden>
Subject: Re: Provision router failed- R3.0-Build-54-Liberty

Can we commit this asap. So is this an issue on 3.0.2 fcs build as well (build 51). 

Ashish 


From: Amudha R
Sent: Thursday, August 25, 2016 9:53:02 PM
To: Shaju V.K
Cc: Deepinder Singh Setia; Sachin Bansal; Ashish Ranjan; Jeba Paulaiyan; Rudra Rugge
Subject: Re: Provision router failed- R3.0-Build-54-Liberty
 
It does seem to be the same issue, and I had not committed the fix into R3.0.
The issue happenswhen openstack node is different than the config node.
Shaju, can you install the memcache packages on your node and re-run provision_vrouter.py,
that should confirm if the issue is fixed.

Thanks,
Amudha

From: "Shaju V.K" <email address hidden>
Date: Thursday, August 25, 2016 at 11:07 PM
To: Amudha R <email address hidden>
Cc: Deepinder Singh Setia <email address hidden>, Sachin Bansal <email address hidden>, Ashish Ranjan <email address hidden>, Jeba Paulaiyan <email address hidden>, Rudra Rugge <email address hidden>
Subject: Re: Provision router failed- R3.0-Build-54-Liberty

+ Amudha
 
Amudha,
 
Looks like this bug is similar to the issue we are seeing now.  Commit for this bug is available only on 3.1 and Trunk not on R3.0. Could you please confirm it?
 
https://bugs.launchpad.net/juniperopenstack/+bug/1602618
 
Thanks,
Shaju
 
From: Ashish Ranjan <email address hidden>
Date: Thursday, August 25, 2016 at 8:24 PM
To: "Shaju V.K" <email address hidden>, Jeba Paulaiyan <email address hidden>
Cc: Deepinder Singh Setia <email address hidden>, Sachin Bansal <email address hidden>
Subject: Re: Provision router failed- R3.0-Build-54-Liberty
 
Deepinder Sachin can you help here..

 
From: Shaju V.K
Sent: Thursday, August 25, 2016 5:58:38 PM
To: Jeba Paulaiyan
Cc: Ashish Ranjan
Subject: Provision router failed- R3.0-Build-54-Liberty
 
Hi Jeba,
 
The below bug is not related the rbac. The provisioning failed due to missing python-memcache from cfgm. I doubt some check-in is missing in R3.0 branch
 
https://bugs.launchpad.net/juniperopenstack/+bug/1617034
 
Thanks,
Shaju