BGPaaS scale: few bgp-client objects not present in control-node after its restarted

[amit surana]

2k BGPaaS sessions are setup. Upon control-node restart, around ~10% of the bgp-client router objects are not present in control-node. Restarting config services (on the node that has the active schema service) fixes the issue.

[Nischal Sheth (nsheth)]

Appears to be a control node issue.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28631
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28633
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28635
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28636
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28631
Committed: http://github.org/Juniper/contrail-controller/commit/fd55ab3c1966c1e4d0c98c98710d548ecd0909c3
Submitter: Zuul (<email address hidden>)
Branch: master

commit fd55ab3c1966c1e4d0c98c98710d548ecd0909c3
Author: Nischal Sheth <email address hidden>
Date: Wed Feb 8 21:05:10 2017 -0800

Fix race condition in BGPaaS peering creation

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-peerings for a bgp-router when processing a notification for
the instance-bgp-router edge on the bgp-router.

Change-Id: Id74dda2573491dd7d8df8d9c09d881dc6aa0e95c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28671
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28636
Committed: http://github.org/Juniper/contrail-controller/commit/7424b482176e4711d35eaff41ec001c15f2fc59f
Submitter: Zuul (<email address hidden>)
Branch: R3.0

commit 7424b482176e4711d35eaff41ec001c15f2fc59f
Author: Nischal Sheth <email address hidden>
Date: Wed Feb 8 21:05:10 2017 -0800

Fix race condition in BGPaaS peering creation

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-peerings for a bgp-router when processing a notification for
the instance-bgp-router edge on the bgp-router.

Change-Id: Id74dda2573491dd7d8df8d9c09d881dc6aa0e95c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28633
Committed: http://github.org/Juniper/contrail-controller/commit/b114626f7a1b7305d8e1307550588656d003b983
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit b114626f7a1b7305d8e1307550588656d003b983
Author: Nischal Sheth <email address hidden>
Date: Wed Feb 8 21:05:10 2017 -0800

Fix race condition in BGPaaS peering creation

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-peerings for a bgp-router when processing a notification for
the instance-bgp-router edge on the bgp-router.

Change-Id: Id74dda2573491dd7d8df8d9c09d881dc6aa0e95c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28671
Committed: http://github.org/Juniper/contrail-controller/commit/73c20ca36395a3e3287c5f245216d6f7d3373c03
Submitter: Zuul (<email address hidden>)
Branch: R3.0.3.x

commit 73c20ca36395a3e3287c5f245216d6f7d3373c03
Author: Nischal Sheth <email address hidden>
Date: Wed Feb 8 21:05:10 2017 -0800

Fix race condition in BGPaaS peering creation

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-peerings for a bgp-router when processing a notification for
the instance-bgp-router edge on the bgp-router.

Change-Id: Id74dda2573491dd7d8df8d9c09d881dc6aa0e95c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28635
Committed: http://github.org/Juniper/contrail-controller/commit/b97f08960b597dff4e1fd586f16e88db2de2ba42
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit b97f08960b597dff4e1fd586f16e88db2de2ba42
Author: Nischal Sheth <email address hidden>
Date: Wed Feb 8 21:05:10 2017 -0800

Fix race condition in BGPaaS peering creation

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-peerings for a bgp-router when processing a notification for
the instance-bgp-router edge on the bgp-router.

Change-Id: Id74dda2573491dd7d8df8d9c09d881dc6aa0e95c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29126
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29127
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29128
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29129
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29130
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29126
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29127
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29128
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29129
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29130
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29126
Committed: http://github.org/Juniper/contrail-controller/commit/32c2cf223ea13ea1b3f6c80699a5d6f6f6814f59
Submitter: Zuul (<email address hidden>)
Branch: master

commit 32c2cf223ea13ea1b3f6c80699a5d6f6f6814f59
Author: Nischal Sheth <email address hidden>
Date: Sat Feb 25 16:04:00 2017 -0800

Fix race condition in BGPaaS peering creation - take 2

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

The previous fix did not cover the case where the notification for a
routing-instance has not been processed when the notification for the
instance-bgp-router link is processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-routers for a routing-instance when processing a notification
for the routing-instance itself. This in turn causes a re-evaluation
of all the bgp-peerings for those bgp-routers.

Change-Id: Idc4425c87037aac1eb96f23394bfae536c17546c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29127
Committed: http://github.org/Juniper/contrail-controller/commit/c2e68576a54c6b6225a705dbc9865961d201c148
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit c2e68576a54c6b6225a705dbc9865961d201c148
Author: Nischal Sheth <email address hidden>
Date: Sat Feb 25 16:04:00 2017 -0800

Fix race condition in BGPaaS peering creation - take 2

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

The previous fix did not cover the case where the notification for a
routing-instance has not been processed when the notification for the
instance-bgp-router link is processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-routers for a routing-instance when processing a notification
for the routing-instance itself. This in turn causes a re-evaluation
of all the bgp-peerings for those bgp-routers.

Change-Id: Idc4425c87037aac1eb96f23394bfae536c17546c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29128
Committed: http://github.org/Juniper/contrail-controller/commit/d5df580728d822ecf1e064edf8155117cd946e62
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit d5df580728d822ecf1e064edf8155117cd946e62
Author: Nischal Sheth <email address hidden>
Date: Sat Feb 25 16:04:00 2017 -0800

Fix race condition in BGPaaS peering creation - take 2

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

The previous fix did not cover the case where the notification for a
routing-instance has not been processed when the notification for the
instance-bgp-router link is processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-routers for a routing-instance when processing a notification
for the routing-instance itself. This in turn causes a re-evaluation
of all the bgp-peerings for those bgp-routers.

Change-Id: Idc4425c87037aac1eb96f23394bfae536c17546c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29129
Committed: http://github.org/Juniper/contrail-controller/commit/557681be3dfc3582c93938c13488089c8da6ac5d
Submitter: Zuul (<email address hidden>)
Branch: R3.0

commit 557681be3dfc3582c93938c13488089c8da6ac5d
Author: Nischal Sheth <email address hidden>
Date: Sat Feb 25 16:04:00 2017 -0800

Fix race condition in BGPaaS peering creation - take 2

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

The previous fix did not cover the case where the notification for a
routing-instance has not been processed when the notification for the
instance-bgp-router link is processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-routers for a routing-instance when processing a notification
for the routing-instance itself. This in turn causes a re-evaluation
of all the bgp-peerings for those bgp-routers.

Change-Id: Idc4425c87037aac1eb96f23394bfae536c17546c
Closes-Bug: 1662678

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/29130
Committed: http://github.org/Juniper/contrail-controller/commit/2009cea7f09ac54add11f2544e985fb35cf25422
Submitter: Zuul (<email address hidden>)
Branch: R3.0.3.x

commit 2009cea7f09ac54add11f2544e985fb35cf25422
Author: Nischal Sheth <email address hidden>
Date: Sat Feb 25 16:04:00 2017 -0800

Fix race condition in BGPaaS peering creation - take 2

If the internal ifmap configuration object for the routing-instance
for the bgp-routers that comprise a bgp-peering does not exist when
processing the bgp-peering, the ifmap config manager does create the
internal ifmap configuration object for the bgp-peering.

This causes the control node to not create the internal ifmap config
object for some bgp-peerings when the control node gets restarted in
a scaled bgpaas configuration. This happens for bgp-peerings which
get processed before the corresponding routing-instance has been
processed.

The previous fix did not cover the case where the notification for a
routing-instance has not been processed when the notification for the
instance-bgp-router link is processed.

Fix this by modifying dependency tracker policy to re-evaluate all of
the bgp-routers for a routing-instance when processing a notification
for the routing-instance itself. This in turn causes a re-evaluation
of all the bgp-peerings for those bgp-routers.

Change-Id: Idc4425c87037aac1eb96f23394bfae536c17546c
Closes-Bug: 1662678