build2740: vrouter provisioning failing

[Sudheendra Rao]

vrouter provisioning failing with the following error on mainline build2740.
The problem is seen on multi-node/multi-interface setup.

Setup details:
Config Nodes : [u'nodea35', u'nodea34']
Control Nodes : [u'nodea35', u'nodea34', u'nodec53']
Compute Nodes : [u'nodec54', u'nodec55', u'nodec56']
Openstack Node : nodea34
WebUI Node : nodec53
Analytics Nodes : [u'nodea35', u'nodec53']

2016-07-12 16:08:39:686937: [root@10.204.217.96] out: [localhost] local: python /opt/contrail/utils/provision_vrouter.py --host_name nodec56 --host_ip 22.22.22.56 --api_server_ip 22.22.22.35 --oper add --admin_user admin --admin_password contrail123 --admin_tenant_name admin --openstack_ip 22.22.22.34
2016-07-12 16:08:39:694508: [root@10.204.217.95] out: Found linux image: /boot/vmlinuz-3.13.0-85-generic
2016-07-12 16:08:40:108627: [root@10.204.217.95] out: Found initrd image: /boot/initrd.img-3.13.0-85-generic
2016-07-12 16:08:40:108724: [root@10.204.217.95] out: File descriptor 3 (/dev/null) leaked on vgs invocation. Parent PID 27236: /usr/sbin/grub-probe
2016-07-12 16:08:40:140525: [root@10.204.217.94] out: Traceback (most recent call last):
2016-07-12 16:08:40:187305: [root@10.204.217.94] out: File "/opt/contrail/utils/provision_vrouter.py", line 190, in <module>
2016-07-12 16:08:40:187399: [root@10.204.217.94] out: main()
2016-07-12 16:08:40:187485: [root@10.204.217.94] out: File "/opt/contrail/utils/provision_vrouter.py", line 186, in main
2016-07-12 16:08:40:187568: [root@10.204.217.94] out: VrouterProvisioner(args_str)
2016-07-12 16:08:40:187650: [root@10.204.217.94] out: File "/opt/contrail/utils/provision_vrouter.py", line 33, in __init__
2016-07-12 16:08:40:187731: [root@10.204.217.94] out: api_server_use_ssl=self._args.api_server_use_ssl)
2016-07-12 16:08:40:187811: [root@10.204.217.94] out: File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 321, in __init__
2016-07-12 16:08:40:187892: [root@10.204.217.94] out: retry_on_error=False)
2016-07-12 16:08:40:187972: [root@10.204.217.94] out: File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 742, in _request
2016-07-12 16:08:40:188052: [root@10.204.217.94] out: content = self._request(op, url, data=data, retry_after_authn=True)
2016-07-12 16:08:40:188146: [root@10.204.217.94] out: File "/usr/lib/python2.7/dist-packages/vnc_api/vnc_api.py", line 769, in _request
2016-07-12 16:08:40:188224: [root@10.204.217.94] out: raise HttpError(status, content)
2016-07-12 16:08:40:188299: [root@10.204.217.94] out: cfgm_common.exceptions.HttpError: HTTP Status: 500 Content: Internal Server Error
2016-07-12 16:08:40:188374: [root@10.204.217.94] out:
2016-07-12 16:08:40:204722: [root@10.204.217.94] out: Fatal error: local() encountered an error (return code 1) while executing 'python /opt/contrail/utils/provision_vrouter.py --host_name nodec54 --host_ip 22.22.22.54 --api_server_ip 22.22.22.35 --oper add --admin_user admin --admin_password contrail123 --admin_tenant_name admin --openstack_ip 22.22.22.34'
2016-07-12 16:08:40:204816: [root@10.204.217.94] out:
2016-07-12 16:08:40:204908: [root@10.204.217.94] out: Aborting.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22069
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22085
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22069
Committed: http://github.org/Juniper/contrail-packaging/commit/f66f0639b97d1bb8b9c63ec951a828be123741c9
Submitter: Zuul
Branch: master

commit f66f0639b97d1bb8b9c63ec951a828be123741c9
Author: Amudha <email address hidden>
Date: Sat Jul 16 09:21:50 2016 -0700

Adding python-memcache to contrail-openstack-config dependency

contrail-config uses keystonemiddleware to authenticate with keystone.
keystonemiddleware has a dependency on memcache, but the dependency is
not defined in the upstream keystonemiddleware package.
Adding the dependency in contrail-openstack-config as the config node
needs memcache installed while using keystonemiddleware.

Change-Id: I2044f0f30b88e44c1db6303738086c729add1841
Closes-Bug: #1602618

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22085
Committed: http://github.org/Juniper/contrail-packaging/commit/c04c0c5c412721615b8098746532f3edea89a713
Submitter: Zuul
Branch: R3.1

commit c04c0c5c412721615b8098746532f3edea89a713
Author: Amudha <email address hidden>
Date: Sun Jul 17 23:25:17 2016 -0700

Adding python-memcache to contrail-openstack-config dependency

contrail-config uses keystonemiddleware to authenticate with keystone.
keystonemiddleware has a dependency on memcache, but the dependency is
not defined in the upstream keystonemiddleware package.
Adding the dependency in contrail-openstack-config as the config node
needs memcache installed while using keystonemiddleware.

Change-Id: Ibd6d24cfb1e6b013078bd383f162a5b1449d60ca
Closes-Bug: #1602618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/23636
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/23636
Committed: http://github.org/Juniper/contrail-packaging/commit/35e8de1eb3d954c52d93d59dfc003fd42232a0b3
Submitter: Zuul
Branch: R3.0

commit 35e8de1eb3d954c52d93d59dfc003fd42232a0b3
Author: Amudha <email address hidden>
Date: Fri Aug 26 07:02:57 2016 -0700

Adding python-memcache to contrail-openstack-config dependency

contrail-config uses keystonemiddleware to authenticate with keystone.
keystonemiddleware has a dependency on memcache, but the dependency is
not defined in the upstream keystonemiddleware package.
Adding the dependency in contrail-openstack-config as the config node
needs memcache installed while using keystonemiddleware.

Change-Id: I42d0bdb4dfa021c0b09e719c66c20f98fea1186e
Closes-Bug: #1602618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24399
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24399
Committed: http://github.org/Juniper/contrail-packaging/commit/d439be178a9c47e0e5d49e2989e5abd6e244f933
Submitter: Zuul
Branch: R3.0

commit d439be178a9c47e0e5d49e2989e5abd6e244f933
Author: Amudha <email address hidden>
Date: Thu Sep 22 17:22:24 2016 -0700

Adding python-memcache to contrail-openstack-config dependency of juno

The issue is seen in juno after updating the keystonemiddleware package
version used in juno, fixing it in control.in for juno.

The latest keystonemiddleware contrail-config uses keystonemiddleware to authenticate with keystone.
keystonemiddleware has a dependency on memcache, but the dependency is
not defined in the upstream keystonemiddleware package.
Adding the dependency in contrail-openstack-config as the config node
needs memcache installed while using keystonemiddleware.

Change-Id: Ie228f37358df1c699859b8b3cb6b26a62893acf2
Closes-Bug: #1602618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/26115
Submitter: amudha ramachandran (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/26115
Committed: http://github.org/Juniper/contrail-packaging/commit/64cee18d7f7e5726af2c78ca1ecd628297acbcb8
Submitter: Zuul
Branch: R3.0.2.x

commit 64cee18d7f7e5726af2c78ca1ecd628297acbcb8
Author: Amudha <email address hidden>
Date: Sat Jul 16 09:21:50 2016 -0700

Adding python-memcache to contrail-openstack-config dependency

contrail-config uses keystonemiddleware to authenticate with keystone.
keystonemiddleware has a dependency on memcache, but the dependency is
not defined in the upstream keystonemiddleware package.
Adding the dependency in contrail-openstack-config as the config node
needs memcache installed while using keystonemiddleware.

Change-Id: Ia863502b23e97e127acc87c8bd6839a1a482a9bf
Closes-Bug: #1602618