[3.2.0.0-8~mitaka] VncApiError: Logical Interface IP address not found

[chhandak]

Description:

While loading scale config following error observed in contrail-api.log. Also loading of config is extremely slow. We have used same script in earlier release and execution was faster. Neutron port POST call is taking around 4 sec to get executed.

Error Log
-------------
VncApiError: Error: 10.3.136.249 ip address not found

What Script is Doing :
1. Creating VN
2. Creating Physical Interface under TOR
3. Creating Logical Interface under physical interface.
4. Associating netroon port (Server) to Logical Interface

This Same Server Ip is showing in error log

Details Log
-------------
127.0.0.1 - - [2016-12-08 16:20:22] "POST /virtual-machine-interfaces HTTP/1.1" 200 555 0.177466
127.0.0.1 - - [2016-12-08 16:20:23] "POST /chown HTTP/1.1" 200 109 0.125695
WARNING:contrail-api:__default__ [SYS_NOTICE]: VncApiNotice: chown: 4d7b503b-ebfa-45f0-92e4-12d5b59d1bd7 owner set to 1c6eb4ffa690412185ec7d7453f57b94
DEBUG:api-0:Sending request(xid=19566): Create(path='/api-server/subnets/default-domain:project1:vn-test1-10452:10.3.136.0/24/0168003834', data='6bb6c468-242a-47d4-ae89-de44dd041b59', acl=[ACL(perms=31, acl_list=['ALL'], id=Id(scheme='world', id='anyone'))], flags=0)
DEBUG:api-0:Received response(xid=19566): u'/api-server/subnets/default-domain:project1:vn-test1-10452:10.3.136.0/24/0168003834'
127.0.0.1 - - [2016-12-08 16:20:23] "POST /instance-ips HTTP/1.1" 200 357 0.742068
ERROR:contrail-api:__default__ [SYS_ERR]: VncApiError: Error: 10.3.136.250 ip address not found
127.0.0.1 - - [2016-12-08 16:20:24] "GET /virtual-machine-interface/4d7b503b-ebfa-45f0-92e4-12d5b59d1bd7?fields=logical_router_back_refs%2Cinstance_ip_back_refs%2Cfloating_ip_back_refs&exclude_hrefs=True HTTP/1.1" 200 2089 0.141627
127.0.0.1 - - [2016-12-08 16:20:24] "POST /chown HTTP/1.1" 200 109 0.722094
WARNING:contrail-api:__default__ [SYS_NOTICE]: VncApiNotice: chown: 6bb6c468-242a-47d4-ae89-de44dd041b59 owner set to 1c6eb4ffa690412185ec7d7453f57b94
127.0.0.1 - - [2016-12-08 16:20:24] "GET /virtual-machine-interface/4d7b503b-ebfa-45f0-92e4-12d5b59d1bd7?fields=logical_router_back_refs%2Cinstance_ip_back_refs%2Cfloating_ip_back_refs&exclude_hrefs=True HTTP/1.1" 200 2089 0.131885
127.0.0.1 - - [2016-12-08 16:20:25] "GET /virtual-network/00f0d248-26f4-4246-91cf-daf4b26fa42c?exclude_back_refs=True&exclude_children=True&exclude_hrefs=True HTTP/1.1" 200 1729 0.127245
127.0.0.1 - - [2016-12-08 16:20:25] "GET /instance-ip/6bb6c468-242a-47d4-ae89-de44dd041b59?exclude_back_refs=True&exclude_children=True&exclude_hrefs=True HTTP/1.1" 200 1384 0.715657
172.17.90.1 - - [2016-12-08 16:20:25] "POST /neutron/port HTTP/1.1" 200 834 4.908077 >>> This Post Call is taking 4 sec
172.17.90.1 - - [2016-12-08 16:20:26] "POST /ref-update HTTP/1.1" 200 156 0.709827
ERROR:contrail-api:__default__ [SYS_ERR]: VncApiError: Error: 10.3.136.249 ip address not found
172.17.90.1 - - [2016-12-08 16:20:31] "GET /logical-interface/3762b915-9104-49a4-afa8-7055aefe2cf5?exclude_back_refs=True&exclude_children=True HTTP/1.1" 200 1432 0.127743
172.17.90.1 - - [2016-12-08 16:20:32] "GET /logical-interface/3762b915-9104-49a4-afa8-7055aefe2cf5?exclude_back_refs=True&exclude_children=True HTTP/1.1" 200 1673 0.137565

[chhandak (chhandak)]

Logs copied to /auto/cores/1648660

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/27090
Submitter: Atul Moghe (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/27092
Submitter: Atul Moghe (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/27090
Committed: http://github.org/Juniper/contrail-controller/commit/7d53da4062164967e8e826a27593f4768aac6bcc
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 7d53da4062164967e8e826a27593f4768aac6bcc
Author: Atul Moghe <email address hidden>
Date: Fri Dec 9 06:57:32 2016 -0800

Remove messages from ipam_alloc_notify, ipam_alloc_req

as its possible that flat-subnet not configured and user gets ip-address
from user-defined subnet
Closes-Bug: #1648660

Change-Id: Ifde8e92cfa896296fd24dcb484467c5f0cd182c9

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/27092
Committed: http://github.org/Juniper/contrail-controller/commit/dbe498be1aa5d3a3d4adcb3ced83d969243aa901
Submitter: Zuul (<email address hidden>)
Branch: master

commit dbe498be1aa5d3a3d4adcb3ced83d969243aa901
Author: Atul Moghe <email address hidden>
Date: Fri Dec 9 06:57:32 2016 -0800

Remove messages from ipam_alloc_notify, ipam_alloc_req

as its possible that flat-subnet not configured and user gets ip-address
from user-defined subnet
Closes-Bug: #1648660

Change-Id: Ifde8e92cfa896296fd24dcb484467c5f0cd182c9