TOR Scale: UI: Delete all using Webui is failing

[chhandak]

I have 4k LIF and 8K VMI associated with one TOR. While trying to delete all the interface using Dleete_all option in WebUI, it is failing with following Error log ""Too many pending updates to RabbitMQ: 4096”]”

Webui Log
—————
2015-06-02T13:03:08.154Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/logical-interface/a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9] returned error ["ID a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9 does not exist"]
2015-06-02T13:03:08.236Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/b4112cf9-b8ad-4ce2-8e0c-ba0ca7f22fe9] returned error ["Too many pending updates to RabbitMQ: 4096"]
2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/virtual-machine-interface/87c36861-5865-4fe8-ad65-f1864af5d16f] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/instance-ip/18499836-0d67-4fd1-b3b4-27a6712a857f] returned error ["NotFoundException(_message=None)"]
2015-06-02T13:03:08.253Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de] returned error ["ID ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de does not exist"]
2015-06-02T13:03:08.256Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/d173d008-9583-4aab-8193-b95b6f697bd7] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.261Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.264Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/1446e62d-6f8e-43ae-9578-01d3de9788ba] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.273Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/c1841abd-04cf-4b19-b1fe-9711aacd92e2] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.321Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to RabbitMQ: 4097"]
2015-06-02T13:03:08.321Z - ^[[31merror^[[39m: Encountered broken link: /logical-interface/0d498491-3c45-4900-8519-8204f4fc9161
2015-06-02T13:03:08.322Z - ^[[31merror^[[39m: REST Server Error: Too many pending updates to RabbitMQ: 4097
    at APIServer.retryMakeCall (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:169:13)
    at Request.APIServer.makeCall (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:280:18)
    at Request.EventEmitter.emit (events.js:99:17)
    at Request.mixin._fireSuccess (/usr/lib64/node_modules/restler/lib/restler.js:226:10)
    at mixin._responseHandler (/usr/lib64/node_modules/restler/lib/restler.js:157:20)

Contrail api log
—————————
ERROR:nodei7:contrail-api:Config:0:__default__ [SYS_ERR]: VncApiError: RabbitMQ connection down
WARNING:nodei7:contrail-api:Config:0:__default__ [SYS_NOTICE]: VncApiError: RabbitMQ connection ESTABLISHED <Connection: amqp://guest@192.168.22.210:5673// at 0x7fc7c4373850>
ERROR:nodei7:contrail-api:Config:0:__default__ [SYS_ERR]: VncApiError: RabbitMQ connection down
WARNING:keystoneclient.middleware.auth_token:Unable to find authentication token in headers
192.168.22.1 - - [2015-06-01 18:58:20] "GET / HTTP/1.1" 401 197 0.000702
192.168.22.1 - - [2015-06-01 18:58:21] "GET / HTTP/1.1" 200 11928 0.025424
WARNING:nodei7:contrail-api:Config:0:__default__ [SYS_NOTICE]: VncApiError: RabbitMQ connection ESTABLISHED <Connection: amqp://guest@192.168.22.210:5673// at 0x7fc7c4373850>
192.168.22.1 - - [2015-06-01 19:10:08] "GET /global-system-config/4ffba3ad-2c68-4ec5-9ed9-05455b9297e3?exclude_back_refs=True&exclude_children=True HTTP/1.1" 200 844 0.037729
192.168.22.1 - - [2015-06-01 19:14:49] "POST /fqname-to-id HTTP/1.1" 200 156 0.008850
ERROR:nodei7:contrail-api:Config:0:__default__ [SYS_ERR]: VncApiError: RabbitMQ connection down
WARNING:nodei7:contrail-api:Config:0:__default__ [SYS_NOTICE]: VncApiError: RabbitMQ connection ESTABLISHED <Connection: amqp://guest@192.168.22.210:5673// at 0x7fc7c4373850>

[chhandak (chhandak)]

Hi,

I am seeing the same issue in my setup . Issue is seen with 2.21.1-22.
Can somebody please have a look ?

Thanks and Regards,
Chhandak

On 12/22/15, 8:58 PM, "<email address hidden> on behalf of Vedamurthy Joshi" <<email address hidden> on behalf of <email address hidden>> wrote:

>** Changed in: juniperopenstack/r2.20
> Importance: Medium => High
>
>--
>You received this bug notification because you are a member of Contrail
>Systems engineering, which is subscribed to Juniper Openstack.
>https://bugs.launchpad.net/bugs/1463786
>
>Title:
> TOR Scale: UI: Delete all using Webui is failing
>
>Status in Juniper Openstack:
> New
>Status in Juniper Openstack r2.20 series:
> New
>Status in Juniper Openstack trunk series:
> New
>
>Bug description:
> I have 4k LIF and 8K VMI associated with one TOR. While trying to
> delete all the interface using Dleete_all option in WebUI, it is
> failing with following Error log ""Too many pending updates to
> RabbitMQ: 4096”]”
>
>
>
> Webui Log
> —————
> 2015-06-02T13:03:08.154Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/logical-interface/a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9] returned error ["ID a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9 does not exist"]
> 2015-06-02T13:03:08.236Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/b4112cf9-b8ad-4ce2-8e0c-ba0ca7f22fe9] returned error ["Too many pending updates to RabbitMQ: 4096"]
> 2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/virtual-machine-interface/87c36861-5865-4fe8-ad65-f1864af5d16f] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/instance-ip/18499836-0d67-4fd1-b3b4-27a6712a857f] returned error ["NotFoundException(_message=None)"]
> 2015-06-02T13:03:08.253Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de] returned error ["ID ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de does not exist"]
> 2015-06-02T13:03:08.256Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/d173d008-9583-4aab-8193-b95b6f697bd7] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.261Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.264Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/1446e62d-6f8e-43ae-9578-01d3de9788ba] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.273Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/c1841abd-04cf-4b19-b1fe-9711aacd92e2] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.321Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to RabbitMQ: 4097"]
> 2015-06-02T13:03:08.321Z - ^[[31merror^[[39m: Encountered broken link: /logical-interface/0d498491-3c45-4900-8519-8204f4fc9161
> 2015-06-02T13:03:08.322Z - ^[[3...

[Jeba Paulaiyan (jebap)]

+Sanju.

Ranjeet is on vacation from today.

On 1/11/16, 5:29 PM, "Chhandak Mukherjee" <email address hidden> wrote:

>Hi,
>
>I am seeing the same issue in my setup . Issue is seen with 2.21.1-22.
>Can somebody please have a look ?
>
>Thanks and Regards,
>Chhandak
>
>
>
>
>On 12/22/15, 8:58 PM, "<email address hidden> on behalf of Vedamurthy Joshi" <<email address hidden> on behalf of <email address hidden>> wrote:
>
>>** Changed in: juniperopenstack/r2.20
>> Importance: Medium => High
>>
>>--
>>You received this bug notification because you are a member of Contrail
>>Systems engineering, which is subscribed to Juniper Openstack.
>>https://bugs.launchpad.net/bugs/1463786
>>
>>Title:
>> TOR Scale: UI: Delete all using Webui is failing
>>
>>Status in Juniper Openstack:
>> New
>>Status in Juniper Openstack r2.20 series:
>> New
>>Status in Juniper Openstack trunk series:
>> New
>>
>>Bug description:
>> I have 4k LIF and 8K VMI associated with one TOR. While trying to
>> delete all the interface using Dleete_all option in WebUI, it is
>> failing with following Error log ""Too many pending updates to
>> RabbitMQ: 4096”]”
>>
>>
>>
>> Webui Log
>> —————
>> 2015-06-02T13:03:08.154Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/logical-interface/a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9] returned error ["ID a345bb15-caad-4a3b-b9e8-e66fcdfb9ff9 does not exist"]
>> 2015-06-02T13:03:08.236Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/b4112cf9-b8ad-4ce2-8e0c-ba0ca7f22fe9] returned error ["Too many pending updates to RabbitMQ: 4096"]
>> 2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/virtual-machine-interface/87c36861-5865-4fe8-ad65-f1864af5d16f] returned error ["Too many pending updates to RabbitMQ: 4097"]
>> 2015-06-02T13:03:08.243Z - ^[[31merror^[[39m: URL [http://192.168.22.1:9100/instance-ip/18499836-0d67-4fd1-b3b4-27a6712a857f] returned error ["NotFoundException(_message=None)"]
>> 2015-06-02T13:03:08.253Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de] returned error ["ID ddc0b34c-f4e6-4f35-8aa2-9677a3ba75de does not exist"]
>> 2015-06-02T13:03:08.256Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/d173d008-9583-4aab-8193-b95b6f697bd7] returned error ["Too many pending updates to RabbitMQ: 4097"]
>> 2015-06-02T13:03:08.261Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to RabbitMQ: 4097"]
>> 2015-06-02T13:03:08.264Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/1446e62d-6f8e-43ae-9578-01d3de9788ba] returned error ["Too many pending updates to RabbitMQ: 4097"]
>> 2015-06-02T13:03:08.273Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/instance-ip/c1841abd-04cf-4b19-b1fe-9711aacd92e2] returned error ["Too many pending updates to RabbitMQ: 4097"]
>> 2015-06-02T13:03:08.321Z - ^[[31merror^[[39m: URL [http://192.168.22.3:9100/logical-interface/0d498491-3c45-4900-8519-8204f4fc9161] returned error ["Too many pending updates to Ra...

[OpenContrail Admin (ci-admin-f)]

bug update...

[Hampapur Ajay (hajay)]

will be tackled as part of
https://bugs.launchpad.net/juniperopenstack/+bug/1589014