07/19/2014 04:57:01 AM [nodec22:Schema:Config:0]: Exception Unknown id: Error: oper 1 url /fqname-to-id body {"fq_name": ["default-domain", "TestBasicVMVN4-45770645", "vn222"], "type": "virtual-network"} response Name [u'default-domain', u'TestBasicVMVN4-45770645', u'vn222'] not found while creating VirtualNetworkST for default-domain:TestBasicVMVN4-45770645:vn222
This means that schema transformer received this message from ifmap, but it wasn't there when it queried api server. On further debugging, I found that API server is not draining rabbit queues:
I see the following in the logs:
07/19/2014 04:57:01 AM [nodec22: Schema: Config: 0]: Exception Unknown id: Error: oper 1 url /fqname-to-id body {"fq_name": ["default-domain", "TestBasicVMVN4 -45770645" , "vn222"], "type": "virtual-network"} response Name [u'default-domain', u'TestBasicVMVN 4-45770645' , u'vn222'] not found while creating VirtualNetworkST for default- domain: TestBasicVMVN4- 45770645: vn222
This means that schema transformer received this message from ifmap, but it wasn't there when it queried api server. On further debugging, I found that API server is not draining rabbit queues:
root@nodec22:~# rabbitmqctl list_queues nodeh4- 9100 950893 nodeh5- 9100 948882 nodec22- 9100 928005
Listing queues ...
vnc_config.
vnc_config.
vnc_config.
Rabbitmq logs show this error:
=ERROR REPORT==== 18-Jul- 2014::14: 06:21 === rabbit@ nodec22) : ** ERROR ** mnesia_event got {inconsistent_ database, starting_ partitioned_ network, rabbit@nodeh5}
Mnesia(
We will have to recluster rabbitmq.