Nova-compute has not sent the delete to agent in this case. The file for the orphan vif was still observed under /var/lib/contrail/ports
Hi Andriy,
Thanks for your analysis. Since the file for orphan vif tapbfe87886-e0 is still present in /var/lib/contrail/ports/, nova-compute has not sent delete to agent and that is the reason why delete has not been sent to vrouter. We need to check as to why nova-compute didn’t send the delete.
Regards,
Ashok
On 2/10/16, 2:46 PM, "Andriy Berestovskyy" <email address hidden> wrote:
Hi Ashok,
Thanks for the very detailed instructions. Answering your questions:
(1)
I run the script and got one orphan vif tapbfe87886-e0.
Sandesh Trace Request Config shows just add messages for this vif
while for other interfaces there are adds and deletes.
(2)
Sure, the file for the vif is present in /var/lib/contrail/ports. The
instance id is 1cd58568-a830-4f45-ac0a-a42e88229516
In the nova-compute.log for the instance id there are few strange logs
(I cut out the dates):
WARNING nova.compute.manager [-] [instance:
1cd58568-a830-4f45-ac0a-a42e88229516] Instance shutdown by itself.
Calling the stop API. Current vm_state: active, current task_state:
None, current DB power_state: 4, current VM power_state: 4
INFO nova.compute.manager [req-d0242ecf-6bf8-4cc1-9836-edb3ca8c0c29
None] [instance: 1cd58568-a830-4f45-ac0a-a42e88229516] Instance is
already powered off in the hypervisor when stop is called.
WARNING nova.virt.libvirt.driver [-] [instance:
1cd58568-a830-4f45-ac0a-a42e88229516] During wait destroy, instance
disappeared
(3)
I guess due to (2) there are no delete messages sent to the vRouter.
Andriy
On Tue, Feb 9, 2016 at 3:30 PM, Ashok Singh R <email address hidden> wrote:
Hi Andriy,
I didn’t get chance to run your script.
Would it be possible for you to check the following in your setup?
This page should have logs which indicate whether the agent is receiving add/delete requests
(2) For Orphan vif, you can also check whether entry is present in directory.
/var/log/contrail/ports/
This directory should have one file for each port. In most cases this file is created/deleted by nova-compute on port-add/delete events. It should create/delete the file even when agent is dead
If the entry for the orphan vif is still present, please check the log file /var/log/nova/nova-compute.log to see if there are any errors/exceptions etc.
Did contrail-vrouter-agent receive the delete message?
If contrail-vrouter-agent received delete message, did it send delete
message to vrouter or not?
Nova-compute has not sent the delete to agent in this case. The file for the orphan vif was still observed under /var/lib/ contrail/ ports
Hi Andriy,
Thanks for your analysis. Since the file for orphan vif tapbfe87886-e0 is still present in /var/lib/ contrail/ ports/, nova-compute has not sent delete to agent and that is the reason why delete has not been sent to vrouter. We need to check as to why nova-compute didn’t send the delete.
Regards,
Ashok
On 2/10/16, 2:46 PM, "Andriy Berestovskyy" <email address hidden> wrote:
Hi Ashok,
Thanks for the very detailed instructions. Answering your questions:
(1)
I run the script and got one orphan vif tapbfe87886-e0.
Sandesh Trace Request Config shows just add messages for this vif
while for other interfaces there are adds and deletes.
(2) contrail/ ports. The a830-4f45- ac0a-a42e882295 16
Sure, the file for the vif is present in /var/lib/
instance id is 1cd58568-
In the nova-compute.log for the instance id there are few strange logs
(I cut out the dates):
WARNING nova.compute. manager [-] [instance: a830-4f45- ac0a-a42e882295 16] Instance shutdown by itself.
1cd58568-
Calling the stop API. Current vm_state: active, current task_state:
None, current DB power_state: 4, current VM power_state: 4
INFO nova.compute. manager [req-d0242ecf- 6bf8-4cc1- 9836-edb3ca8c0c 29 a830-4f45- ac0a-a42e882295 16] Instance is
None] [instance: 1cd58568-
already powered off in the hypervisor when stop is called.
WARNING nova.virt. libvirt. driver [-] [instance: a830-4f45- ac0a-a42e882295 16] During wait destroy, instance
1cd58568-
disappeared
(3)
I guess due to (2) there are no delete messages sent to the vRouter.
Andriy
On Tue, Feb 9, 2016 at 3:30 PM, Ashok Singh R <email address hidden> wrote:
Hi Andriy,
I didn’t get chance to run your script.
Would it be possible for you to check the following in your setup?
(1) node-ip> :8085/Snh_ SandeshTraceReq uest?x= Config
http://<compute-
This page should have logs which indicate whether the agent is receiving add/delete requests
(2) For Orphan vif, you can also check whether entry is present in directory. contrail/ ports/
/var/log/
This directory should have one file for each port. In most cases this file is created/deleted by nova-compute on port-add/delete events. It should create/delete the file even when agent is dead
If the entry for the orphan vif is still present, please check the log file /var/log/ nova/nova- compute. log to see if there are any errors/exceptions etc.
(3) node-ip> :8085/Snh_ SandeshTraceReq uest?x= KSync%20Interfa ce
http://<compute-
This page should have information on whether agent sent the request to vrouter and whether vrouter acked it or not.
Regards,
Ashok
On 2/8/16, 9:19 PM, "Andriy Berestovskyy" <email address hidden> wrote:
Hi Ashok,
I just run my old script and got the "orphan" vifs again, so the issue
is still there. Please find the script attached.
How do we check the messages Agent receives/sends?
Andriy
On Sun, Feb 7, 2016 at 9:32 AM, Ashok Singh R <email address hidden> wrote:
Hi Andriy,
I am working on bug /bugs.launchpad .net/juniperope nstack/ +bug/1495002. One of issues
https:/
mentioned in the bug is vif delete message gets lost. Do you have more
information on that? I have the following questions.
Did contrail- vrouter- agent receive the delete message? vrouter- agent received delete message, did it send delete
If contrail-
message to vrouter or not?
Regards,
Ashok