veth vif status is down after compute reboot

Bug #1506997 reported by Senthilnathan Murugappan
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Fix Committed
Medium
Ashok Singh
R2.21.x
Fix Committed
Medium
Ashok Singh
R2.22.x
Fix Committed
Medium
Ashok Singh
R3.0
Fix Committed
Medium
Ashok Singh
Trunk
Fix Committed
Medium
Ashok Singh

Bug Description

root@b1s21:/var/log/contrail# curl -s http://b1s21.contrail.juniper.net:8085/Snh_ItfReq?name=veth3fd177d9-0 | xmllint --pretty 1 - | grep active
          <active type="string" identifier="5">Inactive &lt; os-state-down ipv4_inactive ipv6_inactive l2_inactive &gt;</active>
          <l2_active type="string" identifier="28">L2 Inactive &lt; os-state-down &gt;</l2_active>
          <ip6_active type="string" identifier="35">Ipv6 Inactive &lt; os-state-down no-ipv6-addr &gt;</ip6_active>
          <ipv4_active type="string" identifier="49">Ipv4 Inactive &lt; os-state-down &gt;</ipv4_active>
root@b1s21:/var/log/contrail#

root@b1s21:/var/log/contrail# ip link show veth3fd177d9-0
23: veth3fd177d9-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 6a:f3:ea:32:6c:30 brd ff:ff:ff:ff:ff:ff

root@b1s21:/var/log/contrail# ip netns exec vrouter-e3b3032f-89f0-48ae-a372-1bbcb6f83165 ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: int-f0ce38f1-d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 02:72:aa:72:89:36 brd ff:ff:ff:ff:ff:ff
3: gw-3fd177d9-0a: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 02:a7:45:34:35:9c brd ff:ff:ff:ff:ff:ff

However the status of the other veth interface bound to netns is active
root@b1s21:/var/log/contrail# curl -s http://b1s21.contrail.juniper.net:8085/Snh_ItfReq?name=vethf0ce38f1-d | xmllint --pretty 1 - | grep active
          <active type="string" identifier="5">Active</active>
          <l2_active type="string" identifier="28">L2 Active</l2_active>
          <ip6_active type="string" identifier="35">Ipv6 Inactive &lt; no-ipv6-addr &gt;</ip6_active>
          <ipv4_active type="string" identifier="49">Active</ipv4_active>

The gcore of the agent will be at /cs-shared/bugs/<bug-id>

Tags: vrouter
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Bug update]

bug update...

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/20464
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/20500
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/20501
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/20503
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/20504
Submitter: Ashok Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/20500
Committed: http://github.org/Juniper/contrail-controller/commit/30f2d1aa28be983dda5d64f323510c5d28b22a88
Submitter: Zuul
Branch: R3.0

commit 30f2d1aa28be983dda5d64f323510c5d28b22a88
Author: ashoksingh <email address hidden>
Date: Fri May 20 23:07:33 2016 +0530

Fix issue of interface becoming inactive even when it was active.

Issue
Agent interface was marked as “os_oper_state = down” even when the state of the interface in OS was Up. This can happen when agent index for interface is reused for a different interface and OS notification for old interface comes after notification for new interface.

Fix
As soon as we see the delete of interface, mark its index as invalid in VnswInterfaceListenerBase so that it cannot be used in updates until Add is seen again. Also when add is seen update the index in VnswInterfaceListenerBase before equeuing notification for resync of OS attributes. Also re-read OS index for interface whenever notification enqueued by VnswInterfaceListenerBase is processed.

Also define tracebuffer for logging VnswIf events. Also increase the size of tracebuffer for Config tracebuffer.

Closes-Bug: #1506997
(cherry picked from commit 81f9ae55b78cfbcbc7ee9eadbc7070743075775c)

Change-Id: I0d69c40370d50e7e9d943184fac8dd08778dfc32

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/20503
Committed: http://github.org/Juniper/contrail-controller/commit/5b5f532ac7638481c0a5129003e4a1bc3670f836
Submitter: Zuul
Branch: R2.20

commit 5b5f532ac7638481c0a5129003e4a1bc3670f836
Author: ashoksingh <email address hidden>
Date: Sat May 21 23:00:48 2016 +0530

Fix issue of interface becoming inactive even when it was active.

Issue
Agent interface was marked as “os_oper_state = down” even when the state of the interface in OS was Up. This can happen when agent index for interface is reused for a different interface and OS notification for old interface comes after notification for new interface.

Fix
As soon as we see the delete of interface, mark its index as invalid in VnswInterfaceListenerBase so that it cannot be used in updates until Add is seen again. Also when add is seen update the index in VnswInterfaceListenerBase before equeuing notification for resync of OS attributes. Also re-read OS index for interface whenever notification enqueued by VnswInterfaceListenerBase is processed.

Also define tracebuffer for logging VnswIf events. Also increase the size of tracebuffer for Config tracebuffer.

Closes-Bug: #1506997
(cherry picked from commit fd4e9378164b09b79207183d2e3a37c96e486805)

Change-Id: I0ca0ebcbd9724eae627b478755bfe274eac4b31c

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/20501
Committed: http://github.org/Juniper/contrail-controller/commit/fd4e9378164b09b79207183d2e3a37c96e486805
Submitter: Zuul
Branch: R2.21.x

commit fd4e9378164b09b79207183d2e3a37c96e486805
Author: ashoksingh <email address hidden>
Date: Sat May 21 23:00:48 2016 +0530

Fix issue of interface becoming inactive even when it was active.

Issue
Agent interface was marked as “os_oper_state = down” even when the state of the interface in OS was Up. This can happen when agent index for interface is reused for a different interface and OS notification for old interface comes after notification for new interface.

Fix
As soon as we see the delete of interface, mark its index as invalid in VnswInterfaceListenerBase so that it cannot be used in updates until Add is seen again. Also when add is seen update the index in VnswInterfaceListenerBase before equeuing notification for resync of OS attributes. Also re-read OS index for interface whenever notification enqueued by VnswInterfaceListenerBase is processed.

Also define tracebuffer for logging VnswIf events. Also increase the size of tracebuffer for Config tracebuffer.

Change-Id: Iec418188faf1bb6839deaad41174ccecdcfc5b04
Closes-Bug: #1506997

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/20504
Committed: http://github.org/Juniper/contrail-controller/commit/5575602d71163b6b463369cc6c1d1f4da4c533c5
Submitter: Zuul
Branch: R2.22.x

commit 5575602d71163b6b463369cc6c1d1f4da4c533c5
Author: ashoksingh <email address hidden>
Date: Sat May 21 23:00:48 2016 +0530

Fix issue of interface becoming inactive even when it was active.

Issue
Agent interface was marked as “os_oper_state = down” even when the state of the interface in OS was Up. This can happen when agent index for interface is reused for a different interface and OS notification for old interface comes after notification for new interface.

Fix
As soon as we see the delete of interface, mark its index as invalid in VnswInterfaceListenerBase so that it cannot be used in updates until Add is seen again. Also when add is seen update the index in VnswInterfaceListenerBase before equeuing notification for resync of OS attributes. Also re-read OS index for interface whenever notification enqueued by VnswInterfaceListenerBase is processed.

Also define tracebuffer for logging VnswIf events. Also increase the size of tracebuffer for Config tracebuffer.

Closes-Bug: #1506997
(cherry picked from commit fd4e9378164b09b79207183d2e3a37c96e486805)

Change-Id: Ia46f308b6cd008c4aa0906a8be1f0fff65280f91

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/20464
Committed: http://github.org/Juniper/contrail-controller/commit/81f9ae55b78cfbcbc7ee9eadbc7070743075775c
Submitter: Zuul
Branch: master

commit 81f9ae55b78cfbcbc7ee9eadbc7070743075775c
Author: ashoksingh <email address hidden>
Date: Fri May 20 23:07:33 2016 +0530

Fix issue of interface becoming inactive even when it was active.

Issue
Agent interface was marked as “os_oper_state = down” even when the state of the interface in OS was Up. This can happen when agent index for interface is reused for a different interface and OS notification for old interface comes after notification for new interface.

Fix
As soon as we see the delete of interface, mark its index as invalid in VnswInterfaceListenerBase so that it cannot be used in updates until Add is seen again. Also when add is seen update the index in VnswInterfaceListenerBase before equeuing notification for resync of OS attributes. Also re-read OS index for interface whenever notification enqueued by VnswInterfaceListenerBase is processed.

Also define tracebuffer for logging VnswIf events. Also increase the size of tracebuffer for Config tracebuffer.

Change-Id: I8811edcc89a809c179d860fb6583324b9816fb69
Closes-Bug: #1506997

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.