br-tun lost ports if openvswitch restart when l2pop enabled

Bug #1332450 reported by Chengli Xu on 2014-06-20
46
This bug affects 7 people
Affects Status Importance Assigned to Milestone
neutron
High
ZongKai LI

Bug Description

When openvswitch restart, ovs agent will reset br-tun, lose all tunnel network related ports/flows, and break all tunnel networks.
If l2 population used, We could maintain all l2 population fdb entries locally and recreate ports/flows . if not, set tunnel_sync = True works.

Tags: ovs Edit Tag help
Eugene Nikanorov (enikanorov) wrote :

are you talking about openvswitch restart or ovs agent restart?

tags: added: ovs
Changed in neutron:
importance: Undecided → High

Fix proposed to branch: master
Review: https://review.openstack.org/101581

Changed in neutron:
assignee: nobody → Chengli Xu (superekcah)
status: New → In Progress

openvswitch restarts and ovs agent not. There is a patch try to fix those problem (bug 1290486), but I think it's not complete yet.

As some guys suggested, I split it to two commits, one is for l2-pop used and the other for l2-pop not used

Change abandoned by Chengli Xu (<email address hidden>) on branch: master
Review: https://review.openstack.org/102532
Reason: covered by other change

Well, the problem would be partially resolved if l2 pop is not used, by bug 1292105

Eugene Nikanorov (enikanorov) wrote :

Reopening as proposed fix was abandoned

Changed in neutron:
status: In Progress → Confirmed
Kyle Mestery (mestery) wrote :

I can triage this one a bit more. Also, when you say OVS restart, do you mean "ovs-vswitchd" or OVS DB?

Aman Kumar (amank) wrote :

"service openvswitch-switch restart"

It restarts both "ovs-vswitchd" as well as "OVSDB"

Change abandoned by Kyle Mestery (<email address hidden>) on branch: master
Review: https://review.openstack.org/101581
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Romil Gupta (romilg) on 2015-02-25
Changed in neutron:
assignee: Chengli Xu (superekcah) → Romil Gupta (romilg)

Fix proposed to branch: master
Review: https://review.openstack.org/159775

Changed in neutron:
assignee: Romil Gupta (romilg) → Aman Kumar (amank)
status: Confirmed → In Progress
Kyle Mestery (mestery) on 2015-03-31
Changed in neutron:
milestone: none → liberty-1

Change abandoned by Kyle Mestery (<email address hidden>) on branch: master
Review: https://review.openstack.org/159775
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Thierry Carrez (ttx) on 2015-06-23
Changed in neutron:
milestone: liberty-1 → liberty-2
Changed in neutron:
assignee: Aman Kumar (amank) → Romil Gupta (romilg)
Thierry Carrez (ttx) on 2015-07-28
Changed in neutron:
milestone: liberty-2 → liberty-3

Can't see how this can merge in the next few days.

Changed in neutron:
milestone: liberty-3 → none

@Romil: are you actively working on this?

Romil Gupta (romilg) wrote :

No, hence unassigned to myself.

Changed in neutron:
assignee: Romil Gupta (romilg) → nobody
ZongKai LI (zongkai) wrote :

Hi guys, is anybody still working this?

ZongKai LI (zongkai) wrote :

I did a little test, yes, ports on br-tun will disappear after restart openvswitch, but not flows on br-tun.

For losing ports on br-tun, I consider the root reason is here:
https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py#L231
If we only restart openvswitch, neutron ovs-agent will keep ofport cache for ports on br-tun. And that will stop neutron ovs-agent to create tunnel to remove IP, so ports on br-tun will not be re-created.

As we have methods like get_port_name_list and get_port_ofport in https://github.com/openstack/neutron/blob/master/neutron/agent/common/ovs_lib.py, so I think we can add some logic to check whether ofport is in br-tun current existing ofports.

This way is not the best way, but will cause less coding changing.

Fix proposed to branch: master
Review: https://review.openstack.org/219807

Changed in neutron:
assignee: nobody → ZongKai LI (lzklibj)
ZongKai LI (zongkai) on 2015-09-06
summary: - br-tun lost ports/flows if openvswitch restart
+ br-tun lost ports/flows if openvswitch restart when l2pop enabled
ZongKai LI (zongkai) on 2015-09-08
summary: - br-tun lost ports/flows if openvswitch restart when l2pop enabled
+ br-tun lost ports if openvswitch restart when l2pop enabled

It this bug still an issue after https://review.openstack.org/#/c/182920 ?

ZongKai LI (zongkai) wrote :

@Rossella Thanks, the patch you mentioned handles flows on tun_br, so I changed this bug summary from "br-tun lost ports/flows" to "br-tun lost ports".

Change abandoned by ZongKai LI (<email address hidden>) on branch: master
Review: https://review.openstack.org/219807
Reason: In https://review.openstack.org/#/c/182920/58 , tun_br.reset_bridge is deprecated, that makes tunnel ports no longer be deleted. So issue #1332450 no more exists.

ZongKai LI (zongkai) wrote :

Per https://review.openstack.org/#/c/182920/58 , to keep tunnel connectivity, reset_bridge
isn't used anymore. And by that, tunnel ports will not be deleted anymore. This issue is no longer exist, mark it as invalid.

Changed in neutron:
status: In Progress → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers