[OVS graceful restart] Tunnel flows are not saved after ovs agent restart

Bug #1488108 reported by Kristina Berezovskaia
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
Critical
Sergey Yudin

Bug Description

On vxlan environment after restarting Open vSwitch agent on all controllers and computes connection to vm failed

Steps to reproduce:
1) Create 2 networks, router between them and boot 2 vms in different nets and on different computes
2) Start Ping between vms
3) Restart Open vSwitch agents on all controllers and computes with script:
ssh node-2 '. openrc && pcs resource disable p_neutron-plugin-openvswitch-agent'

sleep 15

#Enable ovs-agents on controllers
ssh node-2 '. openrc && pcs resource enable p_neutron-plugin-openvswitch-agent $1 $2 $3'

#Start ovs-agents on computes
 fuel nodes | grep compute | awk '{print $1; }' | xargs -I@ ssh node-@ service neutron-plugin-openvswitch-agent restart

#Sleep $time - this value should be tuned manually to be sufficient for all agents going to "alive" state
sleep 5

Expected results: after restarting connection will not brake (1-3 ping packets loss is allowed)
Current result: connections brake, VMs can't renew ip addresses, can't ping gateway.
Root cause: tun-br lacks proper flows. During restart open vswitch agent we can see that flows appeared with 0x0 and then dissapeared

iso:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "187"
  build_id: "2015-08-18_03-05-20"
  nailgun_sha: "4710801a2f4a6d61d652f8f1e64215d9dde37d2e"
  python-fuelclient_sha: "4c74a60aa60c06c136d9197c7d09fa4f8c8e2863"
  fuel-agent_sha: "57145b1d8804389304cd04322ba0fb3dc9d30327"
  fuel-nailgun-agent_sha: "e01693992d7a0304d926b922b43f3b747c35964c"
  astute_sha: "e24ca066bf6160bc1e419aaa5d486cad1aaa937d"
  fuel-library_sha: "0062e69db17f8a63f85996039bdefa87aea498e1"
  fuel-ostf_sha: "17786b86b78e5b66d2b1c15500186648df10c63d"
  fuelmain_sha: "c9dad194e82a60bf33060eae635fff867116a9ce"

neutron vxlan, 3 controllers, 2 computes

Changed in mos:
status: New → Confirmed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/neutron (openstack-ci/fuel-7.0/2015.1.0)

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Eugene Nikanorov <email address hidden>
Review: https://review.fuel-infra.org/10713

Changed in mos:
status: Confirmed → In Progress
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/neutron (openstack-ci/fuel-7.0/2015.1.0)

Reviewed: https://review.fuel-infra.org/10713
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: 192ef6a89e52582c85a8b43f40d67a2244fc6e54
Author: Eugene Nikanorov <email address hidden>
Date: Wed Aug 26 14:30:58 2015

Add correct cookie to the flows added through DeferredBridge

DeferredBridge is used to setup tunnel flows.
Without this fix ovs agent restart leaads to deletion of
flows created for tunelling.

Change-Id: I5f3b189b4f2ae55deb7de00bc5e2556e22aa2d27
Closes-Bug: #1488108

Changed in mos:
status: In Progress → Fix Committed
Sergey Yudin (tsipa740)
tags: added: on-verification
Changed in mos:
assignee: Eugene Nikanorov (enikanorov) → Sergey Yudin (tsipa740)
Revision history for this message
Sergey Yudin (tsipa740) wrote :

I've veryfied the bug using

this env http://jenkins-product.srt.mirantis.net:8080/job/7.0.ubuntu.bvt_2/248/console
with fuel-7.0-242-2015-08-27_09-08-42.iso

Seems like the problem have been resolved.
--- 192.168.113.5 ping statistics ---
172 packets transmitted, 170 packets received, 1% packet loss

Changed in mos:
status: Fix Committed → Fix Released
tags: removed: on-verification
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/neutron (openstack-ci/fuel-8.0/liberty)

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Eugene Nikanorov <email address hidden>
Review: https://review.fuel-infra.org/13307

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/neutron (openstack-ci/fuel-8.0/liberty)

Change abandoned by Ann Kamyshnikova <email address hidden> on branch: openstack-ci/fuel-8.0/liberty
Review: https://review.fuel-infra.org/13307
Reason: This was fixed in upstream patch https://review.openstack.org/#/c/218118/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.