Trace is seen when disabling neutron-openvswitch-agent with of_interface=native

[Inessa Vasilevskaya]

Mos 9.0, 3 controllers/1 compute.

Confuguration:
/etc/neutron/plugins/ml2/openvswitch_agent.ini has of_interface=native on all nodes.

After `pcs resource disable neutron-openvswitch-agent` the following trace is seen in neutron-openvswitch log:

2016-05-24 09:48:32 ERR ryu.lib.hub [-] hub: uncaught exception: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ryu/lib/hub.py", line 52, in _launch
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 532, in close
    self.uninstantiate(app_name)
  File "/usr/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 515, in uninstantiate
    app = self.applications.pop(name)
KeyError: 'ofctl_service'

[Sergey Kolekonov (skolekonov)]

Please note that of_interface=native was set manually and it's not the default option for MOS 9.0

[Inessa Vasilevskaya (ivasilevskaya)]

This is also seen on compute node when stopping neutron-openvswitch-agent via

service neutron-openvswitch-agent stop.

[Inessa Vasilevskaya (ivasilevskaya)]

That's the reason - https://review.openstack.org/#/c/257204/6.
The close is performed in 2 eventlet threads with the applications being uninstantiated twice (see ryu's lib code https://github.com/osrg/ryu/blob/master/ryu/base/app_manager.py#L544)

[Inessa Vasilevskaya (ivasilevskaya)]

https://review.openstack.org/#/c/326437/1 - upstream patch

[Inessa Vasilevskaya (ivasilevskaya)]

The issue was fixed in ryu 4.4 (verified on devstack), will be fixed in Newton by requirements update (https://review.openstack.org/#/c/336799/).