Exceptions in df-controller log file for tempest

[Omer Anson]

The following exception is seen in the log: (Taken from tempest log for patch https://review.openstack.org/#/c/503965/ (DHCP port per lswitch))

For instance:

Oct 29 15:25:26.331040 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology [-] Exception occurred when handling ovs port offline event: AttributeError: 'OvsLportMapping' object has no attribute 'id'
Oct 29 15:25:26.331157 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology Traceback (most recent call last):
Oct 29 15:25:26.331269 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology File "/opt/stack/new/dragonflow/dragonflow/controller/topology.py", line 120, in ovs_port_deleted
Oct 29 15:25:26.331379 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology handler(ovs_port)
Oct 29 15:25:26.331489 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology File "/opt/stack/new/dragonflow/dragonflow/controller/topology.py", line 236, in _vm_port_deleted
Oct 29 15:25:26.331592 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology self._del_from_topic_subscribed(topic, lport.id)
Oct 29 15:25:26.331739 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology AttributeError: 'OvsLportMapping' object has no attribute 'id'
Oct 29 15:25:26.331859 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.controller.topology

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Oct 29 15:31:16.678222 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework [-] Error while calling %(func)r(*%(_args)r, **%(kw)r): AttributeError: 'NoneType' object has no attribute 'version'
Oct 29 15:31:16.678420 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework Traceback (most recent call last):
Oct 29 15:31:16.678578 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/db/model_framework.py", line 108, in _emit
Oct 29 15:31:16.678762 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework cb(self, *args, **kwargs)
Oct 29 15:31:16.678926 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_base.py", line 628, in _add_port_event_handler
Oct 29 15:31:16.679094 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._add_port(lport)
Oct 29 15:31:16.679239 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_proactive.py", line 92, in _add_port
Oct 29 15:31:16.679384 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._add_forward_to_port_flow(dst_ip, dst_mac, network_key, port_key)
Oct 29 15:31:16.679528 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_proactive.py", line 122, in _add_forward_to_port_flow
Oct 29 15:31:16.679689 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework if dst_ip.version == n_const.IP_VERSION_4:
Oct 29 15:31:16.679846 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework AttributeError: 'NoneType' object has no attribute 'version'
Oct 29 15:31:16.679999 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Oct 29 15:31:21.121518 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework [-] Error while calling %(func)r(*%(_args)r, **%(kw)r): AttributeError: 'NoneType' object has no attribute 'version'
Oct 29 15:31:21.121684 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework Traceback (most recent call last):
Oct 29 15:31:21.121832 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/db/model_framework.py", line 108, in _emit
Oct 29 15:31:21.121977 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework cb(self, *args, **kwargs)
Oct 29 15:31:21.122121 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_base.py", line 771, in _remove_port_event_handler
Oct 29 15:31:21.122272 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._remove_port(lport)
Oct 29 15:31:21.122414 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_proactive.py", line 151, in _remove_port
Oct 29 15:31:21.122564 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._remove_forward_to_port_flow(dst_ip, network_key)
Oct 29 15:31:21.122739 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/l3_proactive.py", line 173, in _remove_forward_to_port_flow
Oct 29 15:31:21.122893 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework if dst_ip.version == n_const.IP_VERSION_4:
Oct 29 15:31:21.123038 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework AttributeError: 'NoneType' object has no attribute 'version'
Oct 29 15:31:21.123193 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Oct 29 15:31:40.004350 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework [-] Error while calling %(func)r(*%(_args)r, **%(kw)r): KeyError: u'1c4a265c-2884-4fab-b8f3-745ac717adef'
Oct 29 15:31:40.004490 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework Traceback (most recent call last):
Oct 29 15:31:40.004637 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/db/model_framework.py", line 108, in _emit
Oct 29 15:31:40.004765 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework cb(self, *args, **kwargs)
Oct 29 15:31:40.004911 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/dnat.py", line 540, in _local_port_unbound
Oct 29 15:31:40.005052 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._uninstall_local_floatingip(floatingip, emit_unbind=False)
Oct 29 15:31:40.005181 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/dnat.py", line 499, in _uninstall_local_floatingip
Oct 29 15:31:40.005312 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework port_locator.clear_port_binding(floatingip.floating_lport)
Oct 29 15:31:40.005444 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/port_locator.py", line 40, in clear_port_binding
Oct 29 15:31:40.005567 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework _locations.pop(lport.id)
Oct 29 15:31:40.005699 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework KeyError: u'1c4a265c-2884-4fab-b8f3-745ac717adef'
Oct 29 15:31:40.005833 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Oct 29 15:31:50.437750 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework [-] Error while calling %(func)r(*%(_args)r, **%(kw)r): ReferencedObjectNotFound: Referenced object not found. proxy=LogicalPortProxy(id=a8f5027b-77fe-4d69-9948-91eea8d14e99)
Oct 29 15:31:50.437887 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework Traceback (most recent call last):
Oct 29 15:31:50.438005 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/db/model_framework.py", line 108, in _emit
Oct 29 15:31:50.438122 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework cb(self, *args, **kwargs)
Oct 29 15:31:50.438230 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/dnat.py", line 453, in _update_floatingip
Oct 29 15:31:50.438377 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework self._uninstall_floatingip(orig_floatingip)
Oct 29 15:31:50.438484 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/controller/apps/dnat.py", line 477, in _uninstall_floatingip
Oct 29 15:31:50.438601 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework if floatingip.lport.is_local:
Oct 29 15:31:50.438746 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework File "/opt/stack/new/dragonflow/dragonflow/db/model_proxy.py", line 87, in __getattr__
Oct 29 15:31:50.438865 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework raise exceptions.ReferencedObjectNotFound(proxy=self)
Oct 29 15:31:50.438987 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework ReferencedObjectNotFound: Referenced object not found. proxy=LogicalPortProxy(id=a8f5027b-77fe-4d69-9948-91eea8d14e99)
Oct 29 15:31:50.439120 ubuntu-xenial-inap-mtl01-0000565193 df-local-controller[17324]: ERROR dragonflow.db.model_framework

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/517814

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/519267

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/519273

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/519276

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519267
Committed: https://git.openstack.org/cgit/openstack/dragonflow/commit/?id=1cf402500aeb6a2b279614b8fb452da7bdf6ed1b
Submitter: Zuul
Branch: master

commit 1cf402500aeb6a2b279614b8fb452da7bdf6ed1b
Author: Shachar Snapiri <email address hidden>
Date: Mon Nov 13 10:58:44 2017 +0200

    Resolve an exception in topology.py

    There is a typo in the topology file where there is access to a member
    with incorrect name (probably due to incomplete patch 480195).

    Partial-Bug: #1728518
    Change-Id: Id22318499ac5299c36478a5ea3b7efedf22bb8e8

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519273
Committed: https://git.openstack.org/cgit/openstack/dragonflow/commit/?id=7f9afc7d1460768ecfc44ed7de173f6fc7848508
Submitter: Zuul
Branch: master

commit 7f9afc7d1460768ecfc44ed7de173f6fc7848508
Author: Shachar Snapiri <email address hidden>
Date: Mon Nov 13 11:11:31 2017 +0200

    Add support for multiple/no L3 addresses on port

    As a port may have multiple IP addresses (IPv4/IPv6), we should check
    all of them and not just look at the first one as the current code does.
    Also the new code handles the valid scenario where the port has no L3
    address at all.

    Change-Id: I616731d31e1ec12d50c919d393d0d3990e8d2547
    Partial-Bug: #1728518

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/517814
Committed: https://git.openstack.org/cgit/openstack/dragonflow/commit/?id=4f11bc8f5bbd96d6621569f8c2c42f3e003a426e
Submitter: Zuul
Branch: master

commit 4f11bc8f5bbd96d6621569f8c2c42f3e003a426e
Author: Shachar Snapiri <email address hidden>
Date: Sun Nov 5 11:47:49 2017 +0200

    Resolve dhcp exceptions found in the tempest log

    Fixed issues in the code revealed by exceptions in the tempest build
    logs.
    Ports with IPv6 address only were added to the dhcp, which only handles
          IPv4 packets

    Note - DHCP should be revisited as the code assumes each port may have
    just one address - which is not correct.

    Change-Id: I14039b3e5752b8ecbc7505a5cb2da3965484c1f6
    Closes-Bug: #1728518

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519276
Committed: https://git.openstack.org/cgit/openstack/dragonflow/commit/?id=095c81e0ad1ee018aae3ecc248b43b142415c95d
Submitter: Zuul
Branch: master

commit 095c81e0ad1ee018aae3ecc248b43b142415c95d
Author: Shachar Snapiri <email address hidden>
Date: Mon Nov 13 11:16:25 2017 +0200

    Fix exceptions occuring in the DNAT log

    Use case: Creating a VM, associating a floatin IP with it, and then
    deleting that VM, multiple exceptions occured.

    Two issues found:
    - reference to removed ports
    - uninstall and unbind methods were both calling each other.
      added a cache to identify ports in process so we will avoid these
      loop calls

    Partial-Bug: #1728518
    Change-Id: Ib6c752de33d9356e83dcdb32d9a41dde1faad033