[OVN]- openstack network agent list command doesn't display anything

[Eran Kuris]

The command line: 'openstack network agent list' doesn't display the status of OVN services of each node.

In OVN there are no agents (like in ODL) but I expected to get any indication of the connection status between OVN services of each node (Controller & compute nodes)

Version-Release: Pike

How reproducible: 100%

Steps to Reproduce:
1. Deploy Director HA Pike with OVN
2. type the command: 'openstack network agent list'

Actual results: no output!

Expected results:
Took example from ODL
# openstack network agent list
+--------------------------------------+----------------+--------------------------+-------------------+-------+-------+------------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+----------------+--------------------------+-------------------+-------+-------+------------------------------+
| 2f6fae0d-8cf4-4b99-8c6c-d371fe5827c8 | ODL L2 | controller-0.localdomain | None | :-) | UP | neutron-odlagent-portbinding |
| 4024d452-91cc-4b80-87b4-a771212744e1 | ODL L2 | compute-0.localdomain | None | :-) | UP | neutron-odlagent-portbinding |
| 69d9b928-54fd-42e2-ab4e-46744f98d1fc | Metadata agent | controller-0.localdomain | None | :-) | UP | neutron-metadata-agent |
| b0a28f38-e991-4768-87dc-cedf8a02e721 | DHCP agent | controller-0.localdomain | nova | :-) | UP | neutron-dhcp-agent |
| eecb8ce1-8589-411e-b5c6-72251af100e2 | ODL L2 | compute-1.localdomain | None | :-) | UP | neutron-odlagent-portbinding |

Expect- status of each OVN on each node-
# openstack network agent list
+--------------------------------------+----------------+--------------------------+-------------
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+----------------+--------------------------+-------------
| 2f6fae0d-8cf4-4b99-8c6c-d371fe5827c8 | OVN | controller-0.localdomain | None | :-) | UP | neutron-ovnagent-portbinding |
| 4024d452-91cc-4b80-87b4-a771212744e1 | OVN | compute-0.localdomain | None | :-) | UP | neutron-ovnagent-portbinding | |

[venkata anil (anil-venkata)]

I thought we can use "Connection" TABLE from ovn-sbdb. But unfortunately it is not creating a separate row for each ovn-controller client. So when a ovn-controller is down, we are not sure which ovn-controller is down. May be we need to enhance ovsdb-server to update the particulars of the inactive ovn-controller in external_ids/other_config columns of "Connection" Table.

Another approach can be, ovn-controller has to periodically update their status in ovn-sbdb, like the way neutron agents periodically report their status to neutron server?

Once we have the mechanism to identify the status of ovn-controller, we should work on how we display this information to users.

1) neutron agent-list
+--------------------------------------+--------------------+-------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+--------------------------------------+--------------------+-------------------+-------------------+-------+----------------+---------------------------+
| 4cd5fcdd-4771-48be-9e72-63f2b99f3490 | L3 agent | devstack.rdocloud | nova | xxx | True | neutron-l3-agent |
| 5e0fb37c-1167-4a44-a260-00dd4edc6fd1 | Metadata agent | devstack.rdocloud | | xxx | True | neutron-metadata-agent |
| 6aae495b-c967-4d20-b1bf-f2380ba054ea | DHCP agent | devstack.rdocloud | nova | xxx | True | neutron-dhcp-agent |
| fc6fd801-65dc-43e2-b374-de8e1560b545 | Open vSwitch agent | devstack.rdocloud | | xxx | True | neutron-openvswitch-agent |

We have to decide which agent types OVN support. It doesn't support DHCP agent. It has Metadata agent(how to report its status?). We can use "L3 agent" type for ovn-controller which has "enable-chassis-as-gw" flag. Do we need to have a separate agent type for computes or shall we go ahead with "Open vSwitch agent"?

Alternatively we can define separate agent types for OVN, example
OVN-Gateway
OVN-compute

I think we can also support below commands in OVN.
(neutron) l3-agent-list-hosting-router router1
+--------------------------------------+-------------------+----------------+-------+----------+
| id | host | admin_state_up | alive | ha_state |
+--------------------------------------+-------------------+----------------+-------+----------+
| 4cd5fcdd-4771-48be-9e72-63f2b99f3490 | devstack.rdocloud | True | xxx | |
+--------------------------------------+-------------------+----------------+-------+----------+
(neutron) router-list-on-l3-agent 4cd5fcdd-4771-48be-9e72-63f2b99f3490
+--------------------------------------+---------+----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------...

[venkata anil (anil-venkata)]

Design doc submitted for this bug https://review.openstack.org/#/c/552447/
A related patch https://review.openstack.org/#/c/550097/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/578225
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=1f9eda348666d09a6934a70bea8a9b7c309e70b0
Submitter: Zuul
Branch: master

commit 1f9eda348666d09a6934a70bea8a9b7c309e70b0
Author: Terry Wilson <email address hidden>
Date: Tue Jun 26 15:23:14 2018 +0000

    Add support for Neutron agent api

    Instead of polling and writing updates to the Neutron DB agents
    table, this patch replaces the neutron plugin agent api methods,
    being sure to maintain support for multiple drivers.

    Partial-Bug: #1719574
    Change-Id: I43e8b0a7914e3626dd848b04fdfc54fb0351bae6

[Reedip (reedip-banerjee-deactivatedaccount)]

Related patches [1],[2],[3]

[1]: https://review.openstack.org/#/c/583959/
[2]: https://review.openstack.org/#/c/585027/
[3]: https://review.openstack.org/#/c/578726/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/585027
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=7cf92ee960697d284f58ab9c26dab607b3685f57
Submitter: Zuul
Branch: master

commit 7cf92ee960697d284f58ab9c26dab607b3685f57
Author: Terry Wilson <email address hidden>
Date: Mon Jul 23 20:36:22 2018 +0000

    Use neutron's agent_down_time for agent liveness

    Instead of returning the immediate status, mimick ML2/OVS's
    agent_down_time handling. With this patch, we no longer have an
    issue where an agent will appear down because we didn't wait long
    enough for ovn-controller to update the Chassis and we don't have
    to wait before sending a response to API requests.

    Closes-Bug: #1719574
    Change-Id: Ib08473e4768c94ad3b0a223b72f1dad1d26769dd

[OpenStack Infra (hudson-openstack)]

Change abandoned by Lucas Alvares Gomes (<email address hidden>) on branch: master
Review: https://review.openstack.org/578726
Reason: I've squashed this into the next patch in the series since they were overlapping.

https://review.openstack.org/583959

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/606067

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/583959
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=53a1c87cbf2de25e4d18696987179f40f6c5d67f
Submitter: Zuul
Branch: master

commit 53a1c87cbf2de25e4d18696987179f40f6c5d67f
Author: Daniel Alvarez <email address hidden>
Date: Fri Jun 15 01:36:36 2018 +0000

    Register OVN metadata as an agent

    This patch registers and lists the OVN metadata agent with Neutron.

    Since the OVN Controller / Gateway use the Chassis UUID as its own
    agent UUID, the OVN Metadata creates can't re-use that. Instead, this
    patch generates a random UUID for each Metadata agent and saves it to the
    external_ids' column in the Chassis table. At startup time, the metadata
    agent will check if the "neutron:ovn-metadata-id" key is present in the
    Chassis external_id column and use that UUID if it's there, otherwise
    it will generate a random UUID for it.

    This patch uses the same AgentStats mechanism to identify whether the
    OVN metadata agent is alive or not by relying on the "nb_cfg" from the
    Chassis. The only difference from the OVN controller agent is that the
    metadata will save the "nb_cfg" of it's last update into the Chassis
    external_ids column.

    Co-Authored-By: Daniel Alvarez <email address hidden>
    Closes-Bug: #1719574
    Change-Id: Icfc9e76e8371d8f33b264df65ae3a7cbc0ca099e

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/606904

[OpenStack Infra (hudson-openstack)]

Change abandoned by Miguel Angel Ajo (<email address hidden>) on branch: master
Review: https://review.openstack.org/552447
Reason: Abandoning since this was finally implemented in a different way :) Thanks anil!

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/612306

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/612660

[OpenStack Infra (hudson-openstack)]

Change abandoned by Lucas Alvares Gomes (<email address hidden>) on branch: master
Review: https://review.openstack.org/612306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/606067
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=34fdaa90381b3d56819995aa5113f9f36b6984d0
Submitter: Zuul
Branch: stable/rocky

commit 34fdaa90381b3d56819995aa5113f9f36b6984d0
Author: Terry Wilson <email address hidden>
Date: Mon Jul 23 20:36:22 2018 +0000

    Use neutron's agent_down_time for agent liveness

    Instead of returning the immediate status, mimick ML2/OVS's
    agent_down_time handling. With this patch, we no longer have an
    issue where an agent will appear down because we didn't wait long
    enough for ovn-controller to update the Chassis and we don't have
    to wait before sending a response to API requests.

    Closes-Bug: #1719574
    Change-Id: Ib08473e4768c94ad3b0a223b72f1dad1d26769dd
    (cherry picked from commit 7cf92ee960697d284f58ab9c26dab607b3685f57)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/606904
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=93fccfb62ea0750fd10448b497f7a6cd4f12a828
Submitter: Zuul
Branch: stable/rocky

commit 93fccfb62ea0750fd10448b497f7a6cd4f12a828
Author: Daniel Alvarez <email address hidden>
Date: Fri Jun 15 01:36:36 2018 +0000

    Register OVN metadata as an agent

    This patch registers and lists the OVN metadata agent with Neutron.

    Since the OVN Controller / Gateway use the Chassis UUID as its own
    agent UUID, the OVN Metadata creates can't re-use that. Instead, this
    patch generates a random UUID for each Metadata agent and saves it to the
    external_ids' column in the Chassis table. At startup time, the metadata
    agent will check if the "neutron:ovn-metadata-id" key is present in the
    Chassis external_id column and use that UUID if it's there, otherwise
    it will generate a random UUID for it.

    This patch uses the same AgentStats mechanism to identify whether the
    OVN metadata agent is alive or not by relying on the "nb_cfg" from the
    Chassis. The only difference from the OVN controller agent is that the
    metadata will save the "nb_cfg" of it's last update into the Chassis
    external_ids column.

    Conflicts:
     networking_ovn/common/utils.py

    Co-Authored-By: Daniel Alvarez <email address hidden>
    Closes-Bug: #1719574
    Change-Id: Icfc9e76e8371d8f33b264df65ae3a7cbc0ca099e
    (cherry picked from commit 53a1c87cbf2de25e4d18696987179f40f6c5d67f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/612660
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=6c1ee31257b095fd698c2f3eb113afb64ae48a6b
Submitter: Zuul
Branch: master

commit 6c1ee31257b095fd698c2f3eb113afb64ae48a6b
Author: Daniel Alvarez <email address hidden>
Date: Mon Oct 22 11:16:06 2018 +0100

    Functional: Allow some time for propagating the healthcheck values

    This patch is changing the test_metadata_agent_healthcheck to allow
    some time (up to 5 seconds) for expected SB config value to be
    propagated.

    Prior to this patch, it was checked straight away and sometimes it
    it didn't match the expected value.

    Change-Id: Ia5308cda75571b624f84efa83cffe2acdc04f097
    Related-Bug: #1719574
    Co-Authored-By: Terry Wilson <email address hidden>
    Signed-off-by: Daniel Alvarez <email address hidden>

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/613871

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/613871
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=4a3f497fabca5f420f1a6fec4ffaa6e1ed69806d
Submitter: Zuul
Branch: stable/rocky

commit 4a3f497fabca5f420f1a6fec4ffaa6e1ed69806d
Author: Daniel Alvarez <email address hidden>
Date: Mon Oct 22 11:16:06 2018 +0100

    Functional: Allow some time for propagating the healthcheck values

    This patch is changing the test_metadata_agent_healthcheck to allow
    some time (up to 5 seconds) for expected SB config value to be
    propagated.

    Prior to this patch, it was checked straight away and sometimes it
    it didn't match the expected value.

    Change-Id: Ia5308cda75571b624f84efa83cffe2acdc04f097
    Related-Bug: #1719574
    Co-Authored-By: Terry Wilson <email address hidden>
    Signed-off-by: Daniel Alvarez <email address hidden>
    (cherry picked from commit 6c1ee31257b095fd698c2f3eb113afb64ae48a6b)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 6.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn 5.1.0 release.