neutron agent. list API lacks sort and page feature

Bug #1877254 reported by yong sheng gong on 2020-05-07
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Medium
Unassigned

Bug Description

We have hundreds of neutron agents deployed, but currently, the agent list API does not support sorting and paging, which is very bad.

Slawek Kaplonski (slaweq) wrote :
Download full text (7.5 KiB)

What version are You using? I just checked on RHOSP-16 which is based on stable/train and the result is:

(overcloud) [stack@undercloud-0 ~]$ neutron agent-list --sort-column agent_type
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+--------------------+---------------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+--------------------------------------+--------------------+---------------------------+-------------------+-------+----------------+---------------------------+
| 643a494d-ad06-4455-9c6b-6ad35f7b2f5e | DHCP agent | controller-2.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 8a092f5b-f415-4473-ac8e-9ca333b037bc | DHCP agent | controller-0.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 8a384ede-4ccb-4321-b170-2448344b92fd | DHCP agent | controller-1.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 2ec9c0d6-57d8-4803-9b44-942542e53d6e | L3 agent | controller-0.redhat.local | nova | :-) | True | neutron-l3-agent |
| 758c19ef-0bb3-48bb-a757-90956b830914 | L3 agent | controller-1.redhat.local | nova | :-) | True | neutron-l3-agent |
| 9cec9036-0d0f-432a-a0ea-4420f1e88921 | L3 agent | controller-2.redhat.local | nova | :-) | True | neutron-l3-agent |
| b0d7fb33-607a-4903-9bf3-987743f237a7 | L3 agent | compute-1.redhat.local | nova | :-) | True | neutron-l3-agent |
| bf84115b-fc33-41c6-8734-2cbe1c9a7a5c | L3 agent | compute-0.redhat.local | nova | :-) | True | neutron-l3-agent |
| 386111f1-563c-4a2c-9b41-b292a1aa876e | Metadata agent | compute-0.redhat.local | | :-) | True | neutron-metadata-agent |
| 43d33cd1-c817-4755-8628-e759402acc5b | Metadata agent | controller-0.redhat.local | | :-) | True | neutron-metadata-agent |
| ad765ce7-f2e2-4db3-8ff7-1765e5e5bb93 | Metadata agent | controller-2.redhat.local | | :-) | True | neutron-metadata-agent |
| c832081f-d598-483c-bb57-4f616e631fbb | Metadata agent | controller-1.redhat.local | | :-) | True | neutron-metadata-agent |
| dbd82bdb-43d3-46de-99bc-a8364f260457 | Metadata agent | compute-1.redhat.local | | :-) | True | neutron-metadata-agent |
| 274076b0-56b1-407a-9903-2a6529d6be77 | Open vSwitch agent | compute-1.redhat.local | | :-) | True | neutron-openvswitch-agent |
| 98ed6974-b379-4eb6-8267-d76cd38f9379 | Open vSwitch agent | controller-0.redhat.local | | :-) | True | neutron-openvswitch-...

Read more...

Miguel Lavalle (minsel) on 2020-05-07
Changed in neutron:
status: New → Incomplete
importance: Undecided → Low
yangkai (yangkaiopenstack) wrote :

Check on stable/train and the result is:

`neutron agent-list --sort-column binary` is sorted by neutronclient

`neutron agent-list --sort-key binnary` will be KO

DEBUG: keystoneauth.session REQ: curl -g -i -X GET http://overcloud.internal.vim1.local:9696/v2.0/agents?sort_key=binnary&sort_dir=asc -H "Accept:
 application/json" -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA256}186f3682c2ce0e17e2e3e74077285d319e5347ef6fca7fb78ccd6554fd21473
e"

DEBUG: neutronclient.v2_0.client Error message: {"NeutronError": {"message": "[u'binnary'] is invalid attribute for sort_keys", "type": "HTTPBadRequest", "detail": ""}}

So we need implement it in neutron-server

Miguel Lavalle (minsel) wrote :

I am testing in near master (built it about 6 weeks ago). I can see that the API behavior is inconsistent. If I do:

$ neutron net-list --sort_key name --sort_dir asc

or

$ neutron net-list --sort_key name --sort_dir desc

I get the networks sorted by name correctly, as indicated by the direction argument

If I do:

$ neutron net-list --sort_key name --debug

I get:

DEBUG: keystoneauth.session RESP: [400] Connection: keep-alive Content-Length: 123 Content-Type: application/json Date: Sun, 10 May 2020 17:42:41 GMT X-Openstack-Request-Id: req-3aa8373d-1d00-4a89-b5a5-a14782aa3a46
DEBUG: keystoneauth.session RESP BODY: {"NeutronError": {"type": "HTTPBadRequest", "message": "The number of sort_keys and sort_dirs must be same", "detail": ""}}
DEBUG: keystoneauth.session GET call to network for http://192.168.33.12:9696/v2.0/networks?sort_key=name used request id req-3aa8373d-1d00-4a89-b5a5-a14782aa3a46

Now, in the case of:

$ neutron agent-list --sort_key binary --sort_dir asc

$ neutron agent-list --sort_key binary --sort_dir desc

$ neutron agent-list --sort_key binary

$ neutron agent-list --sort_key binnary

$ neutron agent-list --sort_key binnary --sort_dir asc

I get the exam same list all the time. So the API is just ignoring the sort arguments.

This bug is related to https://bugs.launchpad.net/neutron/+bug/1749820, which was marked as closed. So let's keep this one and continue the conversation here

Changed in neutron:
status: Incomplete → Triaged
Miguel Lavalle (minsel) wrote :
Changed in neutron:
importance: Low → Medium
Slawek Kaplonski (slaweq) wrote :

I marked this bug as low-hanging-fruit bug as IMO it is something which should be easy to fix. If someone wants to work on it, feel free to join #openstack-neutron channel @freenode and ask about any help You need (You can ping e.g. slaweq or mlavalle there).
Also I think that https://github.com/openstack/neutron/blob/master/neutron/db/agents_db.py#L298 is probably place to start looking for a fix as it probably should accepts also sorts, limits, marker and page_reverse parameters.

tags: added: api low-hanging-fruit
Slawek Kaplonski (slaweq) wrote :

Related patch is proposed https://review.opendev.org/#/c/726724/

Changed in neutron:
assignee: nobody → vinay harsha mitta (vinay7)
Changed in neutron:
status: Triaged → In Progress

Reviewed: https://review.opendev.org/738471
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=5b035972e567e1426657218c52b0ebc06860d3ef
Submitter: Zuul
Branch: master

commit 5b035972e567e1426657218c52b0ebc06860d3ef
Author: vinay_m <email address hidden>
Date: Mon Jun 29 21:22:54 2020 +0530

    Neutron agent-list API added with sort_key keyword

    Defining a API Extention for enabling sorting in agent
    -list attributes.

    Added "is_sort_key" keyword for attributes of agent.
    Neutron API can use "sort_key" and "sort_dir" for sorting on resulted
    list.

    Change-Id: I3155a4f1de664e4ff8e32ccc4314927ea462048c
    Related-Bug: #1877254

Change abandoned by vinay harsha mitta (<email address hidden>) on branch: master
Review: https://review.opendev.org/738475

Related fix proposed to branch: master
https://review.opendev.org/#/c/743953/

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: vinay harsha mitta (vinay7) → nobody
status: In Progress → New
tags: added: timeout-abandon

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/743953
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Changed in neutron:
status: New → In Progress
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers