"network-segment-ranges" doesn't return the project_id

[Rodolfo Alonso]

The client call "GET /v2.0/network-segment-ranges" didn't return the project_id in the response [1]. This behavior was seen only in this test execution. Other tests [2] doesn't reproduce this error.

I'm filling this bug just to track a possible error in the API/Client/SDK.

[1] http://logs.openstack.org/27/642527/4/check/neutron-tempest-plugin-api/9aeecb7/controller/logs/tempest_log.txt.gz#_2019-04-15_12_46_03_016
[2] http://logs.openstack.org/27/642527/4/check/neutron-tempest-plugin-api/99cd8c8/controller/logs/tempest_log.txt.gz#_2019-05-07_22_26_39_882

[Lajos Katona (lajos-katona)]

In a fresh devstack with curl I have no project_id in the response:
$ curl -H "X-Auth-Token: $MY_TOKEN" -X GET http://127.0.0.1:9696/v2.0/network-segment-ranges/ade20ea9-c2fe-47c7-805d-591f4e147ddc |json_pp
{
   "network_segment_range" : {
      "used" : {
         ...
      },
      "default" : true,
      "revision_number" : 0,
      "tags" : [],
      "minimum" : 200,
      "physical_network" : "physnet1",
      "id" : "ade20ea9-c2fe-47c7-805d-591f4e147ddc",
      "created_at" : "2019-05-08T11:56:47Z",
      "shared" : true,
      "network_type" : "vlan",
      "available" : [],
      "name" : "",
      "maximum" : 400
   }
}

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/687868

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687868
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=70f7b3222a9805ac1c58c25572b2e4e62de4d90a
Submitter: Zuul
Branch: master

commit 70f7b3222a9805ac1c58c25572b2e4e62de4d90a
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Thu Oct 10 11:48:57 2019 +0000

    Check parameters returned in "list_network_segment_ranges"

    Check that the DB registers returned by listing the network segment
    ranges are the same as in single register retrieval (show vs list).

    Change-Id: I9aee7f2a3108e605c203760a981ac226ecb33ea9
    Related-Bug: #1828205

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

I've detected a curious behavior in the Neutron API response which is causing this error. The tempests FW uses it's own client. This client does not use the pecan interface but the base API. The Neutron API has a set of controllers, one per service plugin enabled (agent, availability_zone, router, flavor, etc).

Service plugin "network_segment_range" API is defined in neutron-lib. In particular this service plugin API definition has "project_id" but not "tenant_id" (newer API definitions do not have "tenant_id"). NOTE: the migration from "tenant_id" to "project_id" is still pending.

When an GET call is done, the plugin retrieves the DB register, creates the OVO object and converts it to dict. In case the network segment range is not shared (the default behavior in the test case), the register MUST have a project ID.

Then the Controller object returns the value in the "show" method [1]. The raw data is transformed in "_view" [2]. This method filters the attributes in the result dictionary. Here [3] is where the dictionary fields are filtered according to the neutron-lib API definition (remember that for this service plugin there is no "tenant_id"). If "tenant_id" is in the dictionary, it is removed from it ALONG WITH "project_id".

Why "tenant_id" is added to this dictionary? Because by default, an OVO with "project_id" in its fields will add automatically "tenant_id" [4] to the OVO "obj_extra_fields". When a NetworkSegmentRange object calls the parent "to_dict", this method will iterate through the fields and extra fields. If "project_id" is set, "tenant_id" will be added to the dict. As seen in [3], this will trigger the deletion of both parameters.

---------------------------------------

Why, sometimes, this test is passing? Well, I still don't have an answer but I saw that, with the Neutron server restarted, the explained behavior is as described. But I don't know why, the "network_segment_range" controller API receives the "tenant_id" definition (the dictionary containing the plugin API is somehow updated with this parameter). I'm still investigating who is doing this.

Regards.

[1]https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/api/v2/base.py#L383-L390
[2]https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/api/v2/base.py#L205-L206
[3]https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/api/v2/base.py#L174-L191
[4]https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/objects/base.py#L345-L350

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/707898

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/707898
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3053caaa02adaadfffa53fb1070aaa60f49419e8
Submitter: Zuul
Branch: master

commit 3053caaa02adaadfffa53fb1070aaa60f49419e8
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Feb 14 18:40:09 2020 +0000

    Workaround in NetworkSegmentRange OVO until "project_id" migration

    The "network_segment_range" service plugin API does not have
    "tenant_id" as parameter. However, because the migration from
    "tenant_id" to "project_id" did not finish, every OVO with
    "project_id" will also have "tenant_id" as an extra field. This
    extra field will be populated with the same value as the
    "project_id" field.

    If the Neutron API WSGI is used and "tenant_id" is not present in
    the API (defined in neutron-lib), both "tenant_id" and "project_id"
    parameters will be removed from the response.

    This patch introduces a workaround for this OVO until the migration
    is over.

    More information about this patch can be found in the bug c#4.

    Change-Id: I85a02d8b59e8a758826c110e00df84672fa93676
    Closes-Bug: #1828205

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/722656

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/722657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/722657
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=995018d7540bb4fdb47acf42cac554da40a5954c
Submitter: Zuul
Branch: stable/stein

commit 995018d7540bb4fdb47acf42cac554da40a5954c
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Feb 14 18:40:09 2020 +0000

    Workaround in NetworkSegmentRange OVO until "project_id" migration

    The "network_segment_range" service plugin API does not have
    "tenant_id" as parameter. However, because the migration from
    "tenant_id" to "project_id" did not finish, every OVO with
    "project_id" will also have "tenant_id" as an extra field. This
    extra field will be populated with the same value as the
    "project_id" field.

    If the Neutron API WSGI is used and "tenant_id" is not present in
    the API (defined in neutron-lib), both "tenant_id" and "project_id"
    parameters will be removed from the response.

    This patch introduces a workaround for this OVO until the migration
    is over.

    More information about this patch can be found in the bug c#4.

    Change-Id: I85a02d8b59e8a758826c110e00df84672fa93676
    Closes-Bug: #1828205
    (cherry picked from commit 3053caaa02adaadfffa53fb1070aaa60f49419e8)
    (cherry picked from commit c945e5ccbadf35d1422828f654a960caf2cd979c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/722656
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c945e5ccbadf35d1422828f654a960caf2cd979c
Submitter: Zuul
Branch: stable/train

commit c945e5ccbadf35d1422828f654a960caf2cd979c
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Feb 14 18:40:09 2020 +0000

    Workaround in NetworkSegmentRange OVO until "project_id" migration

    The "network_segment_range" service plugin API does not have
    "tenant_id" as parameter. However, because the migration from
    "tenant_id" to "project_id" did not finish, every OVO with
    "project_id" will also have "tenant_id" as an extra field. This
    extra field will be populated with the same value as the
    "project_id" field.

    If the Neutron API WSGI is used and "tenant_id" is not present in
    the API (defined in neutron-lib), both "tenant_id" and "project_id"
    parameters will be removed from the response.

    This patch introduces a workaround for this OVO until the migration
    is over.

    More information about this patch can be found in the bug c#4.

    Change-Id: I85a02d8b59e8a758826c110e00df84672fa93676
    Closes-Bug: #1828205
    (cherry picked from commit 3053caaa02adaadfffa53fb1070aaa60f49419e8)