Method "build_segment_queries_for_tenant_and_shared_ranges" returning empty query

[Rodolfo Alonso]

Method "build_segment_queries_for_tenant_and_shared_ranges" returns two queries:
- One for those network segment ranges matching the project_id
- One for those network segment ranges shared (no project_id, but available for everyone)

The first one, if "project_id" is not present in the filter variable, returns an empty list:
https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/plugins/ml2/drivers/helpers.py#L117

The returned queries are used in "allocate_partially_specified_segment": https://github.com/openstack/neutron/blob/6a8277d70ee28ae6fcb68a75634eb508d4e6952a/neutron/plugins/ml2/drivers/helpers.py#L197-L200

If the first object is not a query but an empty list, the code will fail.

UPDATE:
I've found some other issues related to this feature that should be addressed in order to have a healthy functionality. Those issues were found during the implementation of [1]

This service plugin creates, when the drivers are initialized (one per segmentation type: VLAN, VXLAN, GRE or Geneve), a default segment range not assigned to any project, with the min/max values defined statically in the neutron plugin config ("network_vlan_ranges", "vni_ranges", etc).

Then the administrator can create segment ranges for project. Those segment ranges do not overlap among them but can overlap with the default range.

When a network is created, the method "SegmentTypeDriver.allocate_partially_specified_segment" selects a segmentation ID from both the segment ranges assigned to the project AND the shared range.

That means:
- When the the segment ranges are depleted, the project user can always receive a segmentation from the default group. Why is then this feature needed?
- In this case, the user can have assigned a segmentation ID belonging to other project (this segmentation can fall under the interval defined in other segment range). There is no check for this.
- The tests implemented in [2] rely on the current buggy implementation of this method.

Currently this new feature does not perform what is intended to do.

[1] https://review.opendev.org/708027
[2] https://github.com/openstack/neutron-tempest-plugin/blob/b7e0eef8de92f6a70c16c879f6a9a20377e82882/neutron_tempest_plugin/api/admin/test_network_segment_range.py#L91

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/708027

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

More info about BP network-segment-range-management:
* Patches: https://review.opendev.org/#/q/network-segment-range-management
** DB and API patch: https://review.opendev.org/#/c/624708/
** Plugin patch: https://review.opendev.org/#/c/624709/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/710090

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rodolfo Alonso Hernandez (<email address hidden>) on branch: master
Review: https://review.opendev.org/708027
Reason: Superseded by https://review.opendev.org/#/c/710090/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/710090
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=046672247de56bad950e8267a57bd26205f354a0
Submitter: Zuul
Branch: master

commit 046672247de56bad950e8267a57bd26205f354a0
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Feb 26 10:39:19 2020 +0000

    Fix queries to retrieve allocations with network_segment_range

    Fixed the queries to retrieve the segment ID allocations when service
    plugin network_segment_range is enabled. With the previous
    implementation, a project user was able to allocate a segment ID
    belonging to other project segment range.

    The solution implemented was discussed in [1]:
    - A project user will retrieve segments from the project ranges.
    - When depleted, the segment IDs will be retrieved from the shared
      range, never using another project segment ID.

    [1]http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012736.html

    Change-Id: I953062d9ee8ee5ee9a9f07aff4a8222ac63ed525
    Closes-Bug: #1863423

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/712508

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/712508
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=bbe401aaf9bfdd77e1d43d547b2cdb436b1440c8
Submitter: Zuul
Branch: stable/train

commit bbe401aaf9bfdd77e1d43d547b2cdb436b1440c8
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Feb 26 10:39:19 2020 +0000

    Fix queries to retrieve allocations with network_segment_range

    Fixed the queries to retrieve the segment ID allocations when service
    plugin network_segment_range is enabled. With the previous
    implementation, a project user was able to allocate a segment ID
    belonging to other project segment range.

    The solution implemented was discussed in [1]:
    - A project user will retrieve segments from the project ranges.
    - When depleted, the segment IDs will be retrieved from the shared
      range, never using another project segment ID.

    [1]http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012736.html

    Conflicts:
        neutron/objects/network_segment_range.py
        neutron/objects/plugins/ml2/base.py
        neutron/objects/plugins/ml2/vlanallocation.py
        neutron/objects/plugins/ml2/vxlanallocation.py

    Change-Id: I953062d9ee8ee5ee9a9f07aff4a8222ac63ed525
    Closes-Bug: #1863423
    (cherry picked from commit 046672247de56bad950e8267a57bd26205f354a0)

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

The problem described in this bug was introduced by [1]. Those patches are in master (Ussuri), T and S. I953062d9ee8ee5ee9a9f07aff4a8222ac63ed525 is in master (U) and T. We still need to backport it to S.

Along with this patch, we need [2] (already in U and T), to solve a problem introduced by
I953062d9ee8ee5ee9a9f07aff4a8222ac63ed525.

Regards.

[1] https://review.opendev.org/#/q/topic:bp/network-segment-range-management+(status:open+OR+status:merged)
[2] https://review.opendev.org/#/q/Iaff891a48adc811ab114fb03b24ab3da9311eec3

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/721679

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/721679
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3796c03fd1f0e860b102fa8b4198840bc8045edd
Submitter: Zuul
Branch: stable/stein

commit 3796c03fd1f0e860b102fa8b4198840bc8045edd
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Feb 26 10:39:19 2020 +0000

    Fix queries to retrieve allocations with network_segment_range

    Fixed the queries to retrieve the segment ID allocations when service
    plugin network_segment_range is enabled. With the previous
    implementation, a project user was able to allocate a segment ID
    belonging to other project segment range.

    The solution implemented was discussed in [1]:
    - A project user will retrieve segments from the project ranges.
    - When depleted, the segment IDs will be retrieved from the shared
      range, never using another project segment ID.

    [1]http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012736.html

    Conflicts:
          neutron/objects/network_segment_range.py
          neutron/objects/plugins/ml2/base.py
          neutron/objects/plugins/ml2/vxlanallocation.py
          neutron/objects/plugins/ml2/vlanallocation.py
          neutron/tests/unit/objects/test_network_segment_range.py

    Change-Id: I953062d9ee8ee5ee9a9f07aff4a8222ac63ed525
    Closes-Bug: #1863423
    (cherry picked from commit 046672247de56bad950e8267a57bd26205f354a0)
    (cherry picked from commit bbe401aaf9bfdd77e1d43d547b2cdb436b1440c8)