port filter hook for network tenant id matching breaks counting

Bug #1558827 reported by Kevin Benton
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Critical
Kevin Benton

Bug Description

The filter hook added in https://review.openstack.org/#/c/255285 causes SQLAlchemy to add the networks table to the FROM statement without a restricted join condition. This results in many duplicate rows coming back from the DB query. This is okay for normal record retrieval because sqlalchemy would deduplicate the records. However, when calling .count() on the query, it returns a number far too large.

This breaks the quota engine for plugins that don't use the newer method of tracking resources.

Changed in neutron:
assignee: nobody → Kevin Benton (kevinbenton)
importance: Undecided → Critical
milestone: none → mitaka-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/294321

Changed in neutron:
status: New → In Progress
Henry Gessau (gessau)
tags: added: db
Changed in neutron:
milestone: mitaka-1 → newton-1
tags: added: mitaka-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/294321
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ff4067af5ba52cc205f38d12cdf68bd454445ced
Submitter: Jenkins
Branch: master

commit ff4067af5ba52cc205f38d12cdf68bd454445ced
Author: Kevin Benton <email address hidden>
Date: Wed Mar 16 12:28:49 2016 -0700

    Outerjoin to networks for port ownership filter

    Change I55328cb43207654b9bb4cfb732923982d020ab0a
    added a port filter to compare tenant ID to the
    network owner as well. This caused the networks
    table to be added to the FROM statement since
    ports wasn't joined to networks for any other
    reason. This resulted in an explosion of records
    returned (networks * ports). SQLAlchemy would
    de-dup this for us when iterating over results;
    however, it would completely break the 'count()'
    operation required by get_ports_count (which
    the quota engine uses).

    Change-Id: I5b780121ba408fba691fff9304d4a22e5892b85f
    Closes-Bug: #1558827

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/294477

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/294477
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f76056a94000857ac1613d08dee7858373964fa6
Submitter: Jenkins
Branch: stable/mitaka

commit f76056a94000857ac1613d08dee7858373964fa6
Author: Kevin Benton <email address hidden>
Date: Wed Mar 16 12:28:49 2016 -0700

    Outerjoin to networks for port ownership filter

    Change I55328cb43207654b9bb4cfb732923982d020ab0a
    added a port filter to compare tenant ID to the
    network owner as well. This caused the networks
    table to be added to the FROM statement since
    ports wasn't joined to networks for any other
    reason. This resulted in an explosion of records
    returned (networks * ports). SQLAlchemy would
    de-dup this for us when iterating over results;
    however, it would completely break the 'count()'
    operation required by get_ports_count (which
    the quota engine uses).

    Change-Id: I5b780121ba408fba691fff9304d4a22e5892b85f
    Closes-Bug: #1558827
    (cherry picked from commit ff4067af5ba52cc205f38d12cdf68bd454445ced)

tags: added: in-stable-mitaka
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/neutron 8.0.0.0rc2

This issue was fixed in the openstack/neutron 8.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/302570

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)
Download full text (11.4 KiB)

Reviewed: https://review.openstack.org/302570
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=03ccdd6a6d8e4cea9145e39dd7a1614656024aed
Submitter: Jenkins
Branch: master

commit 3213eb124e40b130e174ac3a91067e2b196788dd
Author: Cedric Brandily <email address hidden>
Date: Tue Mar 29 18:06:44 2016 +0200

    Support Routes==2.3

    SubMapper.connect allows in Routes<=2.2 to define routes with/without a
    name:

     map.connect(route_name, route_path)
     map.connect(route_path)

    Routes==2.3 SubMapper.connect now requires a route name (but not
    Mapper.connect). This change adds a route name when needed to support
    Routes<=2.2 and Routes==2.3.

    Closes-Bug: #1563028
    Change-Id: I655bfe7f047945896d63dd94c584acfe60d81b38
    (cherry picked from commit c6efa6292cac9267428366dc3cee00f6fc73e208)

commit 4283a7e36b74823f229400f0f049742f630db28b
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Mar 30 11:48:37 2016 +0200

    Constraint requirements using mitaka upper-constraints.txt file

    Change-Id: I6a908470987ce05769c425c4623df404cef76a10

commit fc690974480611242d4fc6e1deaaf68171fcaed3
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Mar 30 06:21:15 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I78c24969927224277f381532ad2604cfde8741fc

commit 41be555eddb0f9947fdaa4e73fa74a72677d4d11
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 29 06:07:19 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Iafa43bfcfca4d6af0d0866206c07fa817f567de6

commit b435ec56af98d2c04de37dd779ed3613ec4583ad
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Mar 28 06:07:02 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I9e908579b4704c4294367113523b898839586426

commit bec65f6cdd159ec252c4aa0a63aabc9aea8fcfba
Author: YAMAMOTO Takashi <email address hidden>
Date: Fri Mar 25 15:25:39 2016 +0900

    api tests: Check correct extensions

    auto_allocate -> auto-allocated-topology
    rbac_policies -> rbac-policies

    Closes-Bug: #1561858

    (cherry picked from commit 9d3bab6b8d6b2122732faeb66f831927bee0ac7e)
    Change-Id: Ied13e52bf78afe571f7243e93005bf5c68fe44db

commit f0c3b19165a5ca73ea8f1dc0ddaad290eee35039
Author: OpenStack Proposal Bot <email address hidden>
Date: Fri Mar 25 06:06:36 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Idb22eac32944a088d24f4d44e537d9bba09dbcc2

commit 99915fa76bd4ce13e0d36059616ba357f4f54ee2
Author: Hynek Mlnarik <email address hidden>
Date: Thu Mar...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/316304

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/liberty)

Reviewed: https://review.openstack.org/316304
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=13f7d19404cd4d9b941162771df6d3be21438dbf
Submitter: Jenkins
Branch: stable/liberty

commit 13f7d19404cd4d9b941162771df6d3be21438dbf
Author: Kevin Benton <email address hidden>
Date: Wed Mar 16 12:28:49 2016 -0700

    Outerjoin to networks for port ownership filter

    Change I55328cb43207654b9bb4cfb732923982d020ab0a
    added a port filter to compare tenant ID to the
    network owner as well. This caused the networks
    table to be added to the FROM statement since
    ports wasn't joined to networks for any other
    reason. This resulted in an explosion of records
    returned (networks * ports). SQLAlchemy would
    de-dup this for us when iterating over results;
    however, it would completely break the 'count()'
    operation required by get_ports_count (which
    the quota engine uses).

    Change-Id: I5b780121ba408fba691fff9304d4a22e5892b85f
    Closes-Bug: #1558827
    (cherry picked from commit f76056a94000857ac1613d08dee7858373964fa6)

tags: added: in-stable-liberty
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/neutron 7.1.0

This issue was fixed in the openstack/neutron 7.1.0 release.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 9.0.0.0b1

This issue was fixed in the openstack/neutron 9.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.