endpoints not show correctly when using "endpoint_filter.sql" as catalog's backend driver

Bug #1516469 reported by Dave Chen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
High
Dave Chen

Bug Description

If the endpoint group project association was created, and set "endpoint_filter.sql" as catalog's backend driver. All of the endpoints associated with the project and match the criterion defined in the "endpoint group" should be given after a project scoped token was return.

But currently, those endpoints can *only* be show if using call `list_endpoints_for_project` API explicitly by CURL but cannot get back when the project scoped token was issued.

Steps to reproduce this issue.

-Create endpoint group.

$curl -g -i -X POST http://10.239.48.36:5000/v3/OS-EP-FILTER/endpoint_groups -H "X-Auth-Token:a85e07129aa54f61a46395543a3146af" -H "Content-Type: application/json" -d '{"endpoint_group": {"description": "endpoint group description", "filters": {"interface": "admin"}, "name": "endpoint_group_name"}}'

- Create endpoint_group project association

$curl -g -i -X PUT http://10.239.48.36:5000/v3/OS-EP-FILTER/endpoint_groups/ea1af6e153bf4b87a88b5962de8cdae8/projects/927e252fb44d4b5cac9d4fb24d85be41 -H "X-Auth-Token:a85e07129aa54f61a46395543a3146af" -H "Content-Type: application/json"

- Get endpoint for the project, this will return all of the endpoints matched the rule defined in the endpoint group.

$curl -g -i -X GET http://10.239.48.36:5000/v3/OS-EP-FILTER/projects/927e252fb44d4b5cac9d4fb24d85be41/endpoints -H "X-Auth-Token:a85e07129aa54f61a46395543a3146af" -H "Content-Type: application/json"
...
{
    "endpoints": [
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/3f6fb8738db8427a997dbcc791b7901d"
            },
            "url": "http://10.239.48.36:8773/",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "a3338a6847e94766831ea7d9d43598cc",
            "id": "3f6fb8738db8427a997dbcc791b7901d"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/dd69f161f8a24612a7ffe796b45b8cd2"
            },
            "url": "http://10.239.48.36:8774/v2.1/$(tenant_id)s",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "a147aa8896c4429aacf0f2eefd39098e",
            "id": "dd69f161f8a24612a7ffe796b45b8cd2"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/0d70f9fd5a85446c99fee79388adf9dc"
            },
            "url": "http://10.239.48.36:9292",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "4c367805e2a147589a14310d1486ab01",
            "id": "0d70f9fd5a85446c99fee79388adf9dc"
        },
        {
            "region_id": null,
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/5be3023ddf984fcf942b2a396eb0167b"
            },
            "url": "http://127.0.0.0:20",
            "region": null,
            "enabled": true,
            "interface": "internal",
            "service_id": "69da5bbf65aa4565b9833655075e7a8a",
            "id": "5be3023ddf984fcf942b2a396eb0167b"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/9393be9c7eda41d89a28f2ffb486dc7c"
            },
            "url": "http://10.239.48.36:35357/v2.0",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "ef49d941aed34d39b8b49fce27c83a50",
            "id": "9393be9c7eda41d89a28f2ffb486dc7c"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/151b9f8b132f4c26a562872e09389a69"
            },
            "url": "http://10.239.48.36:8774/v2/$(tenant_id)s",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "8bb4bdc9fcac4fb5bec4f6779268f0d0",
            "id": "151b9f8b132f4c26a562872e09389a69"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/bff53486b72c44e9b00cf69184b66ce9"
            },
            "url": "http://10.239.48.36:3333",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "69da5bbf65aa4565b9833655075e7a8a",
            "id": "bff53486b72c44e9b00cf69184b66ce9"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/d0ee548da623477eb73b60018c3e5ab8"
            },
            "url": "http://10.239.48.36:8776/v1/$(tenant_id)s",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "928eb1b536464e238e573284760e656a",
            "id": "d0ee548da623477eb73b60018c3e5ab8"
        },
        {
            "region_id": "RegionOne",
            "links": {
                "self": "http://10.239.48.36:5000/v3/endpoints/44699ffc64274612a0c039531f66096d"
            },
            "url": "http://10.239.48.36:8776/v2/$(tenant_id)s",
            "region": "RegionOne",
            "enabled": true,
            "interface": "admin",
            "service_id": "ab4b7001ccaa4c3896407d4523466183",
            "id": "44699ffc64274612a0c039531f66096d"
        }
    ],
    "links": {
        "self": "http://10.239.48.36:5000/v3/OS-EP-FILTER/projects/927e252fb44d4b5cac9d4fb24d85be41/endpoints",
        "previous": null,
        "next": null
    }
}
...

- Get a project scoped token, this will only return endpoints from endpoint_project table.

$ curl -i -H "Content-Type: application/json" -d '{ "auth": { "identity": {
      "methods": ["password"],
      "password": {
        "user": {
          "name": "admin",
          "domain": { "id": "default" },
          "password": "12345"
        }
      }
    },
    "scope": {
      "project": {
        "name": "admin",
        "domain": { "id": "default" }
      }
    }
  }
}' http://10.239.48.36:5000/v3/auth/tokens; echo

...
        "catalog": [
            {
                "endpoints": [
                    {
                        "region_id": null,
                        "url": "http://127.0.0.0:20",
                        "interface": "internal",
                        "id": "5be3023ddf984fcf942b2a396eb0167b"
                    }
                ],
                "type": "s3",
                "id": "69da5bbf65aa4565b9833655075e7a8a"
            }
        ],
...

Dave Chen (wei-d-chen)
description: updated
description: updated
Changed in keystone:
assignee: nobody → Dave Chen (wei-d-chen)
Dave Chen (wei-d-chen)
description: updated
description: updated
Changed in keystone:
importance: Undecided → High
status: New → Triaged
milestone: none → mitaka-1
tags: added: kilo-backport-potential liberty-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/250032

Changed in keystone:
status: Triaged → In Progress
Changed in keystone:
milestone: mitaka-1 → mitaka-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to keystone (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/255070

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/250032
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=f86448a3113fc594e78d3d9410f44c1f64a9ad58
Submitter: Jenkins
Branch: master

commit f86448a3113fc594e78d3d9410f44c1f64a9ad58
Author: Dave Chen <email address hidden>
Date: Thu Nov 26 05:39:59 2015 +0800

    Ensure endpoints returned is filtered correctly

    This patch move some logic to manager layer, so that endpoints
    filtered by endpoint_group project association will be included
    in catalog when issue a project scoped token and using
    `endpoint_filter.sql` as catalog's backend driver.

    This make sure that call `list_endpoints_for_project` API has
    the same endpoints with that in catalog returned for project
    scoped token.

    Change-Id: I56f4eb6fc524650677b627295dd4338d55164c39
    Closes-Bug: #1516469

Changed in keystone:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to keystone (master)

Reviewed: https://review.openstack.org/255070
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=2a10954b5e2573bd827bade5a7fd9435da3ad1f7
Submitter: Jenkins
Branch: master

commit 2a10954b5e2573bd827bade5a7fd9435da3ad1f7
Author: Dave Chen <email address hidden>
Date: Wed Dec 9 12:59:37 2015 +0800

    refactor: move the common code to manager layer

    There are some methods (`_get_endpoint_groups_for_project` and
    `_get_endpoints_filtered_by_endpoint_group`) needed both by catalog
    controller and manager, move those methods into manager so that
    controller can call these methods directly from manager.

    Change-Id: I3a82c606d62bc2ad54f7454cd6ee4dbce0b88219
    Related-Bug: #1516469

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/256101

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (stable/liberty)

Reviewed: https://review.openstack.org/256101
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9262bb8dcbbfd1b71c3180c2293827a494d4ed6f
Submitter: Jenkins
Branch: stable/liberty

commit 9262bb8dcbbfd1b71c3180c2293827a494d4ed6f
Author: Dave Chen <email address hidden>
Date: Thu Nov 26 05:39:59 2015 +0800

    Ensure endpoints returned is filtered correctly

    Conflicts:
     keystone/catalog/controllers.py
     keystone/catalog/core.py
     keystone/contrib/endpoint_filter/backends/catalog_sql.py
    This patch move some logic to manager layer, so that endpoints
    filtered by endpoint_group project association will be included
    in catalog when issue a project scoped token and using
    `endpoint_filter.sql` as catalog's backend driver.

    This make sure that call `list_endpoints_for_project` API has
    the same endpoints with that in catalog returned for project
    scoped token.

    The difference between this cherry pick and the patch on the master
    branch is massive since the endpoint filter extension has been
    consolidated with keystone catalog on master branch, all the change
    made in the keystone catalog should be made in keystone endpoint filter
    extension instead.

    Closes-Bug: #1516469
    (cherry picked from commit f86448a3113fc594e78d3d9410f44c1f64a9ad58)

    Change-Id: I56f4eb6fc524650677b627295dd4338d55164c39

tags: added: in-stable-liberty
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/keystone 9.0.0.0b2

This issue was fixed in the openstack/keystone 9.0.0.0b2 development milestone.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/keystone 8.1.0

This issue was fixed in the openstack/keystone 8.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.