flaskification

Bug #1776504 reported by Morgan Fainberg on 2018-06-12
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Medium
Morgan Fainberg

Bug Description

Moving keystone to Flask away from it's home-grown WSGI framework is a long-term plan. The major reasons for this is to ensure we have an easy way for folks start contributing.

This will include a number of improvements including:

* moving to flask-restful for API definitions

* all routable paths will be owned by the base prefix (e.g. keystone.api.user will own everything under /user/)

* Paste Deploy removed

Changed in keystone:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Morgan Fainberg (mdrnstm)
milestone: none → ongoing

Reviewed: https://review.openstack.org/574735
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=f18d54fe16dcc91a0d9d144ad15c524a679e3cc5
Submitter: Zuul
Branch: master

commit f18d54fe16dcc91a0d9d144ad15c524a679e3cc5
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 12 06:59:38 2018 -0700

    Store JSON Home Resources off the composing router

    JSON Home Resources must be stored in a location other
    than the router for Flask as we are not composing routers
    in the same way as we used to with the home-grown webob
    based WSGI setup.

    Partial-Bug: #1776504
    Closes-Bug: #1776506
    Change-Id: I292ea9e923ff2f49041dfd417994bcdd797d0520

Fix proposed to branch: master
Review: https://review.openstack.org/576639

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/574953

Fix proposed to branch: master
Review: https://review.openstack.org/578190

Fix proposed to branch: master
Review: https://review.openstack.org/578216

Reviewed: https://review.openstack.org/574414
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8dd2235cbc08f11036a898342e00bc2a5f13f79d
Submitter: Zuul
Branch: master

commit 8dd2235cbc08f11036a898342e00bc2a5f13f79d
Author: Morgan Fainberg <email address hidden>
Date: Mon Jun 11 09:49:05 2018 -0700

    Add Flask-RESTful and update flask minimum(s)

    Keystone's move to flask requires more than the general Flask library
    as Keystone is a RESTful (ish) API. We will be using Flask-RESTful for
    the easier mechanism to implment a REST API than standard flask
    blueprints.

    This also increases the base flask minimum requirements to unblock
    requirements updates (flask has been updated in g-r to minimum of
    1.0.2)

    Partial-Bug: #1776504
    Change-Id: I398acad439f4e525df3ca4e17fdd3e3ba90d58cc

Reviewed: https://review.openstack.org/574502
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ecf721a3c176daf67d00536c48e80e78bded1af6
Submitter: Zuul
Branch: master

commit ecf721a3c176daf67d00536c48e80e78bded1af6
Author: Morgan Fainberg <email address hidden>
Date: Mon Jun 11 17:25:45 2018 -0700

    Keystone adheres to public_endpoint opt only

    With the complete removal of the v2.0 API, keystone no longer
    differentiates between admin and public endpoints. This change
    deprecates the "admin_endpoint" configuration option and converts
    keystone over to only using the public endpoint. The "admin" endpoint
    was only used for unit testing purposes.

    This change does not clean up all code related, it is aimed to make
    the most minimal set of changes eliminating the use of the
    "admin_endpoint" configuration option.

    Partial-Bug: #1776504
    Change-Id: I08f6f8ae078d65203bd95c43c80367dd3489be48

Fix proposed to branch: master
Review: https://review.openstack.org/579746

Fix proposed to branch: master
Review: https://review.openstack.org/579928

Fix proposed to branch: master
Review: https://review.openstack.org/579930

Fix proposed to branch: master
Review: https://review.openstack.org/579985

Reviewed: https://review.openstack.org/574736
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3e3ba18bfaf82711689a45ae03efdba78e6e856e
Submitter: Zuul
Branch: master

commit 3e3ba18bfaf82711689a45ae03efdba78e6e856e
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 12 06:42:21 2018 -0700

    Convert json_home and version discovery to Flask

    Move the JSON Home Document and Version Discovery Documents out of
    the webob-based mapper and into Flask.

    This change removes the keystone.version.controller and
    keystone.version.router modules as they have been moved into
    keystone.api.discovery.

    The keystone.api.discovery module is somewhat specialized as there
    are no "resources" and it must handle multiple types of responses
    based upon the ACCEPTS header (JSON Home or JSON). In lieu of the
    flask-RESTful mechanisms, keystone.api.discovery utilizes bare
    flask blueprint and functions. Minor scaffolding work has been done
    to ensure the discovery blueprint can be loaded via the loader loop
    in keystone.server.flask.application (a stub object in
    keystone.api.discovery).

    Partial-Bug: #1776504
    Change-Id: Ib25380cefdbb7147661bb9853de7872a837322e0

Reviewed: https://review.openstack.org/576637
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3330c228ad030e6d3d987f1f360ba5ce0c460333
Submitter: Zuul
Branch: master

commit 3330c228ad030e6d3d987f1f360ba5ce0c460333
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 19 12:55:24 2018 -0700

    Add support for before and after request functions

    In flask it is possible to register functions that will run before
    and after every request, these are useful for ensuring certain
    things occur (setting up environments) and protecting against
    cases such as un-enforced API calls not explicitly white-listed.

    Partial-Bug: #1776504
    Change-Id: I0138a4676c1e257cd0c238bbe11a520446c299cb

Reviewed: https://review.openstack.org/577587
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1e870abd693088d19db023d731310a1f33f2c831
Submitter: Zuul
Branch: master

commit 1e870abd693088d19db023d731310a1f33f2c831
Author: Morgan Fainberg <email address hidden>
Date: Fri Jun 22 17:27:36 2018 -0700

    Don't replace the whole app just the wsgi_app backing

    Do not replace the entire app when wrapping with middleware. It is
    important to maintain all the flask-functionality on the app object
    and ensure any/all test client calls go through the entire stack of
    app and middleware.

    Partial-Bug: #1776504
    Change-Id: I928d08e96b4c79807ad8c312ba17359c54b67fa0

Reviewed: https://review.openstack.org/577627
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1bcc8a11c8f3fba70cdc4b8c6992b20a423aba48
Submitter: Zuul
Branch: master

commit 1bcc8a11c8f3fba70cdc4b8c6992b20a423aba48
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 23 14:33:24 2018 -0700

    Make it easy to identify a 404 from Flask

    This change makes (for test purposes) the Flask app return a 418
    instead of a 404 if the path is unrouted. This allows easy
    identification (programmatically) if the 404 is issued from Flask
    or is a handled 404 such as "UserNotFound"

    Parital-Bug: #1776504
    Change-Id: I0475d9b6315250d9c3384be63c14a81fbd1c7b7c

Reviewed: https://review.openstack.org/578216
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=b77b5845d58b64babd9111832e239f47fc4766a2
Submitter: Zuul
Branch: master

commit b77b5845d58b64babd9111832e239f47fc4766a2
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 14:46:01 2018 -0700

    Address minor comments to 404 error detection

    Addressing minor comments to the 404 detection mechanism.

    Change-Id: Id8e2b5d5f811495694965d75a32eebb2a59779f2
    Partial-Bug: #1776504

Fix proposed to branch: master
Review: https://review.openstack.org/582450

Reviewed: https://review.openstack.org/576639
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bb3b15bbf02993545d234d827dbb72ca24695e8d
Submitter: Zuul
Branch: master

commit bb3b15bbf02993545d234d827dbb72ca24695e8d
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 23 14:33:50 2018 -0700

    Implement base for new RBAC Enforcer

    Create the base implementation of the RBAC enforcer with compat code
    for the legacy mechanism via @protected decorators.

    Change-Id: I80662d9b23e706b720d56670cb849318e951a3b4
    Parital-Bug: #1776504

Reviewed: https://review.openstack.org/578189
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=fb0299f661f590bcaddcb3d363fbec4ea6df7548
Submitter: Zuul
Branch: master

commit fb0299f661f590bcaddcb3d363fbec4ea6df7548
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 10:26:20 2018 -0700

    Add support for enforce_call to set value on flask.g

    When enforce_call is called, flask.g has the appropriate value set
    in the known attribute location so that it can be determined if
    enforce_call was used within a REST API request.

    Change-Id: I08ecd2be0a80248df7041596437adb6238835153
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/579746
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1caba2a44858ab4a6d405603cd357bb1c5e51a77
Submitter: Zuul
Branch: master

commit 1caba2a44858ab4a6d405603cd357bb1c5e51a77
Author: Morgan Fainberg <email address hidden>
Date: Sun Jul 1 10:26:15 2018 -0700

    Move keystone.server.common to keystone.server

    The common functions are mostly removed and these make sense to
    be part of the __init__.py. This also simplifies imports and
    eliminates odd import errors due to duplicated names and masking
    the names e.g. (and keystone.server.flask needs the functions
    from keystone.server.common):

    keystone.server.flask.common
    keystone.server.common

    Change-Id: Ie586fd45e10c8a1c8db3d9a64f949c97004814d9
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/579342
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=6d1456061defc016529b3c7c89edb7618b117267
Submitter: Zuul
Branch: master

commit 6d1456061defc016529b3c7c89edb7618b117267
Author: Morgan Fainberg <email address hidden>
Date: Sat Jun 30 08:17:33 2018 -0700

    Address minor comments from initial impl RBACEnforcer

    RBACEnforcer had a couple minor comments and this addresses all
    except for adding explicit system-scope enforcement tests.

    * Invalid "action" (policy rule name) now correctly raises
      Forbidden instead of Unauthorized (it can never be authorized
      as the code is incorrect, and a 500 "ISE" should never be expected).
      This is equivalent behavior to an unknown rule in the oslo.policy
      enforcer.

    * Added more expressive errors/warnings/exceptions for invalid actions.

    * Code comments added where missing

    Change-Id: I4895242555d2e9761e035d879ca2dfeb6d9b387c
    Partial-Bug: #1776504

Fix proposed to branch: master
Review: https://review.openstack.org/582725

Fix proposed to branch: master
Review: https://review.openstack.org/582726

Reviewed: https://review.openstack.org/578190
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=16be22b428fd0c7c34dd48417878f7d37ad1e2ac
Submitter: Zuul
Branch: master

commit 16be22b428fd0c7c34dd48417878f7d37ad1e2ac
Author: Morgan Fainberg <email address hidden>
Date: Tue Jun 26 13:02:01 2018 -0700

    Flesh out and add testing for flask_RESTful scaffolding

    Add in support for JSON Home documents, a ResourceBase implementing
    basic functionality, and full testing of the new flask_RESTful
    scaffolding.

    Change-Id: I5bcc8660b68c0b39a2110089f6c67531769d14ef
    Parital-Bug: #1776504

Reviewed: https://review.openstack.org/579928
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9387dfd4cc39fa6fbbf66147ced880dffa6408bd
Submitter: Zuul
Branch: master

commit 9387dfd4cc39fa6fbbf66147ced880dffa6408bd
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 10:10:53 2018 -0700

    Make keystone.server.flask more interesting for importing

    Importing keystone.server.flask now exposes all the relevant bits
    from the sub modules to develop APIs without needing to understand
    all the underlying modules. __all__ has also be setup in a meaningful
    way to allow for `from keystone.server.flask import *` and have
    all the needed objects to start developing APIs for keystone.

    Change-Id: Iab22cabb71c6690e6ffb0c9de68ed8437c4848de
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/579930
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0f4fd2e4f38d4e5e6aea5dfd7c3d124e8965edb0
Submitter: Zuul
Branch: master

commit 0f4fd2e4f38d4e5e6aea5dfd7c3d124e8965edb0
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 10:12:40 2018 -0700

    Fix keystone.common.rbac_enforcer.__init__.py exporting

    keystone.common.rbac_enforcer now exports (and has __all__ setup)
    for the RBACEnforcer object. This limits the underlying module
    knowledge needed to effectively work with the new flask RBACEnforcer.

    Change-Id: I48cafa779ca1895113222a2ebfef6d738a906473
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/579985
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=dbc2ac06b8236c446a4bb2817add4f253bf2034d
Submitter: Zuul
Branch: master

commit dbc2ac06b8236c446a4bb2817add4f253bf2034d
Author: Morgan Fainberg <email address hidden>
Date: Tue Jul 3 13:54:40 2018 -0700

    Do not use flask.g imported as g

    Always explicitly use flask.g as `flask.g` and do not do `from flask
    import g`.

    Change-Id: I3737115717d8a5040c3e299d206fbddd43b33aa6
    Partial-Bug: #1776504

Fix proposed to branch: master
Review: https://review.openstack.org/583356

Fix proposed to branch: master
Review: https://review.openstack.org/583357

Fix proposed to branch: master
Review: https://review.openstack.org/583358

Fix proposed to branch: master
Review: https://review.openstack.org/583373

Fix proposed to branch: master
Review: https://review.openstack.org/583374

Reviewed: https://review.openstack.org/582449
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0d6b427fcc7f6b9b2a06bdd9a7af37064e7ddd51
Submitter: Zuul
Branch: master

commit 0d6b427fcc7f6b9b2a06bdd9a7af37064e7ddd51
Author: Morgan Fainberg <email address hidden>
Date: Thu Jul 12 16:41:10 2018 -0700

    Don't allow legacy and native flask to share paths

    When a path-prefix is moved to flask native dispatching, no longer
    allow that path prefix to be registered with the legacy dispatch
    middleware. This will ensure the entire Keystone path is moved
    and prevent bad behavior due to both dispatchers needing ot handle
    a URL.

    Change-Id: Ice800abf80a725349d6450b742a2c48238e11e6e
    Partial-Bug: #1776504

Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → wangxiyuan (wangxiyuan)
Changed in keystone:
assignee: wangxiyuan (wangxiyuan) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Lance Bragstad (lbragstad)
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → wangxiyuan (wangxiyuan)
Changed in keystone:
assignee: wangxiyuan (wangxiyuan) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Adam Young (ayoung)
Adam Young (ayoung) on 2018-09-10
Changed in keystone:
assignee: Adam Young (ayoung) → nobody
assignee: nobody → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Gage Hugo (gagehugo)
Changed in keystone:
assignee: Gage Hugo (gagehugo) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Gage Hugo (gagehugo)
78 comments hidden view all 158 comments

Reviewed: https://review.openstack.org/597350
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=296f20f0a7e26784b6414ddbe12e0218087a9f51
Submitter: Zuul
Branch: master

commit 296f20f0a7e26784b6414ddbe12e0218087a9f51
Author: Gage Hugo <email address hidden>
Date: Tue Aug 28 20:35:50 2018 -0500

    Convert domains api to flask

    Convert domains api to flask native dispatching.

    Change-Id: I974ab112986e6c242a139f7d7dc14fd54a48ee42
    Partial-Bug: #1776504

Changed in keystone:
assignee: Gage Hugo (gagehugo) → Felipe Monteiro (fm577c)

Fix proposed to branch: master
Review: https://review.openstack.org/603451

Changed in keystone:
assignee: Felipe Monteiro (fm577c) → Harry Rybacki (hrybacki-h)

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/602808

Fix proposed to branch: master
Review: https://review.openstack.org/603461

Changed in keystone:
assignee: Harry Rybacki (hrybacki-h) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Harry Rybacki (hrybacki-h)

Fix proposed to branch: master
Review: https://review.openstack.org/603964

Changed in keystone:
assignee: Harry Rybacki (hrybacki-h) → wangxiyuan (wangxiyuan)
Changed in keystone:
assignee: wangxiyuan (wangxiyuan) → Harry Rybacki (hrybacki-h)
Changed in keystone:
assignee: Harry Rybacki (hrybacki-h) → Morgan Fainberg (mdrnstm)
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Gage Hugo (gagehugo)
Changed in keystone:
assignee: Gage Hugo (gagehugo) → Morgan Fainberg (mdrnstm)

Reviewed: https://review.openstack.org/601881
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8697da7da650e2db29136ce48053ee7c234e3b62
Submitter: Zuul
Branch: master

commit 8697da7da650e2db29136ce48053ee7c234e3b62
Author: Gage Hugo <email address hidden>
Date: Tue Sep 11 17:26:24 2018 -0600

    Add build_target arguement to enforcer

    This change adds in a new arguement "build_target" which takes
    in a passed function to build the enforement target after
    the authentication check. This is to avoid leaking existance
    data when determining scope.

    Change-Id: I9aab71dd0032d40aa2f2e088b529af08b112671f
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/603964
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d027dd2ba9134629e067b24503cc7f61d75b692d
Submitter: Zuul
Branch: master

commit d027dd2ba9134629e067b24503cc7f61d75b692d
Author: wangxiyuan <email address hidden>
Date: Thu Sep 20 11:25:21 2018 +0800

    Add hint back

    When do Flask refactor, some list APIs missed "hints"
    for wrap_collection.

    Add them back for list resource so that filters can work as
    before

    Change-Id: If32ca8d798c95436f72aa55ac7060e8eb7540de7
    Partial-Bug: #1776504

Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Harry Rybacki (hrybacki-h)

Reviewed: https://review.openstack.org/601882
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0c71cdd23bd2a7e4f7ec1a5ecec91f3ed7457d00
Submitter: Zuul
Branch: stable/rocky

commit 0c71cdd23bd2a7e4f7ec1a5ecec91f3ed7457d00
Author: morgan fainberg <email address hidden>
Date: Tue Sep 11 16:03:54 2018 -0700

    Ensure view args is in policy dict

    The policy_dict (in enforcement) was not populating the view args
    in a similar manner to the old style @protected decorator. This
    change ensures that we mirror the old behavior (required for
    proper use of v3cloud policy).

    Conflicts:
        keystone/tests/unit/common/test_rbac_enforcer.py

    Change-Id: Ida9009a95a874be9cc60c3152d4e3225726562eb
    Partial-Bug: #1776504
    Closes-Bug: #1792047
    (cherry picked from commit 4975b79e8174587f7639347939cf679460d4896b)

tags: added: in-stable-rocky

Fix proposed to branch: master
Review: https://review.openstack.org/608756

Changed in keystone:
assignee: Harry Rybacki (hrybacki-h) → Morgan Fainberg (mdrnstm)

Fix proposed to branch: master
Review: https://review.openstack.org/609071

Fix proposed to branch: master
Review: https://review.openstack.org/609159

Fix proposed to branch: master
Review: https://review.openstack.org/609500

Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Harry Rybacki (hrybacki-h)

Fix proposed to branch: master
Review: https://review.openstack.org/609535

Changed in keystone:
assignee: Harry Rybacki (hrybacki-h) → Morgan Fainberg (mdrnstm)

Fix proposed to branch: master
Review: https://review.openstack.org/609548

Reviewed: https://review.openstack.org/603461
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d97832e8e826e37171b727072c720a9b589998dd
Submitter: Zuul
Branch: master

commit d97832e8e826e37171b727072c720a9b589998dd
Author: morgan fainberg <email address hidden>
Date: Tue Sep 18 10:54:59 2018 -0700

    Convert auth to flask native dispatching

    Convert the /auth paths to flask native dispatching.

    A minor change to additional_urls was implemented to ensure all
    urls are added at once instead of individually (causing an over-
    write issue within flask as a single resource may only have a
    single set of URL mappings).

    Alternate URLs now support adding alternate JSON Home rel links.
    This is to support the case of OS-FEDERATION auth routes moving
    to /auth. The old JSON Home entries must exist but reference
    the new paths.

    This port includes the following test changes (needed due to the
    way flask handles requests and the way requests are passed through
    the auth system):

    * Implemented keystone.common.render_token (module)
      containing render_token_response_from_model and use it instead
      of keystone.common.controller.render_token_response_from_model.

      Minor differences occur in render_token_response_from_model in
      the keystone.common.render_token module, this is simply
      for referencing data from flask instead of the request object.

    * Test cases have been modified to no longer rely on the auth
      controller(s) directly

    * Test cases now use "make_request" as a context manager
      since authenticate/authenticate_for_token directly
      reference the flask contexts and must have an explicit
      context pushed.

    * Test cases no longer pass request objects into methods
      such as authenticate/authenticate_for_token or similar
      methods on the auth plugins

    * Test cases for federation reference the token model now
      where possible instead of the rendered token response.
      Rendered token responses are generated where needed.

    * Auth Plugin Configuration is done in test core as well.
      This is because Auth controller does not exist.

    NOTE: This is a massive change, but must of these changes
    were now easily uncoupled because of how far reaching auth
    is.

    Change-Id: I636928102875760726cc3493775a2be48e774fd7
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/608756
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=54b6227c1fdec1e768570ba696841b00f4e0a549
Submitter: Zuul
Branch: master

commit 54b6227c1fdec1e768570ba696841b00f4e0a549
Author: Morgan Fainberg <email address hidden>
Date: Mon Oct 8 12:28:42 2018 -0700

    Auth flask conversion cleanup

    * Superfluous call to setup token authentication has been removed from
      keystone.server.flask.core

    * Base SAML assertion function has been extracted from
      keystone.api.auth and moved to keystone.api._shared.saml

    Change-Id: Idfa62bf1aea81ef5b4c6f564397e6a0d3ae60417
    Partial-Bug: #1776504

Fix proposed to branch: master
Review: https://review.openstack.org/609804

Fix proposed to branch: master
Review: https://review.openstack.org/609805

Fix proposed to branch: master
Review: https://review.openstack.org/609815

Fix proposed to branch: master
Review: https://review.openstack.org/609836

Fix proposed to branch: master
Review: https://review.openstack.org/609837

Fix proposed to branch: master
Review: https://review.openstack.org/609838

Fix proposed to branch: master
Review: https://review.openstack.org/609839

Changed in keystone:
milestone: ongoing → stein-1

Fix proposed to branch: master
Review: https://review.openstack.org/610129

Reviewed: https://review.openstack.org/609071
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=86f968163ea90dfac515b5e59f20edf8c5554cee
Submitter: Zuul
Branch: master

commit 86f968163ea90dfac515b5e59f20edf8c5554cee
Author: Morgan Fainberg <email address hidden>
Date: Mon Oct 8 14:40:56 2018 -0700

    Convert /v3/users to flask native dispatching

    Convert /v3/users to use flask native dispatching.

    The following test changes were required:

    * Application Credentials did not have the plural form
      in the JSON Home document. The JSON Home document was
      corrected both in code and in tests.

    * Application Credentials "patch" test needed to be
      refactored to look for METHOD_NOT_ALLOWED instead
      of NOT FOUND for invalid/unimplemented methods.
      The "assertValidErrorResponse" method was
      insufficient and the test now uses the flask
      test_client mechanism instead.

    Change-Id: Iedaf405d11450b11e2d1fcdfae45ccb8eeb6f255
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/603451
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=46380baeb76fc624bf8b286ce403656767c403cf
Submitter: Zuul
Branch: master

commit 46380baeb76fc624bf8b286ce403656767c403cf
Author: Harry Rybacki <email address hidden>
Date: Tue Sep 18 12:03:52 2018 -0400

    Convert projects API to Flask

    Convert the projects API to Flask native dispatching.

    Change-Id: I3406284acfb7950b701f6a98a3a173a427415f97
    Co-Authored-By: Morgan Fainberg <email address hidden>
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609159
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=0ab08e75b237dd287554e5c86642efcf8292396c
Submitter: Zuul
Branch: master

commit 0ab08e75b237dd287554e5c86642efcf8292396c
Author: Morgan Fainberg <email address hidden>
Date: Tue Oct 9 14:36:20 2018 -0700

    Remove skip for test_locked_out_user_sends_notification

    Remove the explicit skip added pre-user conversion to flask for
    test_locked_out_user_sends_notification.

    Change-Id: I5f2e83f24dca2774fd5abaacf5885a2dab1ae759
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609500
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=35c9bb7eff51ebd6101b4a1d233b984cbf68fca8
Submitter: Zuul
Branch: master

commit 35c9bb7eff51ebd6101b4a1d233b984cbf68fca8
Author: Morgan Fainberg <email address hidden>
Date: Tue Oct 9 17:15:50 2018 -0700

    Convert S3 and EC2 auth to flask native dispatching

    Convert S3 and EC2 auth to flask native dispatching.

    Test changes required:

    * Eliminate direct reference of the EC2 / S3 controllers, originally
      this direct reference was to verify signature checking. Since
      signature checking is an @staticmethod now, direct reference of
      the API resources covers everything.

    * Direct import of keystone.common.controller - due to an oddity in
      how our WSGI code work(s) in test, if nothing imports the common
      controller module, the tests fail using the oslo import_class
      mechanism.

    Change-Id: I06e95957b3ea3a55b0da28959548bd5eb628c70b
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609535
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ee9b035cf17f0070dfb392505e44ea2961e2cf4b
Submitter: Zuul
Branch: master

commit ee9b035cf17f0070dfb392505e44ea2961e2cf4b
Author: Morgan Fainberg <email address hidden>
Date: Wed Oct 10 13:12:32 2018 -0700

    Replace JSON Body middleware with flask-native func

    Replace the JSON Body middleware with flask-native before-request
    function.

    The body filtering and storing data in
    request.environ['openstack.params'] was not used in the code base and
    has been dropped.

    Test Changes:

    * JSON Body middleware has been removed, no testing of the removed code

    * JSON Body Before Request Method has been implemented and associated
      testing (mirroring the JSON Body middleware code).

    * Test entry points no longer looks for JSON Body middleware.

    Change-Id: I84491865870b6bf2b8f094b524ee8b77510f0054
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609548
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=e666839bc170ee3a529e11a78500d34357c8f6ff
Submitter: Zuul
Branch: master

commit e666839bc170ee3a529e11a78500d34357c8f6ff
Author: Morgan Fainberg <email address hidden>
Date: Wed Oct 10 14:23:12 2018 -0700

    Cleanup keystone.server.flask.application

    Remove a chunk of the compat code for legacy dispatching. This moves
    the logging about the request to it's own before_request function.

    Change-Id: I0b1a4ca9a95489e410f055ff47f3399feba3a8f1
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609796
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9b052e4c059893791dd7adc4aa32d40ac098e788
Submitter: Zuul
Branch: master

commit 9b052e4c059893791dd7adc4aa32d40ac098e788
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 12:32:59 2018 -0700

    Register exceptions with a Flask Error Handler

    Exceptions are now handled in the Flask APP instead of in the
    legacy webob Application code (at this point that code was living
    in the URL Normalizing Middleware). All Keystone API exceptions
    (derived from keystone.exception.Error) are automatically
    registered on definition with the
    keystone.exception.KEYSTONE_API_EXCEPTIONS set. This set is
    processed once the app is created in keystone.server.application
    to the flask-friendly handler.

    TypeError and generic Exception are registered to an explicit
    error handler that converts TypeError to ValidationError (BAD_REQUEST)
    and all other Exceptions to UnexpectedError (INTERNAL SERVER ERROR).
    These exceptions are then emitted in a "jsonify-ed" manner to the
    client.

    Two other minor changes were required:

    * Unenforced API decorator had it's core functionality split into
      a dedicated function that can be called in the case of an error
      being raised in a "before_request" function (such as validation
      in the JSON Body before request func.

    * The JSON Body before request func now explicitly sets the
      api to "unenforced_ok" if it is raising an exception. This
      prevents the flask "was this API enforced" assertion from failing
      because @unenforced_api was never run (the ValidationError was
      raised prior to the resource's method being called).

    Change-Id: I0d0ef6a774eb86b4769238ed34d7703232ce86c3
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609804
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1c05f5f88d84d90ca949bab6e3d48c5f1e35a909
Submitter: Zuul
Branch: master

commit 1c05f5f88d84d90ca949bab6e3d48c5f1e35a909
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:05:16 2018 -0700

    Make Request Logging a little better

    Use the flask.request properties instead of direct environ lookups,
    as this is more representative of what is happening in the application.

    Change-Id: Ic16c5ea26b2f526b51ef167e6f6977c72df1d06a
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609805
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=18d597f8e87a7ebee6aba920e11d64bab6dc0295
Submitter: Zuul
Branch: master

commit 18d597f8e87a7ebee6aba920e11d64bab6dc0295
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:20:36 2018 -0700

    Internally defined middleware don't use stevedore

    For internally defined middleware (URL Normalizer and AuthContext)
    Do not use stevedore to load, apply directly. This also cleans up
    a lingering entry in the setup.cfg for token_auth.

    Test Changes:

    * entry points test no longer looks for url_normalize and
    build_auth_context

    Change-Id: I58d3c23ad4f70668ada4eae94a94d3f5fe750b3b
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609815
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=848c8fa6385f690b26d22a28cafeb41c3e36b558
Submitter: Zuul
Branch: master

commit 848c8fa6385f690b26d22a28cafeb41c3e36b558
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:47:18 2018 -0700

    Convert Normalizing filter to flask native Middleware

    Normalizing filter has been converted to a flask-native style
    middleware instead of leaning on the old application logic from
    Webob. We also now strip all trailing slashes, not just a single
    traling slash.

    Test Changes:

    * test_url_middleware now tests the new middleware directly instead
      of leaning on webob and fake requests.

    Change-Id: I5f82817b61a9284b97cf6443105107150d4a1757
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609836
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=595967bba6c19b709743c2991545662fce9f6da7
Submitter: Zuul
Branch: master

commit 595967bba6c19b709743c2991545662fce9f6da7
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 13:59:58 2018 -0700

    Move AuthContextMiddleware

    Move AuthContextMiddleware to keystone.server.flask.request_processing
    to be more in line with the other internally defined middleware.

    Change-Id: I25b6a88f4b0dc3af306360ee4e5ec0abfe3cf812
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609837
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8a6f1759331cef666279192de1fa3ecd10de702f
Submitter: Zuul
Branch: master

commit 8a6f1759331cef666279192de1fa3ecd10de702f
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 14:06:57 2018 -0700

    Flask comment/docstring cleanup

    Address a few nits in docstrings and comments from the flask conversion
    patches.

    Change-Id: I058d50168c8e5fa566bd98d7dba101ae9e4f1684
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609838
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ce93950f44b76b409a59d346758360df2a94cdc5
Submitter: Zuul
Branch: master

commit ce93950f44b76b409a59d346758360df2a94cdc5
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 14:30:54 2018 -0700

    Cleanup test_wsgi

    Cleanup test_wsgi and remove un-needed tests. Rename to test_app_config
    to more closely match what is, in-fact, tested now.

    Change-Id: Ifb9dd311da85597d03671a49cf43db43888ad869
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/610129
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=5c70aef2dacf801ccc147be4450e5985f545a855
Submitter: Zuul
Branch: master

commit 5c70aef2dacf801ccc147be4450e5985f545a855
Author: Morgan Fainberg <email address hidden>
Date: Fri Oct 12 10:30:04 2018 -0700

    Make collection_key and member_key raise if unset

    Instead of populating with __UNUSED__ or other silly string, make
    direct use of "collection_key" or "member_key" raise a ValueError
    if they are unset and referenced.

    Change-Id: Idf4f4df9d933317fff96a474cdf23d758ebdfa8c
    Partial-Bug: #1776504

Reviewed: https://review.openstack.org/609839
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=184c84ae76b1e4dc6b9907caf0cfa0d65bf8ecc5
Submitter: Zuul
Branch: master

commit 184c84ae76b1e4dc6b9907caf0cfa0d65bf8ecc5
Author: Morgan Fainberg <email address hidden>
Date: Thu Oct 11 15:16:02 2018 -0700

    Remove pre-flask legacy code

    This removes common.controller, common.extension, common.router, and
    common.wsgi. Relevant code from common.wsgi (used by AuthContext) was
    moved into keystone.server.flask.request_processing.middleware.auth_context.

    keystone.api.discovery now uses keystone.flask.base_url

    test_middleware and test_exception were modified to reflect the changes
    to the remaining code from keystone.common.wsgi

    keystone.common.authorization only holds a couple constants for auth
    work now.

    Routes is removed from requirements.txt

    Release-Note for migration to flask added.

    Change-Id: I81563b6a49c8f12ecade058a9483f3b6f070dc72
    Closes-Bug: #1776504

Changed in keystone:
status: In Progress → Fix Released
Displaying first 40 and last 40 comments. View all 158 comments or add a comment.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers