oslo.policy policy_dirs support is missing

Bug #1741723 reported by Dmitrii Shcherbakov on 2018-01-07
32
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Gnocchi Charm
Wishlist
Unassigned
OpenStack AODH Charm
Wishlist
Unassigned
OpenStack Designate Charm
Wishlist
Alex Kavanagh
OpenStack Octavia Charm
Wishlist
Alex Kavanagh
OpenStack cinder charm
Wishlist
Alex Kavanagh
OpenStack glance charm
Wishlist
Alex Kavanagh
OpenStack heat charm
Wishlist
Alex Kavanagh
OpenStack keystone charm
Wishlist
Alex Kavanagh
OpenStack neutron-api charm
Wishlist
Alex Kavanagh
OpenStack neutron-gateway charm
Wishlist
Unassigned
OpenStack nova-cloud-controller charm
Wishlist
Alex Kavanagh
OpenStack openstack-dashboard charm
Wishlist
Alex Kavanagh
OpenStack panko charm
Wishlist
Unassigned
OpenStack swift-proxy charm
Wishlist
Unassigned
charms.openstack
Wishlist
Dmitrii Shcherbakov

Bug Description

In order to augment policy definitions either a charm-supplied policy file needs to be modified or a drop-in mechanism needs to be used.

oslo.policy provides policy_dirs option ('policy.d' relative to conf_dir by default) which is inspected if policy_file exists (which is the case for most of the openstack charms but not for every charm).

https://github.com/openstack/oslo.policy/blame/stable/ocata/oslo_policy/opts.py#L25-L47

Projects that use oslo.policy's Enforcers automatically gain support for this functionality.

https://github.com/openstack/oslo.policy/blob/stable/ocata/oslo_policy/policy.py#L428-L445
https://github.com/openstack/oslo.policy/blob/stable/ocata/oslo_policy/policy.py#L502-L542

There needs to be a way for charms to utilize this by rendering a service-specific policy file.

Support for this is needed across classic and reactive openstack charms (primary service charms, plugin charms, subordinate charms). For subordinate charms this would mean that a common mechanism to trigger service restart would be needed across different charms on subordinate policy config change.

Dmitrii Shcherbakov (dmitriis) wrote :

Modified to include an ability to supply a jinja2 template as a drop-in and use the same variables as in policy.json template file.

https://github.com/juju/charm-helpers/pull/87
https://github.com/juju/charm-helpers/pull/88
https://review.openstack.org/#/c/531614/

James Page (james-page) on 2018-01-10
Changed in charm-aodh:
status: New → Triaged
Changed in charm-cinder:
status: New → Triaged
Changed in charm-designate:
status: New → Triaged
Changed in charm-glance:
status: New → Triaged
Changed in charm-heat:
status: New → Triaged
Changed in charm-keystone:
status: New → Triaged
Changed in charm-neutron-api:
status: New → Triaged
Changed in charm-neutron-gateway:
status: New → Triaged
Changed in charm-nova-cloud-controller:
status: New → Triaged
Changed in charm-openstack-dashboard:
status: New → Triaged
Changed in charm-swift-proxy:
status: New → Triaged
Changed in charm-aodh:
importance: Undecided → Wishlist
Changed in charm-cinder:
importance: Undecided → Wishlist
Changed in charm-designate:
importance: Undecided → Wishlist
Changed in charm-glance:
importance: Undecided → Wishlist
Changed in charm-heat:
importance: Undecided → Wishlist
Changed in charm-keystone:
importance: Undecided → Wishlist
Changed in charm-neutron-api:
importance: Undecided → Wishlist
Changed in charm-neutron-gateway:
importance: Undecided → Wishlist
Changed in charm-nova-cloud-controller:
importance: Undecided → Wishlist
Changed in charm-openstack-dashboard:
importance: Undecided → Wishlist
Changed in charm-swift-proxy:
importance: Undecided → Wishlist

Fix proposed to branch: master
Review: https://review.openstack.org/539269

Changed in charm-panko:
assignee: nobody → Dmitrii Shcherbakov (dmitriis)
status: New → In Progress
Changed in charms.openstack:
assignee: nobody → Dmitrii Shcherbakov (dmitriis)
status: New → In Progress
Changed in charm-keystone:
assignee: nobody → Dmitrii Shcherbakov (dmitriis)

Reviewed: https://review.openstack.org/538688
Committed: https://git.openstack.org/cgit/openstack/charms.openstack/commit/?id=1ca4bd0ab941f6c7aebf86bdf5ff47fdc1d2d3e2
Submitter: Zuul
Branch: master

commit 1ca4bd0ab941f6c7aebf86bdf5ff47fdc1d2d3e2
Author: Dmitrii Shcherbakov <email address hidden>
Date: Sat Jan 27 22:42:16 2018 +0300

    add string template rendering capability

    In some cases software deliberately allows drop-in config file usage
    capabilities, for example, when it comes to enforcing policy, the
    desired behavior varies from an operator to operator. For that reason it
    is sometimes desirable to supply custom templates via config options.

    Another use-case is templates that are passed from subordinates for a
    primary charm to render.

    Given that properties and desired adapters can be arbitrary the change
    uses a dict of meta tuples of the following format to render templates
    from strings based on adapter properties:

    {config_file_path: (relation_name, adapter property)}

    relation names must be normalized (lowercase, underscores instead of
    dashes. "options" relation name is used for a config adapter as usual.

    In summary a string config file path should be used:

    1. in the restart_map for a given derived class;
    2. in string_templates dict as a key for a meta tuple

    Change-Id: Ic85b22d0e5d497c49c75243e3c280140f940df66
    Closes-Bug: #1741723

Changed in charms.openstack:
status: In Progress → Fix Released
James Page (james-page) on 2018-02-15
Changed in charm-gnocchi:
importance: Undecided → Wishlist
status: New → Triaged
Changed in charm-panko:
importance: Undecided → Wishlist
Changed in charms.openstack:
importance: Undecided → Wishlist
Dmitrii Shcherbakov (dmitriis) wrote :

openstack-dashboard is a bit unique as it needs to have many per-service policy files and they need to be in sync with individual policy files for other charms.

https://docs.openstack.org/horizon/latest/contributor/topics/policy.html
"The implementation in Horizon is based on copies of policy files found in the service’s source code."

Therefore, horizon charm will need to populate per-service policy dirs and this is something only available in Queens for Horizon project itself:

https://blueprints.launchpad.net/horizon/+spec/policy-dirs

So, for horizon, a single extra-policy option will need to have policy file templates for multiple services which makes it quite complex.

Maybe in case of horizon it would be necessary to get policy files via relation data but it is necessary to get 3 types of policy:

1. a render of policy-in-code for a given service (not applicable for older OpenStack releases);
2. policy files embedded into charms;
3. extra-policy configs from individual charms.

tags: added: canonical-bootstack

Change abandoned by Frode Nordahl (<email address hidden>) on branch: master
Review: https://review.openstack.org/531614
Reason: Marking this review as abandoned due to lack of activity in the past 6 months.
Feel free to restore it again if you want to pick up and continue the work.

Paul Henien (phenien) wrote :

A customer asked asked making changes in the policy.json on the cinder and openstack-dashboard charms to enable consistency groups and consistency group snapshots.

Changed in charm-keystone:
assignee: Dmitrii Shcherbakov (dmitriis) → nobody
Changed in charm-panko:
assignee: Dmitrii Shcherbakov (dmitriis) → nobody
James Page (james-page) on 2019-09-27
Changed in charm-neutron-gateway:
status: Triaged → Invalid

Change abandoned by Alex Kavanagh (tinwood) (<email address hidden>) on branch: master
Review: https://review.opendev.org/685331
Reason: neutron-gateway isn't actually an API charm and so doesn't need this change!

Changed in charm-keystone:
status: Triaged → In Progress
Changed in charm-glance:
status: Triaged → In Progress
Changed in charm-designate:
status: Triaged → In Progress
Changed in charm-cinder:
status: Triaged → In Progress
Changed in charm-neutron-api:
status: Triaged → In Progress
Changed in charm-nova-cloud-controller:
status: Triaged → In Progress
Changed in charm-designate:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-cinder:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-glance:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-keystone:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-neutron-api:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-nova-cloud-controller:
assignee: nobody → Alex Kavanagh (ajkavanagh)
Changed in charm-openstack-dashboard:
assignee: nobody → Alex Kavanagh (ajkavanagh)
status: Triaged → In Progress
Changed in charm-panko:
status: In Progress → Triaged
Changed in charm-designate:
milestone: none → 19.10
Changed in charm-cinder:
milestone: none → 19.10
Changed in charm-glance:
milestone: none → 19.10
Changed in charm-keystone:
milestone: none → 19.10
Changed in charm-neutron-api:
milestone: none → 19.10
Changed in charm-nova-cloud-controller:
milestone: none → 19.10
Changed in charm-openstack-dashboard:
milestone: none → 19.10

Reviewed: https://review.opendev.org/685726
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cloud-controller/commit/?id=f7f6fa295c82ea88953ec08695c68c04694fc1a5
Submitter: Zuul
Branch: master

commit f7f6fa295c82ea88953ec08695c68c04694fc1a5
Author: Alex Kavanagh <email address hidden>
Date: Mon Sep 30 15:47:53 2019 +0100

    Policyd override implementation

    This patchset implements policy overrides for nova-cloud-controler.

    This change includes a charm-helpers sync to bring in the policyd helper
    code.

    Note there are no functional tests for this feature as the charm still
    uses the old style non-zaza amulet framework. The Related-Bug below is
    tracking this issue.

    Change-Id: Ia5f3f8189d4a7b7b46a827707d964ebe40740aeb
    Closes-Bug: #1741723
    Related-Bug: #1845639

Changed in charm-nova-cloud-controller:
status: In Progress → Fix Committed
Changed in charm-neutron-api:
status: In Progress → Fix Committed

Reviewed: https://review.opendev.org/685376
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-api/commit/?id=697ca00bcf29c6a6f3dbca4b13c16a87804592ab
Submitter: Zuul
Branch: master

commit 697ca00bcf29c6a6f3dbca4b13c16a87804592ab
Author: Alex Kavanagh <email address hidden>
Date: Fri Sep 27 16:12:19 2019 +0100

    Policyd override implementation

    This patchset implements policy overrides for neutron-gateway.

    This change includes a charm-helpers sync to bring in the policyd helper
    code.

    Change-Id: I89f1f4b5d58843017e428a8d2cfada840dde14de
    Closes-Bug: #1741723

Reviewed: https://review.opendev.org/679420
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=186769cc056f4306111c1c26e899dd4697ff468f
Submitter: Zuul
Branch: master

commit 186769cc056f4306111c1c26e899dd4697ff468f
Author: Alex Kavanagh <email address hidden>
Date: Fri Aug 30 11:58:04 2019 +0100

    Policyd override implementation

    This patchset implements policy overrides for keystone. It uses the
    code in charmhelpers.

    Closed-Bug: #1741723
    Change-Id: I187f4493392178d87ef7dbd67de841bbeae0c65d

Reviewed: https://review.opendev.org/685973
Committed: https://git.openstack.org/cgit/openstack/charm-glance/commit/?id=97152f55a1f045484ad3fde69a99a84fce8f349b
Submitter: Zuul
Branch: master

commit 97152f55a1f045484ad3fde69a99a84fce8f349b
Author: Alex Kavanagh <email address hidden>
Date: Tue Oct 1 14:55:29 2019 +0100

    Policyd override implementation

    This patchset implements policy overrides for glance. It uses the
    code in charmhelpers.

    Change-Id: I0586326ff87fdf03f2c88e4c459627f4085c3367
    Closed-Bug: #1741723

Change abandoned by Alex Kavanagh (tinwood) (<email address hidden>) on branch: master
Review: https://review.opendev.org/686006
Reason: OpenStack dashboard requires a much more complex set of policy overrides than the system in charm-helpers can support. It'll require a completely different approach, but can be based on the existing code.

Reviewed: https://review.opendev.org/685982
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=6ee32006e54f67e7415a042f00b3489ddbcd113a
Submitter: Zuul
Branch: master

commit 6ee32006e54f67e7415a042f00b3489ddbcd113a
Author: Alex Kavanagh <email address hidden>
Date: Tue Oct 1 15:37:29 2019 +0100

    Policyd override implementation

    This patchset implements policy overrides for cinder. It uses the
    code in charmhelpers.

    It also fixes several bugs in the bundles where the actual version of
    cinder that was being installed was the distro default rather than the
    one that the bundle described.

    Change-Id: Ic979dcb96ddb931fadb1fa4a4b36108244ddf306
    Closed-Bug: #1741723

David Ames (thedac) on 2019-10-24
Changed in charm-neutron-api:
status: Fix Committed → Fix Released
Changed in charm-nova-cloud-controller:
status: Fix Committed → Fix Released
David Ames (thedac) on 2019-10-24
Changed in charm-keystone:
milestone: 19.10 → 20.01
Changed in charm-glance:
milestone: 19.10 → 20.01
Changed in charm-cinder:
milestone: 19.10 → 20.01
Changed in charm-openstack-dashboard:
milestone: 19.10 → 20.01
Changed in charm-designate:
milestone: 19.10 → 20.01
Changed in charm-heat:
assignee: nobody → Alex Kavanagh (ajkavanagh)
status: Triaged → In Progress
Changed in charm-cinder:
milestone: 20.01 → 19.10
status: In Progress → Fix Released
Changed in charm-glance:
milestone: 20.01 → 19.10
status: In Progress → Fix Released
Changed in charm-keystone:
milestone: 20.01 → 19.10
status: In Progress → Fix Released
Changed in charm-designate:
milestone: 20.01 → 19.10
status: In Progress → Fix Released
Alex Kavanagh (ajkavanagh) wrote :

Note that swift software doesn't (yet) support the oslo.policy Enforcer class, which means that policy.d overrides aren't possible with it (yet).

Changed in charm-octavia:
assignee: nobody → Alex Kavanagh (ajkavanagh)
importance: Undecided → Wishlist
milestone: none → 20.01
status: New → In Progress

Reviewed: https://review.opendev.org/693162
Committed: https://git.openstack.org/cgit/openstack/charm-heat/commit/?id=98de623820f58dc562ba770431e3464193d15a27
Submitter: Zuul
Branch: master

commit 98de623820f58dc562ba770431e3464193d15a27
Author: Alex Kavanagh <email address hidden>
Date: Wed Nov 6 11:44:06 2019 +0000

    Policyd override implementation

    This patchset implements policy overrides for heat. It uses the
    code in charmhelpers.

    It also fixes a bug in the actions/domain-setup where it assumes that
    the python2 version of openstackclient should be installed, and corrects
    this via code in hooks/install and hooks/upgrade-charm.

    A sync of charm-helpers is included to bring the latest policyd changes
    through to the charm.

    func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/111

    Change-Id: Ia607dc9120cfb03902efb019041b43cf12ade2d3
    Closed-Bug: #1741723

Changed in charm-heat:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers