Quotas can be set differently on same project if using project name or ID

[Vadym Markov]

OpenStack Designate allows to set quotas on a project with its ID and name independently

Steps to reproduce:

(openstack) project list
+----------------------------------+------------------------------------------------------------------+
| ID | Name |
+----------------------------------+------------------------------------------------------------------+
| 58c7fbb2d90541359a2ee0137755ba0b | admin |
| aa4dcc3bc5ae4729bda30840e984edc6 | demo |
+----------------------------------+------------------------------------------------------------------+

(openstack) dns quota set --zones 3 --project demo +-------------------+-------+
| Field | Value |
+-------------------+-------+
| api_export_size | 1000 |
| recordset_records | 20 |
| zone_records | 500 |
| zone_recordsets | 500 |
| zones | 3 |
+-------------------+-------+

Now set quota for the same project, but address it by id:
(openstack) dns quota set --zones 4 --project aa4dcc3bc5ae4729bda30840e984edc6
+-------------------+-------+
| Field | Value |
+-------------------+-------+
| api_export_size | 1000 |
| recordset_records | 20 |
| zone_records | 500 |
| zone_recordsets | 500 |
| zones | 4 |
+-------------------+-------+

We addressed the same project firstly by name, and secondly by id. Anyway, --project option for client is not supported officially, but supported --project-id gives the same result

In DB we have both quotas records pointing to the same project:
mysql> select * from quotas;
+----------------------------------+---------+---------------------+---------------------+----------------------------------+----------+------------+
| id | version | created_at | updated_at | tenant_id | resource | hard_limit |
+----------------------------------+---------+---------------------+---------------------+----------------------------------+----------+------------+
| 01871011140d47309c73fbe8f6ed0acb | 2 | 2018-04-02 15:16:26 | 2018-04-03 09:29:07 | demo | zones | 3 |
| 57b2dacc2dae49a486d138aa3c7f2600 | 2 | 2018-04-02 15:17:42 | 2018-04-03 09:30:15 | aa4dcc3bc5ae4729bda30840e984edc6 | zones | 4 |
| a0cf6df1d35242839ccba964f669ebf9 | 1 | 2018-04-02 13:20:14 | NULL | 58c7fbb2d90541359a2ee0137755ba0b | zones | 2 |
+----------------------------------+---------+---------------------+---------------------+----------------------------------+----------+------------+
3 rows in set (0.00 sec)

[Graham Hayes (grahamhayes)]

Users can currently set the project-id to any string - we do not verify that the project ID is valid (as we can't - only keystone has the list of projects).

When checking a projects quotas, we only check for the project ID, and do not use the project name at all.

the --project and --project-id issue is a "feature" of the CLI library we use - it will expand shortened flags

[Pavlo Shchelokovskyy (pshchelo)]

Graham, this then might lead to confusion for users/ops, when thinking they've set the quota for project (using name) in fact quota was not set.

[Pavlo Shchelokovskyy (pshchelo)]

I think that even whithout validating the validity of the project/project-id, at least the Designate API could validate that the incoming value is uuid-like if only UUID-like values will have any meaning in further operation.

[Javier Diaz Jr (javierdiazcharles)]

I agree with Pavlo. From an admin perspective this can be problematic if the admin assumes quotas have been set but in fact have not been set properly. Is there any way the Designate API can verify the input value and notify the user to use the project ID?

I am moving this back to confirmed as I think it deserves a second look. If anything this can be an opinion, but absolutely not invalid.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/559305

[Graham Hayes (grahamhayes)]

Not all project IDs are UUIDs unfortunately - in the past I have worked on clouds were they were ints, The keystone validator is very loose [1]

There is work happening on storing quota limit information in keystone, that will ensure the ID is attached to a real project, so my thoughts would be to wait for that work to complete.

If this is required in the meantime, we would need to do vaildation in the quota object [2].
This would need to be off by default, and enabled by a config value, and it should override the the objects validate() method and call keystone to see if the ID is actually a project.

1 - https://github.com/openstack/keystone/blob/b9dee7332374d9a382c8403d646909e0c70a90ac/keystone/common/validation/parameter_types.py#L38-L45
2 - https://github.com/openstack/designate/blob/master/designate/objects/quota.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/560066

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vadym Markov (<email address hidden>) on branch: master
Review: https://review.openstack.org/560066
Reason: Will implement interim solution in Designate itself as Graham proposed

[Dr. Jens Harbott (j-harbott)]

Nova has implemented a similar thing, maybe that approach could work for designate, too.

https://review.openstack.org/435010

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/576219

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/580142

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vadym Markov (<email address hidden>) on branch: master
Review: https://review.openstack.org/559305
Reason: Alternative patch merged

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/588509

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/580142
Committed: https://git.openstack.org/cgit/openstack/designate-tempest-plugin/commit/?id=60e5372a46f588633d68d453c829c397363ed7c2
Submitter: Zuul
Branch: master

commit 60e5372a46f588633d68d453c829c397363ed7c2
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Wed Jul 4 11:21:29 2018 +0000

    Add test for quota set for invalid project

    this also adds a feature flag "api_v2_quotas_verify_project" (False by
    default) enabling which will enable this test.

    Change-Id: Icdc1b9ae419c3634ab2c55005f33fed624706176
    Related-Bug: #1760822

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/576219
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=56651f1fdd58f5692103a5039c6ba54016c7a0b0
Submitter: Zuul
Branch: master

commit 56651f1fdd58f5692103a5039c6ba54016c7a0b0
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Tue Jul 3 21:23:19 2018 +0000

    Verify project-id when setting quota

    this patch adds several things to attempt (on best effort basis)
    to use incoming user token in the quota set request to ask Keystone
    if the project id the quota being set on is actualy valid:

    - added new [keystone] config section to hold session- and
      adapter-realated options for accessing Keystone enndpoint
    - added a token- and service catalog-based user auth plugin to the request
      context
    - use the above to construct a keystoneauth adapter for Identity service
      and attempt to GET on projects/{project_id}

      - only if the Keystone v3 catalog endpoint is not found, or
        the request returns 404 NotFound, we raise an error and return it
        as 400 BadRequest to client when attempting to change quotas for
        project.

    This behavior is enabled by setting a new
    [service:api]quotas-verify-project-id config option to True (default is
    False for backward compatibility).

    Change-Id: Ib14ee5b5628509b6a93be8b7bd10e734ab19ffee
    Depends-On: https://review.openstack.org/580142
    Closes-Bug: #1760822

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/588509
Committed: https://git.openstack.org/cgit/openstack/designate/commit/?id=f3c3dec7dad1471517fb78c2a19bbebf5262094f
Submitter: Zuul
Branch: master

commit f3c3dec7dad1471517fb78c2a19bbebf5262094f
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Fri Aug 3 13:34:29 2018 +0300

    Add docs for project-id verification feature

    Change-Id: Ibe98d50be578b35cb92c70d703c23dd27b192787
    Related-Bug: #1760822

[Bradley Richard Shirley (bshirley223)]

has there been any updates on this

[Graham Hayes (grahamhayes)]

this merged in Stein - https://docs.openstack.org/releasenotes/designate/unreleased.html#relnotes-7-0-0-48

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/designate 8.0.0.0rc1 release candidate.