API: flavors - Cannot list all public and private flavors by default

[Gilles Dubreuil]

The API doesn't return all public and private flavors by default.
Effectively only public flavors are listed even though the default policy rule authorize it.

Here I'm using 'admin' user/project along with no explicit policy therefore relying on the default 'built-in' policy which I believe translate to "compute_extension:flavor_access:addTenantAccess": "rule:admin_api".

$ openstack flavor list --all
+--------------------------------------+-------------+-----+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+-------------+-----+------+-----------+-------+-----------+
| 1 | flavor-tiny | 64 | 0 | 0 | 1 | True |
| a1fec2c4-2f18-422b-977d-c7e2046cfaec | test1 | 1 | 1 | 0 | 1 | False |
+--------------------------------------+-------------+-----+------+-----------+-------+-----------+

# The default flavors list returns only the public ones:
$ curl -s -H "X-Auth-Token: $OS_TOKEN" -H "Content-Type: application/json" http://${OS_HOST}:8774/v2/flavors| python -mjson.tool
{
    "flavors": [
        {
            "id": "1",
            "links": [
                {
                    "href": "http://192.0.2.6:8774/v2/flavors/1",
                    "rel": "self"
                },
                {
                    "href": "http://192.0.2.6:8774/flavors/1",
                    "rel": "bookmark"
                }
            ],
            "name": "flavor-tiny"
        }
    ]
}

$ curl -s -H "X-Auth-Token: $OS_TOKEN" -H "Content-Type: application/json" http://${OS_HOST}:8774/v2/flavors?is_public=false | python -mjson.tool
{
    "flavors": [
        {
            "id": "a1fec2c4-2f18-422b-977d-c7e2046cfaec",
            "links": [
                {
                    "href": "http://192.0.2.6:8774/v2/flavors/a1fec2c4-2f18-422b-977d-c7e2046cfaec",
                    "rel": "self"
                },
                {
                    "href": "http://192.0.2.6:8774/flavors/a1fec2c4-2f18-422b-977d-c7e2046cfaec",
                    "rel": "bookmark"
                }
            ],
            "name": "test1"
        }
    ]
}

[Deepak Mourya (mourya007)]

Hi, I can get the flavor list with --all

openstack flavor list --all
+-----+-----------------------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+-----+-----------------------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 100 | manila-service-flavor | 192 | 0 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 42 | m1.nano | 64 | 0 | 0 | 1 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 84 | m1.micro | 128 | 0 | 0 | 1 | True |
| c1 | cirros256 | 256 | 0 | 0 | 1 | True |
| d1 | ds512M | 512 | 5 | 0 | 1 | True |
| d2 | ds1G | 1024 | 10 | 0 | 1 | True |
| d3 | ds2G | 2048 | 10 | 0 | 2 | True |
| d4 | ds4G | 4096 | 20 | 0 | 4 | True |

[Gilles Dubreuil (gdubreui)]

@mourya007,

Yes OpenStack Client has such option but not nova API.

[Matt Riedemann (mriedem)]

I'm confused by this:

> Yes OpenStack Client has such option but not nova API.

openstack client is using python-novaclient which is just passing is_public=None:

https://github.com/openstack/python-novaclient/blob/1f75c7662d6759354210a63ab7cdc06ba4237a2d/novaclient/v2/flavors.py#L111

And novaclient just omits the is_public param to GET /flavors. This is the API code:

https://github.com/openstack/nova/blob/d4dbb42593893c1d1ed51a127b7183a314bcac2c/nova/api/openstack/compute/flavors.py#L89

So it looks like that accepts 'none' as a string and our schema allows that also:

https://github.com/openstack/nova/blob/d4dbb42593893c1d1ed51a127b7183a314bcac2c/nova/api/openstack/compute/schemas/flavors.py#L22

But the API reference is out of date:

https://developer.openstack.org/api-ref/compute/#list-flavors

[Matt Riedemann (mriedem)]

> And novaclient just omits the is_public param to GET /flavors. This is the API code:

That's wrong, novaclient passes is_public=None specifically as a query parameter which is processed in the API as 'show all flavors' if it's an admin context.

Are you sure you're using an admin context when doing your curl request?

[Matt Riedemann (mriedem)]

Is your bug really about saying that admins shouldn't have to pass is_public=None *by default* and is_public=None should just be the default behavior for admins if the is_public query parameter isn't provided? If so, that's not a bug, and would require a microversion since it's a behavior change to the API.

[Matt Riedemann (mriedem)]

Marked as incomplete since I'm not sure what you're saying is the bug. Please clarify. I'll fix the API reference docs in the meantime.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/588092

[sean mooney (sean-k-mooney)]

based on http://paste.openstack.org/show/727101/ it looks like this works fine on master.

is there any error in the the nova-api log?

[sean mooney (sean-k-mooney)]

just as extra context this is the output of the openstack client as an admin

stack@cloud-3 devstack]$ openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 42 | m1.nano | 64 | 0 | 0 | 1 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 84 | m1.micro | 128 | 0 | 0 | 1 | True |
| c1 | cirros256 | 256 | 0 | 0 | 1 | True |
| d1 | ds512M | 512 | 5 | 0 | 1 | True |
| d2 | ds1G | 1024 | 10 | 0 | 1 | True |
| d3 | ds2G | 2048 | 10 | 0 | 2 | True |
| d4 | ds4G | 4096 | 20 | 0 | 4 | True |
+----+-----------+-------+------+-----------+-------+-----------+
[stack@cloud-3 devstack]$ openstack flavor list --all
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 2b7b0d2d-9981-4554-86a9-ddecc478703f | private | 1024 | 1 | 0 | 1 | False |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 42 | m1.nano | 64 | 0 | 0 | 1 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 84 | m1.micro | 128 | 0 | 0 | 1 | True |
| c1 | cirros256 | 256 | 0 | 0 | 1 | True |
| d1 | ds512M | 512 | 5 | 0 | 1 | True |
| d2 | ds1G | 1024 | 10 | 0 | 1 | True |
| d3 | ds2G | 2048 | 10 | 0 | 2 | True |
| d4 | ds4G | 4096 | 20 | 0 | 4 | True |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+

[Gilles Dubreuil (gdubreui)]

Sorry if the description was confusing.

First off, thank you for all the pointers.

To clarify, I confirm this is about nova API (as mentioned in title).

The API document effectively doesn't provide any information about the option "None" that can be passed to the parameter "is_public".
This was mentioned in https://bugs.launchpad.net/nova/+bug/1784782/comments/3, thank you.

The later solves part of the issue as "all" flavors (public and private) can be returned to Nova API client when using "None".

Meanwhile there are 2 other issues at stake here.

1. I believe that "all" the flavors should be returned by default (attended the user has the right privileges which is admin only by default but could be another tenant if the policy is changed or if flavors access has been granted provided to another tenant).

2. The actual code is not consistent.
In following pasted example http://paste.openstack.org/show/727111/
the behaviour is different between the 'admin' user and the 'demo' user where the 'demo' project has been granted access to a private flavor. By default 'admin' doesn't get returned the private flavor where 'demo' does get it.

Also please note "/flavors/detail" resource faces the same issue.
I've not tested using a difference policy.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/588092
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=eff376b9fc7a6c96ad63c6d9004f902769def01d
Submitter: Zuul
Branch: master

commit eff376b9fc7a6c96ad63c6d9004f902769def01d
Author: Matt Riedemann <email address hidden>
Date: Wed Aug 1 18:15:07 2018 -0400

    api-ref: fix GET /flavors?is_public description

    A couple of things are fixed here:

    1. The type in the schema for the is_public query param
       is string, not boolean.

    2. Since it's a string, the normal 1/yes/0/no types of
       "booleans" are allowed so document that along with
       the default.

    3. Also mention that is_public='None' must be passed for
       an admin user to list both public and private flavors
       in a single request.

    Change-Id: Idcb700b69f13217f68434fd6a54439cc277f8998
    Partial-Bug: #1784782

[Gilles Dubreuil (gdubreui)]

Unfortunately the patch is not providing the right information because private flavors can be accessed by non-admin users when:
1 - Granted access via "flavor access" - see [1]
2 - Policy is changed: "compute_extension:flavor_access:addTenantAccess": ""

The current behavior is not taking the above in consideration.

BTW, it would be easier for users to get all flavors systematically, public and private (with not accessible private ones filtered out).

[1] https://developer.openstack.org/api-ref/compute/#flavors-access-flavors-os-flavor-access