OpenStack Compute (nova)

non-admin users raise KeyError u'project_id'

Bug #1117433 reported by Allan Feid
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
Undecided
Unassigned

Bug Description

Important note: this occurred post-upgrade from essex to folsom. I'm using the CentOS packages provided by EPEL.

steps:

1) obtain an auth-token

curl -H 'Content-Type: application/json' -d '{ "auth": {"tenantName": "Development", "passwordCredentials": {"username": "jenkins", "password": "*****"} } }' http://10.0.80.15:5000/v2.0/tokens
"access": {"token": {"expires": "2013-02-07T15:23:49Z", "id": "e3d266a113a64558801537830b01001d", "tenant": {"enabled": true, "description": "The developer group", "name": "Development", "id": "62b31fa8598a443487d99a79b6ba5547"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.0.80.15:8774/v2/62b31fa8598a443487d99a79b6ba5547", "region": "nyc02", "internalURL": "http://10.0.80.15:8774/v2/62b31fa8598a443487d99a79b6ba5547", "id": "bb7fa36c03bf48589b87109509bfacb0", "publicURL": "http://10.0.80.15:8774/v2/62b31fa8598a443487d99a79b6ba5547"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.0.80.15:9292/v1", "region": "nyc02", "internalURL": "http://10.0.80.15:9292/v1", "id": "a1560797b76d45209af5820c72edf0c3", "publicURL": "http://10.0.80.15:9292/v1"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.0.80.15:8776/v1/62b31fa8598a443487d99a79b6ba5547", "region": "nyc02", "internalURL": "http://10.0.80.15:8776/v1/62b31fa8598a443487d99a79b6ba5547", "id": "214577425ac8411ea114f5d0285d2814", "publicURL": "http://10.0.80.15:8776/v1/62b31fa8598a443487d99a79b6ba5547"}], "endpoints_links": [], "type": "volume", "name": "volume"}, {"endpoints": [{"adminURL": "http://10.0.80.15:8773/services/Admin", "region": "nyc02", "internalURL": "http://10.0.80.15:8773/services/Cloud", "id": "8d0f2bbbd729465eaf92964c728a60db", "publicURL": "http://10.0.80.15:8773/services/Cloud"}], "endpoints_links": [], "type": "ec2", "name": "ec2"}, {"endpoints": [{"adminURL": "http://10.0.80.15:8080/", "region": "nyc02", "internalURL": "http://10.0.80.15:8080/v1/AUTH_62b31fa8598a443487d99a79b6ba5547", "id": "8ab8b4cddd224f8facba3bcaf909b323", "publicURL": "http://10.0.80.15:8080/v1/AUTH_62b31fa8598a443487d99a79b6ba5547"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://10.0.80.15:35357/v2.0", "region": "nyc02", "internalURL": "http://10.0.80.15:5000/v2.0", "id": "1e030df055e54aa2bde029f30a50c79d", "publicURL": "http://10.0.80.15:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "jenkins", "roles_links": [], "id": "f80bbe2743b74f92a85ba61e8f93e62c", "roles": [], "name": "jenkins"}}}

2) attempt to list servers for a tenant

curl -v -H 'X-Auth-Token: e3d266a113a64558801537830b01001d'

Expected:

A list of servers for the Development tenant.

Actual:

Reply from server:
http://10.0.80.15:8774/v2/2201915216d143038d65f61e323caf15/servers
* About to connect() to 10.0.80.15 port 8774 (#0)
* Trying 10.0.80.15...
* connected
* Connected to 10.0.80.15 (10.0.80.15) port 8774 (#0)
> GET /v2/2201915216d143038d65f61e323caf15/servers HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: 10.0.80.15:8774
> Accept: */*
> X-Auth-Token: e3d266a113a64558801537830b01001d
>
< HTTP/1.1 500 Internal Server Error
< Content-Length: 128
< Content-Type: application/json; charset=UTF-8
< Date: Wed, 06 Feb 2013 15:24:38 GMT
<
* Connection #0 to host 10.0.80.15 left intact
{"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}* Closing connection #0

Stack trace:

2013-02-05 16:01:31 6291 ERROR nova.api.openstack [-] Caught error: u'project_id'
2013-02-05 16:01:31 6291 TRACE nova.api.openstack Traceback (most recent call last):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 78, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return req.get_response(self.application)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1053, in get_response
2013-02-05 16:01:31 6291 TRACE nova.api.openstack application, catch_exc_info=False)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1022, in call_application
2013-02-05 16:01:31 6291 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return resp(environ, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/keystone/middleware/auth_token.py", line 278, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return self.app(env, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 147, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack resp = self.call_func(req, *args, **self.kwargs)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 208, in call_func
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return self.func(req, *args, **kwargs)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/api/auth.py", line 117, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack service_catalog=service_catalog)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/context.py", line 70, in __init__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack self.is_admin = policy.check_is_admin(self.roles)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/policy.py", line 115, in check_is_admin
2013-02-05 16:01:31 6291 TRACE nova.api.openstack exception.PolicyNotAuthorized, action=action)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 125, in enforce
2013-02-05 16:01:31 6291 TRACE nova.api.openstack if not _BRAIN.check(match_list, target_dict, credentials_dict):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
                                                                                                                                                      1,1 Top
2013-02-05 16:01:31 6291 ERROR nova.api.openstack [-] Caught error: u'project_id'
2013-02-05 16:01:31 6291 TRACE nova.api.openstack Traceback (most recent call last):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 78, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return req.get_response(self.application)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1053, in get_response
2013-02-05 16:01:31 6291 TRACE nova.api.openstack application, catch_exc_info=False)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/request.py", line 1022, in call_application
2013-02-05 16:01:31 6291 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 159, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return resp(environ, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/keystone/middleware/auth_token.py", line 278, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return self.app(env, start_response)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 147, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack resp = self.call_func(req, *args, **self.kwargs)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg/webob/dec.py", line 208, in call_func
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return self.func(req, *args, **kwargs)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/api/auth.py", line 117, in __call__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack service_catalog=service_catalog)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/context.py", line 70, in __init__
2013-02-05 16:01:31 6291 TRACE nova.api.openstack self.is_admin = policy.check_is_admin(self.roles)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/policy.py", line 115, in check_is_admin
2013-02-05 16:01:31 6291 TRACE nova.api.openstack exception.PolicyNotAuthorized, action=action)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 125, in enforce
2013-02-05 16:01:31 6291 TRACE nova.api.openstack if not _BRAIN.check(match_list, target_dict, credentials_dict):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 260, in _check_rule
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return brain.check(new_match_list, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 204, in check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack for item in and_list]):
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 184, in _check
2013-02-05 16:01:31 6291 TRACE nova.api.openstack return func(self, match_kind, match_value, target_dict, cred_dict)
2013-02-05 16:01:31 6291 TRACE nova.api.openstack File "/usr/lib/python2.6/site-packages/nova/openstack/common/policy.py", line 300, in _check_generic
2013-02-05 16:01:31 6291 TRACE nova.api.openstack match = match % target_dict
2013-02-05 16:01:31 6291 TRACE nova.api.openstack KeyError: u'project_id'

This may be a config problem on my end, but regardless this error is pretty obfuscated. I've tracked it down to an empty target_dict being passed in to _check_generic via the policy.is_admin check. It does directly seem to be related to my policy.json for the nova service, which has the following default_rule:

    "admin_or_owner": [["role:admin"], ["project_id:%(project_id)s"]],
    "default": [["rule:admin_or_owner"]],

Changing project_id:%(project_id)s to any other key causes that to be raised as the KeyError. Removing the secondary part of admin_or_owner causes:

ERROR: Policy doesn't allow compute:get_instance_faults to be performed. (HTTP 403) (Request-ID: req-b61ab676-a9c4-4530-916b-73f5f33211b2)

My full policy.json:

{
    "admin_or_owner": [["role:admin"], ["project_id:%(project_id)s"]],
    "default": [["rule:admin_or_owner"]],

    "compute:create": [],
    "compute:create:attach_network": [],
    "compute:create:attach_volume": [],
    "compute:get_all": [],

    "admin_api": [["role:admin"]],
    "compute_extension:accounts": [["rule:admin_api"]],
    "compute_extension:admin_actions": [["rule:admin_api"]],
    "compute_extension:admin_actions:pause": [["rule:admin_or_owner"]],
    "compute_extension:admin_actions:unpause": [["rule:admin_or_owner"]],
    "compute_extension:admin_actions:suspend": [["rule:admin_or_owner"]],
    "compute_extension:admin_actions:resume": [["rule:admin_or_owner"]],
    "compute_extension:admin_actions:lock": [["rule:admin_api"]],
    "compute_extension:admin_actions:unlock": [["rule:admin_api"]],
    "compute_extension:admin_actions:resetNetwork": [["rule:admin_api"]],
    "compute_extension:admin_actions:injectNetworkInfo": [["rule:admin_api"]],
    "compute_extension:admin_actions:createBackup": [["rule:admin_or_owner"]],
    "compute_extension:admin_actions:migrateLive": [["rule:admin_api"]],
    "compute_extension:admin_actions:migrate": [["rule:admin_api"]],
    "compute_extension:aggregates": [["rule:admin_api"]],
    "compute_extension:certificates": [],
    "compute_extension:cloudpipe": [["rule:admin_api"]],
    "compute_extension:console_output": [],
    "compute_extension:consoles": [],
    "compute_extension:createserverext": [],
    "compute_extension:deferred_delete": [],
    "compute_extension:disk_config": [],
    "compute_extension:extended_server_attributes": [["rule:admin_api"]],
    "compute_extension:extended_status": [],
    "compute_extension:flavorextradata": [],
    "compute_extension:flavorextraspecs": [],
    "compute_extension:flavormanage": [["rule:admin_api"]],
    "compute_extension:floating_ip_dns": [],
    "compute_extension:floating_ip_pools": [],
    "compute_extension:floating_ips": [],
    "compute_extension:hosts": [["rule:admin_api"]],
    "compute_extension:keypairs": [],
    "compute_extension:multinic": [],
    "compute_extension:networks": [["rule:admin_api"]],
    "compute_extension:quotas": [],
    "compute_extension:rescue": [],
    "compute_extension:security_groups": [],
    "compute_extension:server_action_list": [["rule:admin_api"]],
    "compute_extension:server_diagnostics": [["rule:admin_api"]],
    "compute_extension:simple_tenant_usage:show": [["rule:admin_or_owner"]],
    "compute_extension:simple_tenant_usage:list": [["rule:admin_api"]],
    "compute_extension:users": [["rule:admin_api"]],
    "compute_extension:virtual_interfaces": [],
    "compute_extension:virtual_storage_arrays": [],
    "compute_extension:volumes": [],
    "compute_extension:volumetypes": [],

    "volume:create": [],
    "volume:get_all": [],
    "volume:get_volume_metadata": [],
    "volume:get_snapshot": [],
    "volume:get_all_snapshots": [],

    "network:get_all_networks": [],
    "network:get_network": [],
    "network:delete_network": [],
    "network:disassociate_network": [],
    "network:get_vifs_by_instance": [],
    "network:allocate_for_instance": [],
    "network:deallocate_for_instance": [],
    "network:validate_networks": [],
    "network:get_instance_uuids_by_ip_filter": [],

    "network:get_floating_ip": [],
    "network:get_floating_ip_pools": [],
    "network:get_floating_ip_by_address": [],
    "network:get_floating_ips_by_project": [],
    "network:get_floating_ips_by_fixed_address": [],
    "network:allocate_floating_ip": [],
    "network:deallocate_floating_ip": [],
    "network:associate_floating_ip": [],
    "network:disassociate_floating_ip": [],

    "network:get_fixed_ip": [],
    "network:add_fixed_ip_to_instance": [],
    "network:remove_fixed_ip_from_instance": [],
    "network:add_network_to_project": [],
    "network:get_instance_nw_info": [],

    "network:get_dns_domains": [],
    "network:add_dns_entry": [],
    "network:modify_dns_entry": [],
    "network:delete_dns_entry": [],
    "network:get_dns_entries_by_address": [],
    "network:get_dns_entries_by_name": [],
    "network:create_private_dns_domain": [],
    "network:create_public_dns_domain": [],
    "network:delete_dns_domain": []
}

See original description
Allan Feid (crayz)
description: updated
Revision history for this message
Allan Feid (crayz) wrote :

I doubt this is a correct workaround, but a simple try/catch for the KeyError in _check_generic resolved this issue for me.

Revision history for this message
Vish Ishaya (vishvananda) wrote :

it looks like you need context_is_admin your policy.json file:

I think you should replace your admin_or_owner lines with the lines from the default file in folsom:

  "context_is_admin": [["role:admin"]],
  "admin_or_owner": [["is_admin:True"], ["project_id:%(project_id)s"]],

Changed in nova:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Patches

Remote bug watches

Bug watches keep track of this bug in other bug trackers.