Hardcoded tenant name to access swift objects for job execution

Bug #1306713 reported by Luigi Toscano
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Sahara
Fix Released
High
Dmitry Mescheryakov
Icehouse
Fix Released
High
Dmitry Mescheryakov

Bug Description

The scenario is the same described in lp:1305210.
Main points: user admin, tenant/project demo. The Data Sources and the Job Binaries have been configured to use the admin user.
Sahara configuration files contains:
--------------------
os_admin_tenant_name = service
os_admin_username = sahara
--------------------
admin user is not in the service project.

When the proposed fix for the other bug is applied (currently under review), the following backtrace can be observed:

2014-04-11 17:42:41.474 DEBUG keystoneclient.session [-] REQ: curl -i -X POST http://10.34.1.187:5000/v2.0/tokens -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "service", "passwordCredentials": {"username": "admin", "password": "empty"}}}' from (pid=18138) request /opt/stack/python-keystoneclient/keystoneclient/session.py:245
2014-04-11 17:42:41.476 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 10.34.1.187
2014-04-11 17:42:41.559 DEBUG urllib3.connectionpool [-] "POST /v2.0/tokens HTTP/1.1" 401 161 from (pid=18138) _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:330
2014-04-11 17:42:41.560 DEBUG keystoneclient.session [-] RESP: [401] CaseInsensitiveDict({'date': 'Fri, 11 Apr 2014 15:42:41 GMT', 'vary': 'X-Auth-Token', 'content-length': '161', 'content-type': 'application/json', 'www-authenticate': 'Keystone uri="http://10.34.1.187:5000/"'})
RESP BODY: {"error": {"message": "User abab3eb5c92b449d972c2b582c05add6 is unauthorized for tenant d9c50311b94c47b8bed593e422e2b1a3", "code": 401, "title": "Unauthorized"}}
 from (pid=18138) _send_request /opt/stack/python-keystoneclient/keystoneclient/session.py:285
2014-04-11 17:42:41.560 DEBUG keystoneclient.session [-] Request returned failure status: 401 from (pid=18138) request /opt/stack/python-keystoneclient/keystoneclient/session.py:261
2014-04-11 17:42:41.560 DEBUG keystoneclient.v2_0.client [-] Authorization Failed. from (pid=18138) get_raw_token_from_identity_service /opt/stack/python-keystoneclient/keystoneclient/v2_0/client.py:172
2014-04-11 17:42:41.561 ERROR swiftclient [-] Unauthorised. Check username, password and tenant name/id
2014-04-11 17:42:41.561 TRACE swiftclient Traceback (most recent call last):
2014-04-11 17:42:41.561 TRACE swiftclient File "/opt/stack/python-swiftclient/swiftclient/client.py", line 1192, in _retry
2014-04-11 17:42:41.561 TRACE swiftclient self.url, self.token = self.get_auth()
2014-04-11 17:42:41.561 TRACE swiftclient File "/opt/stack/python-swiftclient/swiftclient/client.py", line 1166, in get_auth
2014-04-11 17:42:41.561 TRACE swiftclient insecure=self.insecure)
2014-04-11 17:42:41.561 TRACE swiftclient File "/opt/stack/python-swiftclient/swiftclient/client.py", line 364, in get_auth
2014-04-11 17:42:41.561 TRACE swiftclient insecure=insecure)
2014-04-11 17:42:41.561 TRACE swiftclient File "/opt/stack/python-swiftclient/swiftclient/client.py", line 298, in get_keystoneclient_2_0
2014-04-11 17:42:41.561 TRACE swiftclient raise ClientException('Unauthorised. Check username, password'
2014-04-11 17:42:41.561 TRACE swiftclient ClientException: Unauthorised. Check username, password and tenant name/id
2014-04-11 17:42:41.561 TRACE swiftclient
2014-04-11 17:42:41.684 ERROR sahara.context [-] Thread 'Starting Job Execution 15d34698-582c-40a8-9798-5a6bb323e566' fails with exception: 'Unauthorised. Check username, password and tenant name/id'
2014-04-11 17:42:41.684 TRACE sahara.context Traceback (most recent call last):
2014-04-11 17:42:41.684 TRACE sahara.context File "/opt/stack/sahara/sahara/context.py", line 124, in _wrapper
2014-04-11 17:42:41.684 TRACE sahara.context func(*args, **kwargs)
2014-04-11 17:42:41.684 TRACE sahara.context File "/opt/stack/sahara/sahara/service/edp/job_manager.py", line 144, in run_job
2014-04-11 17:42:41.684 TRACE sahara.context upload_job_files(oozie_server, wf_dir, job, hdfs_user)
2014-04-11 17:42:41.684 TRACE sahara.context File "/opt/stack/sahara/sahara/service/edp/job_manager.py", line 186, in upload_job_files
2014-04-11 17:42:41.684 TRACE sahara.context raw_data = dispatch.get_raw_binary(main)
2014-04-11 17:42:41.684 TRACE sahara.context File "/opt/stack/sahara/sahara/service/edp/binary_retrievers/dispatch.py", line 30, in get_raw_binary
2014-04-11 17:42:41.684 TRACE sahara.context res = i_swift.get_raw_data(context.ctx(), job_binary)
2014-04-11 17:42:41.684 TRACE sahara.context File "/opt/stack/sahara/sahara/service/edp/binary_retrievers/internal_swift.py", line 100, in get_raw_data
2014-04-11 17:42:41.684 TRACE sahara.context raise ex.SwiftClientException(e.message)
2014-04-11 17:42:41.684 TRACE sahara.context SwiftClientException: Unauthorised. Check username, password and tenant name/id
2014-04-11 17:42:41.684 TRACE sahara.context

The tenant name could/should be specified as another parameter for job binaries and data sources.

Changed in sahara:
status: New → Confirmed
Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :

I think we should provide two ways to access binaries:
 * the old way, but also allow user to specify tenant_name in request
 * the new way, which is to use token of the current user

Changed in sahara:
milestone: none → juno-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (master)

Reviewed: https://review.openstack.org/93564
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=b90ea1c7a7578398c5787a5fc7e2aedadf40c8de
Submitter: Jenkins
Branch: master

commit b90ea1c7a7578398c5787a5fc7e2aedadf40c8de
Author: Dmitry Mescheryakov <email address hidden>
Date: Wed May 14 18:07:18 2014 +0400

    Fix hardcoded tenant name for job binaries

    After the fix the current user's tenant is taken instead of one
    from the Sahara config.

    I think we can fix it better and I outlined my thoughts in the bug
    comments. Current fix could be viewed as temporal until we agree on
    something else. BTW it could be ported to stable/icehouse since it
    does not changes Sahara API, unlike the fix I've proposed in the
    comments.

    Change-Id: Ifef76905cab313e451eb9e68f915738f33e11a25
    Partial-Bug: #1306713

Changed in sahara:
importance: Undecided → High
assignee: nobody → Dmitry Mescheryakov (dmitrymex)
Changed in sahara:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to sahara (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/97458

Changed in sahara:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (stable/icehouse)

Reviewed: https://review.openstack.org/97458
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=bd46ba25085124ac18ce2f6b77c00d85c78b82c1
Submitter: Jenkins
Branch: stable/icehouse

commit bd46ba25085124ac18ce2f6b77c00d85c78b82c1
Author: Dmitry Mescheryakov <email address hidden>
Date: Wed May 14 18:07:18 2014 +0400

    Fix hardcoded tenant name for job binaries

    After the fix the current user's tenant is taken instead of one
    from the Sahara config.

    I think we can fix it better and I outlined my thoughts in the bug
    comments. Current fix could be viewed as temporal until we agree on
    something else. BTW it could be ported to stable/icehouse since it
    does not changes Sahara API, unlike the fix I've proposed in the
    comments.

    Change-Id: Ifef76905cab313e451eb9e68f915738f33e11a25
    Partial-Bug: #1306713
    (cherry picked from commit b90ea1c7a7578398c5787a5fc7e2aedadf40c8de)

tags: added: in-stable-icehouse
Thierry Carrez (ttx)
Changed in sahara:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in sahara:
milestone: juno-1 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.