Ironic

glanceclient raise HTTPUnauthorized exception

Bug #1266201 reported by Haomeng,Wang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
High
Chris Krelle
Ironic 2014.1 "icehouse"

Bug Description

With default configurations, our Ironic call glanceclient to get image information, but get HTTPUnauthorized exception as below log messsage:

apply patch - https://review.openstack.org/#/c/61160/

deploy - curl -i -X PUT -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'User-Agent: python-ironicclient' http://127.0.0.1:6385/v1/nodes/7f942ba9-af64-47ef-8c27-ae7870bc30e8/states/provision -d '{"target":"active"}' -H 'X-Auth-Token: MI...'

2014-01-05 03:14:29.458 5267 DEBUG ironic.conductor.manager [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] RPC do_node_deploy called for node 7f942ba9-af64-47ef-8c27-ae7870bc30e8. do_node_deploy /opt/stack/ironic/ironic/conductor/manager.py:236
2014-01-05 03:14:29.511 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Got semaphore "node_resource" lock /opt/stack/ironic/ironic/openstack/common/lockutils.py:170
2014-01-05 03:14:29.513 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Got semaphore / lock "acquire" inner /opt/stack/ironic/ironic/openstack/common/lockutils.py:250
2014-01-05 03:14:29.525 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Semaphore / lock released "acquire" inner /opt/stack/ironic/ironic/openstack/common/lockutils.py:254
2014-01-05 03:14:29.533 5267 DEBUG ironic.common.glance_service.base_image_service [-] Getting image metadata from glance. Image: glance://6eee94d7-c870-4f9e-9b59-0d8fec1a24ca _show /opt/stack/ironic/ironic/common/glance_service/base_image_service.py:184
2014-01-05 03:14:29.534 5267 DEBUG glanceclient.common.http [-] curl -i -X HEAD -H 'Content-Type: application/octet-stream' -H 'User-Agent: python-glanceclient' http://192.168.235.131:9292/v1/images/6eee94d7-c870-4f9e-9b59-0d8fec1a24ca log_curl_request /opt/stack/python-glanceclient/glanceclient/common/http.py:142
2014-01-05 03:14:29.539 5267 DEBUG glanceclient.common.http [-]
HTTP/1.1 401 Unauthorized
date: Sat, 04 Jan 2014 19:14:29 GMT
content-length: 0
content-type: text/html; charset=UTF-8
 log_http_response /opt/stack/python-glanceclient/glanceclient/common/http.py:152
2014-01-05 03:14:29.540 5267 ERROR glanceclient.common.http [-] Request returned failure status.
2014-01-05 03:14:29.552 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Got semaphore "node_resource" lock /opt/stack/ironic/ironic/openstack/common/lockutils.py:170
2014-01-05 03:14:29.553 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Got semaphore / lock "release" inner /opt/stack/ironic/ironic/openstack/common/lockutils.py:250
2014-01-05 03:14:29.554 5267 DEBUG ironic.openstack.common.lockutils [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Semaphore / lock released "release" inner /opt/stack/ironic/ironic/openstack/common/lockutils.py:254
2014-01-05 03:14:29.574 5267 ERROR ironic.openstack.common.rpc.amqp [req-5cb82e04-0bfd-43d0-8ceb-a9c9de387dcc admin admin] Exception during message handling
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp Traceback (most recent call last):
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/openstack/common/rpc/amqp.py", line 434, in _process_data
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp **args)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/conductor/manager.py", line 267, in do_node_deploy
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp node['target_provision_state'] = states.NOSTATE
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/openstack/common/excutils.py", line 70, in __exit__
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp six.reraise(self.type_, self.value, self.tb)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/conductor/manager.py", line 261, in do_node_deploy
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp task.driver.deploy.prepare(task, node)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/drivers/modules/pxe.py", line 538, in prepare
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp pxe_info = _get_tftp_image_info(node)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/drivers/modules/pxe.py", line 400, in _get_tftp_image_info
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp iproperties = glance_service.show(d_info['image_source'])['properties']
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/common/glance_service/v1/image_service.py", line 30, in show
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp return self._show(image_id, method='get')
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 86, in wrapper
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp return func(self, *args, **kwargs)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 188, in _show
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp image = self.call(method, image_id)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 121, in call
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp return getattr(self.client.images, method)(*args, **kwargs)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/python-glanceclient/glanceclient/v1/images.py", line 114, in get
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp % urllib.quote(str(image_id)))
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/python-glanceclient/glanceclient/common/http.py", line 289, in raw_request
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp return self._http_request(url, method, **kwargs)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp File "/opt/stack/python-glanceclient/glanceclient/common/http.py", line 249, in _http_request
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp raise exc.from_response(resp, body_str)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp HTTPUnauthorized: HTTPUnauthorized (HTTP 401)
2014-01-05 03:14:29.574 5267 TRACE ironic.openstack.common.rpc.amqp

However glance CLI works fine with same env:

root@ubuntu40g:~# glance image-list
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+
| 6eee94d7-c870-4f9e-9b59-0d8fec1a24ca | cirros-0.3.1-x86_64-uec | ami | ami | 25165824 | active |
| 8720928a-baa3-42fa-892d-451e2a82e7f9 | cirros-0.3.1-x86_64-uec-kernel | aki | aki | 4955792 | active |
| 1dfd98e2-c19a-4a0c-ab8d-473fba009b4d | cirros-0.3.1-x86_64-uec-ramdisk | ari | ari | 3714968 | active |
+--------------------------------------+---------------------------------+-------------+------------------+----------+--------+
root@ubuntu40g:~# glance image-show 6eee94d7-c870-4f9e-9b59-0d8fec1a24ca
+-----------------------+--------------------------------------+
| Property | Value |
+-----------------------+--------------------------------------+
| Property 'kernel_id' | 8720928a-baa3-42fa-892d-451e2a82e7f9 |
| Property 'ramdisk_id' | 1dfd98e2-c19a-4a0c-ab8d-473fba009b4d |
| checksum | f8a2eeee2dc65b3d9b6e63678955bd83 |
| container_format | ami |
| created_at | 2014-01-03T16:55:33 |
| deleted | False |
| disk_format | ami |
| id | 6eee94d7-c870-4f9e-9b59-0d8fec1a24ca |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.1-x86_64-uec |
| owner | b866b5e7bd7742b395bb34a62c530c67 |
| protected | False |
| size | 25165824 |
| status | active |
| updated_at | 2014-01-03T16:55:33 |
+-----------------------+--------------------------------------+

See original description
Haomeng,Wang (whaom)
description: updated
Revision history for this message
Haomeng,Wang (whaom) wrote :

I think our Ironic did not enable keystone client to get token first, then to pass the token to glance client to get permission.

Revision history for this message
aeva black (tenbrae) wrote :

It looks like ironic.common.glance_service is not passing (or perhaps not receiving) the context.auth_token properly...

Changed in ironic:
status: New → Triaged
importance: Undecided → High
milestone: none → icehouse-2
Ghe Rivero (ghe.rivero)
Changed in ironic:
assignee: nobody → Ghe Rivero (ghe.rivero)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/66008

Changed in ironic:
status: Triaged → In Progress
OpenStack Infra (hudson-openstack)
Changed in ironic:
assignee: Ghe Rivero (ghe.rivero) → Chris Krelle (nobodycam)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/66008
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=79b12ab21c1f2eca2dde5b60a87dc4f0dc1f4d9e
Submitter: Jenkins
Branch: master

commit 79b12ab21c1f2eca2dde5b60a87dc4f0dc1f4d9e
Author: Ghe Rivero <email address hidden>
Date: Fri Jan 10 16:32:36 2014 +0000

    Use correct auth context inside pxe driver

    Until now, general auth context was used inside the pxe driver
    making it fail when authenticating against glance. We should be using
    instead task.context

    Change-Id: I173b912c67ab77dda433d2f1f93e674852b7d054
    Closes-Bug: #1266201

Changed in ironic:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ironic:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: icehouse-2 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.