Ironic

Glance exceptions not handled correctly by ironic's glance image service

Bug #1441727 reported by Vladyslav Drok on 2015-04-08

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Ironic	Fix Released	Medium	Vladyslav Drok	Ironic 2015.1.0 "kilo"

Bug Description

If you specify incorrect image references for node, like for example 'ccc', this href will be considered a Glance image, and Glance image service will try to download it, this is the correct behavior. But this image does not exist in Glance, so ImageNotFound should be thrown. Instead, this happens:

$ ironic node-validate 45d4d35e-127f-4ea7-acc7-23447f3c864a
Remote error: HTTPNotFound HTTPNotFound (HTTP 404)
[u' (HTTP 500)

ir-cond log:

2015-04-08 12:17:19.475 DEBUG ironic.common.glance_service.base_image_service [-] Getting image metadata from glance. Image: ccc from (pid=21342) _show /opt/stack/ironic/ironic/common/glance_service/base_image_service.py:178
2015-04-08 12:17:19.476 DEBUG urllib3.util.retry [-] Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0) from (pid=21342) from_int /usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py:155
2015-04-08 12:17:19.540 ERROR oslo_messaging.rpc.dispatcher [-] Exception during message handling: HTTPNotFound (HTTP 404)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 142, in inner
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher return func(*args, **kwargs)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/conductor/manager.py", line 1241, in validate_driver_interfaces
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher iface.validate(task)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/drivers/modules/pxe.py", line 387, in validate
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher iscsi_deploy.validate_image_properties(task.context, d_info, props)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/drivers/modules/iscsi_deploy.py", line 491, in validate_image_properties
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher image_props = img_service.show(image_href)['properties']
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/common/glance_service/v1/image_service.py", line 28, in show
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher return self._show(image_id, method='get')
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 87, in wrapper
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher return func(self, *args, **kwargs)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 182, in _show
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher image = self.call(method, image_id)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 123, in call
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher return getattr(self.client.images, method)(*args, **kwargs)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 127, in get
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher % urlparse.quote(str(image_id)))
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 259, in head
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher return self._request('HEAD', url, **kwargs)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 230, in _request
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher raise exc.from_response(resp, resp.text)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher HTTPNotFound: HTTPNotFound (HTTP 404)
2015-04-08 12:17:19.540 TRACE oslo_messaging.rpc.dispatcher
2015-04-08 12:17:19.541 DEBUG oslo_messaging._drivers.amqp [-] Pool creating new connection from (pid=21342) create /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:92
2015-04-08 12:17:19.542 INFO oslo_messaging._drivers.impl_rabbit [-] Connecting to AMQP server on 192.168.122.222:5672
2015-04-08 12:17:19.554 INFO oslo_messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.122.222:5672
2015-04-08 12:17:19.554 ERROR oslo_messaging._drivers.common [-] Returning exception HTTPNotFound (HTTP 404) to caller
2015-04-08 12:17:19.554 ERROR oslo_messaging._drivers.common [-] ['Traceback (most recent call last):\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n executor_callback)\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 142, in inner\n return func(*args, **kwargs)\n', ' File "/opt/stack/ironic/ironic/conductor/manager.py", line 1241, in validate_driver_interfaces\n iface.validate(task)\n', ' File "/opt/stack/ironic/ironic/drivers/modules/pxe.py", line 387, in validate\n iscsi_deploy.validate_image_properties(task.context, d_info, props)\n', ' File "/opt/stack/ironic/ironic/drivers/modules/iscsi_deploy.py", line 491, in validate_image_properties\n image_props = img_service.show(image_href)[\'properties\']\n', ' File "/opt/stack/ironic/ironic/common/glance_service/v1/image_service.py", line 28, in show\n return self._show(image_id, method=\'get\')\n', ' File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 87, in wrapper\n return func(self, *args, **kwargs)\n', ' File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 182, in _show\n image = self.call(method, image_id)\n', ' File "/opt/stack/ironic/ironic/common/glance_service/base_image_service.py", line 123, in call\n return getattr(self.client.images, method)(*args, **kwargs)\n', ' File "/usr/local/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 127, in get\n % urlparse.quote(str(image_id)))\n', ' File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 259, in head\n return self._request(\'HEAD\', url, **kwargs)\n', ' File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 230, in _request\n raise exc.from_response(resp, resp.text)\n', 'HTTPNotFound: HTTPNotFound (HTTP 404)\n']

It happens because ironic.common.glance_service.base_image_service handles exceptions incorrectly - it should be catching glanceclient exceptions instead of ironic exceptions.

Vladyslav Drok (vdrok) on 2015-04-08

Changed in ironic:
assignee:	nobody → Vladyslav Drok (vdrok)

Yuriy Zveryanskyy (yzveryanskyy) on 2015-04-08

Changed in ironic:
status:	New → Triaged
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-04-08: Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/171740

Changed in ironic:
status:	Triaged → In Progress

John Stafford (john-stafford) on 2015-04-08

Changed in ironic:
milestone:	none → kilo-rc1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-04-08: Fix merged to ironic (master)

Reviewed: https://review.openstack.org/171740
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=4b53d34c95d78b331d1c47feabd8e6be6441b55e
Submitter: Jenkins
Branch: master

commit 4b53d34c95d78b331d1c47feabd8e6be6441b55e
Author: Vladyslav Drok <email address hidden>
Date: Wed Apr 8 19:52:33 2015 +0300

Fix exception handling in Glance image service

    At the moment there is literally no exception handling in
    glance_service.base_image_service, as it tries to catch Ironic
    exceptions instead of Glance client exceptions. This change fixes
    this behaviour.

Change-Id: Iaaeb234c68088538534f2a455f3678079cc60145
Closes-bug: #1441727

Changed in ironic:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2015-04-14

Changed in ironic:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2015-04-30

Changed in ironic:
milestone:	kilo-rc1 → 2015.1.0

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.