Running image-import call on queued image having valid container and disk formats returns 500 internal server error

Bug #1733813 reported by Abhishek Kekane on 2017-11-22
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
High
Abhishek Kekane

Bug Description

If you run image-import api on any image which is in queued state having valid container-format and disk-format set will return 500 error as it raises IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'. Also image status changes from 'queued' to 'importing'. Ideally transition from queued to importing should not be allowed and it should return HTTP 409 conflict error to the user.

Prerequisites:
1. Ensure you have latest version of python-glanceclient (version 2.8.0) installed
2. Due to isssue [1] to execute taskflow you need to modify line [2] as shown below and restart glance-api service
   - pool.spawn_n(import_task.run, task_executor)
   + import_task.run(task_executor)
   [1] https://bugs.launchpad.net/glance/+bug/1712463
   [2] https://github.com/openstack/glance/blob/master/glance/api/v2/images.py#L106

Steps to reporoduce:
1. Create an image with valid container format and disk-format
    $ glance image-create --container-format ami --disk-format ami --name cirros_image
2. Ensure image is in queued state
3. Run image-import call
   $ glance image-import <ID of image created in 1st step> --import-method glance-direct

Output:
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

Glance API Logs:

Nov 22 09:12:57 devstack <email address hidden>[14229]: pdict['tenant'] = self.tenant
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi [None req-52ea6328-83bf-4c25-a137-a51272308be9 admin admin] Caught error: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be': IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi Traceback (most recent call last):
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1222, in __call__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi request, **action_args)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1261, in dispatch
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return method(*args, **kwargs)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/utils.py", line 363, in wrapped
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/api/v2/images.py", line 107, in import_image
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi import_task.run(task_executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/notifier.py", line 581, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi super(TaskProxy, self).run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/__init__.py", line 439, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi executor.begin_processing(self.task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 143, in begin_processing
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi super(TaskExecutor, self).begin_processing(task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/__init__.py", line 63, in begin_processing
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self._run(task_id, task.type)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 181, in _run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.task_repo.save(task)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.force_reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 172, in _run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi engine.run()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi for _state in self.run_iter(timeout=timeout):
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi failure.Failure.reraise_if_any(er_failures)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi failures[0].reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(*self._exc_info)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi result = task.execute(**arguments)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/flows/api_image_import.py", line 239, in execute
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi image_import.set_image_data(image, file_path or self.uri, self.task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 154, in set_image_data
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi "task_id": task_id})
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.force_reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 145, in set_image_data
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi data_iter = script_utils.get_image_data_iter(uri)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/utils.py", line 139, in get_image_data_iter
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return open(uri, "r")
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi

Changed in glance:
assignee: nobody → Abhishek Kekane (abhishek-kekane)
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/522703

Changed in glance:
status: New → In Progress
Changed in glance:
importance: Undecided → High
milestone: none → queens-2

Reviewed: https://review.openstack.org/522703
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=a19fdb0e5f47eb5ea6f5150e056f1df26fb8b7e9
Submitter: Zuul
Branch: master

commit a19fdb0e5f47eb5ea6f5150e056f1df26fb8b7e9
Author: Abhishek Kekane <email address hidden>
Date: Fri Nov 24 04:08:13 2017 +0000

    Fix 500 from image-import on queued images

    If you run image-import api on any image which is in queued state having
    valid container-format and disk-format set will return 500 error as it
    raises IOError: [Errno 2] No such file or directory:
    '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'. The reason is image
    does not have data stored in staging area.

    Restricted transition from queued to importing state and returned HTTP 409
    conflict error to the user.

    Change-Id: I3d3ab283fd0faf4b63365ff2dbeb714913f02b27
    Closes-Bug: #1733813

Changed in glance:
status: In Progress → Fix Released

This issue was fixed in the openstack/glance 16.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers