Image import fails with python 3.5, image stuck in importing state

Bug #1759510 reported by Abhishek Kekane on 2018-03-28
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Critical
Abhishek Kekane
Queens
Critical
Abhishek Kekane

Bug Description

New image import api with glance-direct or web-download fails to import image if cloud is using python3.5. Image stuck in importing state forever.

Steps:
1. Ensure you are running glance on python 3.5
   Add below lines in your devstack/local.conf
   WSGI_MODE=mod_wsgi
   USE_PYTHON3=True
   PYTHON3_VERSION=3.5

2. Source devstack/openrc using "$ source devstack/openrc admin admin"

3. Create image using new import api
   $ glance image-create-via-import --container-format ami --disk-format ami --name cirros_image --file <file_path>

g-api logs:

Mar 28 08:36:01 ubuntu-3 glance-api[23449]: |__Flow 'api_image_import': TypeError: Unicode-objects must be encoded before hashing
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor Traceback (most recent call last):
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor result = task.execute(**arguments)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/async/flows/api_image_import.py", line 218, in execute
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor image_import.set_image_data(image, file_path or self.uri, self.task_id)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 154, in set_image_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor "task_id": task_id})
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 146, in set_image_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor image.set_data(data_iter)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/domain/proxy.py", line 195, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.base.set_data(data, size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/notifier.py", line 480, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor _send_notification(notify_error, 'image.upload', msg)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/notifier.py", line 427, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.repo.set_data(data, size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/api/policy.py", line 193, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor return self.image.set_data(*args, **kwargs)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/quota/__init__.py", line 304, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.image.set_data(data, size=size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/location.py", line 439, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier=verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/backend.py", line 453, in add_to_backend
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/backend.py", line 426, in store_add_to_backend
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier=verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/capabilities.py", line 225, in op_checker
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor return store_op_fun(store, *args, **kwargs)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/_drivers/filesystem.py", line 697, in add
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self._delete_partial(filepath, image_id)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/_drivers/filesystem.py", line 684, in add
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor checksum.update(buf)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor TypeError: Unicode-objects must be encoded before hashing
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: DEBUG glance.async.taskflow_executor [-] Task 'api_image_import-ImportToStore-a4ef0f11-2e4f-4ed8-ac4a-cbb28293a513' (74506991-d9ed-4fdf-819a-21dcadca3c04) transitioned into state 'REVERTING' from state 'FAILURE' {{(pid=23619) _task_receiver /usr/local/lib/python3.5/dist-packages/taskflow/listeners/logging.py:194}}

Changed in glance:
assignee: nobody → Abhishek Kekane (abhishek-kekane)
summary: - Image import fails with python 3.5, image stuck in uploading state
+ Image import fails with python 3.5, image stuck in importing state
description: updated

Fix proposed to branch: master
Review: https://review.openstack.org/557673

Changed in glance:
status: New → In Progress
tags: added: queens-backport-potential
Changed in glance:
importance: Undecided → Critical

Reviewed: https://review.openstack.org/557673
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=ab4139e30ca56425fb4f62ebf2d5cb48a0455952
Submitter: Zuul
Branch: master

commit ab4139e30ca56425fb4f62ebf2d5cb48a0455952
Author: Abhishek Kekane <email address hidden>
Date: Thu Mar 29 09:54:19 2018 +0000

    Python 3.5: Image Import fails with Unicode Error

    Newly added image import api fails for python 3.5 as it
    raises Unicode error as it fails to open file in binary mode.

    Added 'rb' mode while opening the file which will open the provided
    file in binary mode.

    Change-Id: I297a415d6fa8700c6a131e0931d11bda65265c73
    Closes-Bug: #1759510

Changed in glance:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/558359
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=0da269128ed14096315c9b9e54679fd7e85e9b9b
Submitter: Zuul
Branch: stable/queens

commit 0da269128ed14096315c9b9e54679fd7e85e9b9b
Author: Abhishek Kekane <email address hidden>
Date: Thu Mar 29 09:54:19 2018 +0000

    Python 3.5: Image Import fails with Unicode Error

    Newly added image import api fails for python 3.5 as it
    raises Unicode error as it fails to open file in binary mode.

    Added 'rb' mode while opening the file which will open the provided
    file in binary mode.

    Change-Id: I297a415d6fa8700c6a131e0931d11bda65265c73
    Closes-Bug: #1759510
    (cherry picked from commit ab4139e30ca56425fb4f62ebf2d5cb48a0455952)

Changed in glance:
milestone: none → rocky-1

This issue was fixed in the openstack/glance 17.0.0.0b1 development milestone.

This issue was fixed in the openstack/glance 16.0.1 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers