image-create-via-import needs to fail faster

[Brian Rosmaita]

The image-create-via-import command needs to make sure that it has all and only the correct inputs for the requested method before it performs any actions. Otherwise you get a situation where both a --file and --uri can be specified with no import-method (or with glance-direct, the default), an image record is created and staged but the do_image_import function later fails (because it has a uri and the method is not web-download -- do_image_import must do the check in case it's being invoked directly).

Sure, this can be fixed by making another import call directly, but that presupposes that the user knows how to do it correctly, which is doubtful since they already messed up the create-via-import call. It's better if create-via-import does nothing if the inputs it gets are incorrect. Otherwise, lots of calls to support and sad operators.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/555550

[Jean-Philippe Evrard (jean-philippe-evrard)]

I must say I love the bug description.

[Erno Kuvaja (jokke)]

I'm not convinced that this is the case. Could you please give example command how you can reproduce this bug? I think the lines 126-139 of glanceclient/v2/shell.py are already covering the cases you mentioned, no?

[Brian Rosmaita (brian-rosmaita)]

Yeah, my bug description is incorrect. Here's what really happens:

1. Make this call using web-download and both --file and --uri:
glance image-create-via-import --disk-format raw --container-format bare --name Checking --file fake-image-data.txt --import-method web-download --uri https://www.openstack.org/assets/openstack-logo/2016R/OpenStack-Logo-Horizontal.eps.zip

2. response shows image in status 'uploading'

3. do image-show on the image; response now shows image in status 'importing'

Look in the glance-api log to find out that the task has been picked up and gone into failure mode because there's already stuff in the stage.

[Brian Rosmaita (brian-rosmaita)]

Log extract showing the above:

INFO eventlet.wsgi.server [None req-5efcaf5b-066e-4a56-a66e-b94f5eae1fd3 demo demo] 172.16.132.149,172.16.132.149 - - [28/Mar/2018 11:36:46] "POST /v2/images/d9293fd1-b558-4cc1-9d3a-68a853f030bd/import HTTP/1.1" 202 211 0.616291
DEBUG glance.api.middleware.version_negotiation [None req-5efcaf5b-066e-4a56-a66e-b94f5eae1fd3 demo demo] Determining version of request: GET /v2/images/d9293fd1-b558-4cc1-9d3a-68a853f030bd Accept: */* {{(pid=97459) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:45}}
DEBUG glance.api.middleware.version_negotiation [None req-5efcaf5b-066e-4a56-a66e-b94f5eae1fd3 demo demo] Using url versioning {{(pid=97459) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:57}}
DEBUG glance.api.middleware.version_negotiation [None req-5efcaf5b-066e-4a56-a66e-b94f5eae1fd3 demo demo] Matched version: v2 {{(pid=97459) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:69}}
DEBUG glance.api.middleware.version_negotiation [None req-5efcaf5b-066e-4a56-a66e-b94f5eae1fd3 demo demo] new path /v2/images/d9293fd1-b558-4cc1-9d3a-68a853f030bd {{(pid=97459) process_request /opt/stack/glance/glance/api/middleware/version_negotiation.py:70}}
INFO glance.domain [-] Task [a2ccc934-b99f-48f6-b99d-b68e294b63d1] status changing from pending to processing
DEBUG glance.async.taskflow_executor [-] Taskflow executor picked up the execution of task ID a2ccc934-b99f-48f6-b99d-b68e294b63d1 of task type api_image_import {{(pid=97459) _run /opt/stack/glance/glance/async/taskflow_executor.py:156}}
DEBUG glance.async.taskflow_executor [-] Flow 'api_image_import' (9ff1df84-4716-4fa3-8160-a8fe938a1482) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=97459) _flow_receiver /usr/local/lib/python2.7/dist-packages/taskflow/listeners/logging.py:145}}
DEBUG glance.async.taskflow_executor [-] Task 'api_image_import-WebDownload-a2ccc934-b99f-48f6-b99d-b68e294b63d1' (16b66c30-7478-4cb3-906c-2250a82d5751) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=97459) _task_receiver /usr/local/lib/python2.7/dist-packages/taskflow/listeners/logging.py:194}}
WARNING glance.async.taskflow_executor [-] Task 'api_image_import-WebDownload-a2ccc934-b99f-48f6-b99d-b68e294b63d1' (16b66c30-7478-4cb3-906c-2250a82d5751) transitioned into state 'FAILURE' from state 'RUNNING'
3 predecessors (most recent first):
  Flow 'api_image_import'
  |__Atom 'api_image_import_retry' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': [(None, {})]}
     |__Flow 'api_image_import': Duplicate: Image /opt/stack/data/glance/staging/d9293fd1-b558-4cc1-9d3a-68a853f030bd already exists
ERROR glance.async.taskflow_executor Traceback (most recent call last):
ERROR glance.async.taskflow_executor File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
ERROR glance.async.taskflow_executor result = task.execute(**arguments)
ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/async/flows/_internal_plugins/web_download.py", line 99, in execute
ERROR glance.async.taskflow_executor ...

[Brian Rosmaita (brian-rosmaita)]

See https://review.openstack.org/557382 for a test that shows what happens on the client side.

[Brian Rosmaita (brian-rosmaita)]

Just a quick summary ... the problem is that you can include extra inputs and the import will fail on the server side. I think it's better to make sure everything (or as much as we can check for) is processed on the client side.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/555550
Committed: https://git.openstack.org/cgit/openstack/python-glanceclient/commit/?id=79543a67edd7a15aa23426548d3be9ac5b99366d
Submitter: Zuul
Branch: master

commit 79543a67edd7a15aa23426548d3be9ac5b99366d
Author: Brian Rosmaita <email address hidden>
Date: Thu Mar 22 15:49:44 2018 -0400

    Make image-create-via-import fail faster

    Add checks to the image-create-via-import commmand so that it provides
    better user feedback and doesn't begin the import workflow unless the
    input has a chance of succeeding. Preserves backward compatibility
    with the current image-create command by (1) allowing an image record
    only to be created when no import-method is specified AND no data is
    supplied, and (2) doing the glance-direct workflow when no import-
    method is specified AND data is provided. Also adds the ability for
    the import-method to be set as an env var OS_IMAGE_IMPORT_METHOD.

    Change-Id: I0a225f5471a9311217b5d90ebb5fd415c369129a
    Closes-bug: #1758149

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/559062

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/559062
Committed: https://git.openstack.org/cgit/openstack/python-glanceclient/commit/?id=4211b67e7aa560b97e19945d3bbb07a697628382
Submitter: Zuul
Branch: stable/queens

commit 4211b67e7aa560b97e19945d3bbb07a697628382
Author: Brian Rosmaita <email address hidden>
Date: Thu Mar 22 15:49:44 2018 -0400

    Make image-create-via-import fail faster

    Add checks to the image-create-via-import commmand so that it provides
    better user feedback and doesn't begin the import workflow unless the
    input has a chance of succeeding. Preserves backward compatibility
    with the current image-create command by (1) allowing an image record
    only to be created when no import-method is specified AND no data is
    supplied, and (2) doing the glance-direct workflow when no import-
    method is specified AND data is provided. Also adds the ability for
    the import-method to be set as an env var OS_IMAGE_IMPORT_METHOD.

    Change-Id: I0a225f5471a9311217b5d90ebb5fd415c369129a
    Closes-bug: #1758149
    (cherry picked from commit 79543a67edd7a15aa23426548d3be9ac5b99366d)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-glanceclient 2.10.0 release.