Image Create using the Task API, leaves Image in saving state when Task fails

[Sowmya Rao]

Image Create using the Task API, leaves Image in saving state when Task API fails.

Steps To reproduce :
 - Provide Invalid URL / Provide unsupported URL [Apart from HTTP] to the task API.
 - Task API fails with error message : "Task failed due to Internal Error"
 - Neither Image ID is returned nor the Image is cleaned up from Glance when the Task API fails.

Output from the glance image-list :
+--------------------------------------+----------------------------------------------+-------------+------------------+-------------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+----------------------------------------------+-------------+------------------+-------------+--------+
| 79b600bc-9cff-4754-9bbd-5d0de5f76852 | With_HTTPS | qcow2 | bare | | saving |
| 8f16ce54-f20f-434a-af7d-bb11be4c5b3d | Test_JUNK_URL | qcow2 | bare | | saving |

Expected :
 - Either Image should be cleaned up when the Task API fails (OR) Image ID must be returned back to the user with right error message /status.
 - Appropriate Error Message must be returned stating "HTTPS is NOT Supported / Image URL is not reachable" instead of Generic error message : "Task failed due to Internal Error"

Currently we do see some error in the Glance API logs ONLY when Unsupported URL is provided, but no error logs when the URL is not reachable.

018-07-17 12:00:58.002 36721 INFO glance.domain [-] Task [83f30963-5334-41b0-996b-10d429562178] status changing from processing to processing
2018-07-17 12:00:58.031 36721 ERROR glance.async.flows.base_import [-] Bad task configuration: Task was not configured properly
2018-07-17 12:00:58.155 36721 INFO glance.common.scripts.image_import.main [-] Task 83f30963-5334-41b0-996b-10d429562178: Got image data uri https://10.43.33.103/nfvo_local_repo/ to be imported
2018-07-17 12:00:58.167 36721 WARNING glance.common.scripts.image_import.main [-] Task 83f30963-5334-41b0-996b-10d429562178 failed with exception <urlopen error [Errno 111] ECONNREFUSED>
2018-07-17 12:00:58.167 36721 INFO glance.common.scripts.image_import.main [-] Task 83f30963-5334-41b0-996b-10d429562178: Could not import image file https://10.43.33.103/nfvo_local_repo/
2018-07-17 12:00:58.217 36721 WARNING glance.async.taskflow_executor [-] Task 'import-ImportToStore-83f30963-5334-41b0-996b-10d429562178' (2dd9f3f9-c22c-41b5-a826-602af0c79baa) transitioned into state 'FAILURE' from state 'RUNNING'
3 predecessors (most recent first):
  Atom 'import-CreateImage-83f30963-5334-41b0-996b-10d429562178' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': '79b600bc-9cff-4754-9bbd-5d0de5f76852'}
  |__Atom 'import_retry' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': [(None, {})]}
     |__Flow 'import'

[Abhishek Kekane (abhishek-kekane)]

Could you please add exact steps, commands to reproduce this issue.
Thank you.

[Sowmya Rao (sowmyaraos)]

Steps to reproduce by providing invalid URL [valid URL with HTTPS option]:
1) Task API from PostMan :

{
    "type": "import",
    "input": {
        "import_from": "https://xxxxxxxxxxxxxxx/log.qcow2",
        "import_from_format": "qcow2",
        "image_properties": {
            "disk_format": "qcow",
            "container_format": "bare",
            "name": "test-create2xxx"
        }

    }
}

Response from Task API :

{
    "status": "pending",
    "created_at": "2018-11-05T10:40:10Z",
    "updated_at": "2018-11-05T10:40:10Z",
    "self": "/v2/tasks/2d35894f-24a5-4ab6-8aba-3950008629e6",
    "result": null,
    "owner": "47441f261db046f9967ef099877369e4",
    "input": {
        "image_properties": {
            "container_format": "bare",
            "disk_format": "qcow",
            "name": "test-create2xxx"
        },
        "import_from_format": "qcow2",
        "import_from": "https://xxxxxxxxxxxxxxx/log.qcow2"
    },
    "message": "",
    "type": "import",
    "id": "2d35894f-24a5-4ab6-8aba-3950008629e6",
    "schema": "/v2/schemas/task"
}

2) Check the Task Progress :
{
    "status": "failure",
    "created_at": "2018-11-05T10:40:10Z",
    "updated_at": "2018-11-05T10:42:18Z",
    "expires_at": "2018-11-07T10:42:18Z",
    "self": "/v2/tasks/2d35894f-24a5-4ab6-8aba-3950008629e6",
    "result": null,
    "owner": "47441f261db046f9967ef099877369e4",
    "input": {
        "image_properties": {
            "container_format": "bare",
            "disk_format": "qcow",
            "name": "test-create2xxx"
        },
        "import_from_format": "qcow2",
        "import_from": "https://xxxxxxxxxxxxxxx/log.qcow2"
    },
    "message": "Task failed due to Internal Error",
    "type": "import",
    "id": "2d35894f-24a5-4ab6-8aba-3950008629e6",
    "schema": "/v2/schemas/task"
}

3) Log in to the VIM and view the Image :

[xxxx@undercloud (overcloudrc) ~]$ glance image-show 02503fd9-d29d-4d78-b31d-b6602a229ad1
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2018-11-05T10:40:10Z |
| disk_format | qcow |
| id | 02503fd9-d29d-4d78-b31d-b6602a229ad1 |
| locations | [] |
| min_disk | 0 |
| min_ram | 0 |
| name | test-create2xxx |
| owner | 47441f261db046f9967ef099877369e4 |
| protected | False |
| size | None |
| status | saving |
| tags | [] |
| updated_at | 2018-11-05T10:40:10Z |
| virtual_size | None |
| visibility | private |
+------------------+----------------...

[Sowmya Rao (sowmyaraos)]

Error in Glance API Logs :

2018-11-05 10:42:16.145 5846 INFO eventlet.wsgi.server [req-35badfa3-ef75-426d-baa2-485e3be1efa4 db08886119204f4585405585a156d5c2 47441f261db046f9967ef099877369e4 - default default] 172.17.3.15 - - [05/Nov/2018 10:42:16] "GET /v2/images HTTP/1.1" 200 9646 0.069240
2018-11-05 10:42:17.901 5849 WARNING glance.common.scripts.image_import.main [-] Task 2d35894f-24a5-4ab6-8aba-3950008629e6 failed with exception <urlopen error [Errno 110] ETIMEDOUT>
2018-11-05 10:42:17.904 5849 INFO glance.common.scripts.image_import.main [-] Task 2d35894f-24a5-4ab6-8aba-3950008629e6: Could not import image file https://xxxxxxxxxxxxxxx/log.qcow2
2018-11-05 10:42:18.092 5849 WARNING glance.async.taskflow_executor [-] Task 'import-ImportToStore-2d35894f-24a5-4ab6-8aba-3950008629e6' (371dca54-c94e-46d4-9437-df516c706ec8) transitioned into state 'FAILURE' from state 'RUNNING'
3 predecessors (most recent first):
  Atom 'import-CreateImage-2d35894f-24a5-4ab6-8aba-3950008629e6' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': '02503fd9-d29d-4d78-b31d-b6602a229ad1'}
  |__Atom 'import_retry' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {}, 'provides': [(None, {})]}
     |__Flow 'import'
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor Traceback (most recent call last):
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor result = task.execute(**arguments)
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/glance/async/flows/base_import.py", line 311, in execute
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor image_import.set_image_data(image, file_path or self.uri, self.task_id)
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/glance/common/scripts/image_import/main.py", line 158, in set_image_data
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor "task_id": task_id})
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor self.force_reraise()
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/glance/common/scripts/image_import/main.py", line 149, in set_image_data
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor data_iter = script_utils.get_image_data_iter(uri)
2018-11-05 10:42:18.092 5849 ERROR glance.async.taskflow_executor File "/usr/lib/python2.7/site-packages/glance/common/scripts/utils.py", line 133, in get_image_data_iter
2018-11-0...