cinder client /driver should check service status before taking backup

[satyadev svn]

stop cinder backup service. [c-bak service in devstack]
now try to create cinder backup.
cinder driver /client should throw error saying backup service not present.
current cinder driver not throwing any exception to cinder client hence status enter to 'creating' backup list and 'backing-up' cinder volume and convey same to cinder client so that they can show some meaning full error.

same the case for restore also.

see below commands :

ssatya@pr-devstack:~$ cinder list
+--------------------------------------+-----------+------------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------------+------+-------------+----------+-------------+
| 07edcb91-de05-4f63-b40c-95345866b5d8 | available | small_vol | 1 | None | false | |
| c8fe4e3c-144d-4e88-8a4a-b6e5bd685a0f | available | sparse_vol | 1 | None | true | |
| eab6daa2-28dc-467c-a110-1970f8373faf | available | sparse_vol | 1 | None | true | |
| eccc4d31-478e-4dca-9161-310471ea475c | available | bigvol | 3 | None | false | |
| f4c4076e-c404-492b-82d7-32815878d7ff | available | bigvol | 2 | None | false | |
+--------------------------------------+-----------+------------+------+-------------+----------+-------------+
ssatya@pr-devstack:~$ cinder backup-create 07edcb91-de05-4f63-b40c-95345866b5d8 --name test1
+-----------+--------------------------------------+
| Property | Value |
+-----------+--------------------------------------+
| id | 7b6f0b8d-29fd-4024-9e16-39bb567000a9 |
| name | test1 |
| volume_id | 07edcb91-de05-4f63-b40c-95345866b5d8 |
+-----------+--------------------------------------+
ssatya@pr-devstack:~$ cinder backup-list
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+
| ID | Volume ID | Status | Name | Size | Object Count | Container |
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+
| 16c48e8f-4974-490c-af65-5cd1079484d3 | eab6daa2-28dc-467c-a110-1970f8373faf | available | sparse_vol_snap_backup | 1 | 2 | volumebackups |
| 4200b860-739d-4a8c-8541-c0eda1df5cf2 | 44a9106f-e432-449e-a5fb-f76888f0fad8 | available | image_vol_backup | 1 | 2 | volumebackups |
| 7b6f0b8d-29fd-4024-9e16-39bb567000a9 | 07edcb91-de05-4f63-b40c-95345866b5d8 | creating | test1 | 1 | None | None |
| e8644438-82c5-4d0e-a422-1f6ad1cbdca2 | f4c4076e-c404-492b-82d7-32815878d7ff | available | bigbackup | 2 | 2 | volumebackups |
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+
ssatya@pr-devstack:~$ cinder list
+--------------------------------------+------------+------------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+------------+------------+------+-------------+----------+-------------+
| 07edcb91-de05-4f63-b40c-95345866b5d8 | backing-up | small_vol | 1 | None | false | |
| c8fe4e3c-144d-4e88-8a4a-b6e5bd685a0f | available | sparse_vol | 1 | None | true | |
| eab6daa2-28dc-467c-a110-1970f8373faf | available | sparse_vol | 1 | None | true | |
| eccc4d31-478e-4dca-9161-310471ea475c | available | bigvol | 3 | None | false | |
| f4c4076e-c404-492b-82d7-32815878d7ff | available | bigvol | 2 | None | false | |
+--------------------------------------+------------+------------+------+-------------+----------+-------------+
ssatya@pr-devstack:~$ cinder backup-list
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+
| ID | Volume ID | Status | Name | Size | Object Count | Container |
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+
| 16c48e8f-4974-490c-af65-5cd1079484d3 | eab6daa2-28dc-467c-a110-1970f8373faf | available | sparse_vol_snap_backup | 1 | 2 | volumebackups |
| 4200b860-739d-4a8c-8541-c0eda1df5cf2 | 44a9106f-e432-449e-a5fb-f76888f0fad8 | available | image_vol_backup | 1 | 2 | volumebackups |
| 7b6f0b8d-29fd-4024-9e16-39bb567000a9 | 07edcb91-de05-4f63-b40c-95345866b5d8 | creating | test1 | 1 | None | None |
| e8644438-82c5-4d0e-a422-1f6ad1cbdca2 | f4c4076e-c404-492b-82d7-32815878d7ff | available | bigbackup | 2 | 2 | volumebackups |
+--------------------------------------+--------------------------------------+-----------+------------------------+------+--------------+---------------+

[satyadev svn (svnsatya)]

attached logs

[John Griffith (john-griffith)]

Pretty awful user experience, totally agree and it wouldn't be that difficult to add this check.

[Jin Liu (jin-t)]

I may have a different test result on Icehouse. Following the reproduce steps, I stopped cinder-backup service by 'sudo service cinder-backup stop', then create backup, I got the error message 'ERROR: Service cinder-backup could not be found.'

cloudscaling@z0:~$ cinder list
+--------------------------------------+-----------+------------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------------------+------+-------------+----------+-------------+
| 1ba6d045-eea0-4e8b-afdb-5ebd3958402b | available | image_test_13013 | 2 | None | true | |
| 3361dadd-96cd-4f92-b50b-508ead4e24ca | available | create_test_5141 | 1 | None | false | |
+--------------------------------------+-----------+------------------+------+-------------+----------+-------------+
cloudscaling@z0:~$
cloudscaling@z0:~$ cinder backup-list
+----+-----------+--------+------+------+--------------+-----------+
| ID | Volume ID | Status | Name | Size | Object Count | Container |
+----+-----------+--------+------+------+--------------+-----------+
+----+-----------+--------+------+------+--------------+-----------+
cloudscaling@z0:~$
cloudscaling@z0:~$ cinder backup-create 1ba6d045-eea0-4e8b-afdb-5ebd3958402b
ERROR: Service cinder-backup could not be found.

[Jin Liu (jin-t)]

Update a little bit, the test result I posted earlier was using python-cinderclient 1.0.6 version, after I upgraded to 1.1.1 version, I got this result:
cloudscaling@z0:~$ cinder backup-create 1ba6d045-eea0-4e8b-afdb-5ebd3958402b
ERROR: Internal Server Error (HTTP 500) (Request-ID: req-8aa31507-0e84-4bed-99fb-352d6f1ea6ac)

The error message seems not so clear as 1.0.6 version python-cinderclient.

[Jin Liu (jin-t)]

I found the difference between python-cinderclient versions on this case is in 1.0.6 runs HTTPClient.request, while in 1.1.1 runs SessionClient.request. Cinder drive returns same response with message like this '{"computeFault": {"message": "Service cinder-backup could not be found.", "code": 500}}'. Following different exception handling make different result.

https://review.openstack.org/126713

[Duncan Thomas (duncan-thomas)]

'Service stop' only actually works on cinder-volume services at the moment. The API server should give you a sensible error if you try to stop any other service. That should probably be a separate bug though.

[Jin Liu (jin-t)]

https://review.openstack.org/#/c/126733/

[Sean McGinnis (sean-mcginnis)]

Closing stale bug. If this is still an issue please reopen.