Steps to reproduce:
====================
Test setup:
-----------
* Manage is an administrator operation, so source administrator creds
* Run cinder with a backend that supports multiple storage pools, ex: NetApp ONTAP NFS
* Create a cinder volume, note its name and the host so we can "manage" this volume. See steps here: https://netapp-openstack-dev.github.io/openstack-docs/draft/cinder/examples/openstack_command_line/section_cinder-cli.html#cinder-manage-usage
* Unmanage the volume
Triggering the bug:
-------------------
* Manage the volume
* List the new volume, there is a good chance that this volume's "os-vol-host-attr:host" is set to the wrong pool, though belonging to the same backend
* Observe the host field in the cinder-api service, and the scheduler:
The API sets this host field as "service.host": https://github.com/openstack/cinder/blob/a9e0750/cinder/volume/api.py#L1684
* The "host" attribute of the service object doesn't have pool information
* Any pool in this 'host' may hence pass filtering logic in the scheduler: https://github.com/openstack/cinder/blob/ce3830a/cinder/scheduler/manager.py#L310, and the first pool that passes gets assigned as the "os-vol-host-attr:host" attribute, and this may well be wrong.
Sequence of events in cinder logs:
==================================
The Manage request/response from the API:
-----------------------------------------
2017-10-18 22:53:43,623 3365 INFO [tempest.lib.common.rest_client] Request (NetAppCDOTManageUnmanageTest:test_tc300_qos_applied_on_manage_with_no_preexisting_qos_iops): 202 POST http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/os-volume-manage 0.227s
2017-10-18 22:53:43,624 3365 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'X-Auth-Token': '<omitted>', 'Accept': 'application/json', 'Content-Type': 'application/json'}
Body: {"volume": {"ref": {"source-name": "10.250.119.54:/vol_d4bce8bb_002/volume-bf131ede-cd98-4255-adce-f330f5bfec6e"}, "volume_type": "tempest-type-401201460", "host": "devstack-xenial-cdot-nfs-provider-43-57506@netappcmodenfsd4bce8bb#10.250.119.54:/vol_d4bce8bb_002", "source_name": "10.250.119.54:/vol_d4bce8bb_002/volume-bf131ede-cd98-4255-adce-f330f5bfec6e"}}
Response - Headers: {u'date': 'Wed, 18 Oct 2017 22:53:43 GMT', u'server': 'Apache/2.4.18 (Ubuntu)', u'content-length': '821', 'status': '202', 'content-location': 'http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/os-volume-manage', u'x-compute-request-id': 'req-11caca68-112a-46a0-a927-89ed814bee1b', u'x-openstack-request-id': 'req-11caca68-112a-46a0-a927-89ed814bee1b', u'connection': 'close', u'content-type': 'application/json'}
Body: {"volume": {"status": "creating", "migration_status": null, "user_id": "a4cf00b817a84dc694ddcb277e868b08", "attachments": [], "links": [{"href": "http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263", "rel": "self"}, {"href": "http://192.168.100.126/volume/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263", "rel": "bookmark"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2017-10-18T22:53:43.470183", "description": null, "updated_at": null, "volume_type": "tempest-type-401201460", "name": null, "replication_status": null, "consistencygroup_id": null, "source_volid": null, "snapshot_id": null, "multiattach": false, "metadata": {}, "id": "02320df7-70fe-4f5c-a157-6074a25a4263", "size": 0}}
Observe the requested host: "devstack-xenial-cdot-nfs-provider-43-57506@netappcmodenfsd4bce8bb#10.250.119.54:/vol_d4bce8bb_002"
Requesting the volume information right after:
-----------------------------------------------
2017-10-18 22:53:43.720 3365 INFO tempest.lib.common.rest_client [req-419f774b-4e65-409e-ad20-f5f86fa4f16f ] Request (NetAppCDOTManageUnmanageTest:test_tc300_qos_applied_on_manage_with_no_preexisting_qos_iops): 200 GET http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263 0.095s
2017-10-18 22:53:43.720 3365 DEBUG tempest.lib.common.rest_client [req-419f774b-4e65-409e-ad20-f5f86fa4f16f ] Request - Headers: {'X-Auth-Token': '<omitted>', 'Accept': 'application/json', 'Content-Type': 'application/json'}
Body: None
Response - Headers: {u'date': 'Wed, 18 Oct 2017 22:53:43 GMT', u'server': 'Apache/2.4.18 (Ubuntu)', u'content-length': '1119', 'status': '200', 'content-location': 'http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263', u'x-compute-request-id': 'req-419f774b-4e65-409e-ad20-f5f86fa4f16f', u'x-openstack-request-id': 'req-419f774b-4e65-409e-ad20-f5f86fa4f16f', u'connection': 'close', u'content-type': 'application/json'}
Body: {"volume": {"migration_status": null, "attachments": [], "links": [{"href": "http://192.168.100.126/volume/v2/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263", "rel": "self"}, {"href": "http://192.168.100.126/volume/9d054670fb2a4d06ac4f17962afe2de1/volumes/02320df7-70fe-4f5c-a157-6074a25a4263", "rel": "bookmark"}], "availability_zone": "nova", "os-vol-host-attr:host": "devstack-xenial-cdot-nfs-provider-43-57506@netappcmodenfsd4bce8bb#10.250.119.54:/vol_d4bce8bb_000", "encrypted": false, "updated_at": "2017-10-18T22:53:44.000000", "replication_status": null, "snapshot_id": null, "id": "02320df7-70fe-4f5c-a157-6074a25a4263", "size": 0, "user_id": "a4cf00b817a84dc694ddcb277e868b08", "os-vol-tenant-attr:tenant_id": "9d054670fb2a4d06ac4f17962afe2de1", "os-vol-mig-status-attr:migstat": null, "metadata": {}, "status": "creating", "description": null, "multiattach": false, "source_volid": null, "consistencygroup_id": null, "os-vol-mig-status-attr:name_id": null, "name": null, "bootable": "false", "created_at": "2017-10-18T22:53:43.000000", "volume_type": "tempest-type-401201460"}} _log_request_full tempest/lib/common/rest_client.py:434
Observe the assigned/scheduled host: "os-vol-host-attr:host": "devstack-xenial-cdot-nfs-provider-43-57506@netappcmodenfsd4bce8bb#10.250.119.54:/vol_d4bce8bb_000"
Fix proposed to branch: master
Review: https:/