With a volume that doesn't support multi-attach, you can attach a volume to the same instance more than once.
I'm not sure how easy this is to fix, presumably we need to extend the checks in _attachment_reserve around here:
https://opendev.org/openstack/cinder/src/tag/14.0.0/cinder/volume/api.py#L2092
to fail if we are attaching to the same instance again.
But, doing that alone might interfere with the live migration case where we do a fake multi-attach to two separate hosts. So, maybe the check needed is that we only allow attaching to the same instance id if on two different compute hosts. (?)
Below is a reproduction script and some output of when this happens.
$ cat attach_script.sh
#!/usr/bin/bash -x
OS_TOKEN=$(openstack token issue -c id -f value)
OPENSTACKENDPOINT=10.16.151.37
INSTANCEUUID=c13aa08a-5184-4ce8-ba39-a9ac39067481
VOLUMEUUID=6fb23e37-7a06-4bf1-8a07-3e730ebd3153
#NOVA_URL=https://$OPENSTACKENDPOINT:13774/v2.1
NOVA_URL=https://$OPENSTACKENDPOINT/compute/v2.1/
INSTANCE_ID=$INSTANCEUUID
VOLUME_ID=$VOLUMEUUID
function call {
curl -k -s -H "X-Auth-Token: $OS_TOKEN" -H "X-Subject-Token: $OS_TOKEN" "$@" | jq .
}
echo "Using instance_id: $INSTANCE_ID"
echo "Using volume_id: $VOLUME_ID"
echo
echo "Attachments before test:"
call $NOVA_URL/servers/$INSTANCE_ID/os-volume_attachments
echo
echo "Attempting 10 concurrent attachments..."
for i in {1..10}
do
call -H 'Content-Type: application/json' $NOVA_URL/servers/$INSTANCE_ID/os-volume_attachments -d "{\"volumeAttachment\": {\"volumeId\": \"$VOLUME_ID\"}}" > /dev/null &
done
sleep 15
echo
echo "Attachements after test:"
call $NOVA_URL/servers/$INSTANCE_ID/os-volume_attachments
$ cinder show 6fb23e37-7a06-4bf1-8a07-3e730ebd3153
+--------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+----------------------------------------------------------------------------------+
| attached_servers | ['c13aa08a-5184-4ce8-ba39-a9ac39067481', 'c13aa08a-5184-4ce8-ba39-a9ac39067481'] |
| attachment_ids | ['ac58a579-f2c9-43a7-bda3-f677ba73e4d1', 'f78ff114-ee2b-4c68-953b-cd81fcde9edb'] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2019-06-21T14:14:57.000000 |
| description | None |
| encrypted | False |
| id | 6fb23e37-7a06-4bf1-8a07-3e730ebd3153 |
| metadata | |
| migration_status | None |
| multiattach | False |
| name | None |
| os-vol-host-attr:host | centos7vm1.localdomain@ceph#ceph |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 030d4846a4f9428eae45cdefd1ee2cdf |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2019-06-21T14:15:24.000000 |
| user_id | ed32ac3ec3d64687bd2912021a4d15a3 |
| volume_type | ceph2 |
+--------------------------------+----------------------------------------------------------------------------------+
$ sudo virsh domblklist 1
Target Source
------------------------------------------------
vda vms/c13aa08a-5184-4ce8-ba39-a9ac39067481_disk
vdb volumes/volume-83bc3803-0e7f-49c1-b258-8532a413ac9a
vdc volumes/volume-6fb23e37-7a06-4bf1-8a07-3e730ebd3153
vdd volumes/volume-6fb23e37-7a06-4bf1-8a07-3e730ebd3153
(vdc and vdd are the same volume)
$ sudo virsh dumpxml 1
...snipped
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
<auth username='cinder'>
<secret type='ceph' uuid='c198a0ce-baa3-45fd-bd48-e5c75e75359e'/>
</auth>
<source protocol='rbd' name='volumes/volume-6fb23e37-7a06-4bf1-8a07-3e730ebd3153'>
<host name='10.16.151.37' port='6789'/>
</source>
<target dev='vdc' bus='virtio'/>
<serial>6fb23e37-7a06-4bf1-8a07-3e730ebd3153</serial>
<alias name='virtio-disk2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</disk>
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
<auth username='cinder'>
<secret type='ceph' uuid='c198a0ce-baa3-45fd-bd48-e5c75e75359e'/>
</auth>
<source protocol='rbd' name='volumes/volume-6fb23e37-7a06-4bf1-8a07-3e730ebd3153'>
<host name='10.16.151.37' port='6789'/>
</source>
<target dev='vdd' bus='virtio'/>
<serial>6fb23e37-7a06-4bf1-8a07-3e730ebd3153</serial>
<alias name='virtio-disk3'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</disk>
Interesting note: if you try this with volume encryption, Nova will fail subsequent attaches with:
libvirtError: internal error: a secret with UUID
64311d25-fa9f-41c9-804a-103ddd258cda already defined for use with 83bc3803-0e7f-49c1-b258-8532a413ac9a
I tested in rocky(Ubuntu18.04) but OpenStack returns "ERROR (BadRequest): Invalid volume: volume 7609d0be-