Original volume size is lost when uploading a volume to a qcow2 image

[Takashi Kajinami]

This issue was initially reported in https://bugzilla.redhat.com/show_bug.cgi?id=1984304 .

Currently when uploading a cinder volume to a glance image in qcow2 format, size of the newly created glance image is computed as size of the actual qcow2 file instead of original volume size.

For example if we create an empty 1GiB volume
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack volume create sourcevolume --size 2
...
(overcloud) [stack@undercloud-0 ~]$ openstack volume show sourcevolume
+--------------------------------+---------------------------------------+
| Field | Value |
+--------------------------------+---------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2021-07-21T07:47:18.000000 |
| description | None |
| encrypted | False |
| id | cb2113ce-2768-412c-81c2-190bdd7e7b2c |
| migration_status | None |
| multiattach | False |
| name | sourcevolume |
| os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 942783ae248c4e9eb353a6e6b327bda5 |
| properties | |
| replication_status | None |
| size | 2 |
| snapshot_id | None |
| source_volid | None |
| status | available |
| type | tripleo |
| updated_at | 2021-07-21T07:47:19.000000 |
| user_id | 06b547a0af8f49fd8239c85ce5d9571b |
+--------------------------------+---------------------------------------+
~~~

... and then create a qcow2 image from the volume
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack image create --volume sourcevolume --disk-format qcow2 testimage
...
~~~

... then the image gets 197120 Bi which is the actual size of qcow2 file.
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack image show testimage
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | dc9ef2e7cf57a420ef03a408178a4076 |
| container_format | bare |
| created_at | 2021-07-21T07:48:24Z |
| disk_format | qcow2 |
| file | /v2/images/e1272ef7-f39b-460e-a529-89f5eb0c8e14/file |
| id | e1272ef7-f39b-460e-a529-89f5eb0c8e14 |
| min_disk | 0 |
| min_ram | 0 |
| name | testimage |
| owner | 942783ae248c4e9eb353a6e6b327bda5 |
| properties | direct_url='swift+config://ref1/glance/e1272ef7-f39b-460e-a529-89f5eb0c8e14', os_hash_algo='sha512', os_hash_value='fe600797d0cc7e4a2a598841b129709a327981a52e7de3a1b13dc09427c53136ca42df9bb174b79984f829c1ba8e8ee64891de542194cd9243d149c6593680db', os_hidden='False', stores='default_backend' |
| protected | False |
| schema | /v2/schemas/image |
| size | 197120 |
| status | active |
| tags | |
| updated_at | 2021-07-21T07:48:28Z |
| virtual_size | None |
| visibility | shared |
+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
~~~

If we try to create a 1GB volume form that image, the request is accpeted but the volume eventually becomes error status.
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack volume create smallervolume --image testimage --size 1
...
(overcloud) [stack@undercloud-0 ~]$ openstack volume show smallervolume
+--------------------------------+---------------------------------------+
| Field | Value |
+--------------------------------+---------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2021-07-21T07:52:37.000000 |
| description | None |
| encrypted | False |
| id | a72f04f5-0351-4a3a-856d-2d7a9586a4fc |
| migration_status | None |
| multiattach | False |
| name | smallervolume |
| os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 942783ae248c4e9eb353a6e6b327bda5 |
| properties | |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | error |
| type | tripleo |
| updated_at | 2021-07-21T07:52:39.000000 |
| user_id | 06b547a0af8f49fd8239c85ce5d9571b |
| volume_image_metadata | {'signature_verified': 'False'} |
+--------------------------------+---------------------------------------+
~~~

It is because that virtual size of the image(2GB) doesn't fit in a volume size(1GB), but this error is not exposed to users.
This makes it difficult to let users know actual cause of the error.

~~~
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server [req-30713ed0-564d-4f28-8035-252b63e697a5 06b547a0af8f49fd8239c85ce5d9571b 942783ae248c4e9eb353a6e6b327bda5 - default default] Exception during message handling: cinder.exception.
ImageUnacceptable: Image e1272ef7-f39b-460e-a529-89f5eb0c8e14 is unacceptable: Image virtual size is 2GB and doesn't fit in a volume of size 1GB.
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
...
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/flows/manager/create_volume.py", line 1035, in _create_from_image
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server image_service)
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/flows/manager/create_volume.py", line 915, in _create_from_image_cache_or_download
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server data.virtual_size, volume.size, image_id)
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/image/image_utils.py", line 742, in check_virtual_size
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server reason=reason)
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server cinder.exception.ImageUnacceptable: Image e1272ef7-f39b-460e-a529-89f5eb0c8e14 is unacceptable: Image virtual size is 2GB and doesn't fit in a volume of size 1GB.
2021-07-21 07:52:39.678 74 ERROR oslo_messaging.rpc.server
~~~

Because cinder is aware of the original volume size, it should be able to recognize original volume size and reject creating a new volume smaller than the original volume.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/804584

[Sofia Enriquez (lsofia-enriquez)]

Greetings,

I have one more question that I see it's not on the BZ bug report:
- Are you using RBD for both glance and cinder? If not, do you mind sharing with us the backend you're using?
I'm asking this because RBD implements a copy on write feature that may be related to this.

As you wrote https://bugzilla.redhat.com/show_bug.cgi?id=1984304#c8 the fix should be split in two parts.

Thanks in advance,
Sofia

[Takashi Kajinami (kajinamit)]

Hi Sofia,

Cinder backend is LVM and glance backend is swift.
IIRC the issue is not dependent on backends since the main problem here is related to how size is handled during format conversion and we can't use copy on write when converting image formats.