With the bug #1765182, it was assumed that the ONTAP didn't support the online extend, so a change was implemented [0] to prevent the volume to stay in error_extending status and log a message on cinder scheduler.
Actually, the ONTAP supports the online extend as shown below. The increasing of the size for a lun has limitations on ONTAP. For versions < 9.5 the limit for a resize is based on the initial geometry of the LUN and its limit is approximately 10 times the original size [1]. For versions >= 9.5 the limit is 16TB regardless of its original size [2] after the first resize.
# Environment
- Devstack on Master Branch
- Ubuntu 18.04
- ONTAP NetApp Release 9.6P2
stack@47-throne-dev-openstack:~$ nova list
+--------------------------------------+------+--------+------------+-------------+---------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+---------------------------------------------------------+
| 4899dcd9-18ef-4a59-ada1-98782d078fb1 | srv1 | ACTIVE | - | Running | private=fd9c:4b65:c6c9:0:f816:3eff:fe91:7306, 10.0.0.26 |
+--------------------------------------+------+--------+------------+-------------+---------------------------------------------------------+
stack@47-throne-dev-openstack:~$ cinder list
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| 4c2c2c9a-4e18-44f3-b034-5b0f27172a13 | in-use | vol1 | 1 | netappiscsi | false | 4899dcd9-18ef-4a59-ada1-98782d078fb1 |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
stack@47-throne-dev-openstack:~$ cinder extend vol1 2
stack@47-throne-dev-openstack:~$ cinder list
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| 4c2c2c9a-4e18-44f3-b034-5b0f27172a13 | in-use | vol1 | 2 | netappiscsi | false | 4899dcd9-18ef-4a59-ada1-98782d078fb1 |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
stack@47-throne-dev-openstack:~$ cinder extend vol1 64
stack@47-throne-dev-openstack:~$ cinder list
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
| 4c2c2c9a-4e18-44f3-b034-5b0f27172a13 | in-use | vol1 | 64 | netappiscsi | false | 4899dcd9-18ef-4a59-ada1-98782d078fb1 |
+--------------------------------------+--------+------+------+-------------+----------+--------------------------------------+
- ONTAP NetApp Release 9.4P1
stack@47-throne-dev-openstack:~$ cinder list
+--------------------------------------+--------+------+------+--------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------+------+--------------+----------+--------------------------------------+
| 298c192d-856b-4da5-b755-ad826dc13aed | in-use | vol2 | 1 | netappiscsi2 | false | 2072c0e9-0ad4-4154-b63c-f5a6ad2de1f3 |
+--------------------------------------+--------+------+------+--------------+----------+--------------------------------------+
stack@47-throne-dev-openstack:~$ cinder extend vol2 64
stack@47-throne-dev-openstack:~$ cinder list
+--------------------------------------+-----------------+------+------+--------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------------+------+------+--------------+----------+--------------------------------------+
| 298c192d-856b-4da5-b755-ad826dc13aed | error_extending | vol2 | 1 | netappiscsi2 | false | 2072c0e9-0ad4-4154-b63c-f5a6ad2de1f3 |
+--------------------------------------+-----------------+------+------+--------------+----------+--------------------------------------+
- c-vol log for 9.4
Apr 20 19:59:21.744314 47-throne-dev-openstack cinder-volume[25517]: ERROR cinder.volume.manager cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Volume volume-298c192d-856b-4da5-b755-ad826dc13aed cannot be resized from 1073741824 to 68719476736, because would exceed its max geometry 68718428160 while not being detached.
[0] https://opendev.org/openstack/cinder/commit/792eea0a12bd97f69294ad0570ac330a3f1fe423
[1] https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-cmpr-920%2Flun__resize.html
[2] https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-sanag%2FGUID-E27C32B4-136F-41E2-99B8-0A9D9BE461D0.html
A newly created LUN on ONTAP has a max-resize limit that varies according to
its initial size. For instance, a LUN with initial size of 1GB has max-resize
of 64GB.
ONTAP is unable to directly resize a LUN beyond its max-resize. For instance,
an 1GB LUN cannot be directly extended to 65GB.
When such an extend operation is requested, ONTAP Cinder driver performs the
sub-clone-resize operation, which is done in 3 steps:
1. Create a new LUN with the desired size
2. Copy the data from the old LUN to the new LUN
3. Replace the old LUN with the new LUN
The step 3. would disconnect the LUN from any instance it's attached to. So the
driver prevents this by raising an exception and leaving the Cinder volume in
error_extending state [0].
That's why online extend feature is not supported by ONTAP Cinder driver.
However, after some tests, we realized that this issue can be easily addressed.
The volume create operation can be done in 2 steps:
1. Create a LUN with initial size of 15.97TB
2. Shrink it to the actual desired size
The step 1. will automatically set the LUN's max-resize to 15.97TB, which is
the max LUN size that ONTAP supports. After the step 2., the LUN will remain
with the max-resize of 15.97TB.
For ONTAP >=9.5, there's a better way to achieve this:
1. Create a LUN with initial size of 4MB
2. Extend it to the actual desired size
On ONTAP >=9.5, regardless the initial and final size, a LUN will have
max-resize of 15.97TB after its first resize operation.
By implementing the steps above on volume create method, every new Cinder
volume will be able to be extended to any size up to 15.97TB without the need
to do the sub-clone-resize operation. And therefore, the connection to the
instance will remain untouched.
This solution has another benefit. Currently, an extend operation from 63GB to
64GB requires 63GB of data to be copied from the old LUN to the new one.
As there'll be no more sub-clone-resize operation, the extend operation in
general will have an improved performance.
[0] https:/