Volume quality of service in cinder documentation

[Eric Miller]

The Cinder volume QoS documentation does not provide enough information to understand what the parameters mean.

One example - there are two pages for QoS, one for "Basic Volume" and one for "Capacity-based" quality of service for volumes:
https://docs.openstack.org/cinder/latest/admin/blockstorage-basic-volume-qos.html
https://docs.openstack.org/cinder/latest/admin/blockstorage-capacity-based-qos.html

are they both accurate? or is one deprecated? If both are valid, maybe they should refer to each other, or be combined?

Some issues:

 - it is unclear as to which QoS feature is supported with which release of OpenStack

 - the pages indicate "the values of which should be fairly self-explanatory" - they are not. For example, what is "read_iops_sec"? Read I/O's per second per second? or should it be Read I/O's per second? What is the difference between "read_iops_sec" and "read_iops_sec_max"? If these are already limits, why are there two names? Or is one a minimum and the other a maximum? What is "size_iops_sec"? Is this supposed to provide a maximum I/O payload size? If not, can this be specified somewhere so if a user creates a command with 4MByte chunks, it does not count as a single I/O and 1024 IOPS would be 4GBytes/sec?

 - on the capacity based QoS, read_iops_sec_per_gb should probably be read_iops_per_gb. There is also a "read_iops_sec_per_gb_min", which provides some minimum limit, but what about maximum limit? If a QoS spec was created for 3 IOPS per GiB, a minimum of 10GiB could be specified 10GiB to limit the IOPS to a minimum of 30, but what about the maximum? What limits a user from trying to create a 100TiB volume with 307200 IOPS?

 - many more examples would be incredibly useful - especially "real world" examples.

Eric

-----------------------------------
Release: on 2018-08-07 01:39
SHA: 851f4d9ec46f1ff144066087e47759241ab8c0a2
Source: https://git.openstack.org/cgit/openstack/cinder/tree/doc/source/admin/blockstorage-basic-volume-qos.rst
URL: https://docs.openstack.org/cinder/latest/admin/blockstorage-basic-volume-qos.html

[Eric Miller (erickmiller)]

Just to add some items that might be useful for someone, but please correct me if I'm wrong on any of this.

These are descriptions for the various QoS properties for Cinder, which are really Libvirt properties, except for the "per_gb" properties, which are simply computed values for the respective property based on the volume size in GiB.

# Descriptions for the volume type properties are available here:
# https://libvirt.org/formatdomain.html

# Queens' Libvirt configuration is written here (search for "iotune", which is the tag in the
# Libvirt XML VM configuration file that controls throttling):
# https://github.com/openstack/nova/blob/stable/queens/nova/virt/libvirt/config.py
#
# New properties have been added to Master (currently Rocky beta):
# https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py

# Fixed IOPS per volume, regardless of size, specified unit is IOPS, not IOPS/sec
read_iops_sec
write_iops_sec
total_iops_sec

# Fixed bandwidth per volume, regardless of size, specified unit is bytes/sec
read_bytes_sec
write_bytes_sec
total_bytes_sec

# Burst IOPS per volume for the given "size_iops_sec", specified unit is IOPS, not IOPS/sec
read_iops_sec_max
write_iops_sec_max
total_iops_sec_max

# Burst bandwidth per volume for the given "size_iops_sec", specified unit is bytes/sec
read_bytes_sec_max
write_bytes_sec_max
total_bytes_sec_max

# The burst bucket size, specified unit is bytes
size_iops_sec

# Dynamic IOPS per volume, based on size, specific unit is IOPS, not IOPS/sec
read_iops_sec_per_gb
write_iops_sec_per_gb
total_iops_sec_per_gb

# Dynamic bandwidth per volume, based on size, specific unit is bytes/sec
read_bytes_sec_per_gb
write_bytes_sec_per_gb
total_bytes_sec_per_gb

# Minimum IOPS per volume - used to avoid assigning too few
# IOPS to a volume, such as for tiny volumes.
# NOTE that these have NOT been implemented as of Rocky
# Specified unit is IOPS, not IOPS/sec
read_iops_sec_per_gb_min
write_iops_sec_per_gb_min
total_iops_sec_per_gb_min

# Minimum bandwidth per volume - used to avoid assigning too little
# bandwidth to a volume, such as for tiny volumes.
# NOTE that these have NOT been implemented as of Rocky
# Specified unit is IOPS, not IOPS/sec
read_bytes_sec_per_gb_min
write_bytes_sec_per_gb_min
total_bytes_sec_per_gb_min

[Zhenbo Qiao (qjpqzb)]

@Eric Miller
Hi,

Have "total_bytes_sec_per_gb_min" been implemented as of Ussuri? I have tried setting this in qos settings, but the minimum bandwidth I set can not be achieved. Is there any updates on this? Thanks!

Best,
Zhenbo Qiao