rbd pool stats is reportted inaccurately

[yao ning]

Currently, rbd report pool stats periodically by cinder-volume. It includes allocated_capacity_gb, free_capacity_gb, provisioned_capacity_gb and total_capacity_gb.

The typical value of the stats in my enviroment looks like below:

pool allocated_capacity_gb free_capacity_gb provisioned_capacity_gb total_capacity_gb
hdd 5820 20372.36 7690 29683.46
hybrid 20970 31159.65 21336 46812.57
ssd 13360 5754.2 11553 17559.03

It is apparently that total_capacity_gb - free_capacity_gb > allocated_capacity_gb. This is abnormal because of thin provision storage backend's allocated_capacity_gb should always smaller than used_capacity_gb( total_capacity_gb - free_capacity_gb)

The main reason is that pool_stats['bytes_used'] is the data actually stored in rbd pool, which sums up with the replica size. However, the pool_stats['max_avail'] is saying that the actual size client can write into the pool.

After Nautilus, Ceph provides another metric called pool_stats['stored'] which means the data_size client actually write s.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/852724

[OpenStack Infra (hudson-openstack)]

Change abandoned by "yao ning <[e-mail]>" on branch: master
Review: https://review.opendev.org/c/openstack/cinder/+/852724