Comment 31 for bug 1827119

# RETEST STATUS
PASSED

# CONFIGURATION
2 + 2 Controller Storage Config

# LOAD TESTED
BUILD_ID="2019-10-09_20-00-00"

# TESTING

1. Verify ceph-mon prior to changing
 [sysadmin@controller-0 ~(keystone_admin)]$ system ceph-mon-list
 +--------------------------------------+-------+--------------+------------+------+
 | uuid | ceph_ | hostname | state | task |
 | | mon_g | | | |
 | | ib | | | |
 +--------------------------------------+-------+--------------+------------+------+
 | d225b915-8adf-4b24-961d-df7855e2cebf | 20 | compute-0 | configured | None |
 | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 20 | controller-0 | configured | None |
 | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 20 | controller-1 | configured | None |
 +--------------------------------------+-------+--------------+------------+------+
 [sysadmin@controller-0 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 22G 105M 20G 1% /var/lib/ceph/mon

 controller-1:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 22G 105M 20G 1% /var/lib/ceph/mon

 compute-0:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 22G 107M 20G 1% /var/lib/ceph/mon

2. Increase size of ceph_mon
 [sysadmin@controller-0 ~(keystone_admin)]$ system ceph-mon-modify controller-0 ceph_mon_gib=21
 +--------------------------------------+-------+--------------+------------+------+
 | uuid | ceph_ | hostname | state | task |
 | | mon_g | | | |
 | | ib | | | |
 +--------------------------------------+-------+--------------+------------+------+
 | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None |
 | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None |
 | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None |
 +--------------------------------------+-------+--------------+------------+------+

3. After lock/unlock of controllers, compute still reports config out-of-date
 [sysadmin@controller-1 ~(keystone_admin)]$ fm alarm-list
 +----------+-------------------------------------+----------------+----------+-------------------+
 | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
 +----------+-------------------------------------+----------------+----------+-------------------+
 | 250.001 | compute-0 Configuration is out-of-date. | host=compute-0 | major | 2019-10-11T14:45: |
 | | | | | 05.903134 |
 | | | | | |
 +----------+-------------------------------------+----------------+----------+-------------------+
 [sysadmin@controller-1 ~(keystone_admin)]$ system ceph-mon-list
 +--------------------------------------+-------+--------------+------------+------+
 | uuid | ceph_ | hostname | state | task |
 | | mon_g | | | |
 | | ib | | | |
 +--------------------------------------+-------+--------------+------------+------+
 | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None |
 | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None |
 | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None |
 +--------------------------------------+-------+--------------+------------+------+

 [sysadmin@controller-1 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 23G 103M 21G 1% /var/lib/ceph/mon

 # Ceph mon still showing 20G on compute-0 (worker)
 compute-0:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 22G 103M 20G 1% /var/lib/ceph/mon

4. After lock/unlock compute-0 (worker)
 # Alarm Cleared
 [sysadmin@controller-1 ~(keystone_admin)]$ fm alarm-list

 # Ceph mon shows updated value for all nodes
 [sysadmin@controller-1 ~(keystone_admin)]$ system ceph-mon-list
 +--------------------------------------+-------+--------------+------------+------+
 | uuid | ceph_ | hostname | state | task |
 | | mon_g | | | |
 | | ib | | | |
 +--------------------------------------+-------+--------------+------------+------+
 | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None |
 | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None |
 | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None |
 +--------------------------------------+-------+--------------+------------+------+

 # ceph mon reports correct updated value on each node as well
 compute-0:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 23G 106M 21G 1% /var/lib/ceph/mon

 controller-0:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 23G 108M 21G 1% /var/lib/ceph/mon

 controller-1:~$ df -H | grep /var/lib/ceph/mon
 /dev/mapper/cgts--vg-ceph--mon--lv 23G 106M 21G 1% /var/lib/ceph/mon