# RETEST STATUS PASSED
# CONFIGURATION 2 + 2 Controller Storage Config
# LOAD TESTED BUILD_ID="2019-10-09_20-00-00"
# TESTING
1. Verify ceph-mon prior to changing [sysadmin@controller-0 ~(keystone_admin)]$ system ceph-mon-list +--------------------------------------+-------+--------------+------------+------+ | uuid | ceph_ | hostname | state | task | | | mon_g | | | | | | ib | | | | +--------------------------------------+-------+--------------+------------+------+ | d225b915-8adf-4b24-961d-df7855e2cebf | 20 | compute-0 | configured | None | | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 20 | controller-0 | configured | None | | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 20 | controller-1 | configured | None | +--------------------------------------+-------+--------------+------------+------+ [sysadmin@controller-0 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 22G 105M 20G 1% /var/lib/ceph/mon
controller-1:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 22G 105M 20G 1% /var/lib/ceph/mon
compute-0:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 22G 107M 20G 1% /var/lib/ceph/mon
2. Increase size of ceph_mon [sysadmin@controller-0 ~(keystone_admin)]$ system ceph-mon-modify controller-0 ceph_mon_gib=21 +--------------------------------------+-------+--------------+------------+------+ | uuid | ceph_ | hostname | state | task | | | mon_g | | | | | | ib | | | | +--------------------------------------+-------+--------------+------------+------+ | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None | | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None | | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None | +--------------------------------------+-------+--------------+------------+------+
3. After lock/unlock of controllers, compute still reports config out-of-date [sysadmin@controller-1 ~(keystone_admin)]$ fm alarm-list +----------+-------------------------------------+----------------+----------+-------------------+ | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +----------+-------------------------------------+----------------+----------+-------------------+ | 250.001 | compute-0 Configuration is out-of-date. | host=compute-0 | major | 2019-10-11T14:45: | | | | | | 05.903134 | | | | | | | +----------+-------------------------------------+----------------+----------+-------------------+ [sysadmin@controller-1 ~(keystone_admin)]$ system ceph-mon-list +--------------------------------------+-------+--------------+------------+------+ | uuid | ceph_ | hostname | state | task | | | mon_g | | | | | | ib | | | | +--------------------------------------+-------+--------------+------------+------+ | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None | | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None | | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None | +--------------------------------------+-------+--------------+------------+------+
[sysadmin@controller-1 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 23G 103M 21G 1% /var/lib/ceph/mon
# Ceph mon still showing 20G on compute-0 (worker) compute-0:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 22G 103M 20G 1% /var/lib/ceph/mon
4. After lock/unlock compute-0 (worker) # Alarm Cleared [sysadmin@controller-1 ~(keystone_admin)]$ fm alarm-list
# Ceph mon shows updated value for all nodes [sysadmin@controller-1 ~(keystone_admin)]$ system ceph-mon-list +--------------------------------------+-------+--------------+------------+------+ | uuid | ceph_ | hostname | state | task | | | mon_g | | | | | | ib | | | | +--------------------------------------+-------+--------------+------------+------+ | d225b915-8adf-4b24-961d-df7855e2cebf | 21 | compute-0 | configured | None | | d850eb6e-c880-4188-bc78-43bd162a0ff6 | 21 | controller-0 | configured | None | | df67b4c3-3962-4767-a4fa-dcdeb0f8e53d | 21 | controller-1 | configured | None | +--------------------------------------+-------+--------------+------------+------+
# ceph mon reports correct updated value on each node as well compute-0:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 23G 106M 21G 1% /var/lib/ceph/mon
controller-0:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 23G 108M 21G 1% /var/lib/ceph/mon
controller-1:~$ df -H | grep /var/lib/ceph/mon /dev/mapper/cgts--vg-ceph--mon--lv 23G 106M 21G 1% /var/lib/ceph/mon
# RETEST STATUS
PASSED
# CONFIGURATION
2 + 2 Controller Storage Config
# LOAD TESTED "2019-10- 09_20-00- 00"
BUILD_ID=
# TESTING
1. Verify ceph-mon prior to changing controller- 0 ~(keystone_admin)]$ system ceph-mon-list ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ 8adf-4b24- 961d-df7855e2ce bf | 20 | compute-0 | configured | None | c880-4188- bc78-43bd162a0f f6 | 20 | controller-0 | configured | None | 3962-4767- a4fa-dcdeb0f8e5 3d | 20 | controller-1 | configured | None | ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ controller- 0 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 22G 105M 20G 1% /var/lib/ceph/mon
[sysadmin@
+-----
| uuid | ceph_ | hostname | state | task |
| | mon_g | | | |
| | ib | | | |
+-----
| d225b915-
| d850eb6e-
| df67b4c3-
+-----
[sysadmin@
/dev/mapper/
controller-1:~$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 22G 105M 20G 1% /var/lib/ceph/mon
/dev/mapper/
compute-0:~$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 22G 107M 20G 1% /var/lib/ceph/mon
/dev/mapper/
2. Increase size of ceph_mon controller- 0 ~(keystone_admin)]$ system ceph-mon-modify controller-0 ceph_mon_gib=21 ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ 8adf-4b24- 961d-df7855e2ce bf | 21 | compute-0 | configured | None | c880-4188- bc78-43bd162a0f f6 | 21 | controller-0 | configured | None | 3962-4767- a4fa-dcdeb0f8e5 3d | 21 | controller-1 | configured | None | ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+
[sysadmin@
+-----
| uuid | ceph_ | hostname | state | task |
| | mon_g | | | |
| | ib | | | |
+-----
| d225b915-
| d850eb6e-
| df67b4c3-
+-----
3. After lock/unlock of controllers, compute still reports config out-of-date controller- 1 ~(keystone_admin)]$ fm alarm-list -----+- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -+----- ------- ------- + -----+- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -+----- ------- ------- + -----+- ------- ------- ------- ------- ------- -+----- ------- ----+-- ------- -+----- ------- ------- + controller- 1 ~(keystone_admin)]$ system ceph-mon-list ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ 8adf-4b24- 961d-df7855e2ce bf | 21 | compute-0 | configured | None | c880-4188- bc78-43bd162a0f f6 | 21 | controller-0 | configured | None | 3962-4767- a4fa-dcdeb0f8e5 3d | 21 | controller-1 | configured | None | ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+
[sysadmin@
+-----
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+-----
| 250.001 | compute-0 Configuration is out-of-date. | host=compute-0 | major | 2019-10-11T14:45: |
| | | | | 05.903134 |
| | | | | |
+-----
[sysadmin@
+-----
| uuid | ceph_ | hostname | state | task |
| | mon_g | | | |
| | ib | | | |
+-----
| d225b915-
| d850eb6e-
| df67b4c3-
+-----
[sysadmin@ controller- 1 ~(keystone_admin)]$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 23G 103M 21G 1% /var/lib/ceph/mon
/dev/mapper/
# Ceph mon still showing 20G on compute-0 (worker) cgts--vg- ceph--mon- -lv 22G 103M 20G 1% /var/lib/ceph/mon
compute-0:~$ df -H | grep /var/lib/ceph/mon
/dev/mapper/
4. After lock/unlock compute-0 (worker) controller- 1 ~(keystone_admin)]$ fm alarm-list
# Alarm Cleared
[sysadmin@
# Ceph mon shows updated value for all nodes controller- 1 ~(keystone_admin)]$ system ceph-mon-list ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+ 8adf-4b24- 961d-df7855e2ce bf | 21 | compute-0 | configured | None | c880-4188- bc78-43bd162a0f f6 | 21 | controller-0 | configured | None | 3962-4767- a4fa-dcdeb0f8e5 3d | 21 | controller-1 | configured | None | ------- ------- ------- ------- -----+- ------+ ------- ------- +------ ------+ ------+
[sysadmin@
+-----
| uuid | ceph_ | hostname | state | task |
| | mon_g | | | |
| | ib | | | |
+-----
| d225b915-
| d850eb6e-
| df67b4c3-
+-----
# ceph mon reports correct updated value on each node as well cgts--vg- ceph--mon- -lv 23G 106M 21G 1% /var/lib/ceph/mon
compute-0:~$ df -H | grep /var/lib/ceph/mon
/dev/mapper/
controller-0:~$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 23G 108M 21G 1% /var/lib/ceph/mon
/dev/mapper/
controller-1:~$ df -H | grep /var/lib/ceph/mon cgts--vg- ceph--mon- -lv 23G 106M 21G 1% /var/lib/ceph/mon
/dev/mapper/