timeout deploying openstack-cinder chart if performed system storage-tier-modify operation
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
Fix Released
|
High
|
Daniel Badea |
Bug Description
Bug description:
Brief Description
-----------------
Timeout deploying openstack-cinder chart (stx-openstack apply fails and aborts) if a tier name change performed (system storage-
ceph tier override feature test
Severity
--------
Major
Steps to Reproduce
-----
1. clean system install (platform only)
2. Manually perform stx-openstack apply successfully and configuration completed.
3. storage tier added here
system application-list
system storage-tier-add ceph_cluster goldy
system storage-tier-list ceph_cluster
Run the following to confirm the tree structure with goldy
$ceph osd tree
$ system storage-tier-list ceph_cluster
+------
| uuid | name | status | backend_using |
+------
| 08b8dadc-
| 2f8b52e0-
$ ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-7 0 root goldy-tier
-8 0 chassis group-0-goldy
-9 0 host controller-0-goldy
-1 0.43500 root storage-tier
-2 0.43500 chassis group-0
-3 0.43500 host controller-0
0 ssd 0.43500 osd.0 up 1.00000 1.00000
4. Try to update (or amend) the tier name
eg. system storage-tier-modify ceph_cluster goldy --name gold
bash.log shows the last tier modify operation ~ 16:13
2019-07-
...
2019-07-
Check for the new name
$ceph osd tree
$system storage-tier-list ceph_cluster
$ system storage-tier-list ceph_cluster
+------
| uuid | name | status | backend_using |
+------
| 08b8dadc-
| 2f8b52e0-
$ ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-7 0 root goldy-tier
-8 0 chassis group-0-goldy
-9 0 host controller-0-goldy
-1 0.43500 root storage-tier
-2 0.43500 chassis group-0
-3 0.43500 host controller-0
0 ssd 0.43500 osd.0 up 1.00000 1.00000
Note: The ceph osd tree remains with the original name 'goldy'
5. Try to complete the ceph tier configuration
Host lock
2019-07-
Backend add
2019-07-
2019-07-
host-stor-add
2019-07-
2019-07-
6. Unlock controller
2019-07-
$system application-list
...
| stx-openstack | 1.0-17-
(Note: stx-openstack apply has succeeded in previous install attempts where the tier name was not modifie)
Expected Behavior
-----
Either update osd tree (where the tier is in 'defined' status) if supported or block and provide feedback if it is not supported
(ie. no impact to the stx-openstack apply operation)
Actual Behavior
----------------
In step 4, the system storage-tier-list shows the name changed to gold. The ceph osd tree remains with the original name 'goldy'
Application apply times out and aborts
2019-07-11 17:22:09.804 112674 ERROR sysinv.
2019-07-11 17:22:09.805 112674 INFO sysinv.
2019-07-11 17:22:09.822 112674 ERROR sysinv.
2019-07-11 16:52:09.022 112674 INFO sysinv.
In step 6, unlocking the controller, the armada log report Exception deploying openstack-cinder chart
2019-07-11 16:52:08.838 7505 DEBUG armada.
2019-07-11 16:52:08.838 7505 DEBUG armada.
....
2019-07-11 17:22:08.882 7505 ERROR armada.
2019-07-11 17:22:08.887 7505 ERROR armada.
2019-07-11 17:22:09.624 7505 INFO armada.
2019-07-11 17:22:09.634 7505 ERROR armada.cli [-] Caught internal exception: armada.
2019-07-11 17:22:09.634 7505 ERROR armada.cli Traceback (most recent call last):
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/local/
2019-07-11 17:22:09.634 7505 ERROR armada.cli self.invoke()
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/local/
2019-07-11 17:22:09.634 7505 ERROR armada.cli resp = self.handle(
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/local/
2019-07-11 17:22:09.634 7505 ERROR armada.cli return future.result()
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/lib/
2019-07-11 17:22:09.634 7505 ERROR armada.cli return self.__get_result()
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/lib/
2019-07-11 17:22:09.634 7505 ERROR armada.cli raise self._exception
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/lib/
2019-07-11 17:22:09.634 7505 ERROR armada.cli result = self.fn(*self.args, **self.kwargs)
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/local/
2019-07-11 17:22:09.634 7505 ERROR armada.cli return armada.sync()
2019-07-11 17:22:09.634 7505 ERROR armada.cli File "/usr/local/
2019-07-11 17:22:09.634 7505 ERROR armada.cli raise armada_
2019-07-11 17:22:09.634 7505 ERROR armada.cli armada.
2019-07-11 17:22:09.634 7505 ERROR armada.cli
Reproducibility
---------------
yes
System Configuration
-----
tried on simplex
Branch/Pull Time/Commit
-----
20190703T013000Z
Timestamp/Logs
--------------
see inline
Last Pass
---------
new feature
Test Activity
-------------
[Feature Testing]
tags: | added: stx.retestneeded |
Changed in starlingx: | |
status: | Triaged → In Progress |
Marking as stx.2.0 - it appears that overrides to storage tiers result in application apply failure