Marking this LP as invalid. The TC code will need to be updated to not attempt a controller swact if an application apply is in progress.
Here is the timeline for this issue:
1. controller-1 was active. All other hosts were locked due to the previous TC.
2. controller-0 and the other hosts were unlocked at 20:08:19
3. The system is designed to re-apply applications if needed after hosts are unlocked. The platform-integ-apps app was re-applied after controller-0 went unlocked enabled - sysinv.log:
sysinv 2020-04-26 20:13:21.934 322002 INFO sysinv.conductor.manager [-] There has been an overrides change, setting up reapply of platform-integ-apps
sysinv 2020-04-26 20:13:22.930 322002 INFO sysinv.conductor.manager [-] Reapplying platform-integ-apps app
4. Before the application apply is completed, the TC does a controller swact:
[2020-04-26 20:14:04,488] 314 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://[face::1]:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-swact controller-1'
5. This terminates the application apply in progress.
Marking this LP as invalid. The TC code will need to be updated to not attempt a controller swact if an application apply is in progress.
Here is the timeline for this issue:
1. controller-1 was active. All other hosts were locked due to the previous TC. conductor. manager [-] There has been an overrides change, setting up reapply of platform-integ-apps conductor. manager [-] Reapplying platform-integ-apps app
2. controller-0 and the other hosts were unlocked at 20:08:19
3. The system is designed to re-apply applications if needed after hosts are unlocked. The platform-integ-apps app was re-applied after controller-0 went unlocked enabled - sysinv.log:
sysinv 2020-04-26 20:13:21.934 322002 INFO sysinv.
sysinv 2020-04-26 20:13:22.930 322002 INFO sysinv.
4. Before the application apply is completed, the TC does a controller swact: domain- name Default --os-project- domain- name Default --os-endpoint-type internalURL --os-region-name RegionOne host-swact controller-1'
[2020-04-26 20:14:04,488] 314 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://[face::1]:5000/v3 --os-user-
5. This terminates the application apply in progress.