swact fails with runtime manifest failure

[Yvonne Ding]

Brief Description
-----------------
Runtime manifest failure causes host swact failed.

Severity
--------
Major

Steps to Reproduce
------------------
1. perform system host-swact <controller>
or run testcase mtc::test_swact_controller_platform

Expected Behavior
------------------
host-swact controller success

Actual Behavior
----------------
host-swact controller failed

Reproducibility
---------------
reproducible

System Configuration
--------------------
Regular standard 2+2

Branch/Pull Time/Commit
-----------------------
BUILD_ID="20210306T023358Z"

Timestamp/Logs
--------------
2021-03-11T03:44:29.781 Debug: 2021-03-11 03:44:29 +0000 Exec[signal-mtc-agent](provider=posix): Executing 'pkill -HUP mtcAgent'
2021-03-11T03:44:29.783 Debug: 2021-03-11 03:44:29 +0000 Executing: 'pkill -HUP mtcAgent'
2021-03-11T03:44:29.785 Error: 2021-03-11 03:44:29 +0000 pkill -HUP mtcAgent returned 1 instead of one of [0]
2021-03-11T03:44:29.787 /usr/share/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
2021-03-11T03:44:29.788 /usr/share/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync'
2021-03-11T03:44:29.790 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync'
2021-03-11T03:44:29.792 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'
2021-03-11T03:44:29.794 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes'
2021-03-11T03:44:29.796 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each'
2021-03-11T03:44:29.798 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes'
2021-03-11T03:44:29.800 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'
2021-03-11T03:44:29.801 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply'
2021-03-11T03:44:29.803 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource'
2021-03-11T03:44:29.805 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
2021-03-11T03:44:29.807 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
2021-03-11T03:44:29.808 /usr/share/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark'
2021-03-11T03:44:29.811 /usr/share/ruby/benchmark.rb:296:in `realtime'
2021-03-11T03:44:29.813 /usr/share/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark'
2021-03-11T03:44:29.815 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
2021-03-11T03:44:29.817 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
2021-03-11T03:44:29.819 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
2021-03-11T03:44:29.821 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
2021-03-11T03:44:29.823 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
2021-03-11T03:44:29.825 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'
2021-03-11T03:44:29.827 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
2021-03-11T03:44:29.829 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
2021-03-11T03:44:29.831 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'
2021-03-11T03:44:29.833 /usr/share/ruby/benchmark.rb:296:in `realtime'
2021-03-11T03:44:29.834 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'
2021-03-11T03:44:29.836 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
2021-03-11T03:44:29.838 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
2021-03-11T03:44:29.839 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
2021-03-11T03:44:29.841 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2021-03-11T03:44:29.843 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2021-03-11T03:44:29.845 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
2021-03-11T03:44:29.847 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'
2021-03-11T03:44:29.849 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
2021-03-11T03:44:29.851 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2021-03-11T03:44:29.853 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2021-03-11T03:44:29.854 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
2021-03-11T03:44:29.856 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
2021-03-11T03:44:29.859 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'
2021-03-11T03:44:29.861 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'
2021-03-11T03:44:29.863 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'
2021-03-11T03:44:29.865 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
2021-03-11T03:44:29.867 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
2021-03-11T03:44:29.869 /usr/bin/puppet:5:in `<main>'
2021-03-11T03:44:29.871 Error: 2021-03-11 03:44:29 +0000 /Stage[post]/Platform::Mtce::Reload/Exec[signal-mtc-agent]/returns: change from notrun to 0 failed: pkill -HUP mtcAgent returned 1 instead of one of [0]

[Ghada Khalil (gkhalil)]

As per Yang Liu, this appears to be an intermittent issue. There were some swact TCs on standard systems that passed.

[Yvonne Ding (yding)]

Please share the logs for those TCs passed on STX release, specifically "20210306T023358Z". Thanks.

[Ghada Khalil (gkhalil)]

screening: marking as low priority given the issue is intermittent. There have been many sanities run on the stx master branch since then and the swact TC doesn't fail.

[Ramaswamy Subramanian (rsubrama)]

No progress on this bug for more than 2 years. Candidate for closure.

If there is no update, this issue is targeted to be closed as 'Won't Fix' in 2 weeks.

[Ramaswamy Subramanian (rsubrama)]

Closed as Won't Fix