Comment 0 for bug 1893669

Brief Description
-----------------
When a process fails twice whitin 90 seconds SM does not execute the expected impact.
for instance, I have tested it against dnsmasq process and after killing it twice swact is not triggered.
dnsmasq is an example, but I tested against dcmanager-manager and it has the same behavior.

Severity
--------
Critical

Steps to Reproduce
------------------
I've created a script to test it.

1. Kill process
2. wait 90 seconds.
3. kill it again.

====== script ====
pid_file=$1
t=$2
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
sleep $(expr $t - 5);
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
======= end script ===

run it with: ./script_name.sh PID_FILE INTERVAL
for instance: sh ./script_name.sh /var/run/dnsmasq.pid 90 (same behavior if set 60 seconds for instance)

here are the output logs:

[sysadmin@controller-1 ~(keystone_admin)]$ sh kill_2.sh /var/run/dnsmasq.pid 90
vie ago 28 02:22:21 UTC 2020
killing 3073469
vie ago 28 02:22:26 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-0 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 2:19:28. |
| | | | | 540423 |
| | | | | |
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-1 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 0:07:52. |
| | | | | 947822 |
| | | | | |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
vie ago 28 02:23:52 UTC 2020
killing 3119849
vie ago 28 02:23:57 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-0 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 2:19:28. |
| | | | | 540423 |
| | | | | |
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-1 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 0:07:52. |
| | | | | 947822 |
| | | | | |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
[sysadmin@controller-1 ~(keystone_admin)]$

and swact is not triggered.

Expected Behavior
------------------
after dnsmasq failling twice swact must be triggered.

Actual Behavior
----------------
swact is not triggered.

Reproducibility
---------------
Reproducible

System Configuration
--------------------
IPv4 distributed cloud

Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-08-25_09-07-26"

Last Pass
---------
N/A

Timestamp/Logs
--------------
N/A: logs are above.

Test Activity
-------------
Feature Testing