| 2020-08-31 15:05:13 |
Francisco Giana |
bug |
|
|
added bug |
| 2020-08-31 15:48:49 |
Francisco Giana |
description |
Brief Description
-----------------
When a process fails twice whitin 90 seconds SM does not execute the expected impact.
for instance, I have tested it against dnsmasq process and after killing it twice swact is not triggered.
dnsmasq is an example, but I tested against dcmanager-manager and it has the same behavior.
Severity
--------
Critical
Steps to Reproduce
------------------
I've created a script to test it.
1. Kill process
2. wait 90 seconds.
3. kill it again.
====== script ====
pid_file=$1
t=$2
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
sleep $(expr $t - 5);
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
======= end script ===
run it with: ./script_name.sh PID_FILE INTERVAL
for instance: sh ./script_name.sh /var/run/dnsmasq.pid 90 (same behavior if set 60 seconds for instance)
here are the output logs:
[sysadmin@controller-1 ~(keystone_admin)]$ sh kill_2.sh /var/run/dnsmasq.pid 90
vie ago 28 02:22:21 UTC 2020
killing 3073469
vie ago 28 02:22:26 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-0 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 2:19:28. |
| | | | | 540423 |
| | | | | |
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-1 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 0:07:52. |
| | | | | 947822 |
| | | | | |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
vie ago 28 02:23:52 UTC 2020
killing 3119849
vie ago 28 02:23:57 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-0 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 2:19:28. |
| | | | | 540423 |
| | | | | |
| 400.003 | Evaluation license key will expire on 30-dec-2020; there are 124 | host=controller-1 | minor | 2020-08-28T0 |
| | days remaining in this evaluation | | | 0:07:52. |
| | | | | 947822 |
| | | | | |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
[sysadmin@controller-1 ~(keystone_admin)]$
and swact is not triggered.
Expected Behavior
------------------
after dnsmasq failling twice swact must be triggered.
Actual Behavior
----------------
swact is not triggered.
Reproducibility
---------------
Reproducible
System Configuration
--------------------
IPv4 distributed cloud
Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-08-25_09-07-26"
Last Pass
---------
N/A
Timestamp/Logs
--------------
N/A: logs are above.
Test Activity
-------------
Feature Testing |
Brief Description
-----------------
When a process fails twice whitin 90 seconds SM does not execute the expected impact.
for instance, I have tested it against dnsmasq process and after killing it twice swact is not triggered.
dnsmasq is an example, but I tested against dcmanager-manager and it has the same behavior.
Severity
--------
Critical
Steps to Reproduce
------------------
I've created a script to test it.
1. Kill process
2. wait 60 seconds.
3. kill it again.
====== script ====
pid_file=$1
t=$2
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
sleep $(expr $t - 5);
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
======= end script ===
run it with: ./script_name.sh PID_FILE INTERVAL
for instance: sh ./script_name.sh /var/run/dnsmasq.pid 60
here are the output logs:
[sysadmin@controller-1 ~(keystone_admin)]$ sh kill_2.sh /var/run/dnsmasq.pid 60
vie ago 28 03:17:49 UTC 2020
killing 3410146
vie ago 28 03:17:54 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
vie ago 28 03:18:50 UTC 2020
killing 3434943
vie ago 28 03:18:55 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
and swact was not triggered.
Expected Behavior
------------------
swact has to be triggered if process was killed more than once within 90 seconds.
Actual Behavior
----------------
swact is not triggered.
Reproducibility
---------------
Reproducible
System Configuration
--------------------
IPv4 distributed cloud
Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-08-25_09-07-26"
Last Pass
---------
N/A
Timestamp/Logs
--------------
N/A: logs are above.
Test Activity
-------------
Feature Testing |
|
| 2020-08-31 16:13:39 |
Yang Liu |
summary |
dnsmasq process monitoring expected impact doesn't work. |
swact is not triggered after killing dnsmasq process within 90 seconds |
|
| 2020-08-31 17:29:45 |
Francisco Giana |
description |
Brief Description
-----------------
When a process fails twice whitin 90 seconds SM does not execute the expected impact.
for instance, I have tested it against dnsmasq process and after killing it twice swact is not triggered.
dnsmasq is an example, but I tested against dcmanager-manager and it has the same behavior.
Severity
--------
Critical
Steps to Reproduce
------------------
I've created a script to test it.
1. Kill process
2. wait 60 seconds.
3. kill it again.
====== script ====
pid_file=$1
t=$2
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
sleep $(expr $t - 5);
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
======= end script ===
run it with: ./script_name.sh PID_FILE INTERVAL
for instance: sh ./script_name.sh /var/run/dnsmasq.pid 60
here are the output logs:
[sysadmin@controller-1 ~(keystone_admin)]$ sh kill_2.sh /var/run/dnsmasq.pid 60
vie ago 28 03:17:49 UTC 2020
killing 3410146
vie ago 28 03:17:54 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
vie ago 28 03:18:50 UTC 2020
killing 3434943
vie ago 28 03:18:55 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
and swact was not triggered.
Expected Behavior
------------------
swact has to be triggered if process was killed more than once within 90 seconds.
Actual Behavior
----------------
swact is not triggered.
Reproducibility
---------------
Reproducible
System Configuration
--------------------
IPv4 distributed cloud
Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-08-25_09-07-26"
Last Pass
---------
N/A
Timestamp/Logs
--------------
N/A: logs are above.
Test Activity
-------------
Feature Testing |
Brief Description
-----------------
When a process fails twice whitin 90 seconds SM does not execute the expected impact.
for instance, I have tested it against dnsmasq process and after killing it twice swact is not triggered.
dnsmasq is an example, but I tested against
* dcmanager-audit
* dcmanager-api
* dcmanager-manager
* dcdbsync-api
* dcorch-engine
* hw-mond
* hbsAgent
* rabbitmq
* sysinv-conductor
* mtcAgent
* fmManager
* (it seems all the processes monitored by sm)
and they have the same behavior.
Severity
--------
Critical
Steps to Reproduce
------------------
I've created a script to test it.
1. Kill process
2. wait 60 seconds.
3. kill it again.
====== script ====
pid_file=$1
t=$2
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
sleep $(expr $t - 5);
date;
pid=$(cat $pid_file 2>/dev/null)
echo "killing $pid"
echo "ROOT_PASSWORD" | sudo -S kill -9 $pid &>/dev/null;
sleep 5;
date;
fm alarm-list;
======= end script ===
run it with: ./script_name.sh PID_FILE INTERVAL
for instance: sh ./script_name.sh /var/run/dnsmasq.pid 60
here are the output logs:
[sysadmin@controller-1 ~(keystone_admin)]$ sh kill_2.sh /var/run/dnsmasq.pid 60
vie ago 28 03:17:49 UTC 2020
killing 3410146
vie ago 28 03:17:54 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
vie ago 28 03:18:50 UTC 2020
killing 3434943
vie ago 28 03:18:55 UTC 2020
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
+----------+-------------------------------------------------------------------+-------------------+----------+--------------+
and swact was not triggered.
Expected Behavior
------------------
swact has to be triggered if process was killed more than once within 90 seconds.
Actual Behavior
----------------
swact is not triggered.
Reproducibility
---------------
Reproducible
System Configuration
--------------------
IPv4 distributed cloud
Branch/Pull Time/Commit
-----------------------
BUILD_ID="2020-08-25_09-07-26"
Last Pass
---------
N/A
Timestamp/Logs
--------------
N/A: logs are above.
Test Activity
-------------
Feature Testing |
|
| 2020-08-31 18:32:28 |
Ghada Khalil |
tags |
|
stx.ha |
|
| 2020-09-09 17:36:00 |
Ghada Khalil |
starlingx: importance |
Undecided |
Medium |
|
| 2020-09-09 17:36:02 |
Ghada Khalil |
starlingx: status |
New |
Triaged |
|
| 2020-09-09 17:37:00 |
Ghada Khalil |
tags |
stx.ha |
stx.5.0 stx.ha |
|
| 2020-09-09 17:37:34 |
Ghada Khalil |
starlingx: assignee |
|
Bin Qian (bqian20) |
|
| 2021-03-29 18:48:56 |
Isac Sacchi e Souza |
bug |
|
|
added subscriber Isac Sacchi e Souza |
| 2021-03-30 22:15:42 |
Don Penney |
starlingx: assignee |
Bin Qian (bqian20) |
Don Penney (dpenney) |
|
| 2021-03-30 22:15:47 |
Don Penney |
starlingx: status |
Triaged |
In Progress |
|
| 2021-03-31 16:27:26 |
Dariush Eslimi |
starlingx: assignee |
Don Penney (dpenney) |
Bin Qian (bqian20) |
|
| 2021-04-19 18:05:44 |
OpenStack Infra |
starlingx: status |
In Progress |
Fix Released |
|
| 2021-04-21 15:32:16 |
Ghada Khalil |
tags |
stx.5.0 stx.ha |
in-r-stx50 stx.5.0 stx.ha |
|
| 2021-05-14 13:33:38 |
Ghada Khalil |
tags |
in-r-stx50 stx.5.0 stx.ha |
stx.5.0 stx.cherrypickneeded stx.ha |
|
| 2021-05-14 13:36:47 |
Ghada Khalil |
tags |
stx.5.0 stx.cherrypickneeded stx.ha |
stx.5.0 stx.6.0 stx.cherrypickneeded stx.ha |
|
| 2021-05-20 18:38:28 |
Ghada Khalil |
tags |
stx.5.0 stx.6.0 stx.cherrypickneeded stx.ha |
in-r-stx50 stx.5.0 stx.6.0 stx.ha |
|
| 2021-06-01 19:40:58 |
OpenStack Infra |
tags |
in-r-stx50 stx.5.0 stx.6.0 stx.ha |
in-f-centos8 in-r-stx50 stx.5.0 stx.6.0 stx.ha |
|
| 2021-06-03 18:36:43 |
OpenStack Infra |
cve linked |
|
2020-15705 |
|
