| 2020-01-24 19:50:56 |
Ghada Khalil |
description |
Brief Description
-----------------
After process:hbsClient was killed, fm cmd failed by Unable to establish connection to http://192.168.204.1:5000/v3/auth/tokens
Severity
--------
Major
Steps to Reproduce
------------------
kill hbsClient
run fm cmd
TC-name: mtc/test_process_monitoring.py::test_process_monitoring[hbsClient]
Expected Behavior
------------------
fm cnd is working as usual
Actual Behavior
----------------
fm cmd not working
Reproducibility
---------------
Unknown - first time this is seen in sanity, will monitor
System Configuration
--------------------
One node system
Lab-name: SM-4
Branch/Pull Time/Commit
-----------------------
20200121T000002Z
Last Pass
---------
20191210T000000Z
Timestamp/Logs
--------------
2020-01-21 11:05:03,439] 314 DEBUG MainThread ssh.send :: Send 'cat >/home/sysadmin//test_process.sh <<EOL
true; n=1; last_pid=''; pid=''; for((;n<4;)); do pid=\$(cat /var/run/hbsClient.pid 2>/dev/null); date;
if [ "x\$pid" = "x" -o "\$pid" = "\$last_pid" ]; then echo "stale or empty PID:\$pid, last_pid=\$last_pid";
sleep 0.5; continue; fi; echo "Li69nux*" | sudo -S kill -9 \$pid &>/dev/null;
if [ \$? -eq 0 ]; then echo "OK \$n - \$pid killed"; ((n++)); last_pid=\$pid; pid=''; sleep 15;
else sleep 0.5; fi; done; echo \$pid
EOL'
[2020-01-21 11:05:03,872] 314 DEBUG MainThread ssh.send :: Send 'nohup ./test_process.sh > ./results.txt 2>&1 &'
[2020-01-21 11:11:22,146] 314 DEBUG MainThread ssh.send :: Send 'fm --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne event-list -l 10 -q 'event_log_id=200.006;start=2020-01-21T10:41:48.295439' --nowrap --nopaging --uuid'
[2020-01-21 11:11:22,890] 436 DEBUG MainThread ssh.expect :: Output:
Must provide Keystone credentials or user-defined endpoint and token, error was: Unable to establish connection to http://192.168.204.1:5000/v3/auth/tokens: HTTPConnectionPool(host='192.168.204.1', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa042e60650>: Failed to establish a new connection: [Errno 111] Connection refused',))
controller-0:~$
Test Activity
-------------
Sanity |
Brief Description
-----------------
After process:hbsClient was killed, fm cmd failed by Unable to establish connection to http://192.168.204.1:5000/v3/auth/tokens
Severity
--------
Major
Steps to Reproduce
------------------
kill hbsClient
run fm cmd
TC-name: mtc/test_process_monitoring.py::test_process_monitoring[hbsClient]
Expected Behavior
------------------
fm cnd is working as usual
Actual Behavior
----------------
fm cmd not working
Reproducibility
---------------
Unknown - first time this is seen in sanity, will monitor
System Configuration
--------------------
One node system
Lab-name: SM-4
Branch/Pull Time/Commit
-----------------------
master 20200121T000002Z
Last Pass
---------
20191210T000000Z
Timestamp/Logs
--------------
2020-01-21 11:05:03,439] 314 DEBUG MainThread ssh.send :: Send 'cat >/home/sysadmin//test_process.sh <<EOL
true; n=1; last_pid=''; pid=''; for((;n<4;)); do pid=\$(cat /var/run/hbsClient.pid 2>/dev/null); date;
if [ "x\$pid" = "x" -o "\$pid" = "\$last_pid" ]; then echo "stale or empty PID:\$pid, last_pid=\$last_pid";
sleep 0.5; continue; fi; echo "Li69nux*" | sudo -S kill -9 \$pid &>/dev/null;
if [ \$? -eq 0 ]; then echo "OK \$n - \$pid killed"; ((n++)); last_pid=\$pid; pid=''; sleep 15;
else sleep 0.5; fi; done; echo \$pid
EOL'
[2020-01-21 11:05:03,872] 314 DEBUG MainThread ssh.send :: Send 'nohup ./test_process.sh > ./results.txt 2>&1 &'
[2020-01-21 11:11:22,146] 314 DEBUG MainThread ssh.send :: Send 'fm --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.1:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne event-list -l 10 -q 'event_log_id=200.006;start=2020-01-21T10:41:48.295439' --nowrap --nopaging --uuid'
[2020-01-21 11:11:22,890] 436 DEBUG MainThread ssh.expect :: Output:
Must provide Keystone credentials or user-defined endpoint and token, error was: Unable to establish connection to http://192.168.204.1:5000/v3/auth/tokens: HTTPConnectionPool(host='192.168.204.1', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa042e60650>: Failed to establish a new connection: [Errno 111] Connection refused',))
controller-0:~$
Test Activity
-------------
Sanity |
|
