[astute] supervisorctl do not stop astute

[Vladimir Sharshov]

If we try to stop/restart astute daemon using supervisor in master node by command: 'supervisorctl stop astute', it will not stop process, but do not show any error. Bug consistently reproduced.

Example:
ps uax | grep astute
root 4869 0.0 0.1 108164 1424 ? S 07:21 0:00 /bin/bash /usr/bin/astuted -c /etc/astute/astuted.conf -l /var/log/astute/astute.log -w 7
root 4870 0.0 3.2 225816 33312 ? Sl 07:21 0:00 astute master
root 4896 0.0 2.9 360504 29936 ? Sl 07:21 0:00 astute worker[0]
root 4900 0.0 2.9 360504 29936 ? Sl 07:21 0:00 astute worker[1]
root 4903 0.0 2.9 360508 29940 ? Sl 07:21 0:00 astute worker[2]
root 4908 0.0 2.9 360512 29944 ? Sl 07:21 0:00 astute worker[3]
root 4912 0.0 2.9 360516 29956 ? Sl 07:21 0:00 astute worker[4]
root 4916 0.0 2.9 360520 29960 ? Sl 07:21 0:00 astute worker[5]
root 4919 0.0 2.9 360524 29968 ? Sl 07:21 0:00 astute worker[6]
root 5720 0.0 0.0 103268 888 pts/0 S+ 07:33 0:00 grep astute
[root@fuel astute]# supervisorctl stop astute
astute: stopped
[root@fuel astute]# ps uax | grep astute
root 4870 0.0 3.2 225944 33424 ? Sl 07:21 0:00 astute master
root 4896 0.0 2.9 360504 29936 ? Sl 07:21 0:00 astute worker[0]
root 4900 0.0 2.9 360504 29936 ? Sl 07:21 0:00 astute worker[1]
root 4903 0.0 2.9 360508 29940 ? Sl 07:21 0:00 astute worker[2]
root 4908 0.0 2.9 360512 29944 ? Sl 07:21 0:00 astute worker[3]
root 4912 0.0 2.9 360516 29956 ? Sl 07:21 0:00 astute worker[4]
root 4916 0.0 2.9 360520 29960 ? Sl 07:21 0:00 astute worker[5]
root 4919 0.0 2.9 360524 29968 ? Sl 07:21 0:00 astute worker[6]
root 5751 0.0 0.0 103268 888 pts/0 S+ 07:35 0:00 grep astute

Most interesting part:
Process /bin/bash /usr/bin/astuted -c /etc/astute/astuted.conf -l /var/log/astute/astute.log -w 7 killed successfully by supervisorctl,
but do not kill his children.

[Bogdan Dobrelya (bogdando)]

Please elaborate that ISO should be used to reproduce the issue

[Bogdan Dobrelya (bogdando)]

{"build_id": "2014-04-17_15-38-45", "mirantis": "yes", "build_number": "442", "nailgun_sha": "3c49ef0d840d05f258ae88d39656338b0a45c84a", "production": "dev", "ostf_sha": "1ac04769cc8034438ccf1176860b25d0f5601c4a", "fuelmain_sha": "9890d1afdea85343de40fdfb1d91ddab9340082e", "astute_sha": "0cb0b26773bd269000a14748e8ba49419462ecaf", "release": "5.0", "fuellib_sha": "6b948385aad61486a52f3b24609f1d46aa52853e"}

reproduced on gerrit 442 ISO

[Bogdan Dobrelya (bogdando)]

Looks like restart is also broken, see http://pastebin.com/fmDYvt0c
If I issue supervisorctl restart astute consecutively, new bunch of master and workers would have spawned under /usr/bin/astuted main process, but old ones would not be killed!

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: master
Review: https://review.openstack.org/90077

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/90077
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=843f6fc8552c8414a4f2a16e8be724445a38ede3
Submitter: Jenkins
Branch: master

commit 843f6fc8552c8414a4f2a16e8be724445a38ede3
Author: Vladimir Kuklin <email address hidden>
Date: Thu Apr 24 17:18:40 2014 +0400

    kill all astute processes on stop

    change stopsignal and ask supervisord
    to killall all astute processes on stop

    Change-Id: I9968c0b68c05d63afdaae135fd85fcca6fdb648d
    Closes-Bug: #1310946