When running the ns-metadata-proxy cleanup tasks, we get the following error sometimes:
13:35:48 failed: [jrpcaioiad-3a8_neutron_agents_container-101ec306] => {"changed": true, "cmd": "for ns_pid in $(pgrep neutron-ns-meta); do\n echo $(readlink -f \"/proc/$ns_pid/exe\") | grep -qv \"13.3.4\"\n if [ $? -eq 0 ]; then\n (echo \"old metadata proxy pid found running clean up on $ns_pid\"; kill -9 \"$ns_pid\")\n fi\n done", "delta": "0:00:00.049977", "end": "2016-09-23 13:35:48.950575", "rc": 1, "start": "2016-09-23 13:35:48.900598", "warnings": []}
13:35:48 stderr: /bin/sh: 4: kill: No such process
13:35:48
13:35:48 /bin/sh: 4: kill: No such process
13:35:48 stdout: old metadata proxy pid found running clean up on 1337
13:35:48 old metadata proxy pid found running clean up on 2781
It appears that the PIDs found during the pgrep are sometimes lost before the kill. Either that or the filtering has some subtle bug resulting in an output that's not a valid PID or PIDs.
I suggest that we instead reverse the order of tasks in this handler, replacing the complex pgrep/grep/kill logic with a simple pkill of all the proxies, followed by the service restart. I'll submit a patch for consideration soon.
In which version does this happen? Latest mitaka I guess?
Could you confirm the process were running fine before the neutron playbook run?