neutron-ns-metadata-proxy cannot be killed when running in virtual environment

[Han Zhou]

== Problem ==
When running in virtual env, neutron-ns-metadata-proxy on DHCP node cannot be disabled properly when deleting a network.

== Impact ==
* Corresponding folder and files under /var/lib/neutron/dhcp/ are not cleaned up.
* If neutron-dhcp-agent service is restarted in this unclean state, dnsmasq processes are restarted periodically (resync interval), because the failure of removing the deleted networks. (And because of bug 1345947, huge amount of NAK logs flood for dnsmasq.)

== Root cause ==
Killing neutron-ns-metadata-proxy is failed because of rootwrapper filter not matched in dhcp_filter.conf. Related logs are:
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'kill', '-9', '65832']
Exit code: 99
Stdout: ''
Stderr: '/usr/bin/neutron-rootwrap: Unauthorized command: kill -9 65832 (no filter matched)\n'

neutron-ns-metadata-proxy is started via ProcessManager, and in Virtual Environment, the python binary under virtual env is used instead of /usr/bin/python*.

However, in dhcp_file.conf absolute path is specified for KillerFiler for neutron-ns-metadata-proxy:
kill_metadata: KillFilter, root, /usr/bin/python, -9
kill_metadata7: KillFilter, root, /usr/bin/python2.7, -9
kill_metadata6: KillFilter, root, /usr/bin/python2.6, -9

== Proposed solution ==
Do not specify absolute path in filter: use python (or python2.7, or python2.6) instead
======================================================================
diff --git a/etc/neutron/rootwrap.d/dhcp.filters b/etc/neutron/rootwrap.d/dhcp.filters
index 88d61e8..26c2ffa 100644
--- a/etc/neutron/rootwrap.d/dhcp.filters
+++ b/etc/neutron/rootwrap.d/dhcp.filters
@@ -29,9 +29,9 @@ metadata_proxy_quantum: CommandFilter, quantum-ns-metadata-proxy, root
 metadata_proxy_local: CommandFilter, /usr/local/bin/neutron-ns-metadata-proxy, root
 metadata_proxy_local_quantum: CommandFilter, /usr/local/bin/quantum-ns-metadata-proxy, root
 # RHEL invocation of the metadata proxy will report /usr/bin/python
-kill_metadata: KillFilter, root, /usr/bin/python, -9
-kill_metadata7: KillFilter, root, /usr/bin/python2.7, -9
-kill_metadata6: KillFilter, root, /usr/bin/python2.6, -9
+kill_metadata: KillFilter, root, python, -9
+kill_metadata7: KillFilter, root, python2.7, -9
+kill_metadata6: KillFilter, root, python2.6, -9

 # ip_lib
 ip: IpFilter, ip, root

[Eugene Nikanorov (enikanorov)]

Could you please post the fix on gerrit?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/118296

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/118296
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4de8a3cd94c6d65702dcda457653edc6c93a3722
Submitter: Jenkins
Branch: master

commit 4de8a3cd94c6d65702dcda457653edc6c93a3722
Author: Han Zhou <email address hidden>
Date: Tue Sep 2 09:54:05 2014 +0000

    Remove absolute path in KillFilter for metadata-proxy

    This commit fix the problem of killing neutron-ns-metadata-proxy
    when running in Virtual Environment.

    Change-Id: I987dd87e19c218846a48e58b61679b4153d97f66
    Closes-bug: #1363773

[Gaëtan Trellu (goldyfruit)]

Hi,

I'm still having the same issue with Mitaka (DHCP and L3).

Neutron is running as a VirtualEnv in /srv/data/virtualenvs/neutron/

The only way to get neutron-ns-metadata-proxy killed is to change the command in the l3.filters file:
================
kill_metadata: KillFilter, root, /srv/data/virtualenvs/neutron/bin/python, -9
================

The process looks like this:
================
/srv/data/virtualenvs/neutron/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/40e3c9bf-0108-4f48-b1ae-e39ea8c64ef2.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=40e3c9bf-0108-4f48-b1ae-e39ea8c64ef2 --state_path=/var/lib/neutron --metadata_port=9697 --metadata_proxy_user=2006 --metadata_proxy_group=2006 --debug --log-file=neutron-ns-metadata-proxy-40e3c9bf-0108-4f48-b1ae-e39ea8c64ef2.log --log-dir=/var/log/neutron
================

The systemd unit file used to start neutron-l3-agent:
================
[Unit]
Description=OpenStack Networking (code-named Neutron) L3 Agent
After=syslog.target network.target

[Service]
Type=simple
Restart=on-failure
RestartSec=5
User=neutron
Group=neutron
Environment=VIRTUAL_ENV=/srv/data/virtualenvs/neutron/
Environment=PATH=$VIRTUAL_ENV/bin:$PATH:/bin:/usr/bin:/sbin:/usr/sbin
ExecStart=/srv/data/virtualenvs/neutron/bin/neutron-l3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/l3_agent.ini --log-file /var/log/neutron/l3-agent.log
PrivateTmp=false
KillMode=process

[Install]
WantedBy=multi-user.target
================