Quantum L3 agent is using /usr/share/pyshared for locking

[Endre Karlson]

2013-01-28 15:35:21.702 734 DEBUG quantum.agent.linux.utils [-] Running command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-a4c9fef2-eea2-4a9a-b5b1-53c86b188a16', 'sysctl', '-w', 'net.ipv4.ip_forward=1'] execute /usr/lib/python2.7/dist-packages/quantum/agent/linux/utils.py:41
2013-01-28 15:35:21.853 734 DEBUG quantum.agent.linux.utils [-]
Command: ['sudo', '/usr/bin/quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-a4c9fef2-eea2-4a9a-b5b1-53c86b188a16', 'sysctl', '-w', 'net.ipv4.ip_forward=1']
Exit code: 0
Stdout: 'net.ipv4.ip_forward = 1\n'
Stderr: '' execute /usr/lib/python2.7/dist-packages/quantum/agent/linux/utils.py:58
2013-01-28 15:35:21.855 734 DEBUG quantum.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /usr/lib/python2.7/dist-packages/quantum/openstack/common/lockutils.py:186
2013-01-28 15:35:21.856 734 DEBUG quantum.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /usr/lib/python2.7/dist-packages/quantum/openstack/common/lockutils.py:190
2013-01-28 15:35:21.857 734 DEBUG quantum.openstack.common.lockutils [-] Released file lock "iptables" at /usr/lib/python2.7/dist-packages/quantum/openstack/quantum-iptables for method "_apply"... inner /usr/lib/python2.7/dist-packages/quantum/openstack/common/lockutils.py:227
2013-01-28 15:35:21.858 734 ERROR quantum.agent.l3_agent [-] Failed synchronizing routers
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent Traceback (most recent call last):
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/agent/l3_agent.py", line 603, in _sync_routers_task
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent self._process_routers(routers)
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/agent/l3_agent.py", line 584, in _process_routers
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent self._router_added(r['id'])
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/agent/l3_agent.py", line 228, in _router_added
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent ri.iptables_manager.apply()
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/agent/linux/iptables_manager.py", line 291, in apply
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent self._apply()
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/lockutils.py", line 215, in inner
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent with lock:
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/lockutils.py", line 75, in __enter__
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent self.lockfile = open(self.fname, 'w')
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent IOError: [Errno 13] Permission denied: '/usr/lib/python2.7/dist-packages/quantum/openstack/quantum-iptables'
2013-01-28 15:35:21.858 734 TRACE quantum.agent.l3_agent

gonysh @ freenode had the same problem with the lock occuring in the lib dir instead of in say /var/lock or /var/X...

Is this intentional during development or?

[Endre Karlson (endre-karlson)]

I can see that the "lock_path" option is defaulted to "../" should it be like this?

[Maru Newby (maru)]

A better default might be '' (empty string), since then the lock code will use a temporary directory to store locks in. However, note that the lock_path default is defined by openstack common code, and this issue should probably be addressed in openstack common rather than quantum directly.

[Jason Kölker (jason-koelker)]

Fix incoming to quantum to override the default. Also targeted to oslo.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/20650

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/20651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/20650
Committed: http://github.com/openstack/quantum/commit/d5bfd9106148eec91f1109ea7f88b6fb41cd69d5
Submitter: Jenkins
Branch: master

commit d5bfd9106148eec91f1109ea7f88b6fb41cd69d5
Author: Jason Kölker <email address hidden>
Date: Mon Jan 28 17:06:11 2013 -0600

    Set default lock_path in state_path

    * Set the default lock_path to be within quantum's state_path
    * Fixes bug 1107950

    Change-Id: I346d6088d773b508b4a7ba925fdc91fd4f5bf160

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/20651
Committed: http://github.com/openstack/oslo-incubator/commit/febbd005151e2ade521eed027a7d5372e6e3c0bc
Submitter: Jenkins
Branch: master

commit febbd005151e2ade521eed027a7d5372e6e3c0bc
Author: Jason Kölker <email address hidden>
Date: Mon Jan 28 17:32:02 2013 -0600

    Default lockutils to using a tempdir

    * Previously lockutils attempted to use __file__/../ as the lock path.
      This causes packages to fail if the user running the code doesn't have
      +w access to that dir. Remove the default so it will use mkdtemp
      instead
    * Fixes bug 1107950

    Change-Id: Ie0cd6fac5403a53a768eb4a0ce7da602b3fef755