missed option lock_path in cinder.conf breaks cinder-volume service

[Dennis Dmitriev]

cinder-volume failed to start due to missing lock_path option which is required option in kilo package

10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:42:06.792 14601 TRACE oslo_messaging.rpc.dispatcher ext_lock = external_lock(name, lock_file_prefix, lock_path)
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:42:06.792 14601 TRACE oslo_messaging.rpc.dispatcher lock_file_path = _get_lock_path(name, lock_file_prefix, lock_path)
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:42:06.792 14601 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 325, in _get_lock_path
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:42:06.792 14601 TRACE oslo_messaging.rpc.dispatcher raise cfg.RequiredOptError('lock_path')
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:42:06.792 14601 TRACE oslo_messaging.rpc.dispatcher RequiredOptError: value required for option: lock_path
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:44:26.417 14601 TRACE oslo_messaging.rpc.dispatcher ext_lock = external_lock(name, lock_file_prefix, lock_path)
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:44:26.417 14601 TRACE oslo_messaging.rpc.dispatcher lock_file_path = _get_lock_path(name, lock_file_prefix, lock_path)
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:44:26.417 14601 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 325, in _get_lock_path
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:44:26.417 14601 TRACE oslo_messaging.rpc.dispatcher raise cfg.RequiredOptError('lock_path')
10.109.5.2/var/log/docker-logs/remote/node-5.test.domain.local/cinder-volume.log:2015-08-06 21:44:26.417 14601 TRACE oslo_messaging.rpc.dispatcher RequiredOptError: value required for option: lock_path

Upstream change: https://github.com/openstack/puppet-cinder/commit/d5c38006e8cd2690088555b93db29e69e8b507c1

root@node-2:/var/log/cinder# cinder --version
1.1.2
root@node-2:/var/log/cinder# ceph --version
ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047)

api: '1.0'
astute_sha: e1d3a435e5df5b40cbfb1a3acf80b4176d15a2dc
auth_required: true
build_id: 2015-08-06_06-45-26
build_number: '135'
feature_groups:
- experimental
fuel-agent_sha: 1fe47720ba554818a0be707f2e16281791492d50
fuel-library_sha: 347d51f391a6f2c621006c00e3416718140889ac
fuel-nailgun-agent_sha: 1512b9af6b41cc95c4d891c593aeebe0faca5a63
fuel-ostf_sha: 5cb828f6200e5ae0da63dd39d11fbf910237db95
fuelmain_sha: 858a4ee8336789c10c5c2189c3718f9a605d0eeb
nailgun_sha: c842770d56680d0e0cdf3573d329ce36d4fb2c64
openstack_version: 2015.1.0-7.0
production: docker
python-fuelclient_sha: 28ddd022fd230fde0e88f2f3e4a6a6dddcb58abe
release: '7.0'
release_versions:
  2015.1.0-7.0:
    VERSION:
      api: '1.0'
      astute_sha: e1d3a435e5df5b40cbfb1a3acf80b4176d15a2dc
      build_id: 2015-08-06_06-45-26
      build_number: '135'
      feature_groups:
      - experimental
      fuel-agent_sha: 1fe47720ba554818a0be707f2e16281791492d50
      fuel-library_sha: 347d51f391a6f2c621006c00e3416718140889ac
      fuel-nailgun-agent_sha: 1512b9af6b41cc95c4d891c593aeebe0faca5a63
      fuel-ostf_sha: 5cb828f6200e5ae0da63dd39d11fbf910237db95
      fuelmain_sha: 858a4ee8336789c10c5c2189c3718f9a605d0eeb
      nailgun_sha: c842770d56680d0e0cdf3573d329ce36d4fb2c64
      openstack_version: 2015.1.0-7.0
      production: docker
      python-fuelclient_sha: 28ddd022fd230fde0e88f2f3e4a6a6dddcb58abe
      release: '7.0'

[Marek Zawadzki (mzawadzki-f)]

Isn't it this error related as well: https://bugs.launchpad.net/fuel/+bug/1472675

[Ivan Kolodyazhny (e0ne)]

From cinder-volume logs: http://paste.openstack.org/show/411172/

[Ivan Kolodyazhny (e0ne)]

There is no 'lock_path = /var/lock/cinder' param in cinder conf as we've got in older ISOs[1]. So Cinder can't delete a volume

[1]
  release: "7.0"
 openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "104"

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/209945

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/209945
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=9e664196ba55c538a2bbb85c64c7c9ac0b90dc21
Submitter: Jenkins
Branch: master

commit 9e664196ba55c538a2bbb85c64c7c9ac0b90dc21
Author: vsaienko <email address hidden>
Date: Thu Aug 6 16:59:30 2015 +0300

    add lock_path option for cinder

     The lock_path variable is required by cinder in Kilo
     and cinder-volume will fail to start if it does not exist.

     upstream change-id: d5c38006e8cd2690088555b93db29e69e8b507c1

    Change-Id: I53bede78a6ae36f42d37b18d3d01a106df66e071
    Closes-Bug: #1482192

[Dmitry Borodaenko (angdraug)]

Ivan, comments on https://review.openstack.org/209216 indicate that this should be fixed in the package, too.

[Ivan Kolodyazhny (e0ne)]

Need to verify that we have all required config options

[Aleksander Mogylchenko (amogylchenko)]

If it blocks CI, it should be Critical.

[Dmitry Borodaenko (angdraug)]

Aleksander, CI is unblocked by the fuel-library commit linked above, the bug is re-opened to fix it in the package, too.

[Ivan Kolodyazhny (e0ne)]

We've merged workaround for this issue in puppet manifests, so it won't block CI any more

[Ivan Kolodyazhny (e0ne)]

Fix for this issue affects a lot of cinder code which was merged to Liberty only. Since this feature is not used by our customer, I move this issue to MOS 8

[Ivan Kolodyazhny (e0ne)]

Fix was merged with the latest stable/liberty code

[Timur Nurlygayanov (tnurlygayanov)]

Verified on MOS 8.0 RC1:

root@node-1:~# cat /etc/cinder/cinder.conf | grep lock
lock_path=/var/lock/cinder
root@node-1:~#