puppet-glance

Support for configuring oslo.coordination lock_path is missing

Bug #1980539 reported by Alan Bishop
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
puppet-glance
Fix Released
High
Alan Bishop

Bug Description

The problem is succinctly described in https://bugzilla.redhat.com/show_bug.cgi?id=2101688. The bottom line is that glance's cinder backend uses file locks (in os-brick), but puppet-glance does not support configuring oslo.coordination's lock_path.

The os-brick patch [1] breaks Glance's cinder backend unless the oslo.coordination lock_path is configured (it has no default value).

From the Red Hat bug:

Upstream change[1] made the connect_volume and disconnect_volume methods use external file locks hence making lock_path a mandatory option to be configured by os-brick consumers like glance and nova.

For glance, it is currently unset but also we should be careful while setting it since services residing in the same node should use the same lock path. Eg: if glance and cinder are in the same node then they should use the same lock path.

Following is the stack trace from glance logs:

2022-06-27 07:21:26.030 15 DEBUG os_brick.initiator.connectors.iscsi [req-85f45c14-9bc0-4d02-a225-dc59ad90555e 09319456712d4e3797395aa852ce8ec7 1e59b572049f4fb5a32e9ef81cf92b73 - default default] <== connect_volume: exception (0ms) RequiredOptError('lock_path') trace_logging_wrapper /usr/lib/python3.9/site-packages/os_brick/utils.py:157
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder [req-85f45c14-9bc0-4d02-a225-dc59ad90555e 09319456712d4e3797395aa852ce8ec7 1e59b572049f4fb5a32e9ef81cf92b73 - default default] Exception while accessing to cinder volume b9aac319-7ca2-4d4d-ab0e-e9fd681e6ca9.: oslo_config.cfg.RequiredOptError: value required for option lock_path in group [DEFAULT]
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder Traceback (most recent call last):
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/glance_store/_drivers/cinder.py", line 677, in _open_cinder_volume
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder device = conn.connect_volume(connection_info)
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 154, in trace_logging_wrapper
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder result = f(*args, **kwargs)
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 351, in inner
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder with lock(name, lock_file_prefix, external, lock_path,
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib64/python3.9/contextlib.py", line 119, in __enter__
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder return next(self.gen)
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 269, in lock
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder ext_lock = external_lock(name, lock_file_prefix, lock_path)
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 189, in external_lock
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder lock_file_path = _get_lock_path(name, lock_file_prefix, lock_path)
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder File "/usr/lib/python3.9/site-packages/oslo_concurrency/lockutils.py", line 183, in _get_lock_path
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder raise cfg.RequiredOptError('lock_path')
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder oslo_config.cfg.RequiredOptError: value required for option lock_path in group [DEFAULT]
2022-06-27 07:21:26.031 15 ERROR glance_store._drivers.cinder

[1] https://review.opendev.org/c/openstack/os-brick/+/817708

Alan Bishop (alan-bishop)
Changed in puppet-glance:
status: New → Incomplete
assignee: nobody → Alan Bishop (alan-bishop)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/puppet-glance/+/848468

Changed in puppet-glance:
status: Incomplete → In Progress
Takashi Kajinami (kajinamit)
Changed in puppet-glance:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (master)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/848468
Committed: https://opendev.org/openstack/puppet-glance/commit/6f61ecd9c5b030817d0255fd8fd485722f9556f4
Submitter: "Zuul (22348)"
Branch: master

commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd

Changed in puppet-glance:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/puppet-glance/+/848884

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/848884
Committed: https://opendev.org/openstack/puppet-glance/commit/ba4ada0c70b31277b8569b71d40692b17c2a632b
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit ba4ada0c70b31277b8569b71d40692b17c2a632b
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/puppet-glance/+/848968

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/848968
Committed: https://opendev.org/openstack/puppet-glance/commit/1d37e5400aa1731e28a39ee9bfcbb84173d0690a
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 1d37e5400aa1731e28a39ee9bfcbb84173d0690a
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)
    (cherry picked from commit ba4ada0c70b31277b8569b71d40692b17c2a632b)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/puppet-glance/+/849508

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/849508
Committed: https://opendev.org/openstack/puppet-glance/commit/4ab30d3bc60c29c2d489c8b9a61b9080f1f6c829
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 4ab30d3bc60c29c2d489c8b9a61b9080f1f6c829
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)
    (cherry picked from commit ba4ada0c70b31277b8569b71d40692b17c2a632b)
    (cherry picked from commit 1d37e5400aa1731e28a39ee9bfcbb84173d0690a)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/puppet-glance/+/849712

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/849712
Committed: https://opendev.org/openstack/puppet-glance/commit/8ce29aa7c543f33fff41f3100b1255fe6468cf80
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 8ce29aa7c543f33fff41f3100b1255fe6468cf80
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)
    (cherry picked from commit ba4ada0c70b31277b8569b71d40692b17c2a632b)
    (cherry picked from commit 1d37e5400aa1731e28a39ee9bfcbb84173d0690a)
    (cherry picked from commit 4ab30d3bc60c29c2d489c8b9a61b9080f1f6c829)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/puppet-glance/+/850360

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/850360
Committed: https://opendev.org/openstack/puppet-glance/commit/b3964a75ba9fb1ede8725c3146ca052ed589b940
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit b3964a75ba9fb1ede8725c3146ca052ed589b940
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)
    (cherry picked from commit ba4ada0c70b31277b8569b71d40692b17c2a632b)
    (cherry picked from commit 1d37e5400aa1731e28a39ee9bfcbb84173d0690a)
    (cherry picked from commit 4ab30d3bc60c29c2d489c8b9a61b9080f1f6c829)
    (cherry picked from commit 8ce29aa7c543f33fff41f3100b1255fe6468cf80)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-glance (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/puppet-glance/+/851222

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-glance (stable/train)

Reviewed: https://review.opendev.org/c/openstack/puppet-glance/+/851222
Committed: https://opendev.org/openstack/puppet-glance/commit/d54e9429410bc763d16b63a4cd19148e181558df
Submitter: "Zuul (22348)"
Branch: stable/train

commit d54e9429410bc763d16b63a4cd19148e181558df
Author: Alan Bishop <email address hidden>
Date: Fri Jul 1 10:42:00 2022 -0700

    Configure the oslo.concurrency lock_path

    A recent change in os-brick [1], which is used by cinder backends,
    requires external file locks. This patch adds support for configuring
    the lock path, with a default value that matches the corresponding
    lock paths used by other services that use os-brick (namely cinder
    and nova).

    [1] I6f7f7d19540361204d4ae3ead2bd6dcddb8fcd68

    Closes-Bug: #1980539
    Change-Id: I5ea6766656f4227a1c514777c03bbf78d4ac59cd
    (cherry picked from commit 6f61ecd9c5b030817d0255fd8fd485722f9556f4)
    (cherry picked from commit ba4ada0c70b31277b8569b71d40692b17c2a632b)
    (cherry picked from commit 1d37e5400aa1731e28a39ee9bfcbb84173d0690a)
    (cherry picked from commit 4ab30d3bc60c29c2d489c8b9a61b9080f1f6c829)
    (cherry picked from commit 8ce29aa7c543f33fff41f3100b1255fe6468cf80)
    (cherry picked from commit b3964a75ba9fb1ede8725c3146ca052ed589b940)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance 21.0.0

This issue was fixed in the openstack/puppet-glance 21.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance 18.6.0

This issue was fixed in the openstack/puppet-glance 18.6.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance 19.5.0

This issue was fixed in the openstack/puppet-glance 19.5.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance 20.4.0

This issue was fixed in the openstack/puppet-glance 20.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance victoria-eol

This issue was fixed in the openstack/puppet-glance victoria-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance ussuri-eol

This issue was fixed in the openstack/puppet-glance ussuri-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-glance train-eol

This issue was fixed in the openstack/puppet-glance train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.