kolla-ansible

Cinder-backup iscsi lvm backend

Bug #1838624 reported by Suff on 2019-08-01

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	kolla-ansible	Fix Released	Medium	Mark Goddard	kolla-ansible 9.0.0 "Train"
	Train	Fix Released	Medium	Mark Goddard	kolla-ansible 9.0.0 "Train"

Bug Description

Cinder-backup daemon can't access volumes on lvm backend due to missing iscsi daemon on cinder-backup hostgroup.

Traceback:
2019-07-31 22:30:21.757 6 ERROR os_brick.initiator.connectors.iscsi [req-e3f9c865-c109-4605-a3db-a8b941110cec 71f543e002b345299dca504b1cdba0a0 3ba70903aaf040db8bdc2c3710791151 - default default] Could not login to any iSCSI portal.
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server [req-e3f9c865-c109-4605-a3db-a8b941110cec 71f543e002b345299dca504b1cdba0a0 3ba70903aaf040db8bdc2c3710791151 - default default] Exception during message handling
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 317, in create_backup
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server self._update_backup_error(backup, six.text_type(err))
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in exit
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server self.force_reraise()
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 311, in create_backup
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server self._run_backup(context, backup, volume)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 348, in _run_backup
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server backup_device.is_snapshot)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 818, in _attach_device
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return self._attach_volume(ctxt, backup_device, properties)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 840, in _attach_volume
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server {'volume_id', volume.id})
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in exit
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server self.force_reraise()
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 829, in _attach_volume
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return self._connect_device(conn)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 870, in _conne
ct_device
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server vol_handle = connector.connect_volume(conn['data'])
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 61, in _wrapper
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 223, in call
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server return attempt.get(self._wrap_exception)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server six.reraise(self.value[0], self.value[1], self.value[2])
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 405, in connect_volume
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server connection_properties)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py", line 236, in _get_potential_volume_paths
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server raise exception.FailedISCSITargetPortalLogin(message=msg)
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server FailedISCSITargetPortalLogin: Could not login to any iSCSI portal.
2019-07-31 22:30:22.404 6 ERROR oslo_messaging.rpc.server

Proposal:
This patch is fix it

git diff ansible/roles/iscsi/defaults/main.yml
diff --git a/ansible/roles/iscsi/defaults/main.yml b/ansible/roles/iscsi/defaults/main.yml
index abf68e6..ab74fa0 100644
--- a/ansible/roles/iscsi/defaults/main.yml
+++ b/ansible/roles/iscsi/defaults/main.yml
@@ -61,7 +61,8 @@ tgtd_dimensions: "{{ default_container_dimensions }}"

enable_iscsid_for_cinder: >-
   {{ (inventory_hostname in groups['compute']
- or inventory_hostname in groups['cinder-volume'])
+ or inventory_hostname in groups['cinder-volume']
+ or inventory_hostname in groups['cinder-backup'])
      and enable_cinder | bool
      and enable_cinder_backend_iscsi | bool }}
enable_iscsid_for_ironic: >-

See original description

Tags:

Revision history for this message

Suff (dmitry-a-grachev) wrote on 2019-08-01:

patch Edit (685 bytes, text/plain)

Suff (dmitry-a-grachev) on 2019-08-01

description:

updated

Mark Goddard (mgoddard) on 2019-08-07

Changed in kolla-ansible:
importance:	Undecided → Medium
milestone:	none → 9.0.0

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-07: Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/675052

Changed in kolla-ansible:
assignee:	nobody → Mark Goddard (mgoddard)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-15: Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/675052
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=ec075240541996f812ba39bdb84f21ddfa01d77f
Submitter: Zuul
Branch: master

commit ec075240541996f812ba39bdb84f21ddfa01d77f
Author: Mark Goddard <email address hidden>
Date: Wed Aug 7 09:05:43 2019 +0100

Enable iscsid on cinder-backup hosts

Without this we may see the following error in cinder-backup when using
the LVM backend:

Could not login to any iSCSI portal

Enabling the iscsid container on hosts in the cinder-backup group fixes
this.

Closes-Bug: #1838624

Change-Id: If373c002b0744ce9dbdffed50a02bab55dd0acb9
Co-Authored-By: dmitry-a-grachev <email address hidden>

Changed in kolla-ansible:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-15: Fix proposed to kolla-ansible (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/676635

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-08-16: Fix merged to kolla-ansible (stable/stein)

Reviewed: https://review.opendev.org/676635
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=8cc5ae2f6f72a426dc1c6a334681742341964e23
Submitter: Zuul
Branch: stable/stein

commit 8cc5ae2f6f72a426dc1c6a334681742341964e23
Author: Mark Goddard <email address hidden>
Date: Wed Aug 7 09:05:43 2019 +0100

Enable iscsid on cinder-backup hosts

Without this we may see the following error in cinder-backup when using
the LVM backend:

Could not login to any iSCSI portal

Enabling the iscsid container on hosts in the cinder-backup group fixes
this.

Closes-Bug: #1838624

    Change-Id: If373c002b0744ce9dbdffed50a02bab55dd0acb9
    Co-Authored-By: dmitry-a-grachev <email address hidden>
    (cherry picked from commit ec075240541996f812ba39bdb84f21ddfa01d77f)