Charm does not ensure that iscsid is running

Bug #1816435 reported by Tiago Pasqualini da Silva on 2019-02-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack nova-compute charm
Medium
Tiago Pasqualini da Silva

Bug Description

Currently, nova-compute charm does not ensure that iscsid is being running. If this service is not running, iSCSI volumes cannot attach.

Deploying nova-compute charm on a bionic server reproduces this error:

# systemctl status iscsid
● iscsid.service - iSCSI initiator daemon (iscsid)
   Loaded: loaded (/lib/systemd/system/iscsid.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:iscsid(8)

Log trying to attach a volume:

Failed to attach df4cb304-756c-42be-a7c1-752adf238105 at /dev/vdb: cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid input received: Connector doesn't have required information: initiator). (HTTP 500) (Request-ID: req-42d25dbb-2556-449d-ba87-9a267cfcbd0f)
Traceback (most recent call last):
 File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5506, in _attach_volume
   do_driver_attach=True)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 46, in wrapped
   ret_val = method(obj, context, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 639, in attach
   virt_driver, do_driver_attach)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 617, in _do_attach
   do_driver_attach)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 530, in _volume_attach
   self['mount_device'])['connection_info']
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 383, in wrapper
   res = method(self, ctx, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 419, in wrapper
   res = method(self, ctx, attachment_id, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 826, in attachment_update
   'code': getattr(ex, 'code', None)})
 File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
   self.force_reraise()
 File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
   six.reraise(self.type_, self.value, self.tb)
 File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
   raise value
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 816, in attachment_update
   attachment_id, _connector)
 File "/usr/lib/python3/dist-packages/cinderclient/v3/attachments.py", line 67, in update
   resp = self._update('/attachments/%s' % id, body)
 File "/usr/lib/python3/dist-packages/cinderclient/base.py", line 344, in _update
   resp, body = self.api.client.put(url, body=body, **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 206, in put
   return self._cs_request(url, 'PUT', **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 191, in _cs_request
   return self.request(url, method, **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 177, in request
   raise exceptions.from_response(resp, body)
cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid input received: Connector doesn't have required information: initiator). (HTTP 500) (Request-ID: req-42d25dbb-2556-449d-ba87-9a267cfcbd0f)

After manually starting iscsid, I can attach an iscsi volume without any issues.

James Page (james-page) wrote :

I'm pretty sure that nova has made use of the native userspace support for iscsi built into qemu; this was introduced a long time ago, making any dependency on kernel based integration and associated daemon obsolete.

If you are seeing an issue please provide logs from the nova-compute daemon with a full stacktrace.

Marking Incomplete.

Changed in charm-nova-compute:
status: New → Incomplete

James, I updated the description with the logs from nova-compute.

description: updated
Changed in charm-nova-compute:
status: Incomplete → New
assignee: nobody → Tiago Pasqualini da Silva (tiago.pasqualini)
Changed in charm-nova-compute:
importance: Undecided → Medium
status: New → In Progress
milestone: none → 19.04
David Ames (thedac) on 2019-04-17
Changed in charm-nova-compute:
milestone: 19.04 → 19.07

James, are you referring to the SCSI backend used by QEMU? I think we always have referred to iscsiadm to support transport.

Reviewed: https://review.opendev.org/639411
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=3b91afa9ce69f838789d6c0c3c671c15b061d813
Submitter: Zuul
Branch: master

commit 3b91afa9ce69f838789d6c0c3c671c15b061d813
Author: tpsilva <email address hidden>
Date: Tue Feb 26 14:42:20 2019 -0300

    Added verification to ensure iscsid is running

    Currently, the charm does not verify if iscsid is installed or
    running. This patch adds those verifications on install and
    update hooks.

    Closes-bug: #1816435

    Change-Id: I23992832a82557f406999427fe8d151f6a2b63af

Changed in charm-nova-compute:
status: In Progress → Fix Committed
David Ames (thedac) on 2019-08-12
Changed in charm-nova-compute:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers