Charm does not ensure that iscsid is running

[Tiago Pasqualini da Silva]

Currently, nova-compute charm does not ensure that iscsid is being running. If this service is not running, iSCSI volumes cannot attach.

Deploying nova-compute charm on a bionic server reproduces this error:

# systemctl status iscsid
● iscsid.service - iSCSI initiator daemon (iscsid)
   Loaded: loaded (/lib/systemd/system/iscsid.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:iscsid(8)

Log trying to attach a volume:

Failed to attach df4cb304-756c-42be-a7c1-752adf238105 at /dev/vdb: cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid input received: Connector doesn't have required information: initiator). (HTTP 500) (Request-ID: req-42d25dbb-2556-449d-ba87-9a267cfcbd0f)
Traceback (most recent call last):
 File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 5506, in _attach_volume
   do_driver_attach=True)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 46, in wrapped
   ret_val = method(obj, context, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 639, in attach
   virt_driver, do_driver_attach)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 617, in _do_attach
   do_driver_attach)
 File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 530, in _volume_attach
   self['mount_device'])['connection_info']
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 383, in wrapper
   res = method(self, ctx, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 419, in wrapper
   res = method(self, ctx, attachment_id, *args, **kwargs)
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 826, in attachment_update
   'code': getattr(ex, 'code', None)})
 File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
   self.force_reraise()
 File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
   six.reraise(self.type_, self.value, self.tb)
 File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
   raise value
 File "/usr/lib/python3/dist-packages/nova/volume/cinder.py", line 816, in attachment_update
   attachment_id, _connector)
 File "/usr/lib/python3/dist-packages/cinderclient/v3/attachments.py", line 67, in update
   resp = self._update('/attachments/%s' % id, body)
 File "/usr/lib/python3/dist-packages/cinderclient/base.py", line 344, in _update
   resp, body = self.api.client.put(url, body=body, **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 206, in put
   return self._cs_request(url, 'PUT', **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 191, in _cs_request
   return self.request(url, method, **kwargs)
 File "/usr/lib/python3/dist-packages/cinderclient/client.py", line 177, in request
   raise exceptions.from_response(resp, body)
cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid input received: Connector doesn't have required information: initiator). (HTTP 500) (Request-ID: req-42d25dbb-2556-449d-ba87-9a267cfcbd0f)

After manually starting iscsid, I can attach an iscsi volume without any issues.

[James Page (james-page)]

I'm pretty sure that nova has made use of the native userspace support for iscsi built into qemu; this was introduced a long time ago, making any dependency on kernel based integration and associated daemon obsolete.

If you are seeing an issue please provide logs from the nova-compute daemon with a full stacktrace.

Marking Incomplete.

[Tiago Pasqualini da Silva (tiago.pasqualini)]

James, I updated the description with the logs from nova-compute.

[Tiago Pasqualini da Silva (tiago.pasqualini)]

Patch pushed to gerrit: https://review.openstack.org/#/c/639411/

[Sahid Orentino (sahid-ferdjaoui)]

James, are you referring to the SCSI backend used by QEMU? I think we always have referred to iscsiadm to support transport.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/639411
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=3b91afa9ce69f838789d6c0c3c671c15b061d813
Submitter: Zuul
Branch: master

commit 3b91afa9ce69f838789d6c0c3c671c15b061d813
Author: tpsilva <email address hidden>
Date: Tue Feb 26 14:42:20 2019 -0300

    Added verification to ensure iscsid is running

    Currently, the charm does not verify if iscsid is installed or
    running. This patch adds those verifications on install and
    update hooks.

    Closes-bug: #1816435

    Change-Id: I23992832a82557f406999427fe8d151f6a2b63af