Nexenta driver doesn't use configurations in cinder.conf

[Blake Lai]

Hi,

I just find that the nexenta driver doesn't work when creating volume,
and I found the following error messages in cinder-volume.log.

2013-05-14 14:26:04 INFO [cinder.service] Started child 25710
2013-05-14 14:26:04 AUDIT [cinder.service] Starting cinder-volume node (version 2013.1)
2013-05-14 14:26:14 ERROR [cinder.service] Unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 224, in _start_child
    self._child_process(wrap.server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 201, in _child_process
    launcher.run_server(server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 95, in run_server
    server.start()
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 342, in start
    self.manager.init_host()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 143, in init_host
    self.driver.check_for_setup_error()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 99, in check_for_setup_error
    if not self.nms.volume.object_exists(FLAGS.nexenta_volume):
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/jsonrpc.py", line 66, in __call__
    response_obj = urllib2.urlopen(request)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno -2] Name or service not known>

Then I add some debug code in the function check_for_setup_error() of NexentaDriver and found that
the host IP is not setting right in the URL string.
I am sure my settings in cinder.conf is right. The following is the settings in my cinder.conf.

enabled_backends = nexenta
[nexenta]
volume_driver = cinder.volume.drivers.nexenta.volume.NexentaDriver
volume_backend_name = NexentaStor
nexenta_host = 10.144.138.101
nexenta_rest_port = 2000
nexenta_user = admin
nexenta_password = nexenta
nexenta_volume = cinder

So I try to figure it out what's happened, and here's what I found.
I compared the code of the function check_for_setup_error() in LVMVolumeDriver and NexentaDriver.
I found that LVMVolumeDriver would call self.configuration.append_config_values() when initializing
and use self.configuration to access the settings in the function check_for_setup_error(), but NexentaDriver
doesn't and use FLAGS instead.
Thus I tried modify the NexentaDriver using the same style accessing settings in LVMVolumeDriver and it worked.
I don't know whether this is a bug, hope you could check it and my patch is in attachment. Thanks.

[Thomas Kapolnasi (thomas-kapolnasi)]

Hi Blake,

Thank you for the patch although after I have an other errors in cinder-volumes.log:

2013-05-14 16:51:43 INFO [cinder.service] Child 2261 exited with status 2
2013-05-14 16:51:43 INFO [cinder.service] Started child 2262
2013-05-14 16:51:43 AUDIT [cinder.service] Starting cinder-volume node (version 2013.1)
2013-05-14 16:51:43 INFO [cinder.volume.drivers.nexenta.jsonrpc] Auto switching to HTTPS connection to http://192.168.11.161:2000/rest/nms/
2013-05-14 16:51:43 ERROR [cinder.service] Unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 224, in _start_child
    self._child_process(wrap.server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 201, in _child_process
    launcher.run_server(server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 95, in run_server
    server.start()
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 342, in start
    self.manager.init_host()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 143, in init_host
    self.driver.check_for_setup_error()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 100, in check_for_setup_error
    FLAGS.nexenta_volume)
LookupError: (u'Volume %s does not exist in Nexenta SA', 'cinder')
2013-05-14 16:51:43 INFO [cinder.service] Child 2262 exited with status 2
2013-05-14 16:51:43 INFO [cinder.service] Forking too fast, sleeping

I repaired in nexenta volume.py file to this:

    def check_for_setup_error(self):
        """Verify that the volume for our zvols exists.

        :raise: :py:exc:`LookupError`
        """
        if not self.nms.volume.object_exists(self.configuration.nexenta_volume):
            raise LookupError(_("Volume %s does not exist in Nexenta SA"),
                              self.configuration.nexenta_volume)

2013-05-14 17:06:15 AUDIT [cinder.service] Starting cinder-volume node (version 2013.1)
2013-05-14 17:06:15 INFO [cinder.volume.drivers.nexenta.jsonrpc] Auto switching to HTTPS connection to http://192.168.11.161:2000/rest/nms/
2013-05-14 17:06:16 INFO [cinder.volume.manager] volume volume-48a881d8-bbf5-44d1-9b0d-9d9762d0e927: skipping export
2013-05-14 17:06:16 INFO [cinder.volume.manager] volume volume-965ecf55-f7f9-4c35-a78e-814d7e47d1ef: skipping export
2013-05-14 17:06:16 INFO [cinder.volume.manager] volume volume-c159177d-d1cc-4b7b-a8ae-905cc881b4cb: skipping export
2013-05-14 17:06:16 INFO [cinder.volume.manager] volume volume-ecfc2d58-766e-45eb-a885-b954f2d11102: skipping export
2013-05-14 17:06:16 INFO [cinder.volume.manager] Updating volume status
2013-05-14 17:06:16 INFO [cinder.openstack.common.rpc.common] Connected to AMQP server on 192.168.10.11:5672
2013-05-14 17:06:16 INFO [cinder.openstack.common.rpc.common] Connected to AMQP server on 192.168.10.11:5672
2013-05-14 17:06:19 INFO [cinder.volume.manager] Updating volume status
2013-05-14 17:06:33 INFO [cinder.volume.manager] volume volume-be558ca1-1007-48c6-97e1-03ce2b745afd: creating
2013-05-14 17:06:33 INFO [cinder.volume.driv...

[Blake Lai (blackxwhite)]

Hi Thomas,

I try to reproduce your situation, and found the same result you provide.
Then I try to replace all FLAGS. with self.configuration. in cinder/volume/drivers/nexenta/volume.py and there are new error messages.

2013-05-21 19:14:41 INFO [cinder.volume.manager] volume volume-c3f86e6a-db2b-441c-8507-0f531c1fcafe: creating
2013-05-21 19:14:41 ERROR [cinder.volume.manager] Error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 251, in create_volume\n image_location)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 168, in _create_volume\n model_update = self.driver.create_volume(volume_ref)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 123, in create_volume\n self._get_zvol_name(volume[\'name\']),\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 105, in _get_zvol_name\n return \'%s/%s\' % (self.configuration.nexenta_volume, volume_name)\n', "NameError: global name 'self' is not defined\n"]
2013-05-21 19:14:41 ERROR [cinder.volume.manager] volume volume-c3f86e6a-db2b-441c-8507-0f531c1fcafe: create failed
2013-05-21 19:14:41 ERROR [cinder.openstack.common.rpc.amqp] Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 430, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/dispatcher.py", line 133, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 282, in create_volume
    LOG.error(_("volume %s: create failed"), volume_ref['name'])
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 274, in create_volume
    model_update = self.driver.create_export(context, volume_ref)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 250, in create_export
    loc = self._do_export(_ctx, volume, ensure=False)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 193, in _do_export
    zvol_name = self._get_zvol_name(volume['name'])
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 105, in _get_zvol_name
    return '%s/%s' % (self.configuration.nexenta_volume, volume_name)
NameError: global name 'self' is not defined

So it seems the function with @staticmethod annotation must use FLAGS to access the configurations.
Since FLAGS.register_opts(nexenta_opts) only gets the default value, I suggest you two temporary solutions:
1. Use the default Nexenta Volume name (i.e cinder)
2. Change the default Nexenta Volume name used in cinder/volume/drivers/nexenta/volume.py
    cfg.StrOpt('nexenta_volume',
               default='ZPOOL_TEST',
               help='pool on SA that will hold all volumes'),

Because I am not the author of Cinder nor Nexenta Driver, I can no...

[adapaka bhavaniprasad (adapaka-prasad)]

Hi Blake,

i am also facing issue while nexenta configured with cinder in openstack grizzly version.

i have added few lines of code in my nextenta volume.py file as you suggested. i got resolve the url issue but now i m getting another error as shown below.

2013-07-24 07:11:32 INFO [cinder.service] Starting 1 workers
2013-07-24 07:11:32 INFO [cinder.service] Started child 7672
2013-07-24 07:11:32 AUDIT [cinder.service] Starting cinder-volume node (version 2013.1.2)
2013-07-24 07:11:38 ERROR [cinder.service] Unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 227, in _start_child
    self._child_process(wrap.server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 204, in _child_process
    launcher.run_server(server)
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 95, in run_server
    server.start()
  File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 355, in start
    self.manager.init_host()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 143, in init_host
    self.driver.check_for_setup_error()
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/volume.py", line 106, in check_for_setup_error
    if not self.nms.volume.object_exists(FLAGS.nexenta_volume):
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nexenta/jsonrpc.py", line 80, in __call__
    response = jsonutils.loads(response_data)
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/jsonutils.py", line 127, in loads
    return json.loads(s)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
2013-07-24 07:11:38 INFO [cinder.service] Child 7672 exited with status 2
2013-07-24 07:11:38 INFO [cinder.service] _wait_child 1
2013-07-24 07:11:38 INFO [cinder.service] wait wrap.failed True
~

I came to know that, the data present in "response_data" variable is not clear so that jsonutils package is not able to convert it to JSON format.

any idea on this?

[andrew yongjoon kong (sstrato-kong)]

after patching that, i don't have this type of errors.

[Blake Lai (blackxwhite)]

Hi,

I just spend some time to review the code and made another patch.

This time I substitute all the FLAGS usages by using self.configuration to access the configuration.
But this way would result a problem that the _get_xxx_name() methods in the driver couldn't use self.configuration since they are static methods.
So I change the _get_xxx_name() methods into normal class methods and it works.

Feel free to use the patch and I hope you enjoy that. Please let me know if there is problem.

[Sean McGinnis (sean-mcginnis)]

The Nexenta driver was removed in Kilo as not supported. Close as Invalid?

[Sean McGinnis (sean-mcginnis)]

Nexenta drivers have been readded, and this does not appear to be an issue anymore.