Scenario GlanceImages.create_image_and_boot_instances hangs forever in Rally v.0.8.1 and v0.9.0

Bug #1677031 reported by Tetiana Korchak on 2017-03-28
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Rally
Undecided
Unassigned

Bug Description

Issue: Rally hangs and doesn't throw any exceptions.
This issues happens when I used URL address for image_location:
https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-i386-disk.img
After image was downloaded and in scenarios image_location was changed to path to file - scenario passed.

At the same time when I use Rally v0.7.0 this scenario does not hang but return TimeoutException error
(TimeoutException: Rally tired waiting for Image s_rally_272af06d_q5lpE0F0:28cac912-3bc4-4b84-866a-9a81355bf101 to become ('ACTIVE') current status SAVING)

Scenario
---
  GlanceImages.create_image_and_boot_instances:
    -
      args:
        image_location: "https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-i386-disk.img"
        container_format: "bare"
        disk_format: "qcow2"
        flavor:
            name: "m1.tiny"
        number_instances: 2
      runner:
        type: "constant"
        times: 20
        concurrency: 5
      context:
        users:
          tenants: 5
          users_per_tenant: 5

Here is a list of errors that I could collect
Error 1. Rally scenario hangs with image in a "Queued" status and size 0 KB
Report - No report because task hangs, 'rally task abort' also hangs so report was not generated
Error from Fuel logs
-------------------------------------------------------------------------------------------
INFO eventlet.wsgi.server [req-b3aaacf9-69da-47b7-8ed8-36c512148f1b e8e6098c1b0f4e378565c304be1317cd 9615768486044d64a4399595138c00f2 - - -] Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 512, in handle_one_response
    write(b''.join(towrite))
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 453, in write
    wfile.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 385, in sendall
    tail = self.send(data, flags)
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 379, in send
    return self._send_loop(self.fd.send, data, flags)
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 366, in _send_loop
    return send_method(data, *args)
error: [Errno 32] Broken pipe
glance.api.v2.image_data [req-b3aaacf9-69da-47b7-8ed8-36c512148f1b e8e6098c1b0f4e378565c304be1317cd 9615768486044d64a4399595138c00f2 - - -] Failed to upload image data due to HTTP error
-------------------------------------------------------------------------------------------
Error 2. Rally scenario hangs and shows these errors

2017-03-20 18:32:11.110 22858 INFO rally.task.runner [-] Task d88257da-af03-487f-b778-cb30a9beefd7 | ITER: 3 END: Error ConnectionError: ('Connection aborted.', BadStatusLine("''",))
2017-03-20 21:25:09.041 30661 INFO rally.task.runner [-] Task 247ec505-5c4e-461a-831e-4c08db2ea50c | ITER: 12 END: Error SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rally/task/runner.py", line 72, in _run_scenario_once
    getattr(scenario_inst, method_name)(**scenario_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/glance/images.py", line 146, in run
    **create_image_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/task/atomic.py", line 84, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/glance/utils.py", line 64, in _create_image
    disk_format, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/wrappers/glance.py", line 190, in create_image
    self.client.images.upload(image.id, image_data)
  File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 217, in upload
    self.http_client.put(url, headers=hdrs, data=body)
  File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 185, in put
    return self.request(url, 'PUT', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 331, in request
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 101, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/positional/__init__.py", line 101, in inner
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 616, in request
    resp = send(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 674, in _send_request
    resp = self.session.request(method, url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 446, in send
    low_conn.send(i)
  File "/usr/lib/python2.7/httplib.py", line 873, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 311, in sendall
    sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 301, in _send_until_done
    return self.connection.send(data)
  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 1256, in send
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 1166, in _raise_ssl_error
    raise SysCallError(errno, errorcode.get(errno))
SysCallError: (32, 'EPIPE')

-------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rally/task/runner.py", line 72, in _run_scenario_once
    getattr(scenario_inst, method_name)(**scenario_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/glance/images.py", line 104, in run
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/task/atomic.py", line 84, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/glance/utils.py", line 64, in _create_image
    disk_format, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/wrappers/glance.py", line 188, in create_image
    response = requests.get(image_location, stream=True)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)
-------------------------------------------------------------------------------------------

hai shi (haishi) wrote :

It looks like the ssl's problem. And I have no this site-package of `/usr/local/lib/python2.7/dist-packages/OpenSSL` in my local.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers