see "No space left on device" error while running cinder API concurrently
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Cinder |
Invalid
|
Undecided
|
Unassigned |
Bug Description
I have below scenario:
* Create a nova instance
* Associate floating ip
* Create cinder volume of 1 GB
* Attach cinder volume to above instance
* Perform file system operation like mount, unmount etc
* Detach cinder volume
* Delete instance
* Delete volume.
This above scenario works fine when I run it sequentially. Although if I run this scenario in parallel, with two threads doing this scenario at same time I see below error and traceback. I looked around openstack bug tracker and search on web in general. I could not find this being reported by anyone, so opening a bug for this. Note in my test setup I have enough memory to create more volumes and even compute nodes have plenty of memory.
I have attached cinder-all.log and nova-all.log with this bug also.
Below is the traceback:
<155>Sep 17 23:40:24 node-2 cinder-
Traceback (most recent call last):
File "/usr/lib/
return req.get_
File "/usr/lib/
application, catch_exc_
File "/usr/lib/
app_iter = application(
File "/usr/lib/
return resp(environ, start_response)
File "/usr/lib/
return self.app(env, start_response)
File "/usr/lib/
return resp(environ, start_response)
File "/usr/lib/
return resp(environ, start_response)
File "/usr/lib/
response = self.app(environ, start_response)
File "/usr/lib/
return resp(environ, start_response)
File "/usr/lib/
resp = self.call_func(req, *args, **self.kwargs)
File "/usr/lib/
return self.func(req, *args, **kwargs)
File "/usr/lib/
content_type, body, accept)
File "/usr/lib/
action_result = self.dispatch(meth, request, action_args)
File "/usr/lib/
return method(req=request, **action_args)
File "/usr/lib/
connector)
File "/usr/lib/
return func(self, context, target_obj, *args, **kwargs)
File "/usr/lib/
connector)
File "/usr/lib/
connector=
File "/usr/lib/
wait_
File "/usr/lib/
timeout=
File "/usr/lib/
return self._send(target, ctxt, message, wait_for_reply, timeout)
File "/usr/lib/
raise result
IOError: [Errno 28] No space left on device
<158>Sep 17 23:40:24 node-2 cinder-
nova logs