OpenStack Image Registry and Delivery Service (Glance)

libvirt snapshots fail with: OSError: [Errno 11] Resource temporarily unavailable

Reported by Dan Prince on 2012-03-13
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
High
Eoghan Glynn

Bug Description

Using the latest nova essex code it appears that libvirt snapshots are failing on Fedora. I'm getting the following error in compute.log:

(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/nova/exception.py", line 114, in wrapped
(nova.rpc.amqp): TRACE: return f(*args, **kw)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/connection.py", line 660, in snapshot
(nova.rpc.amqp): TRACE: image_file)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 306, in update
(nova.rpc.amqp): TRACE: _reraise_translated_image_exception(image_id)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 304, in update
(nova.rpc.amqp): TRACE: image_meta = client.update_image(image_id, image_meta, data)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/client.py", line 195, in update_image
(nova.rpc.amqp): TRACE: res = self.do_request("PUT", "/images/%s" % image_id, body, headers)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 58, in wrapped
(nova.rpc.amqp): TRACE: return func(self, *args, **kwargs)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 407, in do_request
(nova.rpc.amqp): TRACE: headers=headers)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 75, in wrapped
(nova.rpc.amqp): TRACE: return func(self, method, url, body, headers)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 503, in _do_request
(nova.rpc.amqp): TRACE: _sendbody(c, iter)
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 475, in _sendbody
(nova.rpc.amqp): TRACE: for sent in iter:
(nova.rpc.amqp): TRACE: File "/usr/lib/python2.7/site-packages/glance/common/client.py", line 135, in __iter__
(nova.rpc.amqp): TRACE: CHUNKSIZE)
(nova.rpc.amqp): TRACE: OSError: [Errno 11] Resource temporarily unavailable
(nova.rpc.amqp): TRACE:

Something about using qemu-img to create snapshots seems to make the SendFileIterator in Glance client unhappy.

Dan Prince (dan-prince) wrote :

This was causing some of the Torpedo/Tempest failures in Smokestack... Once we get it fixed I'll turn those back on.

Changed in nova:
assignee: nobody → Dan Prince (dan-prince)
status: New → In Progress
importance: Undecided → High
Dan Prince (dan-prince) wrote :

Talking to Eoghan and Padraig about this issues.

Looks to be related to just Glance clients sendfile code.

Fix proposed here:

https://review.openstack.org/#change,5333

affects: nova → glance
Changed in glance:
assignee: Dan Prince (dan-prince) → Eoghan Glynn (eglynn)
Eoghan Glynn (eglynn) on 2012-03-14
Changed in glance:
milestone: none → essex-rc1

Reviewed: https://review.openstack.org/5333
Committed: http://github.com/openstack/glance/commit/51a06aab1c153ff30d71b0ce393cc015063ba7da
Submitter: Jenkins
Branch: master

commit 51a06aab1c153ff30d71b0ce393cc015063ba7da
Author: Eoghan Glynn <email address hidden>
Date: Wed Mar 14 12:31:03 2012 +0000

    Retry sendfile on EAGAIN or EBUSY

    Addresses bug 954430

    The sendfile call is redriven after a blocking select()
    to ensure the socket is now writeable.

    Change-Id: I68242c76593405f78e30324c2913cead63463c77

Changed in glance:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-03-21
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in glance:
milestone: essex-rc1 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers