no free nbd devices while spawning many instances per node, injecting data fails

Bug #726244 reported by Christian Berendt
18
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Michael Still

Bug Description

While spawning more instances then max_nbd_devices (or 16 per default) I got the error "2011-02-27 22:45:22,370 WARNING nova.virt.libvirt_conn [-] instance instance-00000091: ignoring error injecting data into image ami-5blouj1l (No free nbd devices)
". I think there sould be some logic to handle this case.

Tags: ops nbd
Thierry Carrez (ttx)
Changed in nova:
importance: Undecided → Low
status: New → Confirmed
Revision history for this message
Mark McLoughlin (markmc) wrote :

I guess the thing to do here would be to wait until an NBD device frees up

Or is this the race condition that Pádraig talks about here:

  https://github.com/openstack/nova/blob/638ea669694f3083145e2300b362ded6749d74a1/nova/virt/disk/nbd.py#L44

Revision history for this message
Mark McLoughlin (markmc) wrote :

Note, this doesn't apply if you use libguestfs

tags: added: nbd
Revision history for this message
Mark McLoughlin (markmc) wrote :

Similar to bug #755854

Michael Still (mikal)
tags: added: ops
Michael Still (mikal)
Changed in nova:
importance: Low → High
status: Confirmed → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/17510

Changed in nova:
assignee: nobody → Michael Still (mikalstill)
status: Triaged → In Progress
Michael Still (mikal)
summary: - no free nbd devices while spawning to many instances per node, injecting
+ no free nbd devices while spawning many instances per node, injecting
data fails
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/17510
Committed: http://github.com/openstack/nova/commit/6da1dbc8498b4c3143221fd4e1b9f59374134fd3
Submitter: Jenkins
Branch: master

commit 6da1dbc8498b4c3143221fd4e1b9f59374134fd3
Author: Michael Still <email address hidden>
Date: Fri Dec 14 10:22:44 2012 +1100

    Retry NBD device allocation.

    Mitigates bug 726244.

    Change-Id: I2f09f3a69c3eb61ca3af90355d5861373b7ae277

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-2 → 2013.1
Revision history for this message
Stanislaw Pitucha (stanislaw-pitucha) wrote :

There may be another reason for this issue. If you're still seeing the problem see bug 1207422

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.