[SRU] [scalability] NC does not detach created pthreads in KVM driver
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Eucalyptus |
Fix Released
|
High
|
Unassigned | ||
eucalyptus (Ubuntu) |
Fix Released
|
High
|
Dustin Kirkland | ||
Lucid |
Fix Released
|
High
|
Dustin Kirkland |
Bug Description
In the KVM NC driver, the pthread attr was set to detach a new pthread on create, but was not being passed to the pthread_create() function leading to the case where the NC will eventually be unable to start a new pthread. Fix is in revno 1223.
======
IMPACT:
* This bug affects anyone running hundreds, or thousands of UEC instances.
ADDRESSED:
* This bug is addressed by cherry-picking an upstream commit from their stable branch that fixes the leak.
REPRODUCE:
* This bug can be hard to reproduce. The most direct way would be to deploy a UEC with one Node controller, and run thousands and thousands of instances on this one NC over time. Eventually, the NC will fail, hitting the limit of attached pthreads in the KVM driver.
REGRESSION POTENTIAL:
* The node could be otherwise affected. However, in our testing of this fix, we have run thousands of UEC instances, and have confidence in the fix's stability.
======
Changed in eucalyptus (Ubuntu): | |
status: | New → In Progress |
importance: | Undecided → High |
assignee: | nobody → Dustin Kirkland (kirkland) |
milestone: | none → lucid-updates |
summary: |
- NC does not detach created pthreads in KVM driver + [scalability] NC does not detach created pthreads in KVM driver |
description: | updated |
summary: |
- [scalability] NC does not detach created pthreads in KVM driver + [SRU] [scalability] NC does not detach created pthreads in KVM driver |
Changed in eucalyptus (Ubuntu Lucid): | |
status: | Fix Released → Fix Committed |
Changed in eucalyptus: | |
status: | Fix Committed → Fix Released |
When this happens, the error message is something like "failed to spawn a VM startup thread" in nc.log.