Bug when running swift processes on a machine that is not in the ring
Bug #812460 reported by
gholt
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Object Storage (swift) |
Expired
|
Medium
|
Unassigned | ||
Bug Description
There is a bug where running the swift processes on a machine that is not in the ring causes it to use up tons of file descriptors (presumably to syslog) eventually making the box unreachable. This was detected by adding a new machine to a cluster by adding it to the the account and container rings, but not the object ring. The machine, using a standard set of configs and scripts, always ran all the processes except the proxy. After a short time, the box would use up all its file descriptors and need rebooting. Short-term workaround was to not start the object processes.
| Changed in swift: | |
| status: | New → Triaged |
| Changed in swift: | |
| status: | Triaged → Confirmed |
| Changed in swift: | |
| importance: | Undecided → Medium |
Revision history for this message
| Samuel Merritt (torgomatic) wrote | #1 |
| Changed in swift: | |
| status: | Confirmed → Incomplete |
| tags: | added: low-hanging-fruit |
Revision history for this message
| Launchpad Janitor (janitor) wrote | #2 |
| Changed in swift: | |
| status: | Incomplete → Expired |
To post a comment you must log in.
I tried to reproduce this with master (c2f8f00), but I couldn't get the object processes to go crazy. As far as I could tell, they were happily doing nothing and using minimal CPU and files.
My setup was a 2-node real Swift cluster with 2 disks per box. I took box #1 and first set its drives' weights to 0 in the object ring, then rebalanced until no partitions were left on them. When that failed to fail, I deleted the drives from the object ring entirely, but that didn't do it either.