Eventlet Log Spamming on Client Disconnect (Broken Pipe)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Identity (keystone) |
Fix Released
|
Critical
|
Morgan Fainberg | ||
Icehouse |
Fix Released
|
Critical
|
Morgan Fainberg |
Bug Description
If a client makes a request to keystone, and then disconnects before keystone responds, it is possible to fill up the logs (INFO) with eventlet tracebacks due to broken pipe:
2014-06-24 23:30:29.729 31440 INFO eventlet.
2014-06-24 23:30:29.731 31440 INFO eventlet.
File "/usr/lib/
write(
File "/usr/lib/
_writelines
File "/usr/lib/
self.flush()
File "/usr/lib/
self.
File "/usr/lib/
tail = self.send(data, flags)
File "/usr/lib/
total_sent += fd.send(
error: [Errno 32] Broken pipe
Example (900k line file) due to this [WARNING THIS LINK MIGHT KILL YOUR BROWSER]:
http://
We should override the required HTTPProtocol class and gracefully handle the traceback. If we would like to keep the information, a single log-line per incident would be sufficient instead of ~14.
This should be considered for a backport to Icehouse to help limit log spam there as well.
Changed in keystone: | |
milestone: | none → juno-2 |
status: | Fix Committed → Fix Released |
tags: | removed: icehouse-backport-potential |
Changed in keystone: | |
milestone: | juno-2 → 2014.2 |
This is being marked as Critical on request of the -infra team, this is causing issues in gate with excessive log-size.