Retry connection to mongodb on AutoReconnect exceptions
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceilometer |
Fix Released
|
Medium
|
Igor Degtiarov | ||
Juno |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
pymongo raises an AutoReconnect exception[1] when a ReplicaSet is used. This is supposed to be intercepted by client libraries which should decide if the operation can be retriggered or not. pymongo will handle reconnecting to a different Replica Set member transparently.
I have only encountered this exception when using ReplicaSets, but the upstream documentation doesn't mention them so it's possible that it is raised in other scenarios as well.
AFAICS this needs to be handled on each mongodb query in ceilometer. Read-only operations can just be re-triggered, but for write operations it might be more difficult.
[1] http://
Changed in ceilometer: | |
assignee: | nobody → Igor Degtiarov (idegtiarov) |
Changed in ceilometer: | |
importance: | Undecided → Medium |
milestone: | none → juno-rc1 |
Changed in ceilometer: | |
milestone: | juno-rc1 → kilo-1 |
Changed in ceilometer: | |
status: | Fix Committed → Fix Released |
tags: | added: canonical-bootstack |
tags: | added: juno-backport-potential |
Changed in ceilometer: | |
milestone: | kilo-1 → 2015.1.0 |
One dumb solution for read-only casses which works fine here looks like this:
Replace:
meters = list(self. db.resource. find(q) )
with:
while True:
meters = list(self. db.resource. find(q) )
time. sleep(1)
break
try:
except AutoReconnect:
else:
This could be wrapped in a reusable function, but I do not have enough ceilometer knowledge to know how to handle database queries involving writes.