Change log level for mailin conflict errors to INFO

Bug #1205651 reported by Paul Everitt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
High
Paul Everitt

Bug Description

Like gsa_sync, mailin is giving a number of errors in the error monitor, causing Nagios to get mad. The conflicts aren't a big deal, so just change the log level to INFO.

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Here is the traceback:

Error in script.

Traceback (most recent call last):
  File "/srv/osfkarl/production/18/eggs/karlserve-1.22-py2.6.egg/karlserve/scripts/main.py", line 204, in wrapper
    return func(args)
  File "/srv/osfkarl/production/18/eggs/karlserve-1.22-py2.6.egg/karlserve/scripts/mailin.py", line 26, in main
    mailin(args, instance)
  File "/srv/osfkarl/production/18/eggs/karlserve-1.22-py2.6.egg/karlserve/scripts/mailin.py", line 52, in mailin
    transaction.commit()
  File "/srv/osfkarl/production/18/eggs/transaction-1.1.1-py2.6.egg/transaction/_manager.py", line 89, in commit
    return self.get().commit()
  File "/srv/osfkarl/production/18/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py", line 329, in commit
    self._commitResources()
  File "/srv/osfkarl/production/18/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py", line 446, in _commitResources
    rm.tpc_vote(self)
  File "/srv/osfkarl/production/18/eggs/ZODB3-3.10.5_perfmetrics-py2.6-linux-i686.egg/ZODB/Connection.py", line 781, in tpc_vote
    s = vote(transaction)
  File "/srv/osfkarl/production/18/eggs/perfmetrics-1.0-py2.6.egg/perfmetrics/__init__.py", line 120, in call_with_metric
    return f(*args, **kw)
  File "/srv/osfkarl/production/18/eggs/RelStorage-1.5.1_perfmetrics2-py2.6.egg/relstorage/storage.py", line 797, in tpc_vote
    return self._vote()
  File "/srv/osfkarl/production/18/eggs/RelStorage-1.5.1_perfmetrics2-py2.6.egg/relstorage/storage.py", line 844, in _vote
    serials = self._finish_store()
  File "/srv/osfkarl/production/18/eggs/RelStorage-1.5.1_perfmetrics2-py2.6.egg/relstorage/storage.py", line 756, in _finish_store
    rdata = self.tryToResolveConflict(oid, prev_tid, serial, data)
  File "/srv/osfkarl/production/18/eggs/ZODB3-3.10.5_perfmetrics-py2.6-linux-i686.egg/ZODB/ConflictResolution.py", line 277, in tryToResolveConflict
    data=newpickle)
ConflictError: database conflict error (oid 0x02ec1a4c, class BTrees.IOBTree.IOBucket, serial this txn started with 0x03a0497094ce38dd 2013-07-26 08:48:34.876302, serial currently committed 0x03a04a7a43567edd 2013-07-26 13:14:15.782314)

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

I'm going to chicken out on this one. It involves a transaction between two ZODB connections (postoffice and KARL.) Giving this one to dad.

Changed in karl3:
assignee: Paul Everitt (paul-agendaless) → Tres Seaver (tseaver)
importance: Medium → High
Revision history for this message
Tres Seaver (tseaver) wrote :

This one needs some discussion. "Put in a little conflict resolution" isn't
feasible, as the conflict is already being raised from an object (an
IOBTree bucket) which does its own resolution when possible.

We could do one of:

- Add some retry-handling to the script. Given that it runs every 60
  seconds, I can't see much win there, but it would be possible.

- Lower the level of the reported error (e.g., log it at INFO level
  w/o any traceback).

Changed in karl3:
status: New → Incomplete
Revision history for this message
Paul Everitt (paul-agendaless) wrote : Re: [Bug 1205651] Re: Give mailin the conflict resolution treatment

Let's go with the latter, log at INFO level.

--Paul

On Jul 29, 2013, at 11:48 AM, Tres Seaver <email address hidden> wrote:

> This one needs some discussion. "Put in a little conflict resolution" isn't
> feasible, as the conflict is already being raised from an object (an
> IOBTree bucket) which does its own resolution when possible.
>
> We could do one of:
>
> - Add some retry-handling to the script. Given that it runs every 60
> seconds, I can't see much win there, but it would be possible.
>
> - Lower the level of the reported error (e.g., log it at INFO level
> w/o any traceback).
>
>
> ** Changed in: karl3
> Status: New => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1205651
>
> Title:
> Give mailin the conflict resolution treatment
>
> Status in KARL3:
> Incomplete
>
> Bug description:
> Like gsa_sync, mailin is giving a number of errors in the error
> monitor, causing Nagios to get mad. Put in a little conflict
> resolution.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/karl3/+bug/1205651/+subscriptions

Tres Seaver (tseaver)
Changed in karl3:
status: Incomplete → In Progress
Revision history for this message
Tres Seaver (tseaver) wrote : Re: Give mailin the conflict resolution treatment

I have pushed changes to the 'mailin-lower_confict_noise-1205651' branch
of the 'karlserve' package which caputure conflict errors and log them
at INFO level.

Changed in karl3:
status: In Progress → Fix Committed
summary: - Give mailin the conflict resolution treatment
+ Change log level for mailin conflict errors to INFO
description: updated
Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Over to me for installation on karlstaging.

Changed in karl3:
assignee: Tres Seaver (tseaver) → Paul Everitt (paul-agendaless)
Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.