Investigate repoze.mailin sqlite locks

Bug #364251 reported by Paul Everitt on 2009-04-20
2
Affects Status Importance Assigned to Milestone
KARL3
Medium
Tres Seaver

Bug Description

We're still getting plagued with SQLite table locks on kdiab. We had a brief discussion with Tres identifying some things to look at.

From Email
==========

The following is the email exchange:

Paul Everitt wrote:
> Hi there. We're a bit wedged on repoze.mailin and the sqlite locks,
> no pun intended.
>
> This morning I checked to see why Will said mailin didn't work during
> tested. The database was locked. It was last locked the morning he
> started testing.
>
> The challenge is that we don't know what the problem is, and can't
> reproduce the problem. However, that puts us in a "do nothing" mode
> that won't really be acceptable.
>
> We could simply *guess* and say daemon-izing it would work. It's
> better than the status quo.

Is the current implementation running as a "one shot" from cron? If so,
we should do two things:

- - Ensure that the cron script uses a well-known dance to keep from
 re-running the process if it is already running.

Doesn't currently do this, but...

- - Figure out if / why it is running so long.

I've never seen any evidence that it takes more than a few seconds to run. (Although there might be anecdotal field evidence that I haven't seen yet.)

- - If it is *not* running long, add some exception handling code
 which ensures that the lock gets released even on some failure
 case (which ought to be logged somewhere).

I implemented this in my first round of trouble shooting this a while back, although it's certainly worth an extra look to see if there's a leak somewhere.

Chris

Changed in karl3:
assignee: nobody → chris-archimedeanco
importance: Undecided → Medium
milestone: none → m12
Paul Everitt (paul-agendaless) wrote :

Shane, if you could also take this one. ChrisR knows the most about it, but I need to get him to work on migration. It appears you won't be starting on the unit tests for karl.peopledir today, so you have a bit of a gap in tickets.

Changed in karl3:
assignee: chris-archimedeanco → shane-hathawaymix
Paul Everitt (paul-agendaless) wrote :

Tres said he would pick this one up.

Changed in karl3:
assignee: Shane Hathaway (shane-hathawaymix) → Tres Seaver (tseaver)
Paul Everitt (paul-agendaless) wrote :

Rolling over to M13.

Changed in karl3:
milestone: m12 → m13
Tres Seaver (tseaver) on 2009-05-04
Changed in karl3:
status: New → In Progress
Tres Seaver (tseaver) wrote :

Please upload a copy of the crontab for the user running the
mail-in processing, and reassign to me.

Changed in karl3:
assignee: Tres Seaver (tseaver) → LarsN (lars-sixfeetup)
LarsN (lars-sixfeetup) wrote :

The current crontab for user "zope" is

*1 * * * * /var/db/karl3/cronscripts/cron-send-alerts.sh >> /var/db/karl3/log/outgoing_mail.log 2>&1
*1 * * * * /var/db/karl3/cronscripts/cron-process-mail.sh >> /var/db/karl3/log/incoming_mail.log 2>&1

Changed in karl3:
assignee: LarsN (lars-sixfeetup) → Tres Seaver (tseaver)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tres Seaver wrote:
> Please upload a copy of the crontab for the user running the
> mail-in processing, and reassign to me.
>
> ** Changed in: karl3
> Assignee: Tres Seaver (tseaver) => LarsN (lars-sixfeetup)
>

Looks like I need to see as much of the log file as possible:

  /var/db/karl3/log/incoming_mail.log

Can you upload that file, plus any rotated older versions still around?

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ/wOd+gerLs4ltQ4RAtyEAJ9UzxBsJ2NAZCcrOvXHkabMLJjzbgCfdoTb
rW8iIEWXA8JC7UdF4Eeux2E=
=rC4a
-----END PGP SIGNATURE-----

Tres Seaver (tseaver) wrote :

Also, I need to see how the script is wired:

  /var/db/karl3/cronscripts/cron-process-mail.sh

Changed in karl3:
assignee: Tres Seaver (tseaver) → LarsN (lars-sixfeetup)
Tres Seaver (tseaver) wrote :

repoze.mailin 0.1.4, now released to the lemonade-dev index, contains a fix
which should resolve this problem.

Changed in karl3:
assignee: LarsN (lars-sixfeetup) → Tres Seaver (tseaver)
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers