WSGIPublisher leaves the request open on error

[Brian Sutherland]

After banging on an erroring URL for 42 times I get to this in the logs:

     2011-05-24T08:09:59 CRITICAL ZODB.DB DB.open() has 42 open connections with a pool_size of 7

It appears that, on error, WSGIPublisher does not close the request. ZPublisher/Publish.py does seem to.

The attached patch causes the request to be closed unconditionally after the transaction is finished. I havn't committed it yet because the current behavior in WSGIPublisher looks intentional.

[Brian Sutherland (jinty)]

Committed patch to trunk (2.14) in r121832: https://mail.zope.org/pipermail/zope-checkins/2011-May/035379.html

And to the 2.13 branch in r121833: https://mail.zope.org/pipermail/zope-checkins/2011-May/035380.html

[stephan_hofmockel (dreagonfly)]

Hello,
the current implementation of this bugfix breaks the following code.
Assuming a request which sends 'a' and 'b' as parameters, eg. URL?a=test&b=test

import transaction

def do_something(self, request):
    self.attribute_a = request.get('a')
    transaction.commit()
    ...... possible other code which raises an Exception .....
   # the b parameter has to occur !
   assert request.get('b') == None

I use this in situations where I would like to be sure that 'self.attrbiute_a' is inside the ZODB regardless if an error happens afterwards or not. However after transaction.commit() the request is closed, because transaction.commit() calls 'afterCompletion' of '_RequestCloserForTransaction'. This means all parameters send in a form are lost/cleared and not available anymore.

Would it make sens to change the Bugfix in ZPublisher a little bit so that
247 _request_closer_for_repoze_tm.add(txn, request)
 is called after
252 response = _publish(request, 'Zope2')

If its a bug in ZOPE, I can work on a patch + test.
If I'm misusing transaction.commit(), plz tell me how it is done right.

[Brian Sutherland (jinty)]

On Sun, Oct 16, 2011 at 02:51:16PM -0000, stephan_hofmockel wrote:
> Hello,
> the current implementation of this bugfix breaks the following code.
> Assuming a request which sends 'a' and 'b' as parameters, eg. URL?a=test&b=test
>
> import transaction
>
> def do_something(self, request):
> self.attribute_a = request.get('a')
> transaction.commit()
> ...... possible other code which raises an Exception .....
> # the b parameter has to occur !
> assert request.get('b') == None

There are also other situations where this code probably will not work
as expected. For example, on serialization errors Zope can retry a
request, a few times.

If you get serialization errors after commit(), it may be that the
request is retried and you get a second or third commit() where you only
wanted one.

As a bonus, serialization errors most often happen on a loaded site.

> I use this in situations where I would like to be sure that 'self.attrbiute_a' is inside the ZODB regardless if an error happens afterwards or not. However after transaction.commit() the request is closed, because transaction.commit() calls 'afterCompletion' of '_RequestCloserForTransaction'. This means all parameters send in a form are lost/cleared and not available anymore.
>
> Would it make sens to change the Bugfix in ZPublisher a little bit so that
> 247 _request_closer_for_repoze_tm.add(txn, request)
> is called after
> 252 response = _publish(request, 'Zope2')

It may help.

> If its a bug in ZOPE, I can work on a patch + test.
> If I'm misusing transaction.commit(), plz tell me how it is done right.

I'm reasonably certain that you almost never want to call commit()
during a transaction. I've never used it, but perhaps savepoint() fits
your usecase better.

> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/787541
>
> Title:
> WSGIPublisher leaves the request open on error
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/zope2/+bug/787541/+subscriptions

--
Brian Sutherland