Zope's transaction behaviour flawed

Bug #142446 reported by Dieter Maurer
2
Affects Status Importance Assigned to Milestone
Zope 2
Fix Released
Critical
Unassigned

Bug Description

Zope's current transaction behaviour is essentially:

  ## request starts
  transaction.begin()
  try:
       object= REQUEST.traverse(...)
       mapply(object,...)
       transaction.commit()
  except:
       transaction.abort()
       handle_error()
  ## request ends

This is flawed as error handling is done outside of a transaction.

   Potential changes during the error handling spill over
   uncontrolled into another request and are there
   either committed or aborted as part of this request.

   Andrew Athan (<mailto:<email address hidden>>) has had lots
   of serious inconsistencies in Zope's session data.
   After extensive analysis, he found out that reading
   the session data during error handling led to these
   error conditions (reading session data causes writes to
   the administrative data).

I suggest, we let Zope perform error handling in its own
transaction after the original transaction had been aborted.
When error handling succeeds, its transaction is committed,
otherwise aborted.

The new behaviour would look something like this:

  ## request starts
  transaction.begin()
  try:
       object= REQUEST.traverse(...)
       mapply(object,...)
       transaction.commit()
  except:
       transaction.abort()
       transaction.begin()
       transaction.note('%s (application error handling)'
                        % '/'.join(object.getPhysicalPath)
   )
       try:
    handle_error()
    transaction.commit()
       except:
           default_handle_error() # Zope's default error handling
                           # it should not have side effects
      # and is executed inside the
      # error handling transaction
    transaction.abort()
  ## request ends

--- See discussion in "zope-dev".

Tags: bug zope
Revision history for this message
Steve Alexander (stevea) wrote :

I'm currently implementing similar behaviour for zope 3.

However, introducing this behaviour in zope2 could break applications that themselves clear cached or computed data on transaction boundaries. This data would not be available for the error page.

I think ZPatterns applications sometimes rely on the current behaviour, so this change would break certain zpatterns applications.

Revision history for this message
Chris McDonough (chrism-plope) wrote :

Michael Dunstan also compiled some notes and fixes for this... see http://mail.zope.org/pipermail/zope-dev/2004-April/022727.html

Revision history for this message
Dieter Maurer (d.maurer) wrote :

Toby convinced me ("zope-dev" discussion) that the correct behaviour would be to abort the transaction after error handling
(thus doing error handling in the same transaction as the request).

Currently, at the start of error handling, "get_transaction().begin()" is called, effectivle aborting the transaction, the request has been running in. This is bad as people expecting to find information in the SESSION are mislead.

I do not understand why implementing the correct behaviour would break application that "clear ... data
themselves at transaction boundaries".

I can see that application will break that perform a commit in the error handling -- but that should be rare...

Revision history for this message
Chris McDonough (chrism-plope) wrote :

Status: Pending => Resolved

A fix for this has been checked in for 2.7.1+ and on the HEAD. I am resolving, although comments on the patch are welcome and encouraged.

Revision history for this message
Chris McDonough (chrism-plope) wrote :

Historical record: See also issue 869 for more discussion/detail.

Revision history for this message
Florent Guillaume (efge) wrote :

Status: Resolved => Pending

I'm reopening because the current fix (and Zope 2.7.2) poses problems. It seems in some cases of error the transaction is not aborted correctly and some objects linger in the cache.

The following external method will help reproduce the problem:

def install(self):
    myfolder = self.myfolder
    nbfold = len(myfolder.objectIds())
    id = "glop"+str(nbfold)
    myfolder.manage_addFolder(id)
    get_transaction().commit(1)
    # The raise should cancel all object creation
    raise Exception(id)

Create a folder 'myfolder', and run the method several times. You'll get the creation of several objects. If however you purge the ZODB cache ("minimize" button), or if for some other reason the cache is cleaned, or another thread is used, you'll see that the objects disappear.

Sometimes we get POSKeyError because the created objects are used by another transaction to create a reference, and later the original object disappears.

The core of the problem seems to be that in Publish.py in the case of an error, the transaction is not always aborted. Reverting to 2.7.0, or just re-adding a
        if transactions_manager:
            transactions_manager.abort()
like there was after the except in 2.7.0 (which I know is the whole point of this bug) makes the problem go away.

Revision history for this message
Michael Dunstan (michael-elyt) wrote :

See link for patch to publisher that may help:

http://mail.zope.org/pipermail/zope-dev/attachments/20040825/e5d7a8e9/publisher-abort.obj

I'm currently running some tests for sessions and transience. And this patch seems to iron out a lot problems that have been reported with transience of recent.

Revision history for this message
Tim Peters (tim-one) wrote :
Revision history for this message
Tim Peters (tim-one) wrote :

As explained in this thread:

http://mail.zope.org/pipermail/zodb-dev/2004-August/007865.html

ZODB isn't helping matters here. The short course is that when a connection is closed without an explicit commit() or abort(), and objects from that connection have pending modifications, things are a mess -- and especially if the pending changes are due to subtransaction commits. The next ZODB 3.2 release will raise an exception if a connection is closed with changes pending (better dead than insane <0.5 wink>).

Revision history for this message
Tim Peters (tim-one) wrote :

As threatened, ZODB in current CVS Zope-2_7-branch raises ConnectionStateError if a Connection is closed while modifications are pending in that connection. Also in ZODB 3.3 branch, and on ZODB trunk.

In addition, there was another bug wherein Transaction.begin() failed to abort the current transaction if the only changes pending were in a subtransaction. That's also been repaired on Zope-2_7-branch, ZODB 3.3 branch, and ZODB trunk.

Revision history for this message
ChrisW (chris-simplistix) wrote :

Tim, have your changes made it into any released version of Zope yet?

Revision history for this message
Tim Peters (tim-one) wrote :

[chrisw]
> Tim, have your changes made it into any released version of Zope yet?

Zope X3b4, I guess. How many Zopes normally get released in a week <wink>?

Note that, if this bug report is correct, Zope on the 2.7 branch still needs changes to live with that ZODB will no longer allow it to close a connection holding pending modifications.

Revision history for this message
ChrisW (chris-simplistix) wrote :

Status: Pending => Resolved

Tim and Michael have apparently fixed this in CVS, and so it will be released in Zope 2.7.3.

Changes have also made the same changes to the Zope HEAD in SVN, and so the fixes will also be present in 2.8.0.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.