Functional testing framework HTTPCaller fails when request is retried because of Retry exception

[Maciej Wisniowski]

I found that in zope/app/testing/functional.py
there is a class HTTPCaller with __call__ method like:

class HTTPCaller(CookieHandler):
    """Execute an HTTP request string via the publisher"""

    def __call__(self, request_string, handle_errors=True, form=None):
        (...)
        response = ResponseWrapper(
            request.response, path,
            omit=('x-content-type-warning', 'x-powered-by'),
            )

        publish(request, handle_errors=handle_errors)
        self.saveCookies(response)
        setSite(old_site)

        # sync Python connection:
        getRootFolder()._p_jar.sync()

        return response

I was dealing with Retry exception and I get
strange errors during test. I found it is because
__call__ method assumes that request and response
are same objects during whole function.
It is not true when Retry exception is raised by
published code because retry creates new request
and response objects.
I think above code should be changed to:

class HTTPCaller(CookieHandler):
    """Execute an HTTP request string via the publisher"""

    def __call__(self, request_string, handle_errors=True, form=None):
        (...)
        request = publish(request, handle_errors=handle_errors)

        response = ResponseWrapper(
            request.response, path,
            omit=('x-content-type-warning', 'x-powered-by'),
            )

        self.saveCookies(response)
        setSite(old_site)

        # sync Python connection:
        getRootFolder()._p_jar.sync()

        return response

I've asked about this bug on zope-dev list: http://mail.zope.org/pipermail/zope3-dev/2007-May/022444.html

This bug causes that it is not possible to execute some tests, eg. these for another bug I've submitted:
https://bugs.launchpad.net/zope3/+bug/111845

Test attached to above bug report may be used as well to check bug in functional.py

[Christian Theune (ctheune)]

Your fix looks weird. A diff would be more helpful. The only change I can see is that you don't call publish() anymore. That can't be right. Maybe I missed something. From your description I have the feeling that using `return request.response` could fix the problem.

[Maciej Wisniowski (pigletto-gmail)]

The problem is that: when conflict error happens and as a result Retry exception is raised (or Retry is raised directly), then whole 'retrying' thing happens in 'publish' method. This includes creation of new(!) request object inside 'publish' method. Finally, this new request object is returned by 'publish' method.

As you can see, in functional.py, request object is created at the begining of this function and the same(!) request object is used thorough it in a call to publish and to create ResponseWrapper object. This is wrong as 'publish' may create new 'request' object.

Because of this my path does:
request = publish(request, handle_errors=handle_errors)

before creating ResponseWrapper object.

Diff is attached.

[Launchpad Janitor (janitor)]

[Expired for Zope 3 because there has been no activity for 60 days.]