XMLRPC + WSGI

[Martijn Faassen]

XMLRPC is broken in a WSGI environment.

We should add a test to grok that demonstrates this brokenness and then fix the bug.

[Michael Haubenwallner (d2m)]

Fix:

In zope.publisher.xmlrpc.XMLRPCRequest.processInputs
replace
    lines = ''.join(self._body_instream.readlines())
by
    lines = ''.join(self._body_instream.readlines(self._body_instream.size))

works with both zopectl (twisted) and paster setup

The problem is with the signature of zope.publisher.http.HTTPInputStream.readlines:

    def readlines(self, hint=0):
        data = self.stream.readlines(hint)
        self.cacheStream.write(''.join(data))
        return data

Apparently reading 0 from twisted's BufferedStream works, but fails on wsgi.input <socket._fileobject object>

How to proceed - should we create a new zope.publisher release and change groks versions.cfg?

[Michael Haubenwallner (d2m)]

The twisted input stream is
wsgi.input <twisted.web2.wsgi.InputStream object>

[Martijn Faassen (faassen)]

Are there any negative performance impacts of doing this change?

I think we should discuss this on zope-dev.

We should also consider what other changes have been made on
zope.publisher that we may not want to pull in. If so, we should
consider branching an earlier version of zope.publisher and applying
the fix there and releasing a bugfix release.

[Michael Haubenwallner (d2m)]

more info for discussion -- looks like we better change Zope3/trunk/src/zope/publisher/http.py instead

======= change of signature from readlines(self, hint=None) to readlines(self, hint=0)
Filename: Zope3/trunk/src/zope/publisher/http.py
Revision 66941 - (view) (download) (as text) - [select for diffs]
Modified Fri Apr 14 13:24:35 2006 UTC (2 years, 10 months ago) by dobe
File length: 34033 byte(s)
Diff to previous 66940

changed the default value of the hint argument to 0, because StringIO.readline requires an int
http://svn.zope.org/Zope3/trunk/src/zope/publisher/http.py?rev=66941&r1=66940&r2=66941

======= twisted api
def readlines(self, hint=None):
    Read until EOF, collecting all lines in a list, and returns that list. The hint argument is ignored (as is allowed in the API specification)
http://python.net/crew/mwh/apidocs/twisted.web2.wsgi.InputStream.html#readlines

======= python api
readlines([sizehint])
    Read until EOF using readline() and return a list containing the lines thus read. If the optional sizehint argument is present, instead of reading up to EOF, whole lines totalling approximately sizehint bytes (possibly after rounding up to an internal buffer size) are read. Objects implementing a file-like interface may choose to ignore sizehint if it cannot be implemented, or cannot be implemented efficiently.
http://www.python.org/doc/2.5.2/lib/bltin-file-objects.html

======= pythonpaste api
def readlines(self, hint=None):
http://svn.pythonpaste.org/Paste/trunk/paste/httpserver.py

Q: how to move the issue to the zope bug collector?

[Martijn Faassen (faassen)]

You can use 'also affects project' with launchpad; I've just added it.

[Michael Haubenwallner (d2m)]

thank you, see same bug in zopeproject
https://bugs.launchpad.net/zope3/+bug/283089

[Michael Haubenwallner (d2m)]

sent mail to zope-dev

[Michael Haubenwallner (d2m)]

Created maintenance branch
http://svn.zope.org/zope.publisher/tags/3.4.6.1/

[Michael Haubenwallner (d2m)]

removed the 3.4.6.1 tag

[Michael Haubenwallner (d2m)]

sent mail to zope-dev

[Michael Haubenwallner (d2m)]

I'll join you in IRC on Fri/Sat and fix these issues
- backport the PUT related bugfix from zope.publisher 3.5.3 to 3.4 branch
- fix the readlines bug
- release a new zope.publisher3.4.8

[Michael Haubenwallner (d2m)]

fix XMLRPC and PUT freeze under WSGI(paster.httpserver)

http://svn.zope.org/zope.publisher/branches/3.4/?rev=101291&view=rev

[Michael Haubenwallner (d2m)]

update grok versions.cfg to use the fixed zope.publisher

http://svn.zope.org/grok/trunk/?rev=101420&view=rev

[Michael Haubenwallner (d2m)]

activating this bug again for the Grok1.1 series
we need to port the bug to the zope.publisher release used by ZTK

[Kevin Teague (kevin-bud)]

The fix in the xmlrpc.py module of zope.publisher was applied to maintenance branches, but never went onto trunk. Was there any objections to applying this to trunk?

It would be good to get this fixed for Grok 1.1. I'm using zope.publisher 3.12 w/ the xmlrpc.py patch applied, and it's working fine for me.

[Michael Haubenwallner (d2m)]

I did not ask about applying to trunk then, because grok used an ancient release of zope.publisher.
But the changes were discussed on the mailing list.

Anyway, if we get the changes into trunk now and get them released,
we also need to get that new release into ztk.cfg as groktoolkit extends from ztk.cfg.

[Kevin Teague (kevin-bud)]

Fixed in zope.publisher 3.12.2, and ztk now uses this release on trunk. This bug should be fixed with the next Grok release if it uses the updated ztk versions.

[Michael Haubenwallner (d2m)]

FYI: Grok1.1 didn't make it to zope.publisher 3.12.2, still using zope.publisher 3.12.0

[Christian Klinger (cklinger)]

Grok 1.2 uses the zope.publisher 3.12.4 from the ztk 1.0.