mailman xmlrpc tests run twice to get good failure tracebacks

[Barry Warsaw]

When using an xmlrpclib.ServerProxy to test Launchpad XMLRPC services, and an exception occurs say in a doctest, the way this error is communicated isn't very helpful for debugging. What happens is that you get an OOPS on stderr and then you have to go hunting for the OOPS file to figure out what actually went wrong on the back end.

This is a problem for two reasons. First, you have to a-priori know where those OOPS files will be live on your file system and this is magical. Second, the actual OOPS information printed on stdout doesn't directly correspond to the file that's written. The latter includes a date stamp that isn't included in the former. So the upshot is that there's a lot of guesswork involved in just figuring out where the traceback information lives, never mind cracking the file open and figuring out what is actually going wrong.

The mailinglist-xmlrpc.txt doctest takes a clever approach around this. It actually runs the doctest twice, once with a ServerProxy and once directly through a faux view. I've actually seen both approaches in Launchpad (I think) but this is the first time they've been combined. The advantage of doing it this way is that the faux view provides very nice Python tracebacks right on stderr when something goes wrong, but the ServerProxy more accurately tests how things will work when deployed.

There are a couple of impedence mismatches you need to deal with to make this work though, and the test harness for mailman-xmlrpc.txt does this. First, you need to create an object that conforms to your interface in the harness and pass it into the globals of the doctest. This is because in one case you'll be binding the global to a ServerProxy instance and in the other case to a view instance. They'll both work though because they expose the same interface.

Except for the second mismatch you need to handle, which comes back exception handling. When an expected fault occurs (not an unexpected traceback requiring debugging as above), in the ServerProxy case you'll get an exception with a traceback, but when going through the view you'll get a Fault instance. These print differently making the doctest impossible to write for both cases. The solution that my doctest setup takes is to wrap the view's method in an impedence matching wrapper (implemented as a decorator) so that when a Fault instance is returned, the wrapper prints something that looks like a traceback. This allows you to write the doctest expected traceback output and everything works.

The third mismatch is that strings returned from the ServerProxy are always 8-bit strings while those returned from the view are going to be unicodes. This breaks the doctest because the latter is printed with a u-prefix while the former is not. My setup solves this by using a helper function to print all string results, and this helper coerces 8-bit strings (which will contain ascii at least in my case) to unicodes so they always print with the u-prefix.

BjornT was my branch reviewer, and had several things to say about this approach. It's a bit fragile since whenever the interface grows a new method, you have to add a wrapper to the setup's impedence matching view class. He's also right that what I've written is very specific to mailinglist-xmlrpc.txt and the approach -- if it's important enough -- should be generalized to be useful for all XMLRPC tests. Finally, he's also right that it doubles the time the test runs because we run it first through the view then through the proxy.

This bug is a placeholder so that someday we may address BjornT's concerns.