Merge lp:~vexo/bzr-webdav/bzr-webdav into lp:bzr-webdav
- bzr-webdav
- Merge into webdav
Status: | Merged | ||||||||
---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 76 | ||||||||
Proposed branch: | lp:~vexo/bzr-webdav/bzr-webdav | ||||||||
Merge into: | lp:bzr-webdav | ||||||||
Diff against target: |
117 lines (+35/-8) 2 files modified
tests/dav_server.py (+28/-2) webdav.py (+7/-6) |
||||||||
To merge this branch: | bzr merge lp:~vexo/bzr-webdav/bzr-webdav | ||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Richard Wilbur | Approve | ||
Vincent Ladeuil | Needs Fixing | ||
Reagan Sanders (community) | Needs Resubmitting | ||
Review via email: mp+177288@code.launchpad.net |
Commit message
Description of the change
Fixes bugs #1204734 and #1204727 by updating the list of status codes we interpret as success for DELETE operations and explicitly sending Overwrite: T with all MOVE operations, respectively.
Reagan Sanders (vexo) wrote : | # |
Tests added via sub-classing DAVServer to exercise the different server behavior to make sure we handle it gracefully. QuirkyDAVServer
Vincent Ladeuil (vila) wrote : | # |
@Reagan: Sorry for the delay and thanks for working on that *and* adding tests.
I have almost nothing to comment, nice job ;)
I think my only question is: did you encounter issues with an existing DAV server (which one ?) or did you get there only by reading the RFC ?
Concretely, the only risk I can see with your patch is that by changing the client behavior we may now fail against servers which worked before... But IIUC, the only case where that could happen is if a server refuse to do a MOVE (with Overwrite: 'T') if the target does not exist. Thoughts ?
For the record:
39 +class QuirkyTestingDA
40 + """
41 + A variant of TesttingDAVRequ
42 + off-spec behaviors to test how gracefully we handle them.
43 + """
PEP8 (or rather the variant we use in the bzr eco-system) asks for a single-line docstring so:
class QuirkyTestingDA
"""Slightly off-spec behaviors.
A variant of TesttingDAVRequ
off-spec behaviors to test how gracefully we handle them.
"""
Or something, the idea is the first line should be a one-liner, more stuff could be added after a blank line.
78 - self._raise_
Ouch !
106 - self._raise_
OUCH !
Damn, that's some lack of test coverage you fixed there :-/
Would you mind adding some to make sure we properly catch that kind of error ?
I guess that's how you discovered the bugs you filed though so 1) sorry
about that, 2) I'd really like to know which server you use ;)
If you can't address the above, just let know and I'll do it while landing, again, good work !
Reagan Sanders (vexo) wrote : | # |
I initially stumbled on the issues testing https:/
With regard to the MOVE behavior, any RFC-compliant server should be totally okay with us sending the Overwrite header whether it's required or not. When it comes to the poorly-written servers, this is just my intuition, but it seems likely that it should be much more common to make the mistake of looking for that Overwrite: T header when it isn't actually required, than to go out of their way to ensure that it isn't set when it doesn't matter. (Looking at the IT HIT server changelog, their front-end apparently used to have exactly this bug, but they claim to have fixed it.)
Though, since the currently packaged version of bzr-webdav doesn't work with the current package of bzr on Quantal+, I get the feeling there might not be too many other users out there anyways, to worry about breaking some other Exotic WebDAV implementation ;). I've tested it against apache2 with mod_dav/davfs without issues. Unfortunately, I don't have an IIS server handy to test probably the most common configuration :/.
I'll look into adding some test-cases to attempt to exercise the various server-failure code paths, since I think that's what was lacking to let those slip through, though I don't have much time during the workweek so you might beat me to it ;).
Richard Wilbur (richard-wilbur) wrote : | # |
@Reagan, thanks for all the work creating the tests for the bugs you fixed.
> Though, since the currently packaged version of bzr-webdav doesn't work with
> the current package of bzr on Quantal+
[...]
Which versions of bzr and bzr-webdav won't work together?
> I'll look into adding some test-cases to attempt to exercise the various
> server-failure code paths, since I think that's what was lacking to let those
> slip through, though I don't have much time during the workweek so you might
> beat me to it ;).
I'll look forward to the server-failure test-cases.
Vincent Ladeuil (vila) wrote : | # |
Thanks for the detailed answer.
>>>>> Reagan Sanders <email address hidden> writes:
> I initially stumbled on the issues testing https:/
> appear to use http://
> MOVE issue might be specific to Cloudsafe in this case, but I think the 200
> on DELETE "quirk" (still within spec, just unusual) should apply to anything
> built off of that IT HIT WebDAV frontend. (I've actually had to abandon that
> project for the moment, as they also don't currently support the
> PUT-with-
> that performance is just abysmal.)
Yeah, I started bzr-webdav when it was the only way for me to host bzr
branches, I've since migrated to ssh which is more secure and performs
better.
> With regard to the MOVE behavior, any RFC-compliant server should be totally
> okay with us sending the Overwrite header whether it's required or not. When
> it comes to the poorly-written servers, this is just my intuition, but it
> seems likely that it should be much more common to make the mistake of
> looking for that Overwrite: T header when it isn't actually required,
/me nods
> Though, since the currently packaged version of bzr-webdav doesn't
> work with the current package of bzr on Quantal+, I get the
> feeling there might not be too many other users out there anyways,
> to worry about breaking some other Exotic WebDAV implementation
> ;).
Yeah, it's always hard to know...
> I've tested it against apache2 with mod_dav/davfs without
> issues.
Yup, using the lp:bzr-local-test-server plugin I did so too, injecting
an apache2 dav server into bzr test suite and running:
bzr selftest -v -s bp.webdav -s bt.per_transport
which should be enough to cover all webdav plugin uses.
> Unfortunately, I don't have an IIS server handy to test probably
> the most common configuration :/.
Same here :-/
> I'll look into adding some test-cases to attempt to exercise the
> various server-failure code paths, since I think that's what was
> lacking to let those slip through,
Exactly, thanks for uncovering them !
> though I don't have much time during the workweek so you might
> beat me to it ;).
Right, let's race :)
But to be honest, this shouldn't block landing your patch so I'll try to
do that first.
Richard Wilbur (richard-wilbur) wrote : | # |
On second thought, I agree with Vincent that we should land this patch first and worry about more test cases in a parallel effort. Thanks for the great work!
+1
- 75. By Vincent Ladeuil
-
Prepare merging Reagan Sanders fixes.
* add some test infrastructure do test bogus servers with canned responses,
* remove some dead code:
** _handle_common_ errors is unused,
** put_bytes_non_atomic( ) and _put_bytes_ranged() can't raise on 200,201
and 204, _perform() already handle them,
* delete() should raise InvalidHttpResponse when receiving a 200, - 76. By Vincent Ladeuil
-
Merge fixes from Reagan Sanders
Vincent Ladeuil (vila) wrote : | # |
This should have landed now, just a few comments for the record:
I've dive in the code again and was able to further clean up various areas:
77 if code not in (200, 201, 204):
78 - self._raise_
79 + self._raise_
80 'expected 200, 201 or 204.')
This was really dead code so I completely removed it (_perform() itself will raise the proper error so the _raise_
I've added a test_delete_
I removed some other dead code and useless imports.
Once again, thanks for your work.
Preview Diff
1 | === modified file 'tests/dav_server.py' | |||
2 | --- tests/dav_server.py 2011-06-14 09:28:40 +0000 | |||
3 | +++ tests/dav_server.py 2013-08-04 16:32:22 +0000 | |||
4 | @@ -50,6 +50,9 @@ | |||
5 | 50 | _RANGE_HEADER_RE = re.compile( | 50 | _RANGE_HEADER_RE = re.compile( |
6 | 51 | r'bytes (?P<begin>\d+)-(?P<end>\d+)/(?P<size>\d+|\*)') | 51 | r'bytes (?P<begin>\d+)-(?P<end>\d+)/(?P<size>\d+|\*)') |
7 | 52 | 52 | ||
8 | 53 | delete_success_code = 204 | ||
9 | 54 | default_should_overwrite = True | ||
10 | 55 | |||
11 | 53 | 56 | ||
12 | 54 | def date_time_string(self, timestamp=None): | 57 | def date_time_string(self, timestamp=None): |
13 | 55 | """Return the current date and time formatted for a message header.""" | 58 | """Return the current date and time formatted for a message header.""" |
14 | @@ -281,7 +284,7 @@ | |||
15 | 281 | # Ok we fail for an unnkown reason :-/ | 284 | # Ok we fail for an unnkown reason :-/ |
16 | 282 | raise | 285 | raise |
17 | 283 | else: | 286 | else: |
19 | 284 | self.send_response(204) # Default success code | 287 | self.send_response(self.delete_success_code) # Default success code |
20 | 285 | self.end_headers() | 288 | self.end_headers() |
21 | 286 | 289 | ||
22 | 287 | def do_MOVE(self): | 290 | def do_MOVE(self): |
23 | @@ -294,8 +297,10 @@ | |||
24 | 294 | overwrite_header = self.headers.get('Overwrite') | 297 | overwrite_header = self.headers.get('Overwrite') |
25 | 295 | if overwrite_header == 'F': | 298 | if overwrite_header == 'F': |
26 | 296 | should_overwrite = False | 299 | should_overwrite = False |
28 | 297 | else: | 300 | elif overwrite_header == 'T': |
29 | 298 | should_overwrite = True | 301 | should_overwrite = True |
30 | 302 | else: | ||
31 | 303 | should_overwrite = self.default_should_overwrite | ||
32 | 299 | (scheme, netloc, rel_to, | 304 | (scheme, netloc, rel_to, |
33 | 300 | params, query, fragment) = urlparse.urlparse(url_to) | 305 | params, query, fragment) = urlparse.urlparse(url_to) |
34 | 301 | trace.mutter("urlparse: (%s) [%s]" % (url_to, rel_to)) | 306 | trace.mutter("urlparse: (%s) [%s]" % (url_to, rel_to)) |
35 | @@ -425,6 +430,14 @@ | |||
36 | 425 | self.end_headers() | 430 | self.end_headers() |
37 | 426 | self.wfile.write(response) | 431 | self.wfile.write(response) |
38 | 427 | 432 | ||
39 | 433 | class QuirkyTestingDAVRequestHandler(TestingDAVRequestHandler): | ||
40 | 434 | """ | ||
41 | 435 | A variant of TesttingDAVRequestHandler implementing various quirky/slightly | ||
42 | 436 | off-spec behaviors to test how gracefully we handle them. | ||
43 | 437 | """ | ||
44 | 438 | delete_success_code = 200 | ||
45 | 439 | default_should_overwrite = False | ||
46 | 440 | |||
47 | 428 | class DAVServer(http_server.HttpServer): | 441 | class DAVServer(http_server.HttpServer): |
48 | 429 | """Subclass of HttpServer that gives http+webdav urls. | 442 | """Subclass of HttpServer that gives http+webdav urls. |
49 | 430 | 443 | ||
50 | @@ -440,6 +453,19 @@ | |||
51 | 440 | # urls returned by this server should require the webdav client impl | 453 | # urls returned by this server should require the webdav client impl |
52 | 441 | _url_protocol = 'http+webdav' | 454 | _url_protocol = 'http+webdav' |
53 | 442 | 455 | ||
54 | 456 | class QuirkyDAVServer(http_server.HttpServer): | ||
55 | 457 | """ | ||
56 | 458 | Variant of DAVServer implementing various quirky/slightly | ||
57 | 459 | off-spec behaviors to test how gracefully we handle them. | ||
58 | 460 | """ | ||
59 | 461 | |||
60 | 462 | def __init__(self): | ||
61 | 463 | # We have special requests to handle that | ||
62 | 464 | # HttpServer_urllib doesn't know about | ||
63 | 465 | super(QuirkyDAVServer,self).__init__(QuirkyTestingDAVRequestHandler) | ||
64 | 466 | |||
65 | 467 | # urls returned by this server should require the webdav client impl | ||
66 | 468 | _url_protocol = 'http+webdav' | ||
67 | 443 | 469 | ||
68 | 444 | class TestCaseWithDAVServer(tests.TestCaseWithTransport): | 470 | class TestCaseWithDAVServer(tests.TestCaseWithTransport): |
69 | 445 | """A support class that provides urls that are http+webdav://. | 471 | """A support class that provides urls that are http+webdav://. |
70 | 446 | 472 | ||
71 | === modified file 'webdav.py' | |||
72 | --- webdav.py 2012-03-26 14:37:07 +0000 | |||
73 | +++ webdav.py 2013-08-04 16:32:22 +0000 | |||
74 | @@ -512,7 +512,7 @@ | |||
75 | 512 | # Intermediate directories missing | 512 | # Intermediate directories missing |
76 | 513 | raise errors.NoSuchFile(abspath) | 513 | raise errors.NoSuchFile(abspath) |
77 | 514 | if code not in (200, 201, 204): | 514 | if code not in (200, 201, 204): |
79 | 515 | self._raise_curl_http_error(abspath, response, | 515 | self._raise_http_error(abspath, response, |
80 | 516 | 'expected 200, 201 or 204.') | 516 | 'expected 200, 201 or 204.') |
81 | 517 | 517 | ||
82 | 518 | try: | 518 | try: |
83 | @@ -627,7 +627,8 @@ | |||
84 | 627 | abs_to = self._remote_path(rel_to) | 627 | abs_to = self._remote_path(rel_to) |
85 | 628 | 628 | ||
86 | 629 | request = _urllib2_wrappers.Request('MOVE', abs_from, None, | 629 | request = _urllib2_wrappers.Request('MOVE', abs_from, None, |
88 | 630 | {'Destination': abs_to}, | 630 | {'Destination': abs_to, |
89 | 631 | 'Overwrite': 'T'}, | ||
90 | 631 | accepted_errors=[201, 204, | 632 | accepted_errors=[201, 204, |
91 | 632 | 404, 409]) | 633 | 404, 409]) |
92 | 633 | response = self._perform(request) | 634 | response = self._perform(request) |
93 | @@ -654,15 +655,15 @@ | |||
94 | 654 | abs_path = self._remote_path(rel_path) | 655 | abs_path = self._remote_path(rel_path) |
95 | 655 | 656 | ||
96 | 656 | request = _urllib2_wrappers.Request('DELETE', abs_path, | 657 | request = _urllib2_wrappers.Request('DELETE', abs_path, |
98 | 657 | accepted_errors=[200, 204, | 658 | accepted_errors=[200, 202, 204, |
99 | 658 | 404, 999]) | 659 | 404, 999]) |
100 | 659 | response = self._perform(request) | 660 | response = self._perform(request) |
101 | 660 | 661 | ||
102 | 661 | code = response.code | 662 | code = response.code |
103 | 662 | if code == 404: | 663 | if code == 404: |
104 | 663 | raise errors.NoSuchFile(abs_path) | 664 | raise errors.NoSuchFile(abs_path) |
107 | 664 | if code != 204: | 665 | if code != 204 and code != 200: |
108 | 665 | self._raise_curl_http_error(curl, 'unable to delete') | 666 | self._raise_http_error(abs_path, response, 'unable to delete') |
109 | 666 | 667 | ||
110 | 667 | def copy(self, rel_from, rel_to): | 668 | def copy(self, rel_from, rel_to): |
111 | 668 | """See Transport.copy""" | 669 | """See Transport.copy""" |
112 | @@ -869,4 +870,4 @@ | |||
113 | 869 | def get_test_permutations(): | 870 | def get_test_permutations(): |
114 | 870 | """Return the permutations to be used in testing.""" | 871 | """Return the permutations to be used in testing.""" |
115 | 871 | import tests.dav_server | 872 | import tests.dav_server |
117 | 872 | return [(HttpDavTransport, tests.dav_server.DAVServer),] | 873 | return [(HttpDavTransport, tests.dav_server.DAVServer),(HttpDavTransport, tests.dav_server.QuirkyDAVServer)] |
Thanks for fixing the problems you noticed in bzr-webdav.
The only issue I see is that you forgot to add tests for these bugs that are resolved by your patches. tests/dav_server.py looks like the most likely place to add them. Vincent Ladeuil <email address hidden> would be the authority on the subject as he wrote this plugin.