Activity log for bug #412244

Date Who What changed Old value New value Message
2009-08-11 23:46:13 Alexander Regueiro bug added bug
2009-08-11 23:49:18 Robert Collins summary Cannot push to Windows FTP server (Microsoft FTP Service IIS6) unlock fails to unlock over FTP with Windows FTP server (Microsoft FTP Service IIS6)
2009-08-12 03:21:50 Robert Collins bzr: importance Undecided Medium
2009-08-12 03:21:55 Robert Collins bzr: status New Confirmed
2009-08-12 03:22:03 Robert Collins tags win32
2009-08-12 11:31:29 Alexander Regueiro tags win32 ftp lock win32
2009-08-18 05:28:51 Robert Collins description I have been trying to use a remote Windows 2003 server running IIS6/Microsoft FTP Service to host my Bazaar repos, but have encountered a lot of issues getting bzr to play nicely with it. Note that in all cases full permissions are granted to the operating FTP user. Firstly, simply pushing a local bzr repo to the FTP server (straight "bzr push ftp://.../repo") generates numerous CHMOD errors followed by a message complaining that the branch has been locked a number of seconds ago (which I have verified is earlier during the same push). FTP Could not set permissions to 448 on /darwindotnet/.bzr. 500 'SITE CHMOD 0700 /darwindotnet/.bzr': command not understood FTP Could not set permissions to 448 on /darwindotnet/.bzr/branch-lock. 500 'SIT E CHMOD 0700 /darwindotnet/.bzr/branch-lock': command not understood FTP Could not set permissions to 384 on /darwindotnet/.bzr/README. 500 'SITE CHM OD 0600 /darwindotnet/.bzr/README': command not understood FTP Could not set permissions to 384 on /darwindotnet/.bzr/branch-format. 500 'S ITE CHMOD 0600 /darwindotnet/.bzr/branch-format': command not understood FTP Could not set permissions to 448 on /darwindotnet/.bzr/repository. 500 'SITE CHMOD 0700 /darwindotnet/.bzr/repository': command not understood FTP Could not set permissions to 448 on /darwindotnet/.bzr/repository/lock. 500 'SITE CHMOD 0700 /darwindotnet/.bzr/repository/lock': command not understood Unable to obtain lock ftp://alexreg-repos@213.175.198.12/darwindotnet/.bzr/repos itory/lock held by foo@bar.com on host Alex-PC [process #4040] locked 4 seconds ago Will continue to try until 00:34:34, unless you press Ctrl-C If you're sure that it's not being modified, use bzr break-lock ftp://username@hostname/darwindotnet/.bzr/repository/lock bzr: interrupted Immediately executing the specified "bzr break-lock" command usually succeeds; however, a subsequent push results in the same types of errors (CHMOD errors followed by "branch locked"). I have checked that the lack of support for CHMOD in Microsoft FTP Service is *not* the cause of the problem, as commenting out the line in the source that performs the action results in no difference. Pushing to an FTP repository (in particular a new one) served by IIS6/Windows 2003 server blocks on obtaining a lock. Debugging has shown that the lock is one created and unlocked by this client, but the server hasn't correctly processed the unlock, A number of unrelated CHMOD errors are reported by the server as well. Debugging has shown that the server is not processing RENAME correctly from the locks held directory to the unlocked state (which is a temporary directory name). The observed symptoms at the FTP level are: the rename is reported successful. deleting the new path of the directory succeeds but the directory is still present when we check later. Various tests have been done, including reading-back from the held/info file to make sure it still exists, and these have shown some very inconsistent behaviour. The most recent test showed this behaviour: 43.900 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp 44.927 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.220 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.499 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp # ERRORS That is, this FTP server lets bzr rename a directory, then read a file from within that old directory path twice, but when we try to rename the directory again it errors. It looks like some sort of fs caching/locking is occuring in the server.
2009-08-21 01:00:22 Robert Collins attachment added log of the minimal trace http://launchpadlibrarian.net/30593955/bzr.log
2009-08-21 01:01:52 Robert Collins attachment added sanitised network trace showing proble, http://launchpadlibrarian.net/30593974/trace.txt
2009-08-21 12:58:59 Alexander Regueiro attachment added ftp script for manually reproducing problem http://launchpadlibrarian.net/30639401/script.txt
2009-08-24 18:51:48 Robert Collins summary unlock fails to unlock over FTP with Windows FTP server (Microsoft FTP Service IIS6) unlock fails to unlock over FTP with Windows IIS6 FTP server which keeps files after delete
2009-08-24 18:59:15 Robert Collins description Pushing to an FTP repository (in particular a new one) served by IIS6/Windows 2003 server blocks on obtaining a lock. Debugging has shown that the lock is one created and unlocked by this client, but the server hasn't correctly processed the unlock, A number of unrelated CHMOD errors are reported by the server as well. Debugging has shown that the server is not processing RENAME correctly from the locks held directory to the unlocked state (which is a temporary directory name). The observed symptoms at the FTP level are: the rename is reported successful. deleting the new path of the directory succeeds but the directory is still present when we check later. Various tests have been done, including reading-back from the held/info file to make sure it still exists, and these have shown some very inconsistent behaviour. The most recent test showed this behaviour: 43.900 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp 44.927 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.220 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.499 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp # ERRORS That is, this FTP server lets bzr rename a directory, then read a file from within that old directory path twice, but when we try to rename the directory again it errors. It looks like some sort of fs caching/locking is occuring in the server. bzr renames and deletes a directory over FTP to cause an unlock. If the server does not process these commands in an immediate, atomic manner, then attempting to rename a new lock directory into place will fail. During bzr push to a new branch bzr locks the repository twice: - creating it - pushing the new content With a server that doesn't process deletes immediately these two behaviours combine to give the user a confusing lock held error - with the user as the lock holder, and the current bzr process id. This should eventually right itself when the server does process the requested deletes. As it is on II6 FTP server, CHMOD errors may be reported as well. Debugging has shown that the server is not processing either RENAME correctly from the locks held directory to the unlocked state (which is a temporary directory name) or the RM of the lock/held file. The observed symptoms at the FTP level are: the rename is reported successful. deleting the new path of the temp/info and the temp directory succeeds but the directory is still present when we check later and the contents of the info file can s till be read from lock/held. Various tests have been done, including reading-back from the held/info file to make sure it still exists, and these have shown some very inconsistent behaviour. The most recent test showed this behaviour: 43.900 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp 44.927 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.220 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.499 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp # ERRORS That is, this FTP server lets bzr rename a directory, then read a file from within that old directory path twice, but when we try to rename the directory again it errors. It looks like some sort of fs caching/locking is occuring in the server. We have speculated about virus scanners and cluster file systems but have no concrete details from the FTP server operator, who insist that the server is operating 'normally'.
2009-08-24 20:00:53 Robert Collins description bzr renames and deletes a directory over FTP to cause an unlock. If the server does not process these commands in an immediate, atomic manner, then attempting to rename a new lock directory into place will fail. During bzr push to a new branch bzr locks the repository twice: - creating it - pushing the new content With a server that doesn't process deletes immediately these two behaviours combine to give the user a confusing lock held error - with the user as the lock holder, and the current bzr process id. This should eventually right itself when the server does process the requested deletes. As it is on II6 FTP server, CHMOD errors may be reported as well. Debugging has shown that the server is not processing either RENAME correctly from the locks held directory to the unlocked state (which is a temporary directory name) or the RM of the lock/held file. The observed symptoms at the FTP level are: the rename is reported successful. deleting the new path of the temp/info and the temp directory succeeds but the directory is still present when we check later and the contents of the info file can s till be read from lock/held. Various tests have been done, including reading-back from the held/info file to make sure it still exists, and these have shown some very inconsistent behaviour. The most recent test showed this behaviour: 43.900 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp 44.927 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.220 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.499 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp # ERRORS That is, this FTP server lets bzr rename a directory, then read a file from within that old directory path twice, but when we try to rename the directory again it errors. It looks like some sort of fs caching/locking is occuring in the server. We have speculated about virus scanners and cluster file systems but have no concrete details from the FTP server operator, who insist that the server is operating 'normally'. bzr renames and deletes a directory over FTP to cause an unlock. If the server does not process these commands in an immediate, atomic manner, then attempting to rename a new lock directory into place will fail. During bzr push to a new branch bzr locks the repository twice:  - creating it  - pushing the new content With a server that doesn't process deletes immediately these two behaviours combine to give the user a confusing lock held error - with the user as the lock holder, and the current bzr process id. This should eventually right itself when the server does process the requested deletes. As it is on II6 FTP server, CHMOD errors may be reported as well. Debugging has shown that the server is not processing either RENAME correctly from the locks held directory to the unlocked state (which is a temporary directory name) or the RM of the lock/held file. The observed symptoms at the FTP level are: the rename is reported successful. deleting the new path of the temp/info and the temp directory succeeds but the directory is still present when we check later and the contents of the info file can s till be read from lock/held. Various tests have been done, including reading-back from the held/info file to make sure it still exists, and these have shown some very inconsistent behaviour. The most recent test showed this behaviour: 43.900 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp 44.927 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.220 FTP get: /texdotnet/.bzr/branch-lock/held/info 45.499 FTP rename: /texdotnet/.bzr/branch-lock/held => /texdotnet/.bzr/branch-lock/releasing.a7rj960i72avjpn2u4qr.tmp # ERRORS That is, this FTP server lets bzr rename a directory, then read a file from within that old directory path twice, but when we try to rename the directory again it errors. It looks like some sort of fs caching/locking is occuring in the server. We have speculated about virus scanners and cluster file systems but have no concrete details from the FTP server operator, who insist that the server is operating 'normally'. RFC 959 section 4.2 specifies that incomplete operations should send a 1xx Positive Preliminary Reply - the server is not doing that, rather it is sending a 2xx - Positive completion reply. This is clearly in violation of the FTP specification.
2009-11-29 23:07:23 Robert Collins bzr: importance Medium Wishlist
2009-11-29 23:07:58 Robert Collins summary unlock fails to unlock over FTP with Windows IIS6 FTP server which keeps files after delete add workaround for Windows/IIS6 FTP server strangeness causing unlock failures.
2017-11-09 00:45:36 Jelmer Vernooij tags ftp lock win32 check-for-breezy ftp lock win32
2017-11-09 01:07:39 Alexander Regueiro removed subscriber Alexander Regueiro
2018-04-02 22:07:25 Launchpad Janitor branch linked lp:~jelmer/brz/no-more-ftp
2018-05-11 12:04:19 Jelmer Vernooij bug task added brz
2018-05-11 12:04:25 Jelmer Vernooij brz: status New Won't Fix
2018-05-11 12:04:30 Jelmer Vernooij tags check-for-breezy ftp lock win32 ftp lock win32