Better ftp support for bzr push

[Martin Packman]

When trying to push to ftp, ran into three seperate issues that caused the operation to fail. After spending some time on irc://irc.freenode.net/bzr have work-around for the first and (minimal) fixes for the second and third.

The command we're trying to get to work:
bzr push ftp://localhost/somewhere
(from in a tiny branch consisting of just a "test.txt" file)

### 1 ###
First it failed with:
bzr: ERROR: socket.error: (10061, 'Connection refused')
and traceback.

This turned out to be a side effect of the server being set up so that it uses 'external' IP for PASV. Using aftp://localhost/somewhere instead for loopback connection worked. Better error would be nice, but perhaps not easy.

### 2 ###
Then failed (after some long time) with:
bzr: ERROR: Could not acquire lock LockDir(ftp://localhost/somewhere/.bzr/branch-lock)
and hundreds of files like:
/somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp/tmp.info.1180989114.875000000.3288.dxh49lcl5e
The .bzr.log recorded lots of:
FTP mkd: /somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp
FTP put: /somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp/info
unable to understand error for path: /somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp/info: 550 Sorry, but that file doesn't exist
contention on LockDir(ftp://localhost/somewhere/.bzr/branch-lock): Generic path error: '/somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp/info': : unable to rename to u'/somewhere/.bzr/branch-lock/pending.wo8fzfxhgjjvc3s41oxy.tmp/tmp.info.1180989114.875000000.3288.dxh49lcl5e': 550 Sorry, but that file doesn't exist)
FTP get: /somewhere/.bzr/branch-lock/held/info
last_info: None, new info: None

So, this is bzr not understanding a real-world 550 error that should indicate NoSuchFile, and then behaving very badly (giving up at some point before 137 failed locks seems sensible).

### 3 ###
Then failed (after a short while) with:
bzr: ERROR: Generic path error: '/somewhere/.bzr/repository/knits/65/test.txt-20070604205240-zbtf1im1xhap227d-1.knit': could not store: 550 test.txt-20070604205240-zbtf1im1xhap227d-1.knit.tmp.1180993553.703000100.660.1962389434: Cannot create file.)
The ftp log shows it did do:
MKD /somewhere/.bzr/repository/knits
but doesn't create any of the sub-directories of /knits/ leading to an error when it tries to put a file in say, /65/ like so:
STOR /somewhere/.bzr/repository/knits/65/test.txt-20070604205240-zbtf1im1xhap227d-1.knit.tmp.1180994162.953000100.3316.1329413278
550 test.txt-20070604205240-zbtf1im1xhap227d-1.knit.tmp.1180994162.953000100.3316.1329413278: Cannot create file.

This is bzr not creating all the directories it needs to do the push, and not falling back to the right make-new-dir command on a particular failure condition.

(note, this is a somewhat simplified history, in the interests of clarity)

[Martin Packman (gz)]

So, solutions:
#1 Use aftp instead
#2 Understand "file doesn't exist" as NoSuchFile
#3 Fallback to mkdir when it's needed

Very simple patch attached, top part by me, bottom part from jam-laptop.

[Martin Pool (mbp)]

The problem of many lock directories being left behind is tracked as bug 109169, and should be fixed in bzr 0.18.

We should perhaps suggest using aftp (or not using it) if we get an error opening a socket. bug 123719

I have put the error heuristic up for review.

[Martin Pool (mbp)]

The patch was merged:

revno: 2571
committer: Canonical.com Patch Queue Manager<email address hidden>
branch nick: +trunk
timestamp: Mon 2007-07-02 19:36:15 +0100
message:
  (John Arbash Meinel) A couple small updates for pushing over ftp.
    ------------------------------------------------------------
    revno: 2570.1.1
    merged: <email address hidden>
    committer: John Arbash Meinel <email address hidden>
    branch nick: jam-integration
    timestamp: Mon 2007-07-02 12:35:53 -0500
    message:
      (John Arbash Meinel) A couple small updates for pushing over ftp.