bazaar internal error on FTP push

[Jorgen Bodde]

Sometimes when pushing my branch to my FTP site, it reports this annoying internal error. When I push a second time it goed through. But I always wonder how much of the files are actually pushed or if the repository on the FTP address is not broken.

bzr: ERROR: ftplib.error_temp: 450 No files found

Traceback (most recent call last):
  File "bzrlib\commands.pyc", line 834, in run_bzr_catch_errors
  File "bzrlib\commands.pyc", line 790, in run_bzr
  File "bzrlib\commands.pyc", line 492, in run_argv_aliases
  File "bzrlib\builtins.pyc", line 825, in run
  File "bzrlib\decorators.pyc", line 127, in read_locked
  File "bzrlib\branch.pyc", line 1569, in push
  File "bzrlib\branch.pyc", line 1611, in _push_with_bound_branches
  File "bzrlib\branch.pyc", line 1630, in _basic_push
  File "bzrlib\decorators.pyc", line 165, in write_locked
  File "bzrlib\branch.pyc", line 1478, in update_revisions
  File "bzrlib\decorators.pyc", line 165, in write_locked
  File "bzrlib\branch.pyc", line 286, in fetch
  File "bzrlib\repository.pyc", line 949, in fetch
  File "bzrlib\decorators.pyc", line 165, in write_locked
  File "bzrlib\repository.pyc", line 2743, in fetch
  File "bzrlib\repofmt\pack_repo.pyc", line 1229, in autopack
  File "bzrlib\repofmt\pack_repo.pyc", line 1248, in _execute_pack_operations
  File "bzrlib\repofmt\pack_repo.pyc", line 1585, in _save_pack_names
  File "bzrlib\transport\ftp.pyc", line 512, in list_dir
  File "ftplib.pyc", line 459, in nlst
  File "ftplib.pyc", line 407, in retrlines
  File "ftplib.pyc", line 356, in transfercmd
  File "ftplib.pyc", line 327, in ntransfercmd
  File "ftplib.pyc", line 241, in sendcmd
  File "ftplib.pyc", line 214, in getresp
error_temp: 450 No files found

bzr 1.2.0 on python 2.5.1.final.0 (win32)
arguments: ['bzr', 'push']
encoding: 'cp1252', fsenc: 'mbcs', lang: None
plugins:
  launchpad C:\PROGRA~1\Bazaar\lib\library.zip\bzrlib\plugins\launchp
ad [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.
---------------------------

Pushing again works as expected:

D:\personal\src\airs>bzr push
Using saved location: ftp://{my-ftp-url}
FTP <email address hidden> password:
Pushed up to revision 45.

[John A Meinel (jameinel)]

This sounds like a response that we didn't expect.

Specifically, when auto-packing, we will clear out any packs that are in "obsolete_packs".

It sounds like your ftp server raises an error when you do "ls" in an empty directory. I would have expected it to just return an empty list.

Can you describe what FTP server you are using, etc?

450 seems to be "requested action not taken" which seems an odd error to return on an ls of an empty directory.
http://www.the-eggman.com/seminars/ftp_error_codes.html

Anyway, it would be easy enough to adjust the FTPTransport.list_dir() code to check for 450 and "No files found" and just return an empty list rather than an exception.

[John A Meinel (jameinel)]

A patch like this should probably fix this:

=== modified file 'bzrlib/transport/ftp.py'
--- bzrlib/transport/ftp.py 2008-03-30 01:41:50 +0000
+++ bzrlib/transport/ftp.py 2008-04-18 19:04:09 +0000
@@ -513,6 +513,12 @@
             paths = f.nlst(basepath)
         except ftplib.error_perm, e:
             self._translate_perm_error(e, relpath, extra='error with list_dir')
+ except ftplib.error_temp, e:
+ if str(e).lower().startswith('450 no files found'):
+ # FTP Server XXXX returns an error code when a directory is
+ # empty
+ return []
+ raise
         # If FTP.nlst returns paths prefixed by relpath, strip 'em
         if paths and paths[0].startswith(basepath):
             entries = [path[len(basepath)+1:] for path in paths]

Could you tell us what server this is, and if possible, would a patch like this prevent the problem?

[Jorgen Bodde (jorgb)]

Hi John,

Thank you very much for looking into it.

> Can you describe what FTP server you are using, etc?

Hard to tell, I am using the FTP server that my provider has. it is xs4all,
but I can't tell you what runs beneath the FTP login that I see. If there is a
way to find out I will look into it for you.

> Anyway, it would be easy enough to adjust the FTPTransport.list_dir()
> code to check for 450 and "No files found" and just return an empty list
> rather than an exception.

If the exception can't hurt I will wait for the next version. Does it have
this patch included then?

Regards,
- Jorgen

[John A Meinel (jameinel)]

Would it be possible for you to test a bzr branch with this patch? I can post a tarball, etc online somewhere if you can download it and test. This patch won't make it into mainline unless we actually know that it works :).

If you can just download a bzr branch, the branch associated with this bug should have the proposed fix in it. You should be able to:
bzr branch lp:~jam/bzr/ftp-215522 test_bzr

and then run bzr from that directory:
cd my_project
../test_bzr push ftp://xs4all/......

you may want to test it with a trivial branch first, and then try a few repeated commits (bzr commit --unchanged helps here), and make sure all the pushes succeed.

Also, you can check your '.bzr.log' to see if the error is being trapped, it should have a comment about "Server returned ... Assuming it means empty directory". (Use 'bzr --version' to check where .bzr.log can be found).

[John A Meinel (jameinel)]

Just waiting to find out if the fix is valid