cannot squash incomplete branch on supermirror sftp

[David Allouche]

In some situations (bzr bugs, sftp server bugs, manual fiddling, connection failure) the sftp server can end up with a branch directory (~user/product/branch) that is not a proper bzr branch branch.

bzr cannot push on this location because it's not a proper branch, and it cannot be deleted because of bug 34540.

Currently, recovering from this situation requires manual surgery using a sftp client. There should be a way to tell bzr "push on this sftp location and delete any pre-existing contents of the directory". This would also be useful in access control scenarios. The deletion of existing files can be racy (ls then rm), and the target location may not be deletable.

Related to bug 45504 and bug 30576.

[John A Meinel (jameinel)]

Some sort of --force or --overwrite to recursively delete a tree would be reasonable. (There already is a transport function 'delete_tree' which would do the work).

[David Allouche (ddaa)]

There is already a "bzr push --overwrite" that does a different thing: it allows pushing over a diverged branch, but retains the existing data to avoid doing a push from scratch.

I think "bzr push --delete" (preferred) or "bzr push --from-scratch" would be good names, as the most prominent characteristic of this operation is to delete remote data before proceeding normally.

That option should cause the deletion of all the contents of the target directory, then do a normal push. Data formats and presence of a working tree on the remote would not preserved (note: that gives a nice alternative to the slow "bzr upgrade" on remote branches). If the branch is part of a repository, the repository's bzr would be left alone (if the repository is corrupt, you have A Bigger Problem).

[Jonathan Lange (jml)]

Another approach (suggested by thumper) is to get the sftp server to check to see if the existing push location constitutes a valid bzr branch. If it doesn't, we could blow it away and let bzr push normally.

[Andrew Bennetts (spiv)]

bzr 0.15 should behave better in this case.

I don't think the SFTP server should try to detect a broken branch. It has the potential to clash badly with two connections trying to push a new branch at the same time, which doesn't seem impossible or even totally unlikely (two devs independently decide to make a "bug-1234" team branch in response to a bug report, and both have slow connections from e.g. Brazil).

If the server does need more capability to help users here, then the smart server will be a much safer system to build this on.

[Bubba Siggler (bud3)]

I not sure if this is a duplicate on the edgy box it was the opposite bzrtools old. I have a bad setup on both.

~$ bzr info
Installed Bazzar version 0.15.0candidate2 is too old to be used with plugin
"Bzrtools" 0.16.0.
Unable to load plugin 'bzrtools' from '/home/name/.bazaar/plugins'
bzr: ERROR: Not a branch: /home/name/

Here is a selftest. I think the GTk is unstable that what making the bzrtools read 0.16

Plugin "gtk" tried to reset default encoding to: utf-8
[145/6132 in 19s] test_bzrdir.TestRemoteSFTP.test_open_containing_tree_or_branc/var/lib/python-support/python2.5/paramiko/message.py:226: DeprecationWarning: integer argument expected, got float
  self.packet.write(struct.pack('>I', n))
[6132/6132 in 1005s, 376 skipped] bzrlib.plugins.launchpad.test_lp_indirect.Ind
----------------------------------------------------------------------
Ran 6132 tests in 1005.968s

OK
376 tests skipped

[Bubba Siggler (bud3)]

I was looking and it say TESTREMOTE_SFTP isn't suppose to be a shared.

[Bubba Siggler (bud3)]

On 3/15/07, Jonathan Lange <email address hidden> wrote:
>
> Another approach (suggested by thumper) is to get the sftp server to
> check to see if the existing push location constitutes a valid bzr
> branch. If it doesn't, we could blow it away and let bzr push normally.
>
> ** Changed in: launchpad-bazaar (upstream)
> Assignee: (unassigned) => Jonathan Lange
>
> --
> cannot squash incomplete branch on supermirror sftp
> https://launchpad.net/bugs/53340
>

   Hi Jonathan
    Bubba worte
     I don't want to slow or damage any pc. I Love it when everything is
running smooth. That said What should we do with this broke pc blow it away
as in format.

  Bubba

--
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFFkcN7yXWcajQQndYRAgbqAKCMyXN9Jx4g0X7jocg+aUSFz0x4LwCgrURW
eGtqLjpzQVYa9+gzpCRtB84=
=zrpM
-----END PGP SIGNATURE-----

[David Allouche (ddaa)]

Note that bug 45504 and bug 30576 are fixed in the upcoming bzr 0.15.

[Alexander Sack (asac)]

bzr 0.11 failed to do initial push to launchpad:

18:49 < asac> asac@hector:~/ubuntu_nobackup/trunk_pkg/firefox-trunk$ bzr push --create-prefix
              sftp://<email address hidden>/~asac/firefox/trunk
18:49 < asac> bzr: ERROR: Transport operation not possible: This is not a LocalTransport, so there is no local
              representation for a path %

after this point i updated to latest 0.15 in order to resume push, which failed like

18:57 < asac> ddaa: asac@hector:~/ubuntu_nobackup/trunk_pkg/firefox-trunk$ bzr push --create-prefix
              sftp://<email address hidden>/~asac/firefox/trunk
18:57 < asac> Unable to load plugin 'svn' from '/usr/lib/python2.4/site-packages/bzrlib/plugins'
18:57 < asac> bzr: ERROR: At sftp://<email address hidden>/~asac/firefox/trunk you have a valid .bzr control
              directory, but not a branch or repository. This is an unsupported configuration. Please move the
              target directory out of the way and try again.

[David Allouche (ddaa)]

This error message is misleading. This bug is about people NOT having to move the target directory away when a push to launchpad failed. Bzr should be able to recover from this situation.

Currently, the user has to sftp manually into launchpad, then delete the broken .bzr, then push again with --use-existing-dir.

At least, the error message should provide those instructions. At best bzr should be able to just ignore and overwrite an existing broken .bzr when --use-existing-dir is specified.

[Bubba Siggler (bud3)]

Now that I formated my box an not so confused. I should of not made a format. It seem like the problem was causing havoc with me an that was a stupid thing to do an not Know what to do was phfeaking me out

Ok if this is still a bug that needs fixing

[Jonathan Lange (jml)]

If the supermirror isn't going to gain Bazaar smarts, then there's little that can be done on the Launchpad side. It's behaving perfectly well as an SFTP server as far as this bug is concerned.

[Jonathan Lange (jml)]

I meant to say "if the supermirror sftp server isn't going to gain Bazaar smarts"

[Robert Collins (lifeless)]

On Wed, 2007-04-25 at 01:34 +0000, Jonathan Lange wrote:
> I meant to say "if the supermirror sftp server isn't going to gain
> Bazaar smarts"

I dont think thats relevant. The problem can occur with any sftp server
so we should fix it in bzr. I thought we had fixed it in bzr though.

Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

[Jonathan Lange (jml)]

It's relevant for why it's not a launchpad-bazaar bug.

[Martin Pool (mbp)]

(I was just confused by Malone's dual statuses on this bug.)

The current state is ok: rejected in launchpad, confirmed in bzr.

In fact this can even happen locally, if you happen to terminate bzr
at just the right moment though the window is much smaller.

--
Martin

[Matthew Paul Thomas (mpt)]

Robert wanted to know what such an error looks like.

19:15:52@2007-04-bug-90545> bzr push --overwrite bzr+ssh://[...]/2007-04-bug-90545 && bzr pqm-submit -m "[...]"
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
bzr: ERROR: Target directory bzr+ssh://[...]/2007-04-bug-90545 already exists, but does not have a valid .bzr directory. Supply --use-existing-dir to push there anyway.
19:16:39@2007-04-bug-90545> bzr push --use-existing-dir bzr+ssh://[...]/2007-04-bug-90545 && bzr pqm-submit -m "[...]"
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
Enter passphrase for key '/home/mpt/.ssh/id_dsa':
bzr: ERROR: File exists: '//[...]/2007-04-bug-90545/.bzr'

[Robert Collins (lifeless)]

I hit a variation of this - and here is what I think makes a complete list:

directory, no .bzr
empty .bzr
.bzr with branch-format etc, no /repository, /branch
.bzr with empty repository dir, no /branch
.bzr with repository with format marker, no /branch
.bzr with repository, empty /branch
.bzr with repository, branch with format marker

We probably have more conditions that a re slight variations. I like aarons proposal to do rename-into-place for each thing to remove drop-link error cases.

[Bubba Siggler (bud3)]

I am glad that the people on this list made me part of this team. I just need to be the first to find it or the last one to fix it.

[Bubba Siggler (bud3)]

Please for give me in what I said. The last to fix it. It's Gods grace that can tie my shoes in the morning. If It's ok to leave this as it is and let the MOTU Hard code it. John has to much to work on as it is, don't need me hang on his heel with mostly it same dum questions.

Regards

BS

[John A Meinel (jameinel)]

Are we able to close this? I know it is something we want to continue to monitor, but I believe the current documentation all uses " not '.

[Bubba Siggler (bud3)]

When I was give a chance to work with MOTU in Bazaar. I responded by trying to learn what bzr was and learn how to set it up. The first place of course was the wiki to find out what it was. The Cathedral an Bazaar is what came up. I then googled it on the wiki ubuntu. I found a old docs on how to set it up create the folders an so. what I started with and the upgraded with bazaar I think 14 then 15. Stupid is that I save the folder I created with the carrot sign and a numeral one. What seem to me is that old howto and the start of dev.bzr got mixed. I am not even sure if it did just a thought. I have not been able to find the old and if you need the name of the setup folders.

[Martin Pool (mbp)]

Is this maybe fixed now?

[Jonathan Lange (jml)]

To be honest, I'm finding it very difficult to understand what this bug is actually about. I'll mark it as invalid, and if it's still a bug then we can file a newer, clearer bug with well-defined success criteria.