Smart server support for upgrading branches/repositories/etc server-side

[Andrew Bennetts]

It would be good to have a smart server request to upgrade the format of a remote branch/repository/bzrdir. This could do all the work on the server, rather than having to transfer all the history back-and-forth across the network so that the client can do the upgrade and then re-upload it.

[Martin Pool (mbp)]

Braindump notes (may be out of date or wrong) from working on this with thumper some time ago:

we want this to turn in to a single rpc to upgrade

changes:

 * just remove the special case for RemoteBzrDir

 * RemoteBzrDir.needs_format_conversion always returns false
   saying no upgrade is needed

 * separate ui code from actual upgrades - maybe separate it
   into a result object? or some kind of view? or just pass the progress
   bar object, as a kind of special facet of the ui, in to the
   backup method, and then the remote one can feed it with data from
   across the network. should that be a separate rpc
   from actually doing the upgrade? could be a place to
   handle removing the backup directory.

 * wonder if there should be a generic concept of the smart server
   giving progress indicators and messages or ui output separate from
   its streamed body.

 * in some cases we do the upgrade in several steps by repeatedly getting
   converters that are supposed to take it closer to the final format -
   presumably that should be done over the network

issues in writing an rpc:

 * how to communicate format objects across the network -- maybe just pass
   the format string through

 * progress indication - just stream back some arbitrary bytes showing
   stuff is happening - or leave this til later. could echo back e.g.
   the revision ids that are being upgraded, or parameters to
   pb.update

other bugs here:

https://bugs.edge.launchpad.net/bzr/+bugs?field.tag=upgrade

http://tinyurl.com/8r7e2e -- all containing "upgrade", many not relevant

* bug 300001 if there's already a backup dir, it makes the next upgrade attempt fail: it
  should either remove it, or make a numbered backup

* bug 317732 it's hard to do anything with the backup directory if you
  can't get at the remote filesystem: you can't remove it if
  it succeeded, and you can't restore it if it failed

* bug 262450: backup directory is world-readable; it should inherit the
  permissions of .bzr (or maybe be inside it)

* bug 145812 it might be nicer to automatically roll back if the upgrade fails
  (but it might fail because we were interrupted)

* poor messages about upgrades of rich-root vs not; the master seems to
  be bug 252908M#

* bug 163908 should tell you what/where to upgrade

* bug 173061 confusing message if no upgrade needs to be done

* bug 59182 upgrade warns that the object needs to be upgraded
  as it opens it, even though it's just about to upgrade it

* and more generally, having shorter names for whole formats

[Dimitri John Ledkov (xnox)]

Remote branch upgrade uses a lot of memory just under 300 MB for a ~100MB repository. It's too much IMHO.