on the fly conversion seems to cause issues for some repositories

Bug #596114 reported by Toshio Kuratomi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
New
Undecided
Unassigned

Bug Description

I'm not really sure what's going on here but I do have a reproducer so I'm going to file and see if anyone has any ideas:

On my server, I have a shared repo with an older format:

bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel

$ bzr info
Shared repository (format: pack-0.92)
Location:
  shared repository:

On my client, I have bzr-.2.1.1:
$ bzr --version
Bazaar (bzr) 2.1.1
  Python interpreter: /usr/bin/python 2.4.3
  Python standard library: /usr/lib/python2.4
  Platform: Linux-2.6.18-194.3.1.el5xen-i686-athlon-with-redhat-5.5-Tikanga
  bzrlib: /usr/lib/python2.4/site-packages/bzrlib
  Bazaar configuration: /home/fedora/toshio/.bazaar
  Bazaar log file: /home/fedora/toshio/.bzr.log

If I do a branch to a regular directory, things work fine:
$ bzr branch bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel
Branched 571 revision(s).

If I first create a shared repository and branch into there, the process gets stuck at some point:
$ bzr init-repo .
Shared repository with trees (format: 2a)
Location:
  shared repository: .
$ bzr branch bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel
Doing on-the-fly conversion from RemoteRepositoryFormat(_network_name='Bazaar pack repository format 1 (needs bzr 0.92)\n') to RepositoryFormat2a().
This may take some time. Upgrade the repositories to the same format for better performance.
/ 16803KB 290KB/s | Fetching revisions:Inserting stream

At this point, CPU usage by the bzr smart server on the server jumps up. Meanwhile, according to strace the client is just waiting for more data to be sent.

Something about shared repositories appear to be causing the issue. Any clue?

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 596114] [NEW] on the fly conversion seems to cause issues for some repositories

If the process hangs for (say) an hour, and server isn't in deep swap
and thrashing, then we should get a backtrace in python of the server
thread and see where it is at.

However, if the process does resume, then its explanable: the server
is doing an on the fly conversion to 2a, and the first bit converted
is the inventory, which is also the crucially bad-big-O component of
the pack-92 series of formats.

-Rob

Revision history for this message
Toshio Kuratomi (toshio) wrote :

Okay, that's what it is::

$ time bzr branch bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel
Doing on-the-fly conversion from RemoteRepositoryFormat(_network_name='Bazaar pack repository format 1 (needs bzr 0.92)\n') to RepositoryFormat2a().
This may take some time. Upgrade the repositories to the same format for better performance.
Branched 571 revision(s).
bzr branch bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel 42.01s user 1.98s system 3% cpu 21:48.86 total

22 minutes to complete the branch.

I'll mark this as a duplicate of #456541 as there's probably not a huge priority on speeding up pack-92 but the changes proposed in that bug would make it clearer that we've hit the stage of the conversion where little progress will be made for a while.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 596114] Re: on the fly conversion seems to cause issues for some repositories

Thanks for confirming; perhaps you could set a nag-flag in the fedora
repos to upgrade to 2a?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.