Refresh branch state takes too long time

Bug #500072 reported by Alexander Belchenko
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Bazaar Explorer
Confirmed
High
Unassigned

Bug Description

I have been experimenting with a large project branch.
The whole project contains several source code projects, some of which have been deleted from this test branch.

I recently updated QBZR and Bazaar Explorer. Now when I start or refresh, the program hangs for around 4 minutes then shows the lower message ' Branch has changes not present in its submit branch' followed by a list of ~5 thousand deleted files and a few modified files. However the top message 'Working tree is up to date at revision #' and there is nothing to commit.

It is quite possible my branch is not configured correctly. I just don't expect to keep seeing the files I've deleted or get the massive pause on each refresh. Whatever version software I had previously been using, I'm sure it did not behave like this.

This is intended to be a general question since I'm confused as to what is going on here.

Here are the program versions anyway:
bzrlib 2.0.0
bzrtools 2.0.0
exporer 0.10.0
qbzr 0.17.0
rebase 0.5.3
Python 2.5.4
PyQt 4.4.3
Qt 4.4.1

Revision history for this message
Alexander Belchenko (bialix) wrote :

As a workaround edit .bzr/branch/branch.conf file and remove line that started with

submit_branch = ...

Revision history for this message
Greg (gregspecialsource) wrote :

removing 'submit_branch = ...' from both source and destination branches had no effect on the 4min delay or the 'Branch has changes not present in its submit branch' message and pseudo change list.
No other references between the two branches exist (that I could find).

Revision history for this message
Alexander Belchenko (bialix) wrote : Re: [Bug 500072] Re: Refresh branch state takes too long time

Greg пишет:
> removing 'submit_branch = ...' from both source and destination branches had no effect on the 4min delay or the 'Branch has changes not present in its submit branch' message and pseudo change list.
> No other references between the two branches exist (that I could find).
>

Something really wrong here. We need more info how to reproduce this
problem.

Does your project is open source?

Revision history for this message
Greg (gregspecialsource) wrote :

I'm afraid the project is not open source.
The behavior I am seeing now only occurred after updating QBZR and Bazaar Explorer from the global release 2.0.
I updated QBZR in order to test a bug for that project and had to update Bazaar Explorer also due to compatibility issues. I have not updated any other modules / plugins.

The main steps I took were:
1) Branch from a large project with thousands of files
2) Delete a few thousand files from the new branch
3) Test updating 2 source code files between the two branches, back and forth
4) Update qbzr, explorer from official v2.0 file set.

The parent branch behaves fine, the new branch behaves strangely as described earlier.

Revision history for this message
Greg (gregspecialsource) wrote :

I just uninstalled bazaar and explorer and reinstalled v2.0.
The issue went away. So definitely related to something new.

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Greg wrote:
> I just uninstalled bazaar and explorer and reinstalled v2.0.
> The issue went away. So definitely related to something new.
>

Bazaar Explorer is effecting doing "bzr status -rsubmit:..-1" in that
feature branch. Try running that at the command line please. How long
does it take?

Ian C.

Revision history for this message
Greg (gregspecialsource) wrote :

Just reinstalled Bazaar Explorer v0.10.0...
Executing "bzr status -rsubmit:..-1" logged (I presume) the same list of deleted and modified files in ~2.5 seconds.
Opening the same project in Bazaar Explorer took ~4 minutes.

Revision history for this message
Craig Hewetson (craighewetson-deactivatedaccount) wrote :

I've noticed that "refresh action" seems to hold up the UI thread, can't this action rather be run in the background?
Also can't the progress bar be shown or some indication that the action is busy (maybe status view shows a message "Reloading..." etc) while the refresh action is running?

I experience similar performance and UI thread hold ups in the embedded qbrowse widget.

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

I wonder if I should make the "submit changes" panel an optional feature rather than always turned on. It would be good to get to the bottom of the slowness regardless.

Changed in bzr-explorer:
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

I've looked into adding a preference here to control whether the submit report is included or not. I think it will be better to make the set of reports configurable rather than make special preferences to enable/disable each report. I'm pretty concerned about the performance issue though. I guess having thousands of files changed in a branch is not a common occurrence though.

Revision history for this message
Greg (gregspecialsource) wrote :

I must admit I haven't got a clue what you are talking about with reports, special preferences etc. :)
I just see a bunch of changes that I don't want to see and get a long pause for some reason.
I have to comment that branches containing thousands of files were quite common with my previous employers projects.

Revision history for this message
Greg (gregspecialsource) wrote :

Clarifying last comment... I meant changes / additions / deletions of thousands of files, not just lots of files themselves.

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

To explain my comment re "reports", my vision is for the bottom half of the repository view and the bottom half (third?) of the status view to show useful "reports". Right now:

* the repo view has 3 of these: Recent History, Local Changes and Missing Revisions
* the status view has one: Submit Delta.

The Submit Delta report only appears for feature branches too.

Looking forward, the idea is that the user will be able to say, for each report, whether they want to see it on each view or not. So if you want to see Recent History on the status view, that will be easy to configure. Furthermore, developers will be able to add their own reports defined in XML. The decision as to where to render those custom reports, if anywhere, will be left to the user to configure. Make sense?

I'm curious about the frequent use of branches with 1000s of changes. What sort of data were you managing?

Revision history for this message
Greg (gregspecialsource) wrote :

Ahh! Thank you for explaining that. I was wondering what that bottom window was doing and why it sometimes showed those file lists.

In that case, selecting reports when needed makes sense and investigating any poor performance is also great.

As for 'frequent uses of branches with 1000s of changes'. Perhaps I exaggerate a little, but those were projects related to video games and CAD. There was a combination of individual developers with their own infrequently merged branches and periodic very large asset changes.

By the way, you are doing a top job Ian :)

Revision history for this message
Greg (gregspecialsource) wrote :

Just installed v2.1 which includes BZR Explorer v1.0rc1
Opening this particular project takes the usual several minutes. I see a progress bar visible in the status region however this bar never showed any progress during the load period. (The bar was empty the whole time.)
Just reporting in, I do not know if you have made changes regarding this issue.

Revision history for this message
Greg (gregspecialsource) wrote :

I did mention this in the first post, but the cause of this issue is the fact that I branched a multi project solution, and with the new branch, deleted a few sub projects totaling 5800 files.

Executing "bzr status -rsubmit:..-1" from the command line takes no significant time, but the explorer reports window still takes the 4 minutes per refresh, making the whole application unusable for the time being.

Just updated to v1.0 final and there is no change, which is expected since no fix was logged.

Revision history for this message
Greg (gregspecialsource) wrote :

I could possibly privately share my file-set, or better yet run a instrumented build to profile or log what is going on, if you want to provide such a build.

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Hi Greg,

Collecting some profiling information would be great. Here's what to do ...

1. Change to the directory holding the branch.
2. Run this command ...

   bzr --lsprof-file callgrind.out.500072 explorer .

3. Wait for the output to appear then close the application.
4. Attach the callgrind.out.500072 file to this bug report.

Thanks.

Revision history for this message
Greg (gregspecialsource) wrote :

I launched with that command, opened the slow project then closed the application.
I've attached the output as callgrind.out.zip

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Hi Greg,

Thanks for the profiling data. It's not telling me much unfortunately. I'm going to be out of the office for most of the next week so I'd like to get things working for you until I can dig deeper. Let's switch off the "Submit delta" report altogether and see if that addresses things. To do that, you'll need to patch the lib/view_workingtree.py file. Around line 40, immediately after this line:

    self.detail_views = []

add this line:

    return

Restart the app and see if things improve or not.

Thanks,
Ian C.

Revision history for this message
Greg (gregspecialsource) wrote :

That work around worked. The bottom window was gone and the slow project opened instantly.

Revision history for this message
Matt Wilkie (maphew) wrote :

I have this same problem. The code I'm using is public: lp:~leo-editor-team/leo-editor/contrib. Screenshot of what I see attached.

Revision history for this message
Alexander Belchenko (bialix) wrote :

Some improvements re speed of refresh landed to trunk. Please test.

tags: added: feel-faster multiprocess performance
Revision history for this message
Alexander Belchenko (bialix) wrote :

As analisys of https://bugs.launchpad.net/bzr-explorer/+bug/779529 reveals there is very poor performance of qt widget we're using for status/submit panes when we need to show many files with many links to different actions.

Can you confirm that you have worse performance when there are many files changed in your tree?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions