exporting to bzr seems broken since a few days

[SirVer]

Since a few days, the daily exports of the launchpad translations to my branch no longer work.

The translations from
https://translations.launchpad.net/widelands/trunk/+translations
should be exported daily to
lp:~sirver/widelands/translations
The last time this has happened was on 2010-05-04. Could this be related to the latest launchpad rollout?

Yet again, I am unsure if this is the correct project to file this bug, but I sure hope it can be quickly fixed.

[Данило Шеган (danilo)]

We are seeing this in our log files as well with the following error message:

2010-05-08 03:59:11 INFO Exporting widelands trunk series.
2010-05-08 03:59:59 ERROR Failure: ConcurrentUpdateError('Branch has been changed. Not committing.',)

This is likely related to branch-scanner problems just before the latest Launchpad rollout. It seems there are a bunch of branches which have remained in the "updating branch" state. Quick workaround for you should be to just push a dummy revision (try using "bzr commit --unchanged" in that branch and then pushing it back to LP), but it's definitely something for our code guys to look into.

Example branches are lp:~sirver/widelands/translations, lp:~walmis/blueman/po, lp:~menesis/schooltool.intervention/translations and so forth (we've got around 30 branches that we are trying to export to that exhibit the same problem). None of them have been changed since something like 2010-05-05.

[Tim Penhey (thumper)]

 https://pastebin.canonical.com/32129/ shows all hosted branches that have a next_mirror_time set.

This will make the pending_writes attribute true. I don't think this should be set, and will investigate further.

[Michael Hudson-Doyle (mwhudson)]

I think what's going on here is that I didn't modify the directbranchcommit code properly when I did the no-hosted-area changes.

In particular DirectBranchCommit.commit should call branchChanged on the branch, not requestMirror -- calling requestMirror will not do anything useful for a hosted branch as the puller never runs for hosted branches (it doesn't mean anything any more!).

The change should be fairly simple.

Sorry about this :/

[Данило Шеган (danilo)]

Ok, the problem seems to definitely be in the code-side of things: it'd be nice to get a fix and get it CPed asap. After fix gets into production we can think about "fixing" all the branches that are still in the inconsistent state.

[Jonathan Lange (jml)]

Does this point to a gap in our test suite?

[Aaron Bentley (abentley)]

Perhaps we should error if requestMirror is called on a hosted branch.

[Michael Hudson-Doyle (mwhudson)]

It means we don't have a completely end-to-end test of translations export: if we'd had a test that set up an export, ran it, pulled and scanned the branch and checked that the revision appeared on the branch index page, it would have caught this. But we don't have any tests like this.

It also doesn't seem the call to requestMirror in directbranchcommit is tested, although if even it was I'd guess it's pretty unlikely that it would have caught this.

I guess requestMirror on hosted branches should explode somehow: in this new world it will never do anything useful.

[Michael Hudson-Doyle (mwhudson)]

Oh, I should say that one thing that probably _would_ have caught this would have been me QAing the translations export on staging. It was an omission of mine to not do this.

[SirVer (sirver)]

When can I as a user expect our branch to be fixed? Could you provide me with a approximate timeframe?

[Michael Hudson-Doyle (mwhudson)]

If you lock and unlock the branch using bzrlib, the next run of the exporter should succeed. I've done that for this branch, so the next run should work.

A real fix should be done in the next day or so.

[SirVer (sirver)]

thanks for the quick help!

[Данило Шеган (danilo)]

I'll be taking care of CPing this fix, but Michael has written an actual fix. I'll do some QA on staging first and then go through CP process.

[SirVer (sirver)]

Just want to mention, that my branch

https://code.launchpad.net/~sirver/widelands/translations

has still not changed. The problem persists: no new translations are merged and the branch is still stuck in the "Updating branch... Please comback in some minutes" state. Could someone please reinvestigate this?

[Данило Шеган (danilo)]

The bug was not yet fixed. We'll be rolling a fix out soon. After that, we'll have to clean up all the branches that have gotten into this inconsistent state.

[Ursula Junque (ursinha)]

Fixed in stable r10858 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/10858>

[Данило Шеган (danilo)]

Fix has been released. bzr lock/unlock trick should work for those that are in a hurry (or pushing a new revision): for the rest we'll have to get some help from the code guys to "reset" other branches.

[Jeroen T. Vermeulen (jtv)]

The fix is still waiting for a cherrypick rollout.

[Данило Шеган (danilo)]

Fix was CPed on 13th, but we haven't reset the branches yet, nor have we applied the DB permissions fix.

[Данило Шеган (danilo)]

DB permissions fixed, we just need to reset branches. For affected users: you can do this yourself by committing a revision to the repo (if you are out of ideas of what to commit, just do a 'bzr commit --unchanged -m "Trigger a branch scan."; bzr push'), but we'll try to get this sorted out today.

[Данило Шеган (danilo)]

Affected branches up to today: https://pastebin.canonical.com/32439/

[Aaron Bentley (abentley)]

All affected branches have had their next_mirror_time cleared, and scans have been triggered. They should now be in a normal state, and ready to accept revisions on the next translations export run.

[Данило Шеган (danilo)]

All branches should have received latest translation updates.