launchpad crashes when trying to view openteacher 2.x code online

[Arvind S Raj]

Click on "Browse the code" in https://code.launchpad.net/~openteachermaintainers/openteacher/2.x. I tried doing it four times-crashed every time.

The branch appears to be damaged:
'
$ bzr branch lp:openteacher/2.x
bzr: ERROR: No such file: ('information.png-20081222091428-1l7jkfpnqphdx5fa-1', '<email address hidden>')
'

and loggerhead - the code browser - failing is fallout from this.

OOPS-2029CBB101.

KeyError: '<email address hidden>'

    Traceback (most recent call last):
  Module launchpad_loggerhead.app, line 345, in wrapped_app
    app_iter = iter(app(environ, wrapped.start_response))
  Module launchpad_loggerhead.debug, line 106, in wrapped_application
    result = application(environ, response_hook)
  Module __main__, line 165, in wrapped
  Module launchpad_loggerhead.debug, line 45, in wrapped
    return app(environ, start_response)
  Module paste.translogger, line 68, in __call__
    return self.application(environ, replacement_start_response)
  Module paste.deploy.config, line 285, in __call__
    return self.app(environ, start_response)
  Module __main__, line 149, in wrapped
  Module paste.wsgilib, line 179, in catch_errors
    app_iter = application(environ, start_response)
  Module launchpad_loggerhead.session, line 53, in __call__
    return self.cookie_handler(environ, start_response)
  Module paste.auth.cookie, line 305, in __call__
    return self.application(environ, response_hook)
  Module launchpad_loggerhead.session, line 80, in _process
    return self.application(environ, response_hook)
  Module paste.httpexceptions, line 636, in __call__
    return self.application(environ, start_response)
  Module launchpad_loggerhead.app, line 230, in __call__
    return view.app(environ, start_response)
  Module loggerhead.apps.branch, line 180, in app
    return c(environ, start_response)
  Module loggerhead.controllers, line 104, in __call__
    values = self.get_values(path, self.kwargs, headers)
  Module loggerhead.controllers.inventory_ui, line 147, in get_values
    filelist = self.get_filelist(rev_tree.inventory, path, sort_type, revno_url)
  Module loggerhead.controllers.inventory_ui, line 91, in get_filelist
    change=change_dict[revid])
KeyError: '<email address hidden>'

[Deryck Hodge (deryck)]

I can't get at the OOPS info to fill in more info or work out if this is a known issue from other bugs.

[Arvind S Raj (arvindsraj-deactivatedaccount)]

Deryck,
Do you need another OOPS? Is that the issue? Here's another one anyway just in case. OOPS-2030CBA106.

[Francesco Del Degan (pr0gg3d)]

Hi, I'm trying to figure out what's wrong with that branch.
The first thing is that trying to get that branch locally, bzr starts to sync then fails with

$ bzr branch lp:openteacher/2.x
bzr: ERROR: No such file: ('information.png-20081222091428-1l7jkfpnqphdx5fa-1', '<email address hidden>')

Only the .bzr directory is present, no other files/directories.

a bzr log shows:

------------------------------------------------------------
revno: 143
committer: Marten de Vries <email address hidden>
branch nick: 2.x
timestamp: Fri 2011-07-08 17:00:02 +0200
message:
  Updated index.html file, needs testing
------------------------------------------------------------
.
.
.
.
------------------------------------------------------------
revno: 122
committer: Marten de Vries <email address hidden>
branch nick: 2.x
timestamp: Fri 2011-05-13 16:41:03 +0200
message:
  Removed accidentally added lines and deleted western charfus out of the cyrillic alphabet.

bzr: ERROR: Revision {<email address hidden>} not present in "Graph(StackedParentsProvider(<bzrlib.repository._LazyListJoin object at 0x9a9d8ec>))".

Seems that it lacks of revisions 1-121, and this leads to some tracebacks into loggerhead, complaining about some RevisionNotFound and other kind of exceptions, hence the oops.

In "files" view, a traceback occurs when loggerhead tries to render the "bin" directory, but it doesn't find the associated revision.

  File "/home/pr0gg3d/.bazaar/plugins/loggerhead/loggerhead/controllers/__init__.py", line 104, in __call__
    values = self.get_values(path, self.kwargs, headers)
  File "/home/pr0gg3d/.bazaar/plugins/loggerhead/loggerhead/controllers/inventory_ui.py", line 147, in get_values
    filelist = self.get_filelist(rev_tree.inventory, path, sort_type, revno_url)
  File "/home/pr0gg3d/.bazaar/plugins/loggerhead/loggerhead/controllers/inventory_ui.py", line 91, in get_filelist
    change=change_dict[revid])
KeyError: '<email address hidden>'
----------------------------------------

And in "changes" view, the revisions are re-numbered, where last revision 143 is shown as 25.

Before looking how to catch that conditions (maybe with a more graceful exit), I'm really curious about what have made that branch like it, any ideas?

[Marten de Vries (marten-de-vries)]

Does anyone know how we can get this branch stable again? It's our main branch for the 2.x series and we use it pretty often. I can just copy the data (we've got plenty of backups) into a new branch, but then we lose all the history which is sometimes useful.

[Robert Collins (lifeless)]

I'm marking the loggerhead task invalid - when dealing with corrupt data like this, oopsing is a valid thing to do :) - the launchpad task needs to stay open on the presumption we'll need to deploy a new bzr asap the cause is fixed in the server [if the server is at fault / can detect and prevent this in principle]

[Marten de Vries (marten-de-vries)]

'Before looking how to catch that conditions (maybe with a more graceful exit), I'm really curious about what have made that branch like it, any ideas?'

I'm just a user of bzr (so no idea about the internal working), but I can say that the local copies of the branch made before the launchpad version was corrupted (with bzr branch lp:openteacher/2.x) work normally and that we can even use them to pull and push from/to the launchpad master branch. (That's how we're developing now, we send each other emails with a non-corrupt version of the branch if someone doesn't have it... :P). But since it's only the master branch, is it possible that that one is corrupted by e.g. a hard disk failure?

[Robert Collins (lifeless)]

Have you tried push --overwrite to the branch?

[Martin Pool (mbp)]

Marten, if you attach a tarball of your working copy of the branch to this bug, perhaps the Launchpad operators can replace the copy on the server with that. I'll save a copy of it so that we can investigate this later.

[Martin Pool (mbp)]

OK, I can confirm I can branch from lp:openteacher ok, and bzr check on the result passes. I get the 'no such file' error branching from lp:openteacher/2.x. I have a local copy over sftp we can use to investigate the bzr bug (if any).

If push --overwrite does not fix this, you can try:

 - rename the branch to say 2.x-broken
 - push a new branch

[Martin Pool (mbp)]

apparently-damaged branches ,copied from launchpad

[Marten de Vries (marten-de-vries)]

> Have you tried push --overwrite to the branch?
Yes, but since the branch was already up-to-date nothing happened.

> If push --overwrite does not fix this, you can try:
>
> - rename the branch to say 2.x-broken
> - push a new branch
I've tried it two times. I once pushed with --overwrite, and once without. Both times it doesn't work, I think because it's still stacked on a (according to the file 814632-branches.tgz damaged) branch: 'Created new stacked branch referring to /+branch-id/450725.' I now reverted it all to the original situation.

> Marten, if you attach a tarball of your working copy of the branch to this bug,
> perhaps the Launchpad operators can replace the copy on the server with that.
> I'll save a copy of it so that we can investigate this later.
Done :)

Thanks everybody for the help!

[Martin Pool (mbp)]

Hi,

As far as I can tell the trunk is fine. Can't you branch from it?

[Marten de Vries (marten-de-vries)]

I didn't know that /+branch-id/450725 was the trunk, but indeed, I can branch from the trunk and it's fine.

Still lp:openteacher/2.x is broken, while the local branches work and are ok according to 'bzr check' as far as I know:

marten@marten-pc:~/Programming/Projects/openteacher/2.x$ bzr check
Checking working tree at '/home/marten/Programming/Projects/openteacher/2.x'.
Checking branch at 'file:///home/marten/Programming/Projects/openteacher/2.x/'.
Checking repository at 'file:///home/marten/Programming/Projects/openteacher/2.x/'.
checked repository file:///home/marten/Programming/Projects/openteacher/2.x/ format RepositoryFormat2a()
   211 revisions
   228 file-ids
checked branch file:///home/marten/Programming/Projects/openteacher/2.x/ format Branch format 7
marten@marten-pc:~/Programming/Projects/openteacher/2.x$

At the moment the branch situation is as follows:
- lp:~openteachermaintainers/openteacher/2.x-broken <- the renamed version we used until the problems started
- lp:~openteachermaintainers/openteacher/2.x (lp:openteacher/2.x) <- my local version pushed to the by the rename freed location.
- lp:~openteachermaintainers/openteacher/3.x (was: lp:~openteachermaintainers/openteacher/trunk , is lp:openteacher and lp:openteacher/3.x) <- fine.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

> At the moment the branch situation is as follows: -
> lp:~openteachermaintainers/openteacher/2.x-broken <- the renamed
> version we used until the problems started -
> lp:~openteachermaintainers/openteacher/2.x (lp:openteacher/2.x) <-
> my local version pushed to the by the rename freed location. -
> lp:~openteachermaintainers/openteacher/3.x (was:
> lp:~openteachermaintainers/openteacher/trunk , is lp:openteacher
> and lp:openteacher/3.x) <- fine.
>

So is your newly pushed-up branch still broken?

The original traceback is indicating that a file is claiming to come
from a revision which isn't in the history of the branch. (Roughly,
I'm not 100% sure on that.)

You're saying that 3.x is fine, I'm wondering if there is just a bad
revision in the history of 2.x that is being processed.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5VIUEACgkQJdeBCYSNAAPLrQCbBbLw/8PWcSDMKPgkdT94nvhu
BUwAoJRBJ5LC/o7GO6ZAv9qY73SGfetp
=BcUC
-----END PGP SIGNATURE-----

[Marten de Vries (marten-de-vries)]

'So is your newly pushed-up branch still broken?'
Yes. (e.g. OOPS-ID OOPS-2062CBA489)

'You're saying that 3.x is fine, I'm wondering if there is just a bad
revision in the history of 2.x that is being processed.'

If that's the case, I've no idea how that could've been introduced. We used that revisions (I saw revno: 122 a few posts back) for a pretty long time before they started giving problems.

[Marten de Vries (marten-de-vries)]

I tried to push the branch to a new location with the --stack-on option. I chose the lp:openteacher branch for that, which has no relation to the 2.x branch (it's completely different code), so I think everything got uploaded from my harddisk (it took longer than it normally does), and now it works :D I updated some names, and now everything works as it did before. Thanks everybody for your help!

[Colin Watson (cjwatson)]

I don't know whether this was fixed in the branch itself or whether it was fixed somewhere along the way by porting Launchpad's code hosting to Breezy, but either way I can no longer reproduce this either by visiting https://bazaar.launchpad.net/~openteachermaintainers/openteacher/2.x/files or by branching this directly using bzr, so it seems best to close this. Feel free to reopen it if the same problem still exists somewhere I didn't notice.