codebrowse fails with infinite redirections

[David Allouche]

When codebrowse.launchpad.net is broken, it fails by causing infinite redirections.

Instead, it should fail with a http 500 error code, like all sane web servers.

[David Allouche (ddaa)]

This failure more is wrong on so many levels: causing the user to wait a long time to get a failure, preventing informative failure page, causing pointless traffic, plain creative silliness. So making it High importance.

[James Henstridge (jamesh)]

Could you get a trace of the requests, or even a URL plus request time? That would help me track this down.

[James Henstridge (jamesh)]

On IRC, Fujitsu mentioned this bug to me. I tracked down the log file entries, and found the Berkeley DB "run recovery" exceptions. The result of this was the $branch/changes URL redirecting to itself.

Looking at all the cases in the log file where $branch/changes redirected, there were DBRunRecoveryError exceptions so I think this is the trigger.

DEB [20070314-05:06:22.914] turbogears.database: Starting SQLObject transaction
ERR [20070314-05:06:24.564] loggerhead.trunk: Exception fetching changes: (-30977, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: Invalid argument')
DEB [20070314-05:06:24.564] loggerhead.trunk: Traceback (most recent call last):
DEB [20070314-05:06:24.564] loggerhead.trunk: File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/controllers/changelog_ui.py", line 70, in default
DEB [20070314-05:06:24.565] loggerhead.trunk: entries = h.get_changes(entry_list)
DEB [20070314-05:06:24.565] loggerhead.trunk: File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
DEB [20070314-05:06:24.565] loggerhead.trunk: return unbound(self, *args, **kw)
DEB [20070314-05:06:24.565] loggerhead.trunk: File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/history.py", line 596, in get_changes
DEB [20070314-05:06:24.565] loggerhead.trunk: changes = self._change_cache.get_changes(revid_list, get_diffs)
DEB [20070314-05:06:24.565] loggerhead.trunk: File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
DEB [20070314-05:06:24.566] loggerhead.trunk: return unbound(self, *args, **kw)
DEB [20070314-05:06:24.566] loggerhead.trunk: File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/changecache.py", line 112, in get_changes
DEB [20070314-05:06:24.566] loggerhead.trunk: cache[sfetch_list.pop(0)] = out[i] = changes.pop(0)
DEB [20070314-05:06:24.566] loggerhead.trunk: File "/usr/lib/python2.4/shelve.py", line 130, in __setitem__
DEB [20070314-05:06:24.566] loggerhead.trunk: self.dict[key] = f.getvalue()
DEB [20070314-05:06:24.566] loggerhead.trunk: File "/usr/lib/python2.4/bsddb/__init__.py", line 224, in __setitem__
DEB [20070314-05:06:24.566] loggerhead.trunk: self.db[key] = value
DEB [20070314-05:06:24.567] loggerhead.trunk: DBRunRecoveryError: (-30977, 'DB_R
UNRECOVERY: Fatal error, run database recovery -- PANIC: Invalid argument')
DEB [20070314-05:06:24.567] loggerhead.trunk:
INF [20070314-05:06:24.568] launchpad_loggerhead: Getting branch for ~bzr/bzr/trunk
INF [20070314-05:06:24.569] launchpad_loggerhead: Getting branch for ~bzr/bzr/trunk
INF [20070314-05:06:24.570] launchpad_loggerhead: Getting branch for ~bzr/bzr/trunk
INF [20070314-05:06:24.570] turbogears.access: 82.211.81.156 - - "GET /%7Ebzr/bzr/trunk/changes HTTP/1.1" 302 163 "" "Mozilla/5.0 (X11; U; Linux i686; en; rv:1.8.1.2) Gecko/20060601 Epiphany/2.16 Firefox/2.0.0.2 (Ubuntu-edgy)"

[James Henstridge (jamesh)]

Looks like the error recovery procedure for a lot of the pages is to redirect to $branch/changes. These should definitely be 500 errors.

[James Henstridge (jamesh)]

Robey: the branch linked to this bug changes the error handling in the loggerhead.controllers to display error pages instead of redirecting to $branch/changes.

When combined with the Berkeley DB corruption issues, this could lead to redirect loops, and in any case it is better to tell the user about the problem.

[James Henstridge (jamesh)]

Fixed for codebrowse.launchpad.net

[Robey Pointer (robey)]

makes sense to me. i was being naive. merged in 120.

[Michael Hudson-Doyle (mwhudson)]

Released in 1.2.