Delay between blob submission and blob availability causes Launchpad to OOPS.

Bug #311690 reported by Dave Jeffery
92
This bug affects 10 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Undecided
Graham Binns
launchpad-integration (Ubuntu)
Critical
Unassigned

Bug Description

Binary package hint: launchpad-integration

1) Ubuntu 8.10 - the Intrepid Ibex

2) rhythmbox:
  Installed: 0.11.6svn20081008-0ubuntu4.2
  Candidate: 0.11.6svn20081008-0ubuntu4.2
  Version table:
 *** 0.11.6svn20081008-0ubuntu4.2 0
        500 http://hu.archive.ubuntu.com intrepid-updates/main Packages
        100 /var/lib/dpkg/status
     0.11.6svn20081008-0ubuntu4 0
        500 http://hu.archive.ubuntu.com intrepid/main Packages

3) A page to appear in Launchpad when I selected "Report A Problem"

4) A Page Not Found message appeared

Related OOPSes: OOPS-1103D2789, OOPS-1103H2820

Related branches

Revision history for this message
Lupine (thelupine) wrote :

I can confirm this is happening on Ubuntu 8.10 for me too. Not only rhythmbox is affected, but so far every app I have tested. For example, FireFox tries to go to this URL if I repeat this process for Evolution:

https://bugs.launchpad.net/ubuntu/+source/evolution/+filebug/6Wfu7sNmvj9mrbDEtCEX5ftjMo3?

Changed in launchpad-integration:
status: New → Confirmed
Revision history for this message
Sarah Kowalik (hobbsee-deactivatedaccount) wrote :

Ouch.

This would be blocking everyone from using these options, for any program. Marking critical.

Changed in launchpad-integration:
importance: Undecided → Critical
Revision history for this message
Graham Binns (gmb) wrote :

I can reproduce this using gedit. I can't tell whether this is a problem with launchpad-integration or with Launchpad Bugs, so I'm adding this task and will investigate tomorrow.

Changed in malone:
assignee: nobody → gmb
milestone: none → 2.2.1
Revision history for this message
Sarah Kowalik (hobbsee-deactivatedaccount) wrote :

Note the last comment of https://bugs.edge.launchpad.net/ubuntu/+source/python-launchpad-bugs/+bug/309307/comments/17 - looks like it's launchpad side, where it hasn't generated the report quickly enough.

Revision history for this message
Graham Binns (gmb) wrote :

As Sarah pointed out, waiting and refreshing makes everything work nicely. The added token at the end is generated by Launchpad and returned to the launchpad-integration package in an X-Launchpad-Blob-Token header (James Westby explained this on IRC).

So it appears that Launchpad is generating a token for the uploaded blob and then pretending that the token doesn't exist, which is positively weird.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

This could be fallout from the replicated db setup?

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 311690] Re: Report A Bug Launchpad page missing for Rhythmbox

On Mon, Jan 05, 2009 at 12:35:13AM -0000, Michael Hudson wrote:
> This could be fallout from the replicated db setup?
>

That's my suspicion, too. I'll investigate and see if that's the case.
If it is, it should be trivial to fix.

Revision history for this message
Graham Binns (gmb) wrote :

Setting this to invalid for launchpad-integration; the problem is with Launchpad itself.

Changed in launchpad-integration:
status: Confirmed → Invalid
Revision history for this message
Connor Imes (ckimes) wrote :

This was opened and Confirmed against Launchpad itself and apport in bug 310016 which has been marked as a duplicate of this one. I'll re-apply that affects/status here since that seems to be the source of the problem (not apport).

Changed in launchpad:
status: New → Confirmed
Revision history for this message
Graham Binns (gmb) wrote :

I'm marking the Launchpad task invalid since the bug is specifically in the Launchpad bug tracker, for which there is already a task.

Changed in launchpad:
status: Confirmed → Invalid
Revision history for this message
Stuart Bishop (stub) wrote :

This is almost certainly fallout from database replication. During normal operation, it will take 30-60 seconds for the information about the upload to propagate to the replica database. It is quite likely that when attempting to view this page with a web browser, the replica database is hit and the information doesn't yet exist.

It may be a simple solution to force the use of the master for this particular page. This would be a malone fix.

An alternative is to update the publisher so, if a 404 is generated and the replica database used, the request is retried using the master database (quite similar to the Retry mechanism used to handle database deadlocks and serialization exceptions). This would be an infrastructure fix.

Revision history for this message
Stuart Bishop (stub) wrote :

Confirming as an infrastructure issue - I think we can preemptively solve similar issues too.

Changed in launchpad:
importance: Undecided → Medium
status: Invalid → Triaged
Revision history for this message
Stuart Bishop (stub) wrote :

I'll see if my approach is sane. It would be nice if we only had to fix this class of problems once (codehosting has a similar race condition - push a branch and go to the URL and it isn't there yet).

Changed in launchpad:
assignee: nobody → stub
milestone: none → 2.2.1
Revision history for this message
Francis J. Lacoste (flacoste) wrote :

It makes sense to fix at the infrastructure level.

Changed in launchpad:
importance: Medium → High
Changed in launchpad:
milestone: 2.2.1 → none
Ursula Junque (ursinha)
description: updated
Revision history for this message
Graham Binns (gmb) wrote :

Marking this Won't Fix for malone since Stuart's solution should take care of the problem across all Launchpad apps.

Changed in malone:
status: New → Won't Fix
Stuart Bishop (stub)
Changed in launchpad-foundations:
milestone: none → 2.2.1
status: Triaged → In Progress
Revision history for this message
Björn Tillenius (bjornt) wrote :

Stuart, will the fix for this also make the Librarian retry using the slave store, if a LibraryFileAlias can't be found? If not, I suspect that the problem that was original reported will still persist, although instead of a 404, it will oops like OOPS-1104F1950.

Or does the Librarian always use the master store?

Revision history for this message
Francis J. Lacoste (flacoste) wrote :

The librarian always use the master store.

Revision history for this message
Stuart Bishop (stub) wrote :

Request for this to be cherry picked, so I guess this is critical.

Changed in launchpad-foundations:
importance: High → Critical
Revision history for this message
Sean Fenton (bobodod) wrote :

I've found that navigating away from the generated Launchpad bug report page & coming back to it causes it to work. I made mention of this happening in a bug report I filed for another package & was directed here. My comments are in bug 311041.

I just tried this again, running "ubuntu-bug gourmet" to generate a new bug report. A Launchpad URL w/gobbledygook was launched & reported: "Lost something? There’s no page with this address in Launchpad. Check that you entered the address correctly." It opened the page for the correct package, however.

I navigated away from that page & used the back button to return to it after a short time & the report then worked. This makes sense to me now in light of what has been said above.

Revision history for this message
Stuart Bishop (stub) wrote :

r7556

Changed in launchpad-foundations:
status: In Progress → Fix Committed
Stuart Bishop (stub)
Changed in launchpad-foundations:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers