bug watches on archived debbugs do not update

Bug #380362 reported by Paul Gevers
86
This bug affects 15 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
High
Unassigned

Bug Description

Several of the bugs that I watch were impossible for Launchpad to import, which is a shame, because they were marked "done". The link in Launchpad does work, and in at least one bug the status was previously updated as well.

No idea why Launchpad could not import those bugs where elsewhere it works.

Bug 340633
Bug 124573
Bug 277483
Bug 200659
Bug 379721
Bug 271994

Tags: lp-bugs
Revision history for this message
Lars Ljung (larslj) wrote :

I've seen this too, e.g. in bug 200659

Changed in launchpad:
status: New → Confirmed
Revision history for this message
Nick Andrik (andrikos) wrote :

Also for this bug
https://bugs.launchpad.net/ubuntu/+source/pidgin-facebookchat/+bug/379721
which has been fixed in debian but launchpad cannot sync

Revision history for this message
Paul Gevers (paul-climbing) wrote : Re: [Bug 380362] Re: Launchpad couldn't import several bugs from Debian Bug tracker.

Just for the record in this bug, I manually set the status of the debian
bugs in my packages, so you won't see this bug anymore in the header of
the bugs in the (original) description of this bug.

Paul

Dara Adib (daradib)
description: updated
Revision history for this message
Sven Eckelmann (sven-eckelmann) wrote : Re: Launchpad couldn't import several bugs from Debian Bug tracker.

I have the same problem with #259333. The most annoying part is that no debian package is set for that bug and I cannot mark it as such because it tells me always "Package mupen64plus not published in Debian". Even if launchpad's search for packages finds mupen64plus and src:mupen64plus (this could also be related to PPAs in ubuntu - not sure about that).
So I cannot set the bug for debian as done.

Revision history for this message
Rockwalrus (rockwalrus) wrote :
Revision history for this message
Artur Rona (ari-tczew) wrote :

I'm reviewing a lot of bugs and I can confirm this bug. It's very oppressive, because I must changing bug status for Debian manually.

Revision history for this message
Pietro Battiston (toobaz) wrote :

If a testcase is needed, a bug which still has its status to "Unkown" because of this bug is Bug 250887 .

Revision history for this message
Bryan McLellan (btm) wrote :

Seeing the same issue with Bug #484756. All of the bugs that are having issues being synced appear to be archived debian bugs. This could be a coincidence that becomes more likely over time, but I do see in the launchpad source that it keeps separate databases for normal and archived bugs in db-h and archive respectively.

Curtis Hovey (sinzui)
affects: launchpad → malone
Revision history for this message
Curtis Hovey (sinzui) wrote :

I thiink someone from the bug team should look at these bugs. I know the debian bug importer was change recently and these issues may be fixed.

Changed in malone:
status: Confirmed → New
Revision history for this message
Deryck Hodge (deryck) wrote :

Graham or Gavin, can one of you look into this please?

Revision history for this message
Graham Binns (gmb) wrote :

Looking at the errors listed on the bug watch pages, it appears that they're all "bug not found" errors. Since the bugs do actually exist in the debbugs DB, that makes me suspect one of three things:

 1. Our copy of the debbugs DB is out of date.
 2. Our copy of the debbugs DB is in the wrong place and the bug watch code can't find the bugs for this reason.
 3. The bugs are in the debbugs archive and for some reason our code* can't find them.

Of these, 1 is the least likely, since we've made sure that our syncing of the DB is up-to-date fairly recently. I would expect 2 to cause a more explosive error than "bug not found" (though I need to check). 3 seems the most likely at this point, and some debugging needs to happen with a copy of the live debbugs DB (rather than the tiny mock version that lives in the LP source tree for testing).

*Note: "our code" includes a lot of Python and one very crummy and under**-tested Perl script that no-one really likes or touches very much. I may be being cynical in thinking this is where the problem lies, but that's what I'm thinking all the same.
**Where "under" is a synonym for "un".

Deryck Hodge (deryck)
Changed in malone:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Graham Binns (gmb)
status: Triaged → In Progress
Revision history for this message
Graham Binns (gmb) wrote :

Some poking around has lead me to the conclusion that this isn't something we can do anything about, at least for these bugs, as the relevant data are no longer available. This is because the bugs are only held in the debbugs archive for 28 days after they're archived. Our copy of the debbugs db hasn't always been updated consistently, so it may be that there have been points where bugs have been deleted from the debbugs archive before we've rsynced the archived data to our mirror. Because we don't have the data and can't get the data I'm not sure that there's a lot that we can do about this problem.

Sorry to have to mark this "Won't fix" (if there was a "Can't fix," I'd use it). I'll keep monitoring debbugs watches to make sure we're not failing to update newer bugs, which would indicate a problem with our scripts.

Changed in malone:
status: In Progress → Won't Fix
Revision history for this message
Nick Andrik (andrikos) wrote :

Actually I think that many of the bugs reported have synced by now.

There is at least one more that hasn't
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497698

Graham, could you please check them?

PS: Thanks for your work

Nick

Revision history for this message
Nick Andrik (andrikos) wrote :

From the original bugs reported, only Bug #200659 still shows this issue

Revision history for this message
Nick Andrik (andrikos) wrote :

I posted the debian bug, the ubuntu one for facebookchat is this:
Bug #248833

PS: Sorry for the many messages

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 380362] Re: Launchpad couldn't import several bugs from Debian Bug tracker.

On 10 December 2010 14:34, Nick Andrik <email address hidden> wrote:
> Actually I think that many of the bugs reported have synced by now.
>
> There is at least one more that hasn't
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497698
>
> Graham, could you please check them?

Which Launchpad bug is this debian bug watched by?

Revision history for this message
Graham Binns (gmb) wrote :

On 10 December 2010 14:34, Nick Andrik <email address hidden> wrote:
> Actually I think that many of the bugs reported have synced by now.
>
> There is at least one more that hasn't
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497698
>
> Graham, could you please check them?

Also, if you click on the little edit icon next to the bug watch
details in the Remote bug watches portlet, you'll be able to see a
list of the most recent activity on the bug watch. I'll wager that for
each of these you'll find "Bug not found" happening a lot, which is
due to the problem I described above. That's certainly the case for
the watch attached to Bug #248833.

Revision history for this message
Nick Andrik (andrikos) wrote : Re: Launchpad couldn't import several bugs from Debian Bug tracker.

Would it be possible to parse the page in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497698 ?

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 380362] Re: Launchpad couldn't import several bugs from Debian Bug tracker.

On 10 December 2010 15:12, Nick Andrik <email address hidden> wrote:
> Would it be possible to parse the page in http://bugs.debian.org/cgi-
> bin/bugreport.cgi?bug=497698 ?
>

It's possible, of course, but that's an absolute last ditch
I-just-don't-want-to-go-there solution. We've used screen-scraping for
other bug trackers and it's a world of pain. My advice would be to
simply select the "I'll update the bug status manually" option for any
bug tasks that are linked to one of these bugs we can't sync.

Revision history for this message
Nick Andrik (andrikos) wrote : Re: Launchpad couldn't import several bugs from Debian Bug tracker.

If you have some pointers on where I could take a look, probably I could come up with the parsing code.
I just need to know where it should fit. Actually we only need the status of the bug, right?

We may also, want to connect to the debian discussion list, like we do for freedesktop (example bug #36812), but this can be a next step.

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 380362] Re: Launchpad couldn't import several bugs from Debian Bug tracker.

On 10 December 2010 16:07, Nick Andrik <email address hidden> wrote:
> If you have some pointers on where I could take a look, probably I could come up with the parsing code.
> I just need to know where it should fit. Actually we only need the status of the bug, right?

I may not have been entirely clear about this: the code is easy enough
to write, in theory, though tying it in with our existing code could
be quite an interesting experience. However I'm definitely -1 on
anyone actually doing this, simply because of the potential for
breakages, which we've worked really hard this year to resolve.

> We may also, want to connect to the debian discussion list, like we do
> for freedesktop (example bug #36812), but this can be a next step.

I'm not sure what you mean by that; can you elaborate?

Revision history for this message
Nick Andrik (andrikos) wrote : Re: Launchpad couldn't import several bugs from Debian Bug tracker.

That example bug #36812 has a reference to freedesktop-bugs #865
If you take a look in the actual comments, the thread is a combination of what is written in the freedesktop bug page and what is written (exclusively) in launchpad.
We could do something similar with debian bugs also.

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 380362] Re: Launchpad couldn't import several bugs from Debian Bug tracker.

On 10 December 2010 17:14, Nick Andrik <email address hidden> wrote:
> That example bug #36812 has a reference to freedesktop-bugs #865
> If you take a look in the actual comments, the thread is a combination of what is written in the freedesktop bug page and what is written (exclusively) in launchpad.
> We could do something similar with debian bugs also.

We already have code in place for this; it's just disabled for the
moment (about which I'll file a bug, since we should probably turn it
on sooner or later).

Revision history for this message
Nick Andrik (andrikos) wrote : Re: Launchpad couldn't import several bugs from Debian Bug tracker.

Could you please send me the number of this bug, since I would be interested in it?

Thank you very much!

Revision history for this message
Graham Binns (gmb) wrote :

Yes, it's bug 688665.

Revision history for this message
Graham Binns (gmb) wrote :

I've re-opened this. We should still be able to sync the old archived bug reports, provided they're still in the archive.

Changed in launchpad:
status: Won't Fix → Triaged
importance: High → Critical
Revision history for this message
Gavin Panella (allenap) wrote :

See https://bugs.launchpad.net/launchpad/+bug/693088/comments/2 for some investigation I did on a duplicate bug.

Revision history for this message
Robert Collins (lifeless) wrote :

Graham, I've dropped this to high as it appears you're not getting timeslices on it and we're not crashing. Also AFAIK we've never supported this. If I'm wrong please mark it regression and make it critical again. (And perhaps remove the assignment if you're busy on other stuff)

Changed in launchpad:
importance: Critical → High
Graham Binns (gmb)
Changed in launchpad:
assignee: Graham Binns (gmb) → nobody
summary: - Launchpad couldn't import several bugs from Debian Bug tracker.
+ bug watches on archived debbugs do not update
Revision history for this message
Olek Wojnar (olekw) wrote :

This seems to apply to bug #1799339 as well.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.