Enable watching Red Hat Bugzilla bugs

Bug #1678486 reported by Neal Gompa on 2017-04-01
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
High
Colin Watson

Bug Description

Currently, Launchpad has bug watching disabled for bugs from Red Hat Bugzilla[1]. This is rather annoying, and seems to be unique among the popular downstream/upstream bug trackers. Can it please be fixed so that it can watch and track Red Hat Bugzilla bugs?

[1]: https://bugs.launchpad.net/bugs/bugtrackers/redhat-bugs

Related branches

Colin Watson (cjwatson) wrote :

I think this must have been done administratively, so we need to trawl through history and figure out why.

tags: added: lp-bugs
Changed in launchpad:
importance: Undecided → High
status: New → Triaged
Colin Watson (cjwatson) wrote :

A small batch on dogfood does seem to exhibit some unhappiness:

launchpad@labbu:~$ http_proxy=http://squid.internal:3128/ ftp_proxy=http://squid.internal:3128/ https_proxy=http://squid.internal:3128/ /srv/launchpad.net/codelines/current/cronscripts/checkwatches.py -t redhat-bugs -v --batch-size=10
2017-10-19 18:33:54 DEBUG Cronscript control file not found at file:cronscripts.ini
2017-10-19 18:33:54 INFO Creating lockfile: /var/lock/launchpad-checkwatches.lock
2017-10-19 18:34:02 DEBUG Using a global batch size of 10
2017-10-19 18:34:23 INFO Updating 10 watches for 10 bugs on https://bugzilla.redhat.com
2017-10-19 18:34:25 INFO Unknown remote importance 'unspecified unspecified' for bug u'1000208' on https://bugzilla.redhat.com.
2017-10-19 18:34:27 DEBUG no rules relevant to 'app.mainsite_only.canonical_url'
2017-10-19 18:34:33 INFO Failure updating bug u'1000208' on https://bugzilla.redhat.com (local bug: 1215656) (OOPS-12ed1702f41f9563fb6bcfdf49491093)
2017-10-19 18:34:33 INFO Unknown remote importance 'unspecified unspecified' for bug u'1001934' on https://bugzilla.redhat.com.
2017-10-19 18:34:34 INFO Unknown remote status 'CLOSED DEFERRED' for bug u'1004673' on https://bugzilla.redhat.com.
2017-10-19 18:34:34 INFO Unknown remote importance 'unspecified unspecified' for bug u'1004673' on https://bugzilla.redhat.com.
2017-10-19 18:34:34 INFO Unknown remote status 'CLOSED EOL' for bug u'1006289' on https://bugzilla.redhat.com.
2017-10-19 18:34:34 INFO Unknown remote importance 'unspecified unspecified' for bug u'1006289' on https://bugzilla.redhat.com.
2017-10-19 18:34:38 INFO Failure updating bug u'1006289' on https://bugzilla.redhat.com (local bug: 1274320) (OOPS-95670b7f6c634dfdbee67561262785d7)
2017-10-19 18:34:40 INFO Unknown remote importance 'unspecified unspecified' for bug u'1010941' on https://bugzilla.redhat.com.
2017-10-19 18:34:41 INFO Unknown remote status 'CLOSED EOL' for bug u'1013005' on https://bugzilla.redhat.com.
2017-10-19 18:34:41 INFO Unknown remote importance 'unspecified unspecified' for bug u'1013110' on https://bugzilla.redhat.com.
2017-10-19 18:34:46 INFO Failure updating bug u'1013110' on https://bugzilla.redhat.com (local bug: 1234612) (OOPS-cbbef56c8ad93e230f52e6d9cf0373e2)
2017-10-19 18:34:46 INFO Unknown remote importance 'unspecified unspecified' for bug u'1014976' on https://bugzilla.redhat.com.
2017-10-19 18:34:46 INFO Time for this run: 44.674 seconds.
2017-10-19 18:34:47 DEBUG checkwatches ran in 44.673754s (excl. load & lock)
2017-10-19 18:34:47 DEBUG Removing lock file: /var/lock/launchpad-checkwatches.lock

This may just be a matter of tweaking our code for the current set of RH statuses and importances.

Colin Watson (cjwatson) wrote :

Also, the OOPSes are things like this:

Traceback (most recent call last):
  Module lp.bugs.scripts.checkwatches.bugwatchupdater, line 85, in updateBugWatch
    self.importBugComments()
  Module lp.bugs.scripts.checkwatches.base, line 261, in wrapper
    return func(self, *args, **kwargs)
  Module lp.bugs.scripts.checkwatches.bugwatchupdater, line 155, in importBugComments
    "when importing comments for %s." % self.bug_watch.title)
  Module lp.bugs.model.bugtracker, line 627, in ensurePersonForSelf
    email, display_name, rationale, creation_comment)
  Module lp.registry.model.person, line 3652, in ensurePerson
    hide_email_addresses=True)
  Module lp.registry.model.person, line 3574, in createPersonAndEmail
    "%s is not a valid email address." % email)
InvalidEmailAddress: triniton is not a valid email address.

Neal Gompa (ngompa13) wrote :

I don't know if you're using a python module for interfacing with Bugzilla or trying to talk to the API directly, but most of us interfacing with Bugzilla systems use python-bugzilla[1], which is adapted to RH/SUSE BZ and works for generic Bugzilla systems, too.

[1]: https://github.com/python-bugzilla/python-bugzilla

Colin Watson (cjwatson) wrote :

I've grepped 6.5 years of IRC logs and I can't find any indication of why this bug tracker was disabled, so I'm going to guess that it was due to high volume of OOPSes, see if I can get things generally working on dogfood, and then re-enable it.

Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → In Progress
William Grant (wgrant) wrote :

It was disabled due to Red Hat Bugzilla's API crashing whenever asked for anything interesting (bug #598575). It looks like the Bugzilla has been upgraded since and is no longer so terribly broken.

Colin Watson (cjwatson) wrote :

Neal: We're using our own implementation, although it's talking to the same XML-RPC backend. I don't know whether the code would get very much shorter if we refactored it to use python-bugzilla; it would still need to fit into our framework for external bug trackers, so it wouldn't surprise me if there ended up being a bit of an impedance mismatch.

Anyway, all of the things that I've identified as going wrong so far (and fixed in my attached merge proposal) are specifically to do with translating Bugzilla's view of the world into something compatible with the Launchpad data model, so an external library wouldn't be able to help here.

Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
Colin Watson (cjwatson) wrote :

This looks fine as far as it goes, but further testing reveals a couple of other things I'd like to fix before enabling this again on production:

 * There are some comments whose author is just a string of all-caps initials. Trying to import these OOPSes because sanitize_name results in the empty string, we then append "-redhat-bugs", and the result of that isn't a valid LP username.
 * There's a lot of noise because we can't push comments to bugzilla.redhat.com; this is because we don't have credentials set up for doing so. IMO if we don't have credentials configured for a Bugzilla instance then we shouldn't consider it eligible for comment pushing (or back-linking).

tags: added: qa-ok
removed: qa-needstesting
Changed in launchpad:
status: Fix Committed → In Progress
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
removed: qa-ok
Changed in launchpad:
status: In Progress → Fix Committed
Colin Watson (cjwatson) wrote :

There are some remaining OOPSes due to comments with 'author': '' in the XML-RPC response, but otherwise this seems to be behaving reasonably now.

tags: added: qa-ok
removed: qa-needstesting
Colin Watson (cjwatson) wrote :

It's importing again now. It will take some time to catch up with six years of backlog, of course!

Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers