Launchpad fails to poll the freedesktop.org Bugzilla

[discont]

Launchpad apparently fails to poll the freedesktop.org Bugzilla, which seems to be completely healthy and responsive (albeit a bit slow). For an example, refer to:
  https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-intel/+bug/876762

Is this a HTTP vs. HTTPS issue (freedesktop.org uses HTTPS, but Launchpad renders all links as HTTP)?

Original report at: https://answers.launchpad.net/launchpad/+question/177094

[Gavin Panella (allenap)]

I've reset the bug watches in bug #876762. With luck they'll update in
a few minutes. I've also changed the base URL from http to
https. Let's see what happens.

[Gavin Panella (allenap)]

Still broken.

[Gavin Panella (allenap)]

$ bin/harness
>>> from lp.bugs.externalbugtracker.bugzilla import Bugzilla
>>> ebt = Bugzilla("https://bugs.freedesktop.org/")
>>> ebt = ebt.getExternalBugTrackerToUse()
>>> ebt.initializeRemoteBugDB(["37700"])
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../lib/lp/bugs/externalbugtracker/bugzilla.py", line 327, in initializeRemoteBugDB
    super(Bugzilla, self).initializeRemoteBugDB(bug_ids)
  File ".../lib/lp/bugs/externalbugtracker/base.py", line 188, in initializeRemoteBugDB
    self.bugs = self.getRemoteBugBatch(bug_ids)
  File ".../lib/lp/bugs/externalbugtracker/bugzilla.py", line 402, in getRemoteBugBatch
    buglist_page, data, repost_on_redirect=True)
  File ".../lib/lp/bugs/externalbugtracker/base.py", line 279, in _postPage
    response = self._post(url, data=post_data)
  File ".../lib/lp/bugs/externalbugtracker/base.py", line 263, in _post
    return self._fetchPage(request, data=data)
  File ".../lib/lp/bugs/externalbugtracker/base.py", line 249, in _fetchPage
    raise BugTrackerConnectError(self.baseurl, val)
BugTrackerConnectError: https://bugs.freedesktop.org: HTTP Error 500: Internal Server Error

[Gavin Panella (allenap)]

$ bin/harness
>>> from lp.bugs.externalbugtracker.bugzilla import Bugzilla
>>> ebt = Bugzilla("https://bugs.freedesktop.org/")
>>> ebt.getExternalBugTrackerToUse()
<lp.bugs.externalbugtracker.bugzilla.Bugzilla object at 0x7b796d0>

I might have expected a BugzillaAPI object there.

[Gavin Panella (allenap)]

Trying with BugzillaAPI directly:

$ bin/harness
>>> from lp.bugs.externalbugtracker.bugzilla import BugzillaAPI
>>> ebt = BugzillaAPI("https://bugs.freedesktop.org/")
>>> ebt.getExternalBugTrackerToUse()
<lp.bugs.externalbugtracker.bugzilla.BugzillaAPI object at 0x7d0f6d0>
>>> ebt.initializeRemoteBugDB(["37700"])
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../lib/lp/services/database/isolation.py", line 65, in wrapper
    return func(*args, **kwargs)
  File ".../lib/lp/bugs/externalbugtracker/bugzilla.py", line 689, in initializeRemoteBugDB
    'permissive': True,
  File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File ".../lib/lp/bugs/externalbugtracker/xmlrpc.py", line 117, in request
    return self._parse_response(StringIO(response), None)
  File "/usr/lib/python2.6/xmlrpclib.py", line 1387, in _parse_response
    p.feed(response)
  File "/usr/lib/python2.6/xmlrpclib.py", line 601, in feed
    self._parser.Parse(data, 0)
ExpatError: mismatched tag: line 90, column 4

I suspect that's an error page that can't be parsed as XML.

[Gavin Panella (allenap)]

Indeed, it's an error message, containing the following:

  Bugzilla has suffered an internal error. Please save this page and
  send it to <email address hidden> with details of what
  you were doing at the time this message appeared.

  The XML-RPC Interface feature is not available in this Bugzilla.

[Bryce Harrington (bryce)]

Is it a failure on the fdo side, or is launchpad doing something incorrect on our end?

I'd really like to see this working again.

[Gavin Panella (allenap)]

Bryce, I'm not really sure yet. I'll see if I can spend a little time
tomorrow to poke some more.

[Lucy Llewellyn (lucyllewy)]

I _think_ this also affects https://bugzilla.kernel.org (kernel bugzilla) watches, as I just entered a watch on the upstream kernel bug to https://bugs.launchpad.net/linux/+bug/1336541 which also fails with the same message as the bug above relating to freedesktop's bugzilla.