Cannot query freedesktop.org's bugzilla

[Bryce Harrington]

[Problem]
bzutils-get is able to parse csv text, but was only doing this for type 'text/plain', and would fail if used against bugzillas configured to use the (correct) type 'text/csv'.

[Impact]
Makes tool useless for a variety of well known bugzilla instances.

[Addressed]
Fix uploaded to Lucid directly, and the same patch has been sponsored for karmic-proposed.

[Patch]
http://launchpadlibrarian.net/35132404/bzutils.debdiff

[Reproduction]
bryce@chideok:~/src/python-bzutils/python-bzutils-0.2/src$ ./bzutils-get https://bugs.freedesktop.org 21190

Usage: bzutils-get <url> <bugnumber>

<url>: The base url of the bugzilla server.
<query>: Number/ID of the bug

Traceback (most recent call last):
  File "./bzutils-get", line 37, in <module>
    print bz.get(sys.argv[2])
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/bugzilla.py", line 43, in get
    return self.cgi.query_bchart(charts=charts)[0]
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/cgi.py", line 367, in query_bchart
    return self.parse(f)
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/cgi.py", line 127, in parse
    raise BugListParseError, "Couldn't parse bug list: unknown format"
bzutils.exceptions.BugListParseError: Couldn't parse bug list: unknown format

[Regression Potential]
Probably none. This tool also has a very narrow userbase so the potential impact on users is similarly quite small.

[Bryce Harrington (bryce)]

I get the same thing even running against a locally installed bugzilla from Karmic ubuntu:

bryce@chideok:~/src/python-bzutils/python-bzutils-0.2/src$ ./bzutils-get http://localhost/cgi-bin/bugzilla3/ 1

Usage: bzutils-get <url> <bugnumber>

<url>: The base url of the bugzilla server.
<query>: Number/ID of the bug

Traceback (most recent call last):
  File "./bzutils-get", line 37, in <module>
    print bz.get(sys.argv[2])
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/bugzilla.py", line 43, in get
    return self.cgi.query_bchart(charts=charts)[0]
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/cgi.py", line 367, in query_bchart
    return self.parse(f)
  File "/home/bryce/src/python-bzutils/python-bzutils-0.2/src/bzutils/cgi.py", line 127, in parse
    raise BugListParseError, "Couldn't parse bug list: unknown format"
bzutils.exceptions.BugListParseError: Couldn't parse bug list: unknown format

[temugen (temugen)]

Not sure how the original developer would want to take care of the sanitization of the strings, but here's a simple fix.
The original code, although it parsed csv text, was not checking for csv in the content-type of the bug's page, and that's why the exception was being raised.

[Bryce Harrington (bryce)]

I've sponsored the upload. Only issue I saw was that karmic-proposed was misspelled (fixed).

[Bryce Harrington (bryce)]

Uploaded fix to Lucid as well (same version is used)

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-bzutils - 0.2-1ubuntu1

---------------
python-bzutils (0.2-1ubuntu1) lucid; urgency=low

  * Fixed text/csv scraping in cgi.py (LP: #389289)
 -- Brad Misik <email address hidden> Wed, 04 Nov 2009 20:57:26 -0800

[Martin Pitt (pitti)]

Please subscribe ubuntu-sru to SRU bugs, otherwise nobody will look at them. Subscribed now.

[Martin Pitt (pitti)]

Can you please forward this upstream as well?

[Martin Pitt (pitti)]

Accepted python-bzutils into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Anyone who can test the proposed package? If not, the proposed update will be removed again. Thanks!

[Jean-Baptiste Lallement (jibel)]

SRU Verification:

The original issue seems to be fixed but I'm unable to confirm it due to another problem with the package. bzutils-get now fails with the error:
$ ./bzutils-get https://bugs.freedesktop.org 28576

Usage: bzutils-get <url> <bugnumber>

<url>: The base url of the bugzilla server.
<query>: Number/ID of the bug

Traceback (most recent call last):
  File "./bzutils-get", line 37, in <module>
    print bz.get(sys.argv[2])
  File "/tmp/python-bzutils-0.2/src/bzutils/bugzilla.py", line 43, in get
    return self.cgi.query_bchart(charts=charts)[0]
  File "/tmp/python-bzutils-0.2/src/bzutils/cgi.py", line 369, in query_bchart
    return self.parse(f)
  File "/tmp/python-bzutils-0.2/src/bzutils/cgi.py", line 125, in parse
    return self.parse_csv(handle)
  File "/tmp/python-bzutils-0.2/src/bzutils/cgi.py", line 150, in parse_csv
    bug.set_product(report[7])
  File "/tmp/python-bzutils-0.2/src/bzutils/bug.py", line 85, in set_product
    raise exceptions.InvalidProductError("Invalid product: %s" % product)
bzutils.exceptions.InvalidProductError: Invalid product: Add the Sony Ericsson Xperia X10 Android phone

The same problem exists in Maverick too. I haven't tested with other bugzilla instances but at least bugs.f.d.o
doesn't returns the format expected by bzutils. The fields product, component and version are missing from the csv returned by bugs.fdo.

According to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=557481 the package is orphaned and, unless someone wants to maintain, it should be removed from the archive.

Marking as verification-failed.

[Martin Pitt (pitti)]

Failed SRU, I removed the proposed package again.