Jenkins returning 404 cannot be distinguished

[Kevin L. Mitchell]

If a non-GET request is made for a resource which doesn't exist—such as cancelling a non-existant queue item—no exception or error is reported to the user. At the very least, a distinguishable error should be raised. (Distinguishable in that it should not be JenkinsException, although it should be a subclass of JenkinsException).

[Khai Do (zaro0508)]

Kevin, does this change fixes this issue? https://review.openstack.org/#/c/106314/

[Kevin L. Mitchell (klmitch)]

No, it does not; the exception raised isn't even raised for a 404 error. I had to override jenkins_open() with the following code to get the behavior I needed:

    def jenkins_open(self, req):
        try:
            if self.auth:
                req.add_header('Authorization', self.auth)
            return urllib2.urlopen(req).read()
        except urllib2.HTTPError, e:
            # Jenkins's funky authentication means its nigh impossible to
            # distinguish errors.
            if e.code in [401, 403, 500]:
                raise jenkins.JenkinsException(
                    'Error in request.' +
                    'Possibly authentication failed [%s]' % (e.code)
                )
            elif e.code == 404:
                raise NotFoundException('Requested item could not be found')

            # Log the error
            self.ctxt.log.warn("HTTPError raised with code %d: %s" %
                               (e.code, e.msg))

[Khai Do (zaro0508)]

Kevin, I'm using jenkins LTS 1.565.3 and your solution doesn't work for me. In your implementation I always get a NotFoundException. I seem to always a 404 whether I pass in a valid build id or not.

[Kevin L. Mitchell (klmitch)]

Well, my point is I want an exception raised when a 404 is returned by the Jenkins API. With the current code, I get *nothing* raised when the API returns a 404.

[Khai Do (zaro0508)]

I've also tried to add a try/except block in cancel_queue method similar to the get_build_info method however the response from from a CANCEL_QUEUE REST call always seems to return None therefore I'm not sure how we could implement catching a 404. It seems like we are limited by the Jenkins REST api here.

[Kevin L. Mitchell (klmitch)]

I'm afraid I still don't understand your complaint. If the API returns a 404, I want a distinguishable exception thrown, regardless of why Jenkins returned that 404. I can tell you that Jenkins does return 404 responses for, e.g., jobs that don't exist.

[Khai Do (zaro0508)]

What I'm trying to say, probably not very clearly, is that when I use the Jenkins CANCEL_QUEUE REST endpoint I always get a 404 error. I don't know why but that's what jenkins is giving me. Your implementation above raises a NotFoundException on a 404 so I would get an exception every time i make a call to cancel_queue() wether I pass in a valid ID or not. So in this case how do I determine wether there's a failure or not?

I think you may be correct that 404 is returned for jobs that don't exist, but I assume that you would use the JOB_NAME endpoint for that therefore I think raising an exception for 404 may work for some endpoints but not for all.

It might be more clear and more productive if you push a patch to gerrit suggesting your change for this fix then we can discuss the patch.

[Kevin L. Mitchell (klmitch)]

OK, now I understand what you're trying to say, and yeah, it does sound like Jenkins' API is at fault here. (Have I ever mentioned just how much I hate Jenkins? :) What do you think about calling cancel_queue() a best-effort operation? After all, there's usually a tiny window between when a build is queued to when it's popped off the queue, assigned a build number, and started.

[Kevin L. Mitchell (klmitch)]

Odd that a comment hasn't been logged yet. Review is up: https://review.openstack.org/#/c/139158/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/139158
Committed: https://git.openstack.org/cgit/stackforge/python-jenkins/commit/?id=8c3e979810ce8b0256465e9a8fe373d8d9832859
Submitter: Jenkins
Branch: master

commit 8c3e979810ce8b0256465e9a8fe373d8d9832859
Author: Kevin L. Mitchell <email address hidden>
Date: Tue Jan 13 16:38:15 2015 -0600

    Raise an exception on 404

    Currently, when Jenkins returns a 404 error, the jenkins_open()
    method gives no indication. This adds a new NotFoundException,
    descending from JenkinsException, which can be used to distinguish
    this case. This causes a problem with cancel_queue(), however,
    as the Jenkins API always returns a 404; since cancel_queue() is
    a best-effort mechanism, this change also modifies that method to
    explicitly ignore a NotFoundException. The maybe_add_crumb()
    method also has to be updated to properly handle 404s, for the
    case where crumbs are disabled. The get_job_name() method is also
    adjusted to return None if jenkins_open() raises a NotFoundException.

    A few quick google searches shows that only the cancelItem
    endpoint will always return 404.

    Change-Id: I98320d174659fc83d48614b31a5a054fd0773dfe
    Closes-Bug: #1246468