build_job() always returns an empty string, no way to track resulting ID

[Ken Dreyer (Red Hat)]

Since Jenkins version 1.519 (released 2013/06/17), build_job() always returns an empty string (the empty HTTP body from the POST request).

The Jenkins REST API currently returns an HTTP response with a "Location" header that includes a queue item number for the job. This queue item number is only valid for a few minutes until Jenkins removes it from memory, but in the meantime, while it's available, clients can poll it to find the eventual job ID / URL.

Instead of returning the HTTP response body, we should parse this Location HTTP header and return the queue ID. This will require a new jenkins_urlopen() method, since the main jenkins_open() method only returns the body without regard for any headers.

We can also implement a get_queue_item() method to discover a queued job's status, and possibly a job URL.

[Ken Dreyer (Red Hat) (kdreyer-redhat)]

https://review.openstack.org/#/q/project:openstack/python-jenkins+topic:bug/1724932

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/511939
Reason: Not sure if this complex example is really needed. Also there were too many changes in between. If you want this merged please restore the review once you get it rebased.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/511938
Reason: Please rework this change to make it mergeable with current codebase, now it depends on already abandoned change.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/500103
Reason: incompatible with request, obsolete.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/511937

[Jacob Smith (thechao)]

I, also, would like a change like this. However, I couldn't figure out how to apply the previous patch, nor do I know how to use Gerrit. I'm inlining the patch I'm maintaining, locally. I'm about 90% sure I don't match style (the PEP...?), there's probably insufficient error handling, etc. Sorry if this doesn't help.

    421c421
    < def jenkins_open(self, req, add_crumb=True):
    ---
    > def jenkins_open(self, req, add_crumb=True, return_qid=False):
    431c431,432
    < response = urlopen(req, timeout=self.timeout).read()
    ---
    > rresponse = urlopen(req, timeout=self.timeout)
    > response = rresponse.read()
    436c437,444
    < return response.decode('utf-8')
    ---
    > if not return_qid:
    > return response.decode('utf-8')
    > else:
    > if rresponse is None or 'location' not in rresponse.info():
    > raise EmptyResponseException(
    > "Error communicating with server[%s]:"
    > "bad response" % self.server)
    > return int(rresponse.info()['location'].strip('/').split('/')[-1])
    1051c1059
    < self.build_job_url(name, parameters, token), b''))
    ---
    > self.build_job_url(name, parameters, token), b''), return_qid=True)

[Jacob Smith (thechao)]

... and the commenting mechanism, here, stripped the whitespace.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/561835

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/561836

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/561837

[Ken Dreyer (Red Hat) (kdreyer-redhat)]

Sure, I've rebase and resubmitted to Gerrit at https://review.openstack.org/#/q/topic:bug/1724932+(status:open+OR+status:merged)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/561835
Committed: https://git.openstack.org/cgit/openstack/python-jenkins/commit/?id=f3ba366a69c53910747dcc412fb693d0287bb5cb
Submitter: Zuul
Branch: master

commit f3ba366a69c53910747dcc412fb693d0287bb5cb
Author: Ken Dreyer <email address hidden>
Date: Tue Apr 17 02:52:35 2018 -0600

    add jenkins_request() method

    Move the main pieces of the jenkins_open() method out into a new
    jenkins_request() method.

    This makes the raw requests.Response object available to other callers,
    so they can access things that were not in the response's body string,
    such as the response headers.

    Change-Id: I1f4049899cf3fa25e6159c416a325419b91e322e
    Closes-Bug: #1724932

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/561836
Committed: https://git.openstack.org/cgit/openstack/python-jenkins/commit/?id=91ab3d90576b3394d18b5c3963bbde74cda4d60f
Submitter: Zuul
Branch: master

commit 91ab3d90576b3394d18b5c3963bbde74cda4d60f
Author: Ken Dreyer <email address hidden>
Date: Tue Apr 17 02:52:48 2018 -0600

    build_job: return queue item identifier

    Since Jenkins version 1.519 (released 2013/06/17), build_job() would
    always return an empty string (the empty HTTP body from the POST
    request).

    Read the HTTP response's Location header and return the queue ID number.

    Clients can use this number to query server for the the upcoming job's
    status.

    Change-Id: I2a1ef3abb7e675e0ad37dd8eb4a377af22f27a03
    Closes-Bug: #1724932

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/561837
Committed: https://git.openstack.org/cgit/openstack/python-jenkins/commit/?id=284c3d37b70770462cb32b34026f6ef5e8140ec2
Submitter: Zuul
Branch: master

commit 284c3d37b70770462cb32b34026f6ef5e8140ec2
Author: Ken Dreyer <email address hidden>
Date: Tue Apr 17 02:52:51 2018 -0600

    add get_queue_item() method

    Pass in a queue ID number to discover a job's status, and possibly a job
    URL.

    Change-Id: I20541ec49cc30e5c74a6c596e02b3f42b2567fa5
    Closes-Bug: #1724932

[Ken Dreyer (Red Hat) (kdreyer-redhat)]

This feature is now in the v1.0.0 release of python-jenkins.