trigger jobs on unwanted slaves

[Antoine "hashar" Musso]

The Jenkins slaves can be configured to only run jobs which are tied to it either by the node name or the labels applied to it. When a Gearman jobs is received without a label (ie: build:$projectName), it is added to a worker regardless of that configuration. To say it otherwise, an unlabeled job can end up running on a slave that can not properly process it.

I have upgraded Wikimedia Zuul setup earlier today and prompty had to revert back because of that. An abstracted out version of the Wikimedia setup is:

Jenkins servers:

master: does not execute any job

slave-tester:
 - no label
 - testing utilities installed

slave-deployer:
 - has label 'deploy'
 - node configuration has Usage: Leave this machine for tied jobs only

Jobs:
- project-test (no specific label)
- project-deploy (labeled 'deploy')

When Zuul got upgraded, the `build:project-test` (ie no label in Zuul nor in Jenkins job config) ended up being triggered on the slave-deployers. Which fails the job since that slave is missing the utilities.

I haven't tried out what would happen if a Jenkins job is tied

I suspect the issue is in ExecutorWorkerThread::egisterJobs(), the nodes being configured with 'Leave this machine for tieds jobs only' should not be considered if the job labels does not match the slave labels.

[Antoine "hashar" Musso (hashar)]

Following a conversation with Khai Do:

There is an option on slave nodes to prevent them from running jobs that are not explicitly tied for them. That shows up in the node configuration as Usage: …

A real life use case is having a slave with node.js package manager npm. That slave does not have any other utilities and thus is dedicated to run javascript tests. The slave receives label hasNpm and is restricted to only run jobs explicitly tied to it.

In Jenkins config.xml the Usage: menu generate a <mode /> entry that takes either NORMAL or EXCLUSIVE has a value.

The Jenkins model Node has a canTake( Task ) method which is used by the queue to find out whether a node is able to launch it. Might want to reuse that logic?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/58140

[Antoine "hashar" Musso (hashar)]

Khai asked me yesterday whether I tested this out. I am still fighting with my dev environment to reproduce the slave / labels being used.

Will do my best to validate it is now working properly. Hopefully by the end of this week.

[Antoine "hashar" Musso (hashar)]

Sorry Khai, my dev environment has been screwed for the last few days. It finally got fixed up tonight so I will be able to play test your change seriously.

At first glance, it seems the list of jobs registered in German server match my expectations :-]

[Antoine "hashar" Musso (hashar)]

Thank you very much Khai Do. I got the fix deployed on Wikimedia Zuul instance and it is working properly.

[Akash (akash-redhat)]

I have setup multiple jenkins master using gearman plugin. On gearmand server I am able to see the workers. As according to gearman plugin documentation they said to test the setup using python client gearman-plugin-client. Now as per my requirement I want to trigger jenkins build from the gearman-plugin-client(it is also on same server i.e., gearmand) but I am able to trigger the jenkins builds. Please help!!!

[Akash (akash-redhat)]

Actually I have setup 2 jenkins master(with gearman plugin) and to support them I have one gearman server. As according to the gearman plugin link https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin, I am trying to trigger my created jobs on jenkins from my gearmand server. As according to there documentation on the given link My gearmand job server and gearman client(gear_client.py) is on same machine. I used there given command to trigger my jenkins build (python gear_client.py -s MyGearmanSever --function=build:uptime --params='{"OFFLINE_NODE_WHEN_COMPLETE":"false","param1":"moon","param1":"sun"}')

I am giving --function=build:uptime i.e. uptime is the name of job which I created on jenkins master.

BUT my builds on jenkin machine not starting up. Please help me on this and tell me what to do to trigger jenkins build through gearman client.