disconnect from a slave does not kill worker threads

[Khai Do]

Not all executor workers threads get killed when disconnecting a slave. It looks like only the 1st is stopped.

steps to repro:
1. setup jenkins with a slave that has more than 1 executor on it
2. install jenkins gearman-plugin
3. enable the gearman plugin from jenkins global config page
4. check the workers on the gearman server (i.e. gearadmin --workers)

I have master node with one executor and a slave with 2 executors, so mine looks like this..
36 127.0.0.1 oneiric-668621_exec-0 : build:guava
37 127.0.0.1 oneiric-668621_exec-1 : build:guava
33 127.0.0.1 EliteBook.home_manager : stop:EliteBook.home
34 127.0.0.1 EliteBook.home_exec-0 : build:guava10

5. On jenkins click the link to 'Disconnect' the slave. check the workers on the gearman server again. Both executor workers should have stopped.

Expected should like this..
33 127.0.0.1 EliteBook.home_manager : stop:EliteBook.home
34 127.0.0.1 EliteBook.home_exec-0 : build:guava10

However the actual result looks like this..
37 127.0.0.1 oneiric-668621_exec-1 : build:guava
33 127.0.0.1 EliteBook.home_manager : stop:EliteBook.home
34 127.0.0.1 EliteBook.home_exec-0 : build:guava10

[Khai Do (zaro0508)]

I was able to repro after the commit for adding a custom worker class (MyGearmanWorkerImpl). The commit is 20844c7e46504adbfa81fdc2bed538b353068b46

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/31089

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/31089
Committed: http://github.com/openstack-infra/gearman-plugin/commit/78abc3d90375aed87c9d9c9dd163d00cd9667874
Submitter: Jenkins
Branch: master

commit 78abc3d90375aed87c9d9c9dd163d00cd9667874
Author: zaro0508 <email address hidden>
Date: Thu May 30 11:07:27 2013 -0700

    Added a Stop() method to handle stopping executor worker threads.

    fix for bug 1181569

    Refactored how stop was handled. Removing thread and stopping thread
    in the same syncronized loop was causing concurrent modification error.
    Needed to seperate the actions for it to work.

    Change-Id: I99b676f377842826f9e6a847cfeff76be91c8299