Selector not being closed

Bug #513180 reported by Norberto Leite on 2010-01-27
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Gearman Java
High
Eric Lambert

Bug Description

I've been checking this question performed by my college Raul. It seems that the fix he purposes bellow hasn't been committed on trunk. Are you considering other implementations? Is there a branch where this problem is already fixed ? Can we perform the commit for board approval ??

Cheers,
N.

I asked because i'm testing client gearman with N threads.

I've test 2 options.

1 - Using only one GearmanClientImpl and passing it to all the threads, but then i think that the client implemention don't recognize the jobs well. I see the next Warning:
WARNING: Client received packet from server for unknown job ( job_handle = org.gearman.client.GearmanClientImpl$JobHandle@3f65305d packet = WORK_COMPLETE )
org.gearman.common.GearmanException: Encountered Fatal Exception while driving the following ServerRequest: REQ:SUBMIT_JOB:1615: [66756c6c5f666565640036663836643736362d366430632d343230622d386139372d306330343538303338646639006172726179282769645f736f75726365273d3e2734272c2769645f6462273d3e273439272c27636f756e747279273d3e276573272c27766572746963616c273d3e2731272c276164273d3e617272617928617272617928276164273d3e6172726179282776616c7565273d3e27272c2761747472273d3e272729292c617272617928276964273d3e6172726179282776616c7565273d3e273539355f564134373036272c2761747472273d3e272729292c6172726179282775726c273d3e6172726179282776616c7565273d3e27687474703a2f2f7777772e757262616e697a612e636f6d2f6275736361722f696e6d7565626c65732f6669636861456e6c6163652e6a73703f635f6167656e6369613d35393526616d703b635f696e6d7565626c653d564134373036272c2761747472273d3e272729292c617272617928277469746c65273d3e6172726179282776616c7565273d3e274c6f63616c657320436f6d65726369616c657320656e20416c7175696c65722c203230206d322c204752414e414441204341504954414c20535552202856455247454c455329272c2761747472273d3e272729292c6172726179282774797065273d3e6172726179282776616c7565273d3e27466f722072656e74272c2761747472273d3e272729292c617272617928276167656e6379273d3e6172726179282776616c7565273d3e27557262616e697a612e636f6d272c2761747472273d3e272729292c61727261792827636f6e74656e74273d3e6172726179282776616c7565273d3e274c6f63616c657320436f6d65726369616c657320656e20416c7175696c65722c203230206d322c204752414e414441204341504954414c20535552202856455247454c4553292e20272c2761747472273d3e272729292c617272617928277072696365273d3e6172726179282776616c7565273d3e27323030272c2761747472273d3e272729292c6172726179282770726f70657274795f74797065273d3e6172726179282776616c7565273d3e274c6f63616c657320436f6d65726369616c6573272c2761747472273d3e272729292c61727261792827666c6f6f725f61726561273d3e6172726179282776616c7565273d3e273230272c2761747472273d3e61727261792827756e6974273d3e276d6574657273272929292c61727261792827726f6f6d73273d3e6172726179282776616c7565273d3e2730272c2761747472273d3e272729292c6172726179282762617468726f6f6d73273d3e6172726179282776616c7565273d3e2730272c2761747472273d3e272729292c617272617928277061726b696e67273d3e6172726179282776616c7565273d3e27272c2761747472273d3e272729292c6172726179282761646472657373273d3e6172726179282776616c7565273d3e27272c2761747472273d3e272729292c6172726179282763697479273d3e6172726179282776616c7565273d3e274752414e414441204341504954414c20535552272c2761747472273d3e272729292c61727261792827636974795f61726561273d3e6172726179282776616c7565273d3e2756455247454c4553272c2761747472273d3e272729292c61727261792827706f7374636f6465273d3e6172726179282776616c7565273d3e273138303038272c2761747472273d3e272729292c61727261792827726567696f6e273d3e6172726179282776616c7565273d3e274772616e616461272c2761747472273d3e272729292c617272617928276c61746974756465273d3e6172726179282776616c7565273d3e2733372e313831313038303837393135272c2761747472273d3e272729292c617272617928276c6f6e676974756465273d3e6172726179282776616c7565273d3e272d332e3630323331333939353336313332272c2761747472273d3e272729292c617272617928277069637475726573273d3e6172726179282776616c7565273d3e270a272c2761747472273d3e272729292c6172726179282764617465273d3e6172726179282776616c7565273d3e2732362f30392f32303039272c2761747472273d3e272729292c6172726179282774696d65273d3e6172726179282776616c7565273d3e2730323a3039272c2761747472273d3e272729292929]
        at org.gearman.common.GearmanTask.handleGearmanIOEvent(GearmanTask.java:114)
        at org.gearman.common.GearmanJobServerSession.handleResSessionEvent(GearmanJobServerSession.java:308)
        at org.gearman.common.GearmanJobServerSession.handleSessionEvent(GearmanJobServerSession.java:240)
        at org.gearman.common.GearmanJobServerSession.driveSessionIO(GearmanJobServerSession.java:203)
        at org.gearman.client.GearmanClientImpl.driveClientIO(GearmanClientImpl.java:547)
        at org.gearman.client.GearmanClientImpl.driveRequestTillState(GearmanClientImpl.java:599)
        at org.gearman.client.GearmanClientImpl.submit(GearmanClientImpl.java:269)
        at com.trovit.parsing.feeds.executers.ExternalExecuter.executeGearman(ExternalExecuter.java:124)

2- Using one GearmanClientImpl for earch Thread. Establishing connection. I test it with one thread and 10 threads.

The error is the same:
java.lang.InternalError
        at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:752)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:168)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:192)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at sun.misc.Launcher$ExtClassLoader.findClass(Launcher.java:229)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:300)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.util.ResourceBundle$RBClassLoader.loadClass(ResourceBundle.java:445)
        at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2383)
        at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1406)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:1365)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:1292)
        at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1234)
        at java.util.ResourceBundle.getBundle(ResourceBundle.java:715)
        at java.util.logging.Level.getLocalizedName(Level.java:223)
        at java.util.logging.SimpleFormatter.format(SimpleFormatter.java:64)
        at java.util.logging.StreamHandler.publish(StreamHandler.java:179)
        at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:88)
        at java.util.logging.Logger.log(Logger.java:472)
        at java.util.logging.Logger.doLog(Logger.java:494)
        at java.util.logging.Logger.log(Logger.java:583)
        at org.gearman.client.GearmanClientImpl.addJobServer(GearmanClientImpl.java:168)
        at com.trovit.parsing.feeds.executers.ExternalExecuter.executeGearman(ExternalExecuter.java:106)
        at com.trovit.parsing.feeds.executers.FullFeedExecuter.run(FullFeedExecuter.java:147)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:114)
        at java.util.jar.JarFile.<init>(JarFile.java:133)
        at java.util.jar.JarFile.<init>(JarFile.java:70)
        at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:643)
        at sun.misc.URLClassPath$JarLoader.access$600(URLClassPath.java:538)
        at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:605)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:597)
        at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:750)
        ... 29 more

It seems problem with Log from java:
            LOG.log(Level.WARNING,"Failed to connect to job server " + newconn + ".",ioe);
i change it to test -> System.out.println("Failed to connect to job server " + newconn + ".") ;

And then i receive the error:
Failed to connect to job server GearmanNIOJobServerConnection:/10.0.0.106:4730.
java.io.IOException: Too many open files
        at sun.nio.ch.IOUtil.initPipe(Native Method)
        at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49)
        at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18)
        at java.nio.channels.Selector.open(Selector.java:209)
        at org.gearman.client.GearmanClientImpl.addJobServer(GearmanClientImpl.java:161)
        at com.trovit.parsing.feeds.executers.ExternalExecuter.executeGearman(ExternalExecuter.java:106)
        at com.trovit.parsing.feeds.executers.FullFeedExecuter.run(FullFeedExecuter.java:147)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

The code i use from this in the thread is:
GearmanClientImpl gearman = new GearmanClientImpl();
gearman.addJobServer(new GearmanNIOJobServerConnection(hostname, port));
GearmanJob job = GearmanJobImpl.createJob(this.gearmanFunction, org.gearman.util.ByteUtils.toUTF8Bytes(message) , null);
gearman.submit(job);
...
gearman.shutdown() ;

I'm doing something wrong?

Thanks,
Raul

Related branches

Eric Lambert (elambert) wrote :

Norberto:

Thanks for filing this bug. I some how managed to miss Raul's original question on this topic. I'll get the fix in as soon as I can.

Cheers,

Eric

Changed in gearman-java:
assignee: nobody → Eric Lambert (elambert)
milestone: none → 0.04
importance: Undecided → High
Eric Lambert (elambert) on 2010-08-12
Changed in gearman-java:
status: New → Confirmed
Eric Lambert (elambert) on 2010-08-12
Changed in gearman-java:
status: Confirmed → Fix Committed
Eric Lambert (elambert) on 2010-09-10
Changed in gearman-java:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Related questions