Comment 7 for bug 480775

Revision history for this message
Eric Day (eday) wrote : Re: [Bug 480775] Re: a blank unique_key means gearmand fails to add any background jobs when mysql persistent queues are used

No, gearmand doesn't store foreground jobs persistently because the
connected client can failover if something should happen. Persistence
really only makes sense for background jobs.

You can pass a unique key for background jobs with Gearman::Client:

use Gearman::Client;
my $client = Gearman::Client->new;
$client->job_servers('127.0.0.1:4730');
$ref= $client->dispatch_background('reverse', 'Hello World', {'uniq' => 'my_unique_key'});
print "$$ref\n";

-Eric

On Wed, Nov 11, 2009 at 08:20:27PM -0000, Dave Webb wrote:
> gearmand doesn't appear to use the persistent storage for jobs where the
> client waits for a result. And these are the only jobs where you can
> pass in a explicit unique key (using the perl client anyway).
>
> --
> a blank unique_key means gearmand fails to add any background jobs when mysql persistent queues are used
> https://bugs.launchpad.net/bugs/480775
> You received this bug notification because you are a member of Gearman-
> developers, which is the registrant for Gearman C Server and Library.
>
> Status in Gearman C Server and Library: Confirmed
>
> Bug description:
> When running gearmand backed onto a mysql database for queue persistence, only one job of any type can be added. This appears to be because the SQL used to insert the jobs has a blank value for "unique_key" which is the primary key of the queue table.
>
> I'm running gearmand 0.10 with a command like the following:
>
> /usr/local/sbin/gearmand --daemon --port=4730 --user=gearmand \
> --log-file=/logs/local_gearman.log \
> --queue-type=libdrizzle --libdrizzle-mysql --libdrizzle-user=gearman \
> --libdrizzle-db=gearman_queues --libdrizzle-table=gearman_queue1 -v -v -v
>
> I do not get a job handle back when calling the perl client with Gearman::Client->dispatch_background().
>
> Looking at the gearmand logs I see lines like:
>
> DEBUG [ 0] :::43190 Received SUBMIT_JOB_BG
> DEBUG libdrizzle add:
> CRAZY libdrizzle query: INSERT INTO gearman_queue1 SET priority=1,unique_key='',function_name='my_function',data='^E^F^C\
> FATAL _libdrizzle_query:drizzle_query:Duplicate entry '' for key 'PRIMARY'
> INFO [ 0] :::43190 Disconnected
>
> This seems to indicate that the SQL being build up by the gearmand has a blank value for unique_key. This causes all jobs subsequent to the first to fail to be added.
>
> I don't program in c, but the problem seems to be in the libgearman-server/job.c file, where the SQL is assembled.
>
> I'm using libdrizzle-0.4 if that makes a difference.
>
> Any ideas?
>
> Dave.