Persistent Queue does not update fast enough

[symphonyx]

In my test environment, I am running 0.27 via supervisord:

command=gearmand -vvvvvvvv -q libsqlite3 --user=www --libsqlite3-db=/tmp/gearman_queue --libsqlite3-table=gearman_queue

For simplicity, I'm running a single worker, also managed by supervisord.

The jobs are getting inserted into the database properly from what I can see. However, the worker is running the same task multiple times, which can be anywhere between 2 and 10 times before the job is finally removed from the persistent queue.

The worker code is pretty simple:

<?php

$worker = new GearmanWorker();
$worker->addServer();

$worker->addFunction('updateCredits', 'adjustCredits');

while ($worker->work()) {
    if ($worker->returnCode() != GEARMAN_SUCCESS) {
        echo "return_code: " . $worker->returnCode() . "\n";
        break;
    }
}

function adjustCredits(GearmanJob $job)
{
    $job->sendStatus(0,1);
    $workload = unserialize($job->workload());

    $player = new Account();
    $player->setMerchantAccountId($workload['player']);
    $player->setName($workload['player']);

    $token = new Token();
    $token->setMerchantTokenId('CREDIT');

    $credit_adjustment = $workload['amount'];

    $amount = new TokenAmount();
    $amount->setToken($token);
    $amount->setAmount($credit_adjustment);

    if($credit_adjustment < 0) {
        $response = $player->decrementTokens(array($amount));
    }
    else {
        $response = $player->incrementTokens(array($amount));
    }

    switch($response['returnCode']) {
        case VIN_SOAP_CODE_SUCCESS:
            $job->sendStatus(1,1);
            $job->sendComplete(serialize($response));
            return GEARMAN_SUCCESS;
        default:
            $job->sendFail();
            exit(255);
    }
}
?>

The client code is also simple:

<?php
$gearman = new GearmanClient();
$gearman->addServer();

$job = array(
  'player' => $adjustment->getPlayerName(),
  'amount' => $adjustment->getAmount()
);

$job_params = serialize($job);
$job_id = sha1(json_encode($job));

$rc = $gearman->doBackground('updateCredits', $job_params, $job_id);
?>

I can confirm that only one job gets inserted into the sqlite persistent queue.

[symphonyx (erich-beyrent)]

I ran gearmand through strace while this issue was happening. I don't know if this will help debug what's happening or not.

http://pastebin.com/h85gL42L

[symphonyx (erich-beyrent)]

Also having this problem with 0.28.

This time, I ran gearmand directly without the persistent queue, and observed the same behavior. I'm using 0.8.1 of the PHP extension.

[Brian Aker (brianaker)]

Can you confirm that your client is correctly exiting with SUCCESS?

[symphonyx (erich-beyrent)]

Yes - I added the following to my code:

<?php syslog(LOG_CRIT, 'GEARMAN RETURN CODE = '.$gearman->returnCode()); ?>

Here's the matching output in the log:

Feb 23 08:12:43 usbost-fenvli15 php: WORKER RETURN CODE = 0

[Brian Aker (brianaker)]

BTW we are looking at a solution for this for the next major release.