Persistent Queue does not update fast enough

Bug #938059 reported by symphonyx on 2012-02-21
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Gearman
Wishlist
Unassigned

Bug Description

In my test environment, I am running 0.27 via supervisord:

command=gearmand -vvvvvvvv -q libsqlite3 --user=www --libsqlite3-db=/tmp/gearman_queue --libsqlite3-table=gearman_queue

For simplicity, I'm running a single worker, also managed by supervisord.

The jobs are getting inserted into the database properly from what I can see. However, the worker is running the same task multiple times, which can be anywhere between 2 and 10 times before the job is finally removed from the persistent queue.

The worker code is pretty simple:

<?php

$worker = new GearmanWorker();
$worker->addServer();

$worker->addFunction('updateCredits', 'adjustCredits');

while ($worker->work()) {
    if ($worker->returnCode() != GEARMAN_SUCCESS) {
        echo "return_code: " . $worker->returnCode() . "\n";
        break;
    }
}

function adjustCredits(GearmanJob $job)
{
    $job->sendStatus(0,1);
    $workload = unserialize($job->workload());

    $player = new Account();
    $player->setMerchantAccountId($workload['player']);
    $player->setName($workload['player']);

    $token = new Token();
    $token->setMerchantTokenId('CREDIT');

    $credit_adjustment = $workload['amount'];

    $amount = new TokenAmount();
    $amount->setToken($token);
    $amount->setAmount($credit_adjustment);

    if($credit_adjustment < 0) {
        $response = $player->decrementTokens(array($amount));
    }
    else {
        $response = $player->incrementTokens(array($amount));
    }

    switch($response['returnCode']) {
        case VIN_SOAP_CODE_SUCCESS:
            $job->sendStatus(1,1);
            $job->sendComplete(serialize($response));
            return GEARMAN_SUCCESS;
        default:
            $job->sendFail();
            exit(255);
    }
}
?>

The client code is also simple:

<?php
$gearman = new GearmanClient();
$gearman->addServer();

$job = array(
  'player' => $adjustment->getPlayerName(),
  'amount' => $adjustment->getAmount()
);

$job_params = serialize($job);
$job_id = sha1(json_encode($job));

$rc = $gearman->doBackground('updateCredits', $job_params, $job_id);
?>

I can confirm that only one job gets inserted into the sqlite persistent queue.

symphonyx (erich-beyrent) wrote :

I ran gearmand through strace while this issue was happening. I don't know if this will help debug what's happening or not.

http://pastebin.com/h85gL42L

symphonyx (erich-beyrent) wrote :

Also having this problem with 0.28.

This time, I ran gearmand directly without the persistent queue, and observed the same behavior. I'm using 0.8.1 of the PHP extension.

Brian Aker (brianaker) wrote :

Can you confirm that your client is correctly exiting with SUCCESS?

symphonyx (erich-beyrent) wrote :

Yes - I added the following to my code:

<?php syslog(LOG_CRIT, 'GEARMAN RETURN CODE = '.$gearman->returnCode()); ?>

Here's the matching output in the log:

Feb 23 08:12:43 usbost-fenvli15 php: WORKER RETURN CODE = 0

Brian Aker (brianaker) wrote :

BTW we are looking at a solution for this for the next major release.

Brian Aker (brianaker) on 2012-12-13
Changed in gearmand:
importance: Undecided → Wishlist
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers