Persistent Queue does not update fast enough

Bug #938059 reported by symphonyx
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Gearman
Confirmed
Wishlist
Unassigned

Bug Description

In my test environment, I am running 0.27 via supervisord:

command=gearmand -vvvvvvvv -q libsqlite3 --user=www --libsqlite3-db=/tmp/gearman_queue --libsqlite3-table=gearman_queue

For simplicity, I'm running a single worker, also managed by supervisord.

The jobs are getting inserted into the database properly from what I can see. However, the worker is running the same task multiple times, which can be anywhere between 2 and 10 times before the job is finally removed from the persistent queue.

The worker code is pretty simple:

<?php

$worker = new GearmanWorker();
$worker->addServer();

$worker->addFunction('updateCredits', 'adjustCredits');

while ($worker->work()) {
    if ($worker->returnCode() != GEARMAN_SUCCESS) {
        echo "return_code: " . $worker->returnCode() . "\n";
        break;
    }
}

function adjustCredits(GearmanJob $job)
{
    $job->sendStatus(0,1);
    $workload = unserialize($job->workload());

    $player = new Account();
    $player->setMerchantAccountId($workload['player']);
    $player->setName($workload['player']);

    $token = new Token();
    $token->setMerchantTokenId('CREDIT');

    $credit_adjustment = $workload['amount'];

    $amount = new TokenAmount();
    $amount->setToken($token);
    $amount->setAmount($credit_adjustment);

    if($credit_adjustment < 0) {
        $response = $player->decrementTokens(array($amount));
    }
    else {
        $response = $player->incrementTokens(array($amount));
    }

    switch($response['returnCode']) {
        case VIN_SOAP_CODE_SUCCESS:
            $job->sendStatus(1,1);
            $job->sendComplete(serialize($response));
            return GEARMAN_SUCCESS;
        default:
            $job->sendFail();
            exit(255);
    }
}
?>

The client code is also simple:

<?php
$gearman = new GearmanClient();
$gearman->addServer();

$job = array(
  'player' => $adjustment->getPlayerName(),
  'amount' => $adjustment->getAmount()
);

$job_params = serialize($job);
$job_id = sha1(json_encode($job));

$rc = $gearman->doBackground('updateCredits', $job_params, $job_id);
?>

I can confirm that only one job gets inserted into the sqlite persistent queue.

Revision history for this message
symphonyx (erich-beyrent) wrote :

I ran gearmand through strace while this issue was happening. I don't know if this will help debug what's happening or not.

http://pastebin.com/h85gL42L

Revision history for this message
symphonyx (erich-beyrent) wrote :

Also having this problem with 0.28.

This time, I ran gearmand directly without the persistent queue, and observed the same behavior. I'm using 0.8.1 of the PHP extension.

Revision history for this message
Brian Aker (brianaker) wrote :

Can you confirm that your client is correctly exiting with SUCCESS?

Revision history for this message
symphonyx (erich-beyrent) wrote :

Yes - I added the following to my code:

<?php syslog(LOG_CRIT, 'GEARMAN RETURN CODE = '.$gearman->returnCode()); ?>

Here's the matching output in the log:

Feb 23 08:12:43 usbost-fenvli15 php: WORKER RETURN CODE = 0

Revision history for this message
Brian Aker (brianaker) wrote :

BTW we are looking at a solution for this for the next major release.

Brian Aker (brianaker)
Changed in gearmand:
importance: Undecided → Wishlist
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.