Queued jobs in redis backend are orphaned if Gearman server dies

Bug #1596303 reported by Danny Kopping on 2016-06-26

This bug report was converted into a question: question #509172: Queued jobs in redis backend are orphaned if Gearman server dies.

6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Gearman
Undecided
Unassigned

Bug Description

When using Redis as the persistence backend, queued jobs are set in Redis like so:

1466949136.176673 [0 127.0.0.1:61635] "SET" "_gear_-process-job-fdaa984bed6075f02e9971734e57c20f260a704d\x00\x00\x00" "workload"

redis-cli info | grep db0

-> db0:keys=1,expires=0,avg_ttl=0

(echo 'status'; sleep 0.1) | nc 127.0.0.1 4730

-> process-job 1 0 0

This shows there is 1 job in the queue, and that job is stored in Redis - all fine at this point.

Now, let's say the Gearman server is killed for whatever reason... When Gearman starts up again, it doesn't know about these queued jobs and they are effectively "orphaned".

Is there a workaround for this?

Kiril Angov (kiril.angov) wrote :

There is a pretty nasty bug in the Redis "persistence" which pretty much makes this plugin a noop. You put stuff in but it can never replay them because there is an error in the KEYS command it uses for replaying from Redis.

https://github.com/gearman/gearmand/commit/40028f8ba1a1d4f610a55a6d0a08683525372483#diff-7580c1cc3895914f180088198d7f8c92R255

This is the fix, you might need to just update to the latest version.

Changed in gearmand:
status: New → Invalid
Clint Byrum (clint-fewbar) wrote :

To be clear, the fix was released in version 1.1.14:

https://github.com/gearman/gearmand/releases/tag/1.1.14

Changed in gearmand:
status: Invalid → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers