queue-type sqlite3 crashes on startup if data blob in saved job is empty

[Andy B]

version 1.1.12

Using queue-type sqlite3, gearman crashes on startup when it tries to load the saved jobs from the sqlite3 data file if a job has an empty data blob.

Steps to reproduce:

1. start gearmand: gearmand --verbose INFO --queue-type libsqlite3 --libsqlite3-db /tmp/gearman.sqlite3
2. create a background job with an empty data blob: gearman -P -f junk -b </dev/null
3. stop gearmand
4. start gearmand
5. witness "column 3 is not type TEXT replay_loop(QUEUE_ERROR)" error.

Expected outcome:

1. gearmand starts and is able to read persistent jobs from the sqlite3 database.

See log attached from a sample run same as the above steps to reproduce.

I suspect this is caused by sqlite3's manifest typing. The empty field's type isn't SQLITE_BLOB. Since the data field is user/client provided and could be empty, the type of its value is mostly meaningless. It's probably best to only check the types of gearmand generated columns, if the types must be checked at all.

Alternative is to set the data column to be not null, and then attempting to queue a job without a data value fails on creation, but doesn't keep the server from running. This may not be backward compatible.

Also, this error message is confusing because the table is created with the data column having type BLOB, but the error message says it is not type TEXT. Additionally (unrelated to the above), the check of column 4's type looks like it will display "column 3 is not type INTEGER", when it should say "column 4 is not type INTEGER".

[Andy B (u-launchpad-x)]

After further testing and examination of the created sqlite3 database, it seems that empty data field is being inserted into the table as NULL, and sqlite3 is able to make a distinction between NULL and the empty string for blob fields, so maybe the fix is to set the data field not NULL and only ever insert the empty string into it.

[Andy B (u-launchpad-x)]

Some additional notes.

1)
Creating the field as

    data BLOB not null default (cast('' as BLOB))

avoids the NULL insert issue and maintains the type of the empty data column so the replay reading works.

2)
Should this crop up in an sqlite3 database that won't load, it can be fixed with the following query:

    update gearman_queue set data = cast('' as BLOB) where data is NULL

3)
The gearman command line client enforces having a data payload, but gearmand will accept an empty payload. It's odd that libgearman will enforce something differently than what the server will accept. But, there's also an inconsistency in the gearman command line client.

$ gearman -P -f junk -b -N
<hit enter, attempting to create an empty data payload after the newline is stripped>
gearman: gearman_client_add_task : add_task(GEARMAN_INVALID_ARGUMENT) invalid workload -> libgearman/add.cc:194: pid(26893)
=> no job created

$ echo -n | gearman -P -f junk -b
=> job created with data=NULL

$ gearman -P -f junk -b </dev/null
=> job created with data=NULL

$ gearman -P -f junk -b -N </dev/null
=> immediately exit, since EOF is detected on STDIN.

I'd expect these to all act the same, either creating a job with an empty payload or being rejected.