redis storage claims exception

Bug #1495567 reported by Ronald Rees
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
zaqar
New
Undecided
Unassigned

Bug Description

when using the redis storage backend, found the following sequence causes an exception in the zaqar server.

- create queue
- post messages
- create claim
- get stats on queue

Below is the output in the debug logs of the zaqar server.

2015-09-14 15:08:52.583 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-498fd05f-6737-4a02-ade7-f88b9a47977a - - - - -] [project_id:None] Queue item PUT - queue: workflow5, on_put /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:39
135.109.209.112 - - [14/Sep/2015 15:08:52] "PUT /v1/queues/workflow5 HTTP/1.1" 204 0
2015-09-14 15:08:52.600 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-1ec023f1-9c51-45cc-a393-8f2f0c60df81 - - - - -] [project_id:None] Queue collection GET on_get /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:91
2015-09-14 15:08:52.600 26916 WARNING zaqar.common.pipeline [req-1ec023f1-9c51-45cc-a393-8f2f0c60df81 - - - - -] [project_id:None] Stage <zaqar.storage.redis.messages.MessageQueueHandler object at 0x7fa3bd354750> does not implement list
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues HTTP/1.1" 200 352
2015-09-14 15:08:52.618 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-776e1ac1-a813-49cb-bd32-d856f9e24a62 - - - - -] [project_id:None] Queue item exists - queue: workflow5, project: None on_head /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:56
2015-09-14 15:08:52.618 26916 WARNING zaqar.common.pipeline [req-776e1ac1-a813-49cb-bd32-d856f9e24a62 - - - - -] [project_id:None] Stage <zaqar.storage.redis.messages.MessageQueueHandler object at 0x7fa3bd354750> does not implement exists
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5 HTTP/1.1" 204 0
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5/stats HTTP/1.1" 200 51
2015-09-14 15:08:52.650 26916 DEBUG zaqar.transport.wsgi.v1_0.messages [req-acba3a87-e203-48bd-8936-74b5c873b1de - - - - -] [project_id:None] Messages collection POST - queue: workflow5, project: None on_post /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/messages.py:133
135.109.209.112 - - [14/Sep/2015 15:08:52] "POST /v1/queues/workflow5/messages HTTP/1.1" 201 173
2015-09-14 15:08:52.669 26916 ERROR zaqar.transport.wsgi.v1_0.stats [req-864ff994-c007-42ed-9f0f-faa85890cb58 - - - - -] [project_id:None] int() argument must be a string or a number, not 'NoneType'
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats Traceback (most recent call last):
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/transport/wsgi/v1_0/stats.py", line 38, in on_get
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats project=project_id)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/common/pipeline.py", line 97, in consumer
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats tmp = target(*args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 153, in wrapper
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats return func(*args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 186, in wrapper
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats return func(self, *args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/messages.py", line 601, in stats
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats claimed = self._claim_ctrl._count_messages(name, project)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 178, in _count_messages
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats num_claimed += int(info[0])
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats TypeError: int() argument must be a string or a number, not 'NoneType'
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5/stats HTTP/1.1" 503 138

I also found that if I tried to make subsequent posts/claims on that queue, that would not work either and would get the following exception.

2015-09-14 15:03:32.048 26916 DEBUG zaqar.transport.wsgi.v1_0.claims [req-95e6b65e-429d-4a51-b500-e9388016940a - - - - -] [project_id:None] Claims collection POST - queue: workflow3, project: None on_post /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/claims.py:47
2015-09-14 15:03:32.049 26916 ERROR zaqar.transport.wsgi.v1_0.claims [req-95e6b65e-429d-4a51-b500-e9388016940a - - - - -] [project_id:None] Error running script (call to f_f8726c7ad2f323131fb1bbd395fff6b501b316c8): @user_script:59: user_script:59: attempt to compare nil with number
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims Traceback (most recent call last):
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/transport/wsgi/v1_0/claims.py", line 65, in on_post
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims **claim_options)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/common/pipeline.py", line 97, in consumer
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims tmp = target(*args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 153, in wrapper
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(*args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 186, in wrapper
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(self, *args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 268, in create
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims msg_ttl, msg_expires)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 115, in _claim_messages
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(keys=[msgset_key], args=args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 2646, in __call__
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return client.evalsha(self.sha, len(keys), *args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1911, in evalsha
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 565, in execute_command
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return self.parse_response(connection, command_name, **options)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 577, in parse_response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims response = connection.read_response()
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 574, in read_response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims raise response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims ResponseError: Error running script (call to f_f8726c7ad2f323131fb1bbd395fff6b501b316c8): @user_script:59: user_script:59: attempt to compare nil with number
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims
135.109.209.112 - - [14/Sep/2015 15:03:32] "POST /v1/queues/workflow3/claims HTTP/1.1" 503 135

I found that if you issue a DELETE of the claimed messages prior to issuing subsequent post/claims or get stats, then these exceptions do not occur.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.