redis storage claims exception

Bug #1495567 reported by Ronald Rees on 2015-09-14
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
zaqar
Undecided
Unassigned

Bug Description

when using the redis storage backend, found the following sequence causes an exception in the zaqar server.

- create queue
- post messages
- create claim
- get stats on queue

Below is the output in the debug logs of the zaqar server.

2015-09-14 15:08:52.583 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-498fd05f-6737-4a02-ade7-f88b9a47977a - - - - -] [project_id:None] Queue item PUT - queue: workflow5, on_put /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:39
135.109.209.112 - - [14/Sep/2015 15:08:52] "PUT /v1/queues/workflow5 HTTP/1.1" 204 0
2015-09-14 15:08:52.600 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-1ec023f1-9c51-45cc-a393-8f2f0c60df81 - - - - -] [project_id:None] Queue collection GET on_get /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:91
2015-09-14 15:08:52.600 26916 WARNING zaqar.common.pipeline [req-1ec023f1-9c51-45cc-a393-8f2f0c60df81 - - - - -] [project_id:None] Stage <zaqar.storage.redis.messages.MessageQueueHandler object at 0x7fa3bd354750> does not implement list
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues HTTP/1.1" 200 352
2015-09-14 15:08:52.618 26916 DEBUG zaqar.transport.wsgi.v1_0.queues [req-776e1ac1-a813-49cb-bd32-d856f9e24a62 - - - - -] [project_id:None] Queue item exists - queue: workflow5, project: None on_head /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/queues.py:56
2015-09-14 15:08:52.618 26916 WARNING zaqar.common.pipeline [req-776e1ac1-a813-49cb-bd32-d856f9e24a62 - - - - -] [project_id:None] Stage <zaqar.storage.redis.messages.MessageQueueHandler object at 0x7fa3bd354750> does not implement exists
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5 HTTP/1.1" 204 0
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5/stats HTTP/1.1" 200 51
2015-09-14 15:08:52.650 26916 DEBUG zaqar.transport.wsgi.v1_0.messages [req-acba3a87-e203-48bd-8936-74b5c873b1de - - - - -] [project_id:None] Messages collection POST - queue: workflow5, project: None on_post /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/messages.py:133
135.109.209.112 - - [14/Sep/2015 15:08:52] "POST /v1/queues/workflow5/messages HTTP/1.1" 201 173
2015-09-14 15:08:52.669 26916 ERROR zaqar.transport.wsgi.v1_0.stats [req-864ff994-c007-42ed-9f0f-faa85890cb58 - - - - -] [project_id:None] int() argument must be a string or a number, not 'NoneType'
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats Traceback (most recent call last):
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/transport/wsgi/v1_0/stats.py", line 38, in on_get
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats project=project_id)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/common/pipeline.py", line 97, in consumer
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats tmp = target(*args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 153, in wrapper
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats return func(*args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 186, in wrapper
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats return func(self, *args, **kwargs)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/messages.py", line 601, in stats
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats claimed = self._claim_ctrl._count_messages(name, project)
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 178, in _count_messages
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats num_claimed += int(info[0])
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats TypeError: int() argument must be a string or a number, not 'NoneType'
2015-09-14 15:08:52.669 26916 TRACE zaqar.transport.wsgi.v1_0.stats
135.109.209.112 - - [14/Sep/2015 15:08:52] "GET /v1/queues/workflow5/stats HTTP/1.1" 503 138

I also found that if I tried to make subsequent posts/claims on that queue, that would not work either and would get the following exception.

2015-09-14 15:03:32.048 26916 DEBUG zaqar.transport.wsgi.v1_0.claims [req-95e6b65e-429d-4a51-b500-e9388016940a - - - - -] [project_id:None] Claims collection POST - queue: workflow3, project: None on_post /opt/stack/zaqar/zaqar/transport/wsgi/v1_0/claims.py:47
2015-09-14 15:03:32.049 26916 ERROR zaqar.transport.wsgi.v1_0.claims [req-95e6b65e-429d-4a51-b500-e9388016940a - - - - -] [project_id:None] Error running script (call to f_f8726c7ad2f323131fb1bbd395fff6b501b316c8): @user_script:59: user_script:59: attempt to compare nil with number
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims Traceback (most recent call last):
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/transport/wsgi/v1_0/claims.py", line 65, in on_post
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims **claim_options)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/common/pipeline.py", line 97, in consumer
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims tmp = target(*args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 153, in wrapper
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(*args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/utils.py", line 186, in wrapper
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(self, *args, **kwargs)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 268, in create
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims msg_ttl, msg_expires)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/opt/stack/zaqar/zaqar/storage/redis/claims.py", line 115, in _claim_messages
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return func(keys=[msgset_key], args=args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 2646, in __call__
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return client.evalsha(self.sha, len(keys), *args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1911, in evalsha
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 565, in execute_command
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims return self.parse_response(connection, command_name, **options)
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 577, in parse_response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims response = connection.read_response()
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 574, in read_response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims raise response
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims ResponseError: Error running script (call to f_f8726c7ad2f323131fb1bbd395fff6b501b316c8): @user_script:59: user_script:59: attempt to compare nil with number
2015-09-14 15:03:32.049 26916 TRACE zaqar.transport.wsgi.v1_0.claims
135.109.209.112 - - [14/Sep/2015 15:03:32] "POST /v1/queues/workflow3/claims HTTP/1.1" 503 135

I found that if you issue a DELETE of the claimed messages prior to issuing subsequent post/claims or get stats, then these exceptions do not occur.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers