notification/eventlet: ack message error

[Mehdi Abaakouk]

Hi,

Since 1.17, ack/requeue of messages is a bit broken when we use the eventlet executor:

2015-07-15 10:06:18.926 17191 ERROR concurrent.futures Traceback (most recent call last):
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 298, in _invoke_callbacks
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures callback(self)
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/oslo_messaging/_executors/impl_pooledexecutor.py", line 83, in <lambda>
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures fut.add_done_callback(lambda f: callback.done())
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/oslo_messaging/_utils.py", line 96, in done
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures self._post(self._incoming, self._result)
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/oslo_messaging/notify/dispatcher.py", line 80, in _post_dispatch
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures incoming.acknowledge()
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 106, in acknowledge
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures self.acknowledge_callback()
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 180, in acknowledge
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures self._raw_message.ack()
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures File "/usr/lib/python2.7/site-packages/kombu/message.py", line 87, in ack
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures self))
2015-07-15 10:06:18.926 17191 ERROR concurrent.futures MessageStateError: Message already acknowledged with state: ACK

Cheers,

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mehdi Abaakouk (sileht) (<email address hidden>) on branch: master
Review: https://review.openstack.org/202187
Reason: Don't worry let take the other patch: https://review.openstack.org/#/c/202202/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/202202
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=02a3a398145c4923915f7eaa5e601d6f970df403
Submitter: Jenkins
Branch: master

commit 02a3a398145c4923915f7eaa5e601d6f970df403
Author: Joshua Harlow <email address hidden>
Date: Wed Jul 15 10:26:06 2015 -0700

    Ensure callback variable capture + cleanup is done correctly

    It appears the the callback variable that was being called on
    future done was not the right one, due to the lambda capture
    mechanism referring to a lazy variable which would potentially
    be a different callback by the time the future would finish so
    make sure we capture the right one and ensure the future has access
    to it.

    This adds a helper method that submission will go through
    to ensure that the callback variable is correctly captured
    in the created lambda and also ensures that the incomplete futures
    list is cleaned up (when the future is done).

    The notification listener tests use now eventlet to show up this
    issue that doesn't occur with blocking executor.

    Closes-bug: #1474943

    Closes-bug: #1475307

    Change-Id: I23e393d504662532572b5b344b87387be6d7bcb1