list.remove(x): x not in list when creating a thread

Bug #1662939 reported by Thomas Herve on 2017-02-08
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Thomas Herve

Bug Description

Seen in heat engine:

Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/heat/engine/", line 783, in _action_recorder
     self.state_set(action, self.FAILED, msg)
   File "/usr/lib/python2.7/site-packages/heat/engine/", line 2015, in state_set
     self._add_event(action, status, reason)
   File "/usr/lib/python2.7/site-packages/heat/engine/", line 1769, in _add_event
   File "/usr/lib/python2.7/site-packages/heat/engine/", line 907, in dispatch_event
   File "/usr/lib/python2.7/site-packages/heat/engine/", line 143, in start
     func, *args, **kwargs)
   File "/usr/lib/python2.7/site-packages/oslo_service/", line 96, in add_thread
     th = Thread(gt, self)
   File "/usr/lib/python2.7/site-packages/oslo_service/", line 46, in __init__, group, self)
   File "/usr/lib/python2.7/site-packages/eventlet/", line 197, in link
   File "/usr/lib/python2.7/site-packages/eventlet/", line 232, in _resolve_links
     f(self, *ca, **ckw)
   File "/usr/lib/python2.7/site-packages/oslo_service/", line 34, in _on_thread_done
   File "/usr/lib/python2.7/site-packages/oslo_service/", line 101, in thread_done
 ValueError: list.remove(x): x not in list

It looks like a race condition: if the callback is fast enough, the thread can exit before the link, which is not accounted for.

Thomas Herve (therve) on 2017-02-08
Changed in oslo.service:
assignee: nobody → Thomas Herve (therve)
Thomas Herve (therve) wrote :

See for the reason. Links can be resolved right away, but we only add the thread afterwards.

Thomas Herve (therve) on 2017-02-09
Changed in oslo.service:
status: New → In Progress
tags: added: ocata-backport-potential
Changed in oslo.service:
importance: Undecided → High

This issue was fixed in the openstack/oslo.service 1.20.0 release.

Changed in oslo.service:
status: In Progress → Fix Released
Thomas Herve (therve) on 2017-05-30
tags: added: newton-backport-potential

This issue was fixed in the openstack/oslo.service 1.19.1 release.

This issue was fixed in the openstack/oslo.service 1.16.1 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers