Completion cache write exception when threads are in-use

[Ryan Hsu]

I have a few threads that are doing searches for nova instances and I occasionally hit "ValueError: I?O operation on closed file" (full traceback below). This seems to be similar to bug #1213958, which had a fix committed not too long ago. Perhaps the root cause was never fixed?

VERSION:
python-novaclient==2.15.0.156.g72f6258

TRACEBACK:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/polarbear/stack.py", line 438, in worker
    target(item)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/polarbear/stack.py", line 453, in _wait_for_server_deleted
    self.nova.servers.find(name=server.name)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/base.py", line 189, in find
    matches = self.findall(**kwargs)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/base.py", line 236, in findall
    found.append(self.get(obj.id))
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/v1_1/servers.py", line 391, in get
    return self._get("/servers/%s" % base.getid(server), "server")
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/base.py", line 147, in _get
    return self.resource_class(self, body[response_key], loaded=True)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/base.py", line 433, in __init__
    self.manager.write_to_completion_cache('uuid', self.id)
  File "/var/lib/jenkins/workspace/prototype/venv/local/lib/python2.7/site-packages/novaclient/base.py", line 143, in write_to_completion_cache
    cache.write("%s\n" % val)
ValueError: I/O operation on closed file

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/63725

[Ryan Hsu (rhsu)]

Abhishek, I noticed your patch has been abandoned. is this bug still in-progress?

[melanie witt (melwitt)]

So it looks like this should be fixed based on this patch:

https://review.openstack.org/#/c/101373/

"This patch was tested in multithreaded mode to prevent any regression
      from:
        https://bugs.launchpad.net/python-novaclient/+bug/1213958"

Please re-open if you still encounter this issue.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/63725
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.