swiftclient.service upload leaks filehandles for large files

[Chris Stratford]

When uploading large files (i.e. multi-segment ones) using the swiftclient.service functions, if you pass a file path instead of a file object it seems to leave the files open after upload.

For example:
tmpfile = "/tmp/wibble"
obj = SwiftUploadObject(tmpfile, object_name=tmpfile)
swift.upload(container="mycontainer", objects=[obj])

After this, lsof shows /tmp/wibble is still open (multiple times)

This approach works around the problem:
with open(tmpfile, "r") as f:
    obj = SwiftUploadObject(f, object_name=tmpfile)
    swift.upload(container=container, objects=[obj]):

While I've seen this on the wily packaged version, it also appears to be a problem in the upstream (3.0) python module.

$ lsb_release -rd
Description: Ubuntu 15.10
Release: 15.10

$ apt-cache policy python-swiftclient
python-swiftclient:
  Installed: 1:2.6.0-1ubuntu1
  Candidate: 1:2.6.0-1ubuntu1
  Version table:
 *** 1:2.6.0-1ubuntu1 0
        500 http://gb.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages
        100 /var/lib/dpkg/status

[Tim Burke (1-tim-z)]

Sure enough, we've got an open() at https://github.com/openstack/python-swiftclient/blob/3.3.0/swiftclient/service.py#L1718 with no close()

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446527

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446527
Committed: https://git.openstack.org/cgit/openstack/python-swiftclient/commit/?id=809e4cf98fceed1ddef033c86199cca48708d710
Submitter: Jenkins
Branch: master

commit 809e4cf98fceed1ddef033c86199cca48708d710
Author: Kazufumi Noto <email address hidden>
Date: Wed Mar 15 20:47:33 2017 +0000

    Close file handle after upload job

    The opened file for upload is not closed.
    This fix prevents possible file handle leak.

    Closes-Bug: #1559079
    Change-Id: Ibc58667789e8f54c74ae2bbd32717a45f7b30550

[Tim Burke (1-tim-z)]

Fixed in https://github.com/openstack/python-swiftclient/commit/809e4cf

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-swiftclient 3.4.0 release.