python-swiftclient 2.4.0 broke WAL-E

[Vladimir Mihailenco]

WAL-E is not able to restore DB from backup using python-swiftclient 2.4.0. Version 2.3.1 works.

wal_e.worker.swift.swift_worker INFO MSG: beginning partition download
        DETAIL: The partition being downloaded is part_00000004.tar.lzo.
        HINT: The absolute Swift object name is basebackups_005/base_000000010000069300000083_00232840/tar_partitions/part_00000004.tar.lzo.
        STRUCTURED: time=2015-03-29T14:47:44.911781-00 pid=18002
requests.packages.urllib3.connectionpool INFO Starting new HTTPS connection (2): snet-storage101.ord1.clouddrive.com
/var/lib/postgresql/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:79: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
wal_e.worker.swift.swift_worker INFO MSG: beginning partition download
        DETAIL: The partition being downloaded is part_00000005.tar.lzo.
        HINT: The absolute Swift object name is basebackups_005/base_000000010000069300000083_00232840/tar_partitions/part_00000005.tar.lzo.
        STRUCTURED: time=2015-03-29T14:47:45.278930-00 pid=18002
lzop: <stdin>: Compressed data violation
lzop: <stdin>: Compressed data violation
wal_e.retries WARNING MSG: retrying after encountering exception
        DETAIL: Exception information dump:
        Traceback (most recent call last):
          File "/var/lib/postgresql/venv/local/lib/python2.7/site-packages/wal_e/retries.py", line 62, in shim
            return f(*args, **kwargs)
          File "/var/lib/postgresql/venv/local/lib/python2.7/site-packages/wal_e/worker/swift/swift_worker.py", line 78, in fetch_partition
            raise exc
        OSError: [Errno 32] Broken pipe

        HINT: A better error message should be written to handle this exception. Please report this output and, if possible, the situation under which it arises.
        STRUCTURED: time=2015-03-29T14:47:45.839374-00 pid=18002

[Christian Schwede (cschwede)]

Just to make sure I understood this correctly: you tried with python-swiftclient 2.4.0 and wal-e fails, and afterwards retried successfully with 2.3.1?

[Vladimir Mihailenco (vmihailenco)]

@Christian, correct.

[Christian Schwede (cschwede)]

I installed python-swiftclient 2.4.0 and wal-e 0.8.0 and was able to successfully backup and restore a postgresql DB on my saio.

Can you share some more information with us? For example, what arguments where used?

[Vladimir Mihailenco (vmihailenco)]

Sorry, but I don't maintain WAL-E or python-swiftclient to debug this for you. WAL-E itself was used without any arguments: `var/lib/postgresql/venv/bin/wal-e backup-fetch /var/lib/postgresql/9.4 LATEST`