Duplicity Hangs intermittently when using multiprocess uploads to S3

Bug #1243246 reported by someone1 on 2013-10-22
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
Medium
Unassigned

Bug Description

Python Version 2.7.5
Duplicity Latest Dev (from my s3-glacier branch) Though I believe this issue exists in 0.6.22
OS: Arch Linux (64-bit) fully updated

Randomly during full backups duplicity hangs. My usual incremental backup has under 100 volumes, I've seen the hangups starting to occur anywhere from volume 300 all the way to volume 8000.

Options I give:
/bin/duplicity --full-if-older-than=27D20h --allow-source-mismatch --s3-unencrypted-connection --s3-use-multiprocessing --s3-multipart-chunk-size=10 -vdebug --asynchronous-upload --volsize=30 --encrypt-key=<removed> --sign-key=<removed> /path/to/my/Backup/ s3+http://<bucket_removed>

I added additional logging and saw that the multiprocess pool is spawned and the main process is waiting for the pool to finish, although the pool processes never start. Looks like they all hang while waiting to access the internal Queue of the pool. I found other people reporting similar behavior with Python: http://bugs.python.org/msg118867

When I issue a `kill -2` to the processes that spawned, I get the following stack trace:

Process PoolWorker-1209:
Process PoolWorker-1208:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-1210:
Traceback (most recent call last):
Process PoolWorker-1207:
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self.run()
    self.run()
    self.run()
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
    task = get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
    task = get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
    racquire()
    racquire()
KeyboardInterrupt
    return recv()
KeyboardInterrupt
KeyboardInterrupt
    racquire()
KeyboardInterrupt

Related branches

Changed in duplicity:
milestone: none → 0.6.24
status: New → Fix Committed
importance: Undecided → Medium
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers