util.ignore_missing() does not work

Bug #1216921 reported by Matthias Witte
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
Fix Released
High
Unassigned
duplicity (Ubuntu)
Fix Released
High
Michael Terry
Precise
Fix Released
High
Louis Bouchard
Quantal
Fix Released
High
Louis Bouchard
Raring
Fix Released
High
Louis Bouchard

Bug Description

SRU justification :

Without this fix, there is a potential for crash during execution of duplicity

Impact :

Renders duplicity potentially unusable dues to spurious crashes

Fix :

Backport upstream fix for this problem merged in
https://code.launchpad.net/~mterry/duplicity/ignore-missing

Test Case :

A session must be run within the python debugger to systematically reproduce the context.

1) Run a duplicity session as outlined in comment #8 inside the debugger
2) break at duplicity/path:568 instead of 567
3) When the program breaks, manually remove the file that ends in "...manifest.gpg"
4) continue execution

Without the fix, duplicity will crash with the outlined backtrace. With the fix, duplicity will terminate normally.

Regression :

Minimal as the modification changes exception handling only for a function only used twice to delete files/directories (path.py & tempdir.py)

Description of the problem :

Duplicity can potentially crash while attempting to delete a file that no longer exists.

Original description

duplicity version: 0.6.18-0ubuntu3
python version: 2.7.3
Distro: ubuntu precise 12.04
target file system: ftp

I happen to encounter failed backups with tracebacks like this:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1403, in <module>
     with_tempdir(main)
  File "/usr/bin/duplicity", line 1396, in with_tempdir
     fn()
   File "/usr/bin/duplicity", line 1366, in main
     full_backup(col_stats)
   File "/usr/bin/duplicity", line 504, in full_backup
     sig_outfp.to_remote()
   File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 184, in to_remote
     globals.backend.move(tgt) #@UndefinedVariable
   File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 364, in move
     source_path.delete()
   File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567, in delete
     util.ignore_missing(os.unlink, self.name)
   File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116, in ignore_missing
     fn(filename)
 OSError: [Errno 2] No such file or directory: '/BACKUP/.duplycache/duply_foo/duplicity-full-signatures.20130825T140002Z.sigtar.gpg'

Now running test code like

#!/usr/bin/env python
#
# Do what util.ignore_missing(os.unlink, self.name) tries to do and
# fails to achieve

import os
import sys
import errno

if __name__ == "__main__":
        try:
                os.unlink("/tmp/doesnotexist")
        except Exception:
                # "type" is a reserved keyword, replaced with "mytype"
                mytype, value, tb = sys.exc_info()
                print "-" * 78
                print "mytype: ", mytype
                print "value: ", value
                print "value[0]:", value[0]
                print "errno.ENOENT: ", errno.ENOENT
                print "isinstance(mytype, OSError): ", isinstance(mytype, OSError)
                print "-" * 78
                if isinstance(mytype, OSError) and value[0] == errno.ENOENT:
                        print "Gotcha!"
                        pass
                print "Ooops, missed it ..."
                raise

will always raise the exception and not ignore it, because isinstance(mytype, OSError) is always False.

What I expect ignore_missing to look like is:

def ignore_missing(fn, filename):
    """
    Execute fn on filename. Ignore ENOENT errors, otherwise raise exception.

    @param fn: callable
    @param filename: string
    """
    try:
        fn(filename)
    except OSError, ex:
        if ex.errno == errno.ENOENT:
            pass
        else:
            raise
    else:
        raise

Revision history for this message
Matthias Witte (matthias-witte-7) wrote :
Revision history for this message
Matthias Witte (matthias-witte-7) wrote :

Added patch

Revision history for this message
Matthias Witte (matthias-witte-7) wrote :

Uploade a patch version without tabs.

Revision history for this message
Michael Terry (mterry) wrote :

Confirmed, I filed a branch against trunk with your patch. Thanks!

Louis Bouchard (louis)
Changed in duplicity:
status: New → In Progress
assignee: nobody → Louis Bouchard (louis-bouchard)
Louis Bouchard (louis)
affects: duplicity → duplicity (Ubuntu)
Changed in duplicity (Ubuntu):
importance: Undecided → High
Louis Bouchard (louis)
Changed in duplicity (Ubuntu Precise):
status: New → Confirmed
status: Confirmed → Triaged
Changed in duplicity (Ubuntu Quantal):
status: New → Triaged
Changed in duplicity (Ubuntu Raring):
status: New → Triaged
Changed in duplicity:
status: New → Confirmed
Changed in duplicity (Ubuntu):
status: In Progress → Triaged
Changed in duplicity (Ubuntu Precise):
assignee: nobody → Louis Bouchard (louis-bouchard)
Changed in duplicity (Ubuntu Quantal):
assignee: nobody → Louis Bouchard (louis-bouchard)
Changed in duplicity (Ubuntu Raring):
assignee: nobody → Louis Bouchard (louis-bouchard)
Louis Bouchard (louis)
Changed in duplicity (Ubuntu Precise):
importance: Undecided → High
Changed in duplicity (Ubuntu Quantal):
importance: Undecided → High
Changed in duplicity (Ubuntu Raring):
importance: Undecided → High
Louis Bouchard (louis)
Changed in duplicity (Ubuntu):
status: Triaged → In Progress
assignee: Louis Bouchard (louis-bouchard) → Michael Terry (mterry)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package duplicity - 0.6.21-0ubuntu4

---------------
duplicity (0.6.21-0ubuntu4) saucy; urgency=low

  * debian/patches/02ignoremissing.dpatch:
    - Fix patch to not try to handle non-OSError exceptions
 -- Michael Terry <email address hidden> Thu, 12 Sep 2013 11:16:32 -0400

Changed in duplicity (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Louis Bouchard (louis) wrote :

@mathias:

I have run over 75 iteration of a full backup to FTP target without any kind of python issue.

Is there some more specific steps to be used in order to reproduce the issue ? If so, please share it with me so I can use it for the SRU

Changed in duplicity:
milestone: none → 0.6.23
importance: Undecided → Medium
status: Confirmed → Fix Committed
importance: Medium → High
Louis Bouchard (louis)
Changed in duplicity (Ubuntu Precise):
status: Triaged → In Progress
Changed in duplicity (Ubuntu Quantal):
status: Triaged → In Progress
Changed in duplicity (Ubuntu Raring):
status: Triaged → In Progress
Revision history for this message
Matthias Witte (matthias-witte-7) wrote :

@Louis:

To reproduce this error you would have to monitor the file transfers to the ftp storage and on completion
remove the source files from the archieve directory, before duplicity reaches the unlink. You could probably
run duplicity in a debugger and set a breakpoint in line 567 path.py, print the file name it wants to delete and
remove it manually. Then continue.

Revision history for this message
Matthias Witte (matthias-witte-7) wrote :

Ok, checked it:

TMPDIR='/BACKUP/.duplytemp' PASSPHRASE= SIGN_PASSPHRASE=YOURSIGN_PASSPHRASE FTP_PASSWORD=YOUR_FTP_PASSWORD pydb /usr/bin/duplicity --archive-dir '/BACKUP/.duplycache' --name 'duply_foo' --encrypt-key DEADBEEF0 --sign-key DEADBEEF1 --verbosity '5' --gpg-options '--trust-model always --compress-algo=zip --compress-level=1' --volsize 100 --full-if-older-than 1M --exclude-globbing-filelist /root/.duply/foo/exclude /DATA0 ftp://<email address hidden>/foo/
(Pydb) b duplicity/path:567
(Pydb) c
...
(/usr/share/pyshared/duplicity/path.py:567): delete
567 util.ignore_missing(os.unlink, self.name)
(Pydb) p self.name
'/BACKUP/.duplycache/duply_foo/duplicity-new....20130916T144629Z.to.20130916T144916Z.sigtar.gpg'

Now remove this file manually and tell the debugger to continue:

(Pydb) c

Revision history for this message
Louis Bouchard (louis) wrote :

debdiff for precise

description: updated
Revision history for this message
Louis Bouchard (louis) wrote :

debdiff for quantal

Revision history for this message
Louis Bouchard (louis) wrote :

debdiff for raring

Revision history for this message
Louis Bouchard (louis) wrote :

Updated precise debdiff with typo fixed

Revision history for this message
Louis Bouchard (louis) wrote :

Updated quantal debdiff with typo fixed

Revision history for this message
Louis Bouchard (louis) wrote :

Updated raring debdiff with typo fixed

Revision history for this message
Chris J Arges (arges) wrote :

uploaded to P/Q/R

Revision history for this message
Stéphane Graber (stgraber) wrote : Please test proposed package

Hello Matthias, or anyone else affected,

Accepted duplicity into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/duplicity/0.6.18-0ubuntu3.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in duplicity (Ubuntu Precise):
status: In Progress → Fix Committed
tags: added: verification-needed
Changed in duplicity (Ubuntu Quantal):
status: In Progress → Fix Committed
Revision history for this message
Stéphane Graber (stgraber) wrote :

Hello Matthias, or anyone else affected,

Accepted duplicity into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/duplicity/0.6.19-0ubuntu2.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in duplicity (Ubuntu Raring):
status: In Progress → Fix Committed
Revision history for this message
Stéphane Graber (stgraber) wrote :

Hello Matthias, or anyone else affected,

Accepted duplicity into raring-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/duplicity/0.6.21-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Sebastien Bacher (seb128) wrote :

Those got uploaded, unsubscribing sponsors

Louis Bouchard (louis)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package duplicity - 0.6.21-0ubuntu1.2

---------------
duplicity (0.6.21-0ubuntu1.2) raring-proposed; urgency=low

  * debian/patches/02fixignoremissing.dpatch
    - Fix a utility function that tried to ignore missing files, but
      didn't. Also fixed not try to handle non-OSError exceptions
      Fixes a possible crasher. (LP: #1216921)
      Patch by Matthias Witte.
 -- Louis Bouchard <email address hidden> Fri, 25 Oct 2013 10:50:54 -0500

Changed in duplicity (Ubuntu Raring):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package duplicity - 0.6.19-0ubuntu2.3

---------------
duplicity (0.6.19-0ubuntu2.3) quantal-proposed; urgency=low

  * debian/patches/10fixignoremissing.dpatch
    - Fix a utility function that tried to ignore missing files, but
      didn't. Also fixed not try to handle non-OSError exceptions
      Fixes a possible crasher. (LP: #1216921)
      Patch by Matthias Witte.
 -- Louis Bouchard <email address hidden> Fri, 25 Oct 2013 10:45:37 -0500

Changed in duplicity (Ubuntu Quantal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package duplicity - 0.6.18-0ubuntu3.3

---------------
duplicity (0.6.18-0ubuntu3.3) precise-proposed; urgency=low

  * debian/patches/10fixignoremissing.dpatch
    - Fix a utility function that tried to ignore missing files, but
      didn't. Also fixed not try to handle non-OSError exceptions
      Fixes a possible crasher. (LP: #1216921)
      Patch by Matthias Witte.
 -- Louis Bouchard <email address hidden> Fri, 25 Oct 2013 10:27:19 -0500

Changed in duplicity (Ubuntu Precise):
status: Fix Committed → Fix Released
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.