0.7.0 to 0.7.01 breaks gs:// (Google Cloud Storage)

[troy engel]

Arch (current)

python2 2.7.9-1
python2-boto 2.35.1-1

Upgrading duplicity from 0.7.0 (working) to 0.7.01 seems to break the parsing of Google Cloud Storage GS buckets on the commandline. The result in -v9 (slightly redacted to protect private info):

/usr/bin/duplicity --name duply_rocinante --encrypt-key XXXXXXXX --sign-key XXXXXXXX --verbosity 9 --gpg-options --pinentry-mode loopback --exclude-if-present .duplicity-ignore --exclude-globbing-filelist /path/to/exclude /path gs://rocinante

[...]

Import of duplicity.backends.webdavbackend Succeeded
Using temporary directory /tmp/duplicity-ihzuBN-tempdir
Backend error detail: Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1500, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1494, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1327, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/site-packages/duplicity/commandline.py", line 1047, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 221, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 207, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/site-packages/duplicity/backends/_boto_single.py", line 145, in __init__
    raise BackendException('Boto requires a bucket name.')
BackendException: Boto requires a bucket name.

If I downgrade duplicity to 0.7.0 everything works fine as expected. When diff'ing the two releases, I think the problem is that "gs://" was added to the "uses_netloc" array when it did not exist before, and it's not supported by Boto this way.

Change #1 I saw adding 'gs':
=====================
duplicity-0.7.01/duplicity/backends/botobackend.py

 duplicity.backend.register_backend("gs", BotoBackend)
 duplicity.backend.register_backend("s3", BotoBackend)
 duplicity.backend.register_backend("s3+http", BotoBackend)
+duplicity.backend.uses_netloc.extend([ 'gs', 's3', 's3+http' ])
=====================

Change #2 that did *not* have 'gs' before the above change
=====================
duplicity-0.7.01/duplicity/backend.py

 # This looks similar to urlparse's 'uses_netloc' list, but urlparse doesn't use
 # that list for parsing, only creating urls. And doesn't include our custom
 # schemes anyway. So we keep our own here for our own use.
-uses_netloc = ['ftp',
- 'ftps',
- 'hsi',
- 's3',
- 'scp', 'ssh', 'sftp',
- 'webdav', 'webdavs',
- 'gdocs',
- 'http', 'https',
- 'imap', 'imaps',
- 'mega',
- 'copy']
+#
+# NOTE: this is filled by the respective backends during registering
+uses_netloc = []
=====================

Given the above finds, if I simply edit /usr/lib/python2.7/site-packages/duplicity/backends/botobackend.py on version 0.7.01 and remove the 'gs' in the extended array like so:

duplicity.backend.uses_netloc.extend([ 's3', 's3+http' ])
# duplicity.backend.uses_netloc.extend([ 'gs', 's3', 's3+http' ])

...and then run the same duplicity command, it all works as expected. I am not sure why 'gs' was added, it's possible the bug is actually in ParseUrl() but I think it's because Boto doesn't support gs:// in this fashion. Either way, Google Cloud Storage is non-operational on 0.7.01 due to this issue.

[Kenneth Loafman (kenneth-loafman)]

This bug is a dupe of 1411803 and is fixed in the trunk. Target is 0.7.02.