B2 backend fails on string concatenation

Bug #1843995 reported by Stefano Marengo
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Duplicity
Fix Released
Medium
Unassigned

Bug Description

Using the b2 backend the `b2backend.py` fails on string concatenation.

Duplicity version: 0.8.03
Python version: 3.7.4
OS Distro and version: macOS Mojave 10.14.3
Type of target filesystem: macOS

$ pip show b2
Name: b2
Version: 1.4.0

Log output from -v9 option - Include the command line, the first 200 lines of the log, and the last 200 lines of the log (anonymized):

$ duplicity \
-v9 \
--encrypt-key=$GPG_KEYS_ENC \
--sign-key=$GPG_KEY_SIGN \
--volsize 50 \
--full-if-older-than 30D \
/Users/me/source_folder/ b2://[keyID]:[application_key]@[B2_bucket_name][/path/to/target_folder];

Using archive dir: /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b
Using backup name: 4ab78363b97d4f981c0a9b0192c8a99b
GPG binary is gpg, version (2, 2, 10)
Import of duplicity.backends.adbackend Succeeded
Import of duplicity.backends.azurebackend Succeeded
Import of duplicity.backends.b2backend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.hubicbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.jottacloudbackend Succeeded
Import of duplicity.backends.lftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.mediafirebackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.multibackend Succeeded
Import of duplicity.backends.ncftpbackend Succeeded
Import of duplicity.backends.onedrivebackend Succeeded
Import of duplicity.backends.par2backend Succeeded
Import of duplicity.backends.pcabackend Succeeded
Import of duplicity.backends.pydrivebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ssh_paramiko_backend Succeeded
Import of duplicity.backends.ssh_pexpect_backend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.sxbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
B2 Backend (path= path/to/target_folder/, bucket= B2_bucket_name, minimum_part_size= 100000000)
Bucket found
Main action: inc
Acquiring lockfile b'/Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/lockfile'
================================================================================
duplicity $version ($reldate)
Args: /usr/local/bin/duplicity -v9 --encrypt-key=0000000000000000000000000000000000000000 --sign-key=1111111111111111111111111111111111111111 --volsize 50 --full-if-older-than 30D /Users/me/source_folder/ b2://keyID:application_key@B2_bucket_name/path/to/target_folder
Darwin Mac 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64 i386
/usr/local/Cellar/duplicity/0.8.03/libexec/bin/python 3.7.4 (default, Sep 7 2019, 18:27:02)
[Clang 10.0.1 (clang-1001.0.46.4)]
================================================================================
Using temporary directory /var/folders/bj/44q8_hl565s18nqk4fdt76f80000gn/T/duplicity-dq0gcj9o-tempdir
Registering (mkstemp) temporary file /var/folders/bj/44q8_hl565s18nqk4fdt76f80000gn/T/duplicity-dq0gcj9o-tempdir/mkstemp-cgnl642u-1
Temp has 19106807808 available, backup will use approx 68157440.
0 files exist on backend
1 file exists in cache
Extracting backup chains from list of files: []
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
Last full backup is too old, forcing full backup
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive dir: /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Using temporary directory /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-wcyw31kg-tempdir
Registering (mktemp) temporary file /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-wcyw31kg-tempdir/mktemp-f14umhy1-1
Using temporary directory /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-_d8autpm-tempdir
Registering (mktemp) temporary file /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-_d8autpm-tempdir/mktemp-xjznuxqa-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /var/folders/bj/44q8_hl565s18nqk4fdt76f80000gn/T/duplicity-dq0gcj9o-tempdir/mktemp-3yh390fd-2
Selecting /Users/me/source_folder
Comparing . and None
Getting delta of (. dir) and None
A .
Selection: examining path /Users/me/source_folder/FILE_1
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_1
Comparing .FILE_1
Getting delta of (FILE_1 reg) and None
A FILE_1
Selection: examining path /Users/me/source_folder/FILE_2
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_2
Comparing FILE_2
Getting delta of (FILE_2 dir) and None
A FILE_2
Selection: examining path /Users/me/source_folder/FILE_3
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_3
Comparing FILE_3 None
Getting delta of (FILE_3 reg) and None
A FILE_3
Selection: examining path /Users/me/source_folder/FILE_4
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_4
Comparing FILE_4.pdf and None
Getting delta of (FILE_4 reg) and None
A FILE_4
Selection: examining path /Users/me/source_folder/FILE_5
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_5
Comparing FILE_5-25 13-56.pdf and None
Getting delta of (FILE_5 reg) and None
A FILE_5
Selection: examining path /Users/me/source_folder/FILE_6
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_6
Comparing FILE_6 and None
Getting delta of (FILE_6 reg) and None
A FILE_6
Selection: examining path /Users/me/source_folder/FILE_7
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_7
Comparing FILE_7 and None
Getting delta of (FILE_7 reg) and None
A FILE_7
Selection: examining path /Users/me/source_folder/FILE_8
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_8
Comparing FILE_8 and None
Getting delta of (FILE_8 reg) and None
A FILE_8
Selection: examining path /Users/me/source_folder/FILE_9
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_9
Comparing FILE_9 and None
Getting delta of (FILE_9 reg) and None
A FILE_9
Selection: examining path /Users/me/source_folder/FILE_10
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_10
Comparing FILE_10 and None
Getting delta of (FILE_10 reg) and None
A FILE_10
Selection: examining path /Users/me/source_folder/FILE_11
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_11
Comparing FILE_11 and None
Getting delta of (FILE_11 reg) and None
A FILE_11
Selection: examining path /Users/me/source_folder/FILE_12
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_12
Comparing FILE_12 and None
Getting delta of (FILE_12 reg) and None
A FILE_12
Selection: examining path /Users/me/source_folder/FILE_13
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_13
Comparing FILE_13 and None
Getting delta of (FILE_13 reg) and None
A FILE_13
Selection: examining path /Users/me/source_folder/FILE_14
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_14
Comparing FILE_14 and None
Getting delta of (FILE_14 reg) and None
A FILE_14
Selection: examining path /Users/me/source_folder/FILE_15
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_15
Comparing FILE_15 and None
Getting delta of (FILE_15 reg) and None
A FILE_15
Selection: examining path /Users/me/source_folder/FILE_16
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_16
Comparing FILE_16 and None
Getting delta of (FILE_16 reg) and None
A FILE_16
Selection: examining path /Users/me/source_folder/FILE_17
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_17
Comparing FILE_17 and None
Getting delta of (FILE_17 reg) and None
A FILE_17
Selection: examining path /Users/me/source_folder/FILE_18
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_18
Comparing FILE_18 and None
Getting delta of (FILE_18 reg) and None
A FILE_18
Selection: examining path /Users/me/source_folder/FILE_19
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_19
Comparing FILE_19 and None
Getting delta of (FILE_19 reg) and None
A FILE_19
Selection: examining path /Users/me/source_folder/FILE_20
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_20
Comparing FILE_20 and None
Getting delta of (FILE_20 reg) and None
A FILE_20
Selection: examining path /Users/me/source_folder/FILE_21
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_21
Comparing FILE_21 and None
Getting delta of (FILE_21 reg) and None
A FILE_21
Selection: examining path /Users/me/source_folder/FILE_22
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_22
Comparing FILE_22 and None
Getting delta of (FILE_22 reg) and None
A FILE_22
Selection: examining path /Users/me/source_folder/FILE_23
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_23
Comparing FILE_23 and None
Getting delta of (FILE_23 reg) and None
A FILE_23
Selection: examining path /Users/me/source_folder/FILE_24
Selection: + no selection functions found. Including
Selecting /Users/me/source_folder/FILE_24
Comparing FILE_24 and None
Getting delta of (FILE_24 reg) and None
A FILE_24
Removing still remembered temporary file /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-wcyw31kg-tempdir/mktemp-f14umhy1-1
Removing still remembered temporary file /Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/duplicity-_d8autpm-tempdir/mktemp-xjznuxqa-1
AsyncScheduler: running task synchronously (asynchronicity disabled)
Writing duplicity-full.20190914T133203Z.vol1.difftar.gpg
Backtrace of previous error: Traceback (innermost last):
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 375, in inner_retry
    return fn(self, *args)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 535, in put
    self.__do_put(source_path, remote_filename)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 521, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backends/b2backend.py", line 118, in _put
    log.Log(u"Put: %s -> %s" % (source_path.name, self.path + remote_filename), log.INFO)
 TypeError: can only concatenate str (not "bytes") to str

Attempt 1 failed. TypeError: can only concatenate str (not "bytes") to str
Writing duplicity-full.20190914T133203Z.vol1.difftar.gpg
Backtrace of previous error: Traceback (innermost last):
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 375, in inner_retry
    return fn(self, *args)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 535, in put
    self.__do_put(source_path, remote_filename)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backend.py", line 521, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/usr/local/Cellar/duplicity/0.8.03/libexec/lib/python3.7/site-packages/duplicity/backends/b2backend.py", line 118, in _put
    log.Log(u"Put: %s -> %s" % (source_path.name, self.path + remote_filename), log.INFO)
 TypeError: can only concatenate str (not "bytes") to str

Attempt 2 failed. TypeError: can only concatenate str (not "bytes") to str
^CReleasing lockfile b'/Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/lockfile'
Removing still remembered temporary file /var/folders/bj/44q8_hl565s18nqk4fdt76f80000gn/T/duplicity-dq0gcj9o-tempdir/mkstemp-cgnl642u-1
Removing still remembered temporary file /var/folders/bj/44q8_hl565s18nqk4fdt76f80000gn/T/duplicity-dq0gcj9o-tempdir/mktemp-3yh390fd-2
INT intercepted...exiting.
Releasing lockfile b'/Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/lockfile'
Releasing lockfile b'/Users/me/.cache/duplicity/4ab78363b97d4f981c0a9b0192c8a99b/lockfile'

Tags: b2
Revision history for this message
Colin Darie (colinux) wrote :

I have the same error with swift backend.

Duplicity version: 0.8.03
Python version: 3.7.4
OS Distro and version: macOS Mojave 10.14.6

Changed in duplicity:
importance: Undecided → Medium
milestone: none → 0.8.05
status: New → Fix Committed
Changed in duplicity:
status: Fix Committed → Fix Released
Revision history for this message
Graham Cobb (g+launchpad) wrote :

I reported this bug in debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=942593), and was told it had been reported here. I have subsequently tested the fix in 0.8.05 but the bug remains, although in a different function.

Here is the update I provided in the debian bug report:

The bug still exists in the new version, although in a slightly different form:
(during "duplicity full" command)
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 371, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 606, in _do_delete
    self.backend._delete(filename)
  File "/usr/lib/python3/dist-packages/duplicity/backends/b2backend.py", line 138, in _delete
    log.Log(u"Delete: %s" % self.path + filename, log.INFO)
 TypeError: can only concatenate str (not "bytes") to str

In this case the problem is the use of filename in _delete.

From my earlier investigation (which may be wrong, of course, as I am no expert on either Python or this code), I believe the following references also have to be "decoded", in addition to the changes already made:

In _get: the use of local_path.name
In _put: the use of source_path.name
In _delete: the two uses of filename
In _query: the two uses of filename

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.