B2 Python APIs are missing

[Andrew Berry]

When I try to use the B2 backend with 0.7.16-0ubuntu1ppa1347~ubuntu16.04.1, I get the following error:

> BackendException: B2 backend requires B2 Python APIs (pip install b2)

It seems odd to me that the library would be installed outside of apt. I had an old package from trusty (duplicity_0.7.14-0ubuntu0ppa1316~ubuntu14.04.1_amd64.deb) with 0.7.14 which works fine on 16.04, which makes me think this regression was in 0.7.15 or 0.7.16.

[Kenneth Loafman (kenneth-loafman)]

Duplicity runs on many distros and needs close to 30 packages if you were to supply them all at install time. The basic install is around 15. I have found that some distro's, Ubuntu included, lock the version of the package and will not update it except to fix a critical error. I recommend pip because it supplies the most current version as well as allowing easy regression to a previous version of a Python package. If you wish to use apt to get the B2 package, I see no harm, but I have not tested against whatever version they supply.

I will be providing better documentation in version 0.8.0, but I hope the comments above help for now.

[Andrew Berry (andrewberry)]

Thanks. The main advantage of sticking with apt is unattended-upgrades makes it easy to be sure security updates are applied automatically. I’m not familiar with a similar solution for pip, and it sounds like “upgrade everything” isn’t that straightforward (according to https://github.com/pypa/pip/issues/3819).

Any idea why the 0.7.14 release works? Was there a refactoring that extracted out the B2 code? Perhaps a warning when upgrading about any removed libraries would help future upgraders.

[PDJB (philbradley)]

The pip command didn't work for me, and nor did the pip2 equivalent. The same b2 missing backend message always comes up, despite pip ostensibly having installed the required backend as requested (at least, without error meassages). Downgrading to 0.7.06-2ubuntu2 works, but that's a pretty old package!

Running Ubuntu 16.04.3.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kenneth Loafman (kenneth-loafman)]

0.7.15 included a rewrite of the b2 backend to be much faster. The rewrite uses the official b2 API, not web access, thus the need for the package.

If you issue the commands below, you should see similar results:

ken@dione:~$ pip --version
pip 9.0.1 from /usr/local/lib/python2.7/site-packages (python 2.7)
ken@dione:~$ pip2 --version
pip 9.0.1 from /usr/local/lib/python2.7/site-packages (python 2.7)
ken@dione:~$ pip3 --version
pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6)

Please do the commands and post the results, just copy/paste from the console.

[PDJB (philbradley)]

~$ pip --version
pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)
~$ pip2 --version
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
~$ pip3 --version
pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)

[PDJB (philbradley)]

Here's what "pip2 install b2" gives (each time I run it):

 Collecting b2
Collecting requests>=2.9.1 (from b2) Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting futures>=3.0.5 (from b2) Using cached futures-3.2.0-py2-none-any.whl
Collecting logfury>=0.1.2 (from b2) Using cached logfury-0.1.2-py2.py3-none-any.whl
Collecting arrow>=0.8.0 (from b2) Collecting six>=1.10 (from b2)
  Using cached six-1.11.0-py2.py3-none-any.whl Collecting tqdm>=4.5.0 (from b2) Using cached tqdm-4.19.5-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests>=2.9.1->b2) Using cached urllib3-1.22-py2.py3-none-any.whl Collecting idna<2.7,>=2.5 (from requests>=2.9.1->b2)
  Using cached idna-2.6-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.9.1->b2)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests>=2.9.1->b2)
  Using cached certifi-2017.11.5-py2.py3-none-any.whl
Collecting funcsigs (from logfury>=0.1.2->b2)
  Using cached funcsigs-1.0.2-py2.py3-none-any.whl
Collecting python-dateutil (from arrow>=0.8.0->b2)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Installing collected packages: urllib3, idna, chardet, certifi, requests, futures, six, funcsigs, logfury, python-dateutil, arrow, tqdm, b2
Successfully installed arrow-0.12.0 b2-1.1.0 certifi-2017.11.5 chardet-3.0.4 funcsigs-1.0.2 futures-3.2.0 idna-2.6 logfury-0.1.2 python-dateutil-2.6.1 requests-2.18.4 six-1.10.0 tqdm-4.19.5 urllib3-1.22
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[PDJB (philbradley)]

I solved my problem; b2 was failing to import because of a missing "backports.functools_lru_cache"

I installed it with
~$ sudo -H pip install backports.functools_lru_cache

[Jean-Marc Le Roux (jeanmarc-leroux)]

Same issue here.

# pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
# pip2 --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
# pip3 --version
pip 1.5.4 from /usr/lib/python3/dist-packages (python 3.4)

I tried:

pip install b2
pip2 install b2
pip3 install b2

And I still have the same error:

BackendException: B2 backend requires B2 Python APIs (pip install b2)

It was working fine until now (Ubuntu 14.04 64bit).

[Jean-Marc Le Roux (jeanmarc-leroux)]

Reinstalling duplicity after installing b2 (pip install b2) apparently did the trick.

[Erik (eleftrik)]

Same issue here, on Mac OS 10.11.6 and on Debian 9.
Reinstalling duplicity for me didn't the trick.

[Erik (eleftrik)]

I'm using duplicity 0.7.17.

None of these solutions worked for me (and I got no errors):

sudo -H pip install backports.functools_lru_cache
pip install b2
pip2 install b2

[Kenneth Loafman (kenneth-loafman)]

You may want to always specify pip2 since pip could be symlinked to pip2 or pip3. The install would not fail, but if pip --> pip3, then duplicity would not have it available in Python2.

[James Bradbury (jamesbradbury)]

Can anyone tell me what's going on here or how I can workaround this problem?

$ duplicity --version
duplicity 0.7.17

$ pip list | grep b2
b2 (1.3.4)

$ pip2 list | grep b2
b2 (1.3.4)

$ duplicity LocalDir/ b2://xxxxx:yyyyy@bucketname/local_dir
BackendException: B2 backend requires B2 Python APIs (pip install b2)

$ python
Python 2.7.6 (default, Nov 23 2017, 15:50:55)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import b2
>>> dir(b2)
['NullHandler', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'logging', 'sys']

[James Bradbury (jamesbradbury)]

I think it's this issue: https://github.com/henrysher/duplicity/issues/14#issue-283710635

[Stefano Marengo (ste-m)]

Solution that worked for me on macOS, for anyone searching:

$ /location/of/duplicity/libexec/bin/pip3 install b2

(
if installed with brew that would most probably be:
$ /usr/local/Cellar/duplicity/[VERSION]/libexec/bin/pip3 install b2
)

Use the pip coming from the duplicity package, and b2 will be located.
Of course this is a quick-and-dirty, as you'll have to re-patch for every update.

[Kenneth Loafman (kenneth-loafman)]

I marked this as Won't Fix. Duplicity runs on many distros and needs close to 30 packages if you were to supply them all at install time. The basic install is around 15. I have found that some distro's, Ubuntu included, lock the version of the package and will not update it except to fix a critical error. I recommend pip because it supplies the most current version as well as allowing easy regression to a previous version of a Python package. If you wish to use apt to get the B2 package, I see no harm, but I have not tested against whatever version they supply.

[Andrew Berry (andrewberry)]

If pip is the recommended installation method, and apt packages aren't complete or tested, should https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa be marked as deprecated?

[Kenneth Loafman (kenneth-loafman)]

Actually, no. The latest and best duplicity versions are in the ppa package. Ubuntu, and others, lock the version when a release is made. For example, version 0.7.19 is the current version in the ppa, but it's not in any apt repository. Instead, much earlier versions are there and do not get fixes or improvements, so I normally recommend upgrading using the duplicity ppa with apt.

As to pip, much the same goes there. There are versions on pip that far exceed what is on the distro's apt repository. Some of those are needed to keep the duplicity backends running, since the hosts may randomly change the API after successfully getting the last version working.

It's not apt I don't like, it does it's job. It's the policy of locked versions that causes me a lot of grief and needless bug reports for packages that are years older than they should be.

[Andrew Berry (andrewberry)]

This bug report was originally against the PPA package (0.7.16-0ubuntu1ppa1347~ubuntu16.04.1) and not those included in Ubuntu's repositories.

[Kenneth Loafman (kenneth-loafman)]

Two options for now:
- upgrade to the _latest_ (dev) version of duplicity,
- revert to b2sdk==1.7.0 until duplicity 0.8.20 is released

To revert do: _sudo pip install -U b2sdk==1.7.0_

There are multiple options for latest:

- Latest snap builds - “sudo snap install duplicity —classic —edge"
- Latest pip3 builds - “sudo pip3 install --pre duplicity"
- Latest duplicity PPA - https://code.launchpad.net/\~duplicity-team/+archive/ubuntu/duplicity-develop-git

Note 1: UNINSTALL duplicity first if it was installed from a different source..  This is due to divergent install locations, especially between repository installs and the other forms.

Note 2: Launchpad PPAs contain builds for Bionic 18.04, Eoan 19.10, Focal 20.04, Hirsute 20.10 and Impish 21.04.

Note 3: Xenial 16.04 works with Snap and Pip installs, but cannot be built under Launchpad PPAs at the moment.

[Noxy (noxypaws)]

I'm on 21.04 but I had success resolving this by installing python3-b2sdk with apt