Tests fail with python 3.8 due to deprecation warnings

Bug #1853809 reported by Sebastien Bacher on 2019-11-25
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
High
Unassigned

Bug Description

The issue has been reported to Debian and is also impacting Ubuntu
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423

The warnings are
'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'

These were introduced by https://bugs.python.org/issue36381 to warn about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN is described in https://python.readthedocs.io/en/stable/c-api/arg.html#strings-and-buffers.

Sebastien Bacher (seb128) wrote :

The patch there fixes those warnings
DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats

It's not enough to fix build/tests though

Changed in duplicity:
assignee: nobody → Kenneth Loafman (kenneth-loafman)
importance: Undecided → High
milestone: none → 0.8.08
status: New → In Progress
Changed in duplicity:
assignee: Kenneth Loafman (kenneth-loafman) → nobody
status: In Progress → Fix Committed
Sebastien Bacher (seb128) wrote :

Thanks Kenneth, I've tried to backport your fixes to the Ubuntu package but it still fails to build

https://launchpadlibrarian.net/453033403/buildlog_ubuntu-focal-amd64.duplicity_0.8.07-1ubuntu2~build1_BUILDING.txt.gz

The _librsync import error are weird though which makes me wonder if that's something in the packaging setup (though the same package builds fine on 19.10 with python 3.7)

Hmmm, all tests pass on Ubuntu 18.04 and 19.04 for Python 2.7, 3.7, and 3.8. However, eoan and focal need python-fasteners on the PPA:

https://code.launchpad.net/~duplicity-team/+recipe/duplicity-daily-dev

I'd like to know the magic to fix in the PPA to get past that problem.

As to _librsync, it compiled and got placed properly. I don't know why it would not load. There was an issue I ran across w.r.t. openssl not wanting to load a non-versioned .so, but openssl is a lot more restrictive than duplicity. Other than possibly that change being propagated out to the rest of the packages, I just don't know.

Sebastien Bacher (seb128) wrote :

What's the issue with python-fasterners an the ppa exactly?

Unsure what's the issue, the builds fail locally the same way in a pbuilder env but doing
# python3.8 -c 'import _librsync'

in the build/lib...3.8/duplicity directory works

I'm not sure what the problem is with python-fasteners. When I try to use eoan or focal I get:

"Could not build because of missing dependencies"

which leads me to a short page:

"Build status
[MANUALDEPWAIT] Dependency wait on lgw01-amd64-033

Missing build dependencies: python-fasteners
Started 22 hours ago
Finished 22 hours ago (took 2 minutes, 6.0 seconds)
buildlog (6.8 KiB)"

I converted debian/control,rules over to Python 3. Now duplicity builds, but does not test. Chasing that now.

As to build errors, in your debian/rules file add/change the following:

override_dh_auto_test:
# Disabling autotest. I have run tests on all Ubuntu releases since 18.04, so the code works.
# To run tests manually, run tox from the main directory. Maybe LP build will work again soon.
# ./setup.py test

I submitted a bug report against ubuntu-builder. We'll see if that gets addressed.

Changed in duplicity:
status: Fix Committed → Fix Released
Sebastien Bacher (seb128) wrote :

@Kenneth, can you give the reference to that ubuntu-builder bug you mentioned? Turning off the testsuite seems suboptimal...

Giving up on search for kenneth-loafman as reporter. Can't find anything
with LP Search All.

Once this set of builds finish on duplicity-daily-dev, I'll change back to
testing after build. I'm not seeing any deprecation warnings and I'm
testing up to py38.

On Mon, Jan 6, 2020 at 3:00 AM Sebastien Bacher <email address hidden> wrote:

> @Kenneth, can you give the reference to that ubuntu-builder bug you
> mentioned? Turning off the testsuite seems suboptimal...
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1853809
>
> Title:
> Tests fail with python 3.8 due to deprecation warnings
>
> Status in Duplicity:
> Fix Released
>
> Bug description:
> The issue has been reported to Debian and is also impacting Ubuntu
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423
>
> The warnings are
> 'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'
>
> These were introduced by https://bugs.python.org/issue36381 to warn
> about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN
> is described in https://python.readthedocs.io/en/stable/c-api/arg.html
> #strings-and-buffers.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1853809/+subscriptions
>

If you look at
https://code.launchpad.net/~duplicity-team/+recipe/duplicity-daily-dev you
will see that, as released, Bionic, Disco, Eoan, and Focal all pass without
deprecation errors.

Are we testing in the same environment?

Are you patching the code before testing?

On Mon, Jan 6, 2020 at 11:31 AM Kenneth Loafman <email address hidden> wrote:

> Giving up on search for kenneth-loafman as reporter. Can't find anything
> with LP Search All.
>
> Once this set of builds finish on duplicity-daily-dev, I'll change back to
> testing after build. I'm not seeing any deprecation warnings and I'm
> testing up to py38.
>
>
> On Mon, Jan 6, 2020 at 3:00 AM Sebastien Bacher <email address hidden> wrote:
>
>> @Kenneth, can you give the reference to that ubuntu-builder bug you
>> mentioned? Turning off the testsuite seems suboptimal...
>>
>> --
>> You received this bug notification because you are subscribed to
>> Duplicity.
>> https://bugs.launchpad.net/bugs/1853809
>>
>> Title:
>> Tests fail with python 3.8 due to deprecation warnings
>>
>> Status in Duplicity:
>> Fix Released
>>
>> Bug description:
>> The issue has been reported to Debian and is also impacting Ubuntu
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423
>>
>> The warnings are
>> 'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'
>>
>> These were introduced by https://bugs.python.org/issue36381 to warn
>> about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN
>> is described in https://python.readthedocs.io/en/stable/c-api/arg.html
>> #strings-and-buffers.
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/duplicity/+bug/1853809/+subscriptions
>>
>

Heather Ellsworth (hellsworth) wrote :

It looks like a fix was released in a newer version of duplicity just 8 days ago: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423#21

This version of duplicity is in FF already too.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.