Tests fail with python 3.8 due to deprecation warnings

[Sebastien Bacher]

The issue has been reported to Debian and is also impacting Ubuntu
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423

The warnings are
'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'

These were introduced by https://bugs.python.org/issue36381 to warn about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN is described in https://python.readthedocs.io/en/stable/c-api/arg.html#strings-and-buffers.

[Sebastien Bacher (seb128)]

The patch there fixes those warnings
DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats

It's not enough to fix build/tests though

[Sebastien Bacher (seb128)]

Thanks Kenneth, I've tried to backport your fixes to the Ubuntu package but it still fails to build

https://launchpadlibrarian.net/453033403/buildlog_ubuntu-focal-amd64.duplicity_0.8.07-1ubuntu2~build1_BUILDING.txt.gz

The _librsync import error are weird though which makes me wonder if that's something in the packaging setup (though the same package builds fine on 19.10 with python 3.7)

[Kenneth Loafman (kenneth-loafman)]

Hmmm, all tests pass on Ubuntu 18.04 and 19.04 for Python 2.7, 3.7, and 3.8. However, eoan and focal need python-fasteners on the PPA:

https://code.launchpad.net/~duplicity-team/+recipe/duplicity-daily-dev

I'd like to know the magic to fix in the PPA to get past that problem.

As to _librsync, it compiled and got placed properly. I don't know why it would not load. There was an issue I ran across w.r.t. openssl not wanting to load a non-versioned .so, but openssl is a lot more restrictive than duplicity. Other than possibly that change being propagated out to the rest of the packages, I just don't know.

[Sebastien Bacher (seb128)]

What's the issue with python-fasterners an the ppa exactly?

Unsure what's the issue, the builds fail locally the same way in a pbuilder env but doing
# python3.8 -c 'import _librsync'

in the build/lib...3.8/duplicity directory works

[Kenneth Loafman (kenneth-loafman)]

I'm not sure what the problem is with python-fasteners. When I try to use eoan or focal I get:

"Could not build because of missing dependencies"

which leads me to a short page:

"Build status
[MANUALDEPWAIT] Dependency wait on lgw01-amd64-033

Missing build dependencies: python-fasteners
Started 22 hours ago
Finished 22 hours ago (took 2 minutes, 6.0 seconds)
buildlog (6.8 KiB)"

[Kenneth Loafman (kenneth-loafman)]

I converted debian/control,rules over to Python 3. Now duplicity builds, but does not test. Chasing that now.

[Kenneth Loafman (kenneth-loafman)]

As to build errors, in your debian/rules file add/change the following:

override_dh_auto_test:
# Disabling autotest. I have run tests on all Ubuntu releases since 18.04, so the code works.
# To run tests manually, run tox from the main directory. Maybe LP build will work again soon.
# ./setup.py test

I submitted a bug report against ubuntu-builder. We'll see if that gets addressed.

[Sebastien Bacher (seb128)]

@Kenneth, can you give the reference to that ubuntu-builder bug you mentioned? Turning off the testsuite seems suboptimal...

[Kenneth Loafman (kenneth-loafman)]

Giving up on search for kenneth-loafman as reporter. Can't find anything
with LP Search All.

Once this set of builds finish on duplicity-daily-dev, I'll change back to
testing after build. I'm not seeing any deprecation warnings and I'm
testing up to py38.

On Mon, Jan 6, 2020 at 3:00 AM Sebastien Bacher <email address hidden> wrote:

> @Kenneth, can you give the reference to that ubuntu-builder bug you
> mentioned? Turning off the testsuite seems suboptimal...
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1853809
>
> Title:
> Tests fail with python 3.8 due to deprecation warnings
>
> Status in Duplicity:
> Fix Released
>
> Bug description:
> The issue has been reported to Debian and is also impacting Ubuntu
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423
>
> The warnings are
> 'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'
>
> These were introduced by https://bugs.python.org/issue36381 to warn
> about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN
> is described in https://python.readthedocs.io/en/stable/c-api/arg.html
> #strings-and-buffers.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1853809/+subscriptions
>

[Kenneth Loafman (kenneth-loafman)]

If you look at
https://code.launchpad.net/~duplicity-team/+recipe/duplicity-daily-dev you
will see that, as released, Bionic, Disco, Eoan, and Focal all pass without
deprecation errors.

Are we testing in the same environment?

Are you patching the code before testing?

On Mon, Jan 6, 2020 at 11:31 AM Kenneth Loafman <email address hidden> wrote:

> Giving up on search for kenneth-loafman as reporter. Can't find anything
> with LP Search All.
>
> Once this set of builds finish on duplicity-daily-dev, I'll change back to
> testing after build. I'm not seeing any deprecation warnings and I'm
> testing up to py38.
>
>
> On Mon, Jan 6, 2020 at 3:00 AM Sebastien Bacher <email address hidden> wrote:
>
>> @Kenneth, can you give the reference to that ubuntu-builder bug you
>> mentioned? Turning off the testsuite seems suboptimal...
>>
>> --
>> You received this bug notification because you are subscribed to
>> Duplicity.
>> https://bugs.launchpad.net/bugs/1853809
>>
>> Title:
>> Tests fail with python 3.8 due to deprecation warnings
>>
>> Status in Duplicity:
>> Fix Released
>>
>> Bug description:
>> The issue has been reported to Debian and is also impacting Ubuntu
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423
>>
>> The warnings are
>> 'DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats'
>>
>> These were introduced by https://bugs.python.org/issue36381 to warn
>> about an upcoming Python C API change. The meaning of PY_SSIZE_T_CLEAN
>> is described in https://python.readthedocs.io/en/stable/c-api/arg.html
>> #strings-and-buffers.
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/duplicity/+bug/1853809/+subscriptions
>>
>

[Heather Ellsworth (hellsworth)]

It looks like a fix was released in a newer version of duplicity just 8 days ago: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945423#21

This version of duplicity is in FF already too.