tox lowest-supported-dev tests fail

Bug #1980854 reported by pd9370
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released
James Falcon

Bug Description

I'm working on cloud-init with a view to adding support for the devuan distro (debian without systemd). I'm working on a local machine rather than a cloud provider, and I'm doing the development on devuan v4 (chimaera) based on debian 11 (bullseye) - so I can't run the commands ubuntu-bug or journalctl. The version of python 3 on devuan chimaera is 3.9.

When I run the tests, using tox, I get 140 failures from the lowest-supported-dev environment. I've identified that they're caused by two problems:

1) jinja2 tries to import soft_unicode() from the package markupsafe. This has been removed from recent versions of markupsafe, and this causes jinja2 to fail to initialise properly, resulting in errors like:

WARNING Jinja not available as the selected renderer for desired template, reverting to the basic renderer.

The problem has been flagged up in markupsafe:

- but is apparently still unresolved. The recommended workaround is to use markupsafe 2.0.1, which was the last version to include soft_unicode(). This can be done by adding


to the deps in the [lowest-supported-deps] section of tox.ini, directly below jinja==2.10

2) In python 3.9, threading.Thread.isAlive has been removed:

The recommendation is to use threading.Thread.is_alive instead.

This breaks httpretty 0.9.5. The earliest version of httpretty in which this is fixed is 0.9.7

Changing "httpretty==0.9.5" to "httpretty==0.9.7" in tox.ini works around this.

With these two changes to tox.ini in place, tox on cloud-init under devuan 4 runs without failures.

Hope this is useful. Happy to provide further information if required.

pd9370 (f-peter-9)
description: updated
description: updated
Revision history for this message
Alberto Contreras (aciba) wrote :

Thanks, pd9370, for the detailed description.

lowest-supported-dev is supposed to run on the latest supported Python version, which may vary depending on the cloud-init version you are hacking with. One can find the minimum python version changes in the following documentation:

Revision history for this message
Brett Holman (holmanb) wrote :

Hi pd9370, thanks for reporting this.

I know little about Devuan (besides the tagline), but I assume that tox is not actually used during the package build process, right? On Ubuntu/Debian, I think sbuild just invokes a python3 -m pytest -v tests/unittests with system dependencies. The failing test exists in tox to verify the oldest supported distro dependency versions (using our CI infrastructure or locally with pyenv/deadsnakes/etc).

Agreed, the issues you described may be worked around on newer versions, but the intended use case currently is running on 3.6 so I'm not sure working around this issue on other versions really makes sense - what does it buy us? For what it's worth, that test env fails when executed with system-provided python on the latest Ubuntu release (Jammy).

It might make sense to drop that environment from the tox envlist to reduce noise.


James Falcon (falcojr)
Changed in cloud-init:
status: New → In Progress
importance: Undecided → Low
assignee: nobody → James Falcon (falcojr)
Revision history for this message
pd9370 (f-peter-9) wrote (last edit ):

Alberto, Brett and James - thanks for looking into this.

One clarification: as I'm hoping eventually to submit changes to add devuan support to cloud-init, I'm working on a clone of the github repo and keeping the clone up-to-date - so hopefully at the moment I'm doing my investigations at the tip of the 'main' branch. When I know what's required for devuan support, I'll create a fork in my github account and eventually create a pull request.

Re. the jinja2 problem - I've just tried editing tox.ini to remove the markupsafe==2.0.1 line and changing the jinja2 line to a later version. It appears that jinja2 version 3.0.0a1 was the first one in which the soft_unicode problem was fixed (with the deps line in tox.ini set to "jinja2==3.0.0a1", the lowest-supported-dev tests run clean; with it set to "jinja2==2.11.3" (the last version before v3) they fail).

Note that I'm using the following command to find all the available releases of jinja2 (apparently it's the usual way of doing this in pip3):

pip3 install --use-deprecated=legacy-resolver jinja2==

I haven't as yet tried testing this on a distro/release which uses python<3.9 (I'll do this soon). I've also been adding devuan support to walinuxagent, and unfortunately that requires python 3.9 (using lower versions of python 3, walinuxagent can't distinguish between devuan and debian).

Brett Holman (holmanb)
Changed in cloud-init:
status: In Progress → Fix Committed
Revision history for this message
Brett Holman (holmanb) wrote :

When you're ready to upstream requirements for Devuan, feel free to open a PR on Github. In the meantime, we've removed[1] that particular test from the envlist.

With respect to changing that particular set of test requirements: Again, I don't think we want to. It currently serves its purpose and works as is for its intended platform. However, if you feel the need to add a new environment to tox.ini for testing the oldest supported Devuan dep equivalents from pypi, I don't see a reason why we couldn't do that. Either way, we can have that discussion when a PR is opened to upstream support for Devuan.

I don't think there is anything else for upstream to do, so I've marked this a "fix committed".


Revision history for this message
pd9370 (f-peter-9) wrote :

Hi Brett - thanks for that. I can confirm that with the latest changes, under devuan chimaera, the tox test suite now runs clean. I'm happy to concur that this is now fixed.

Revision history for this message
Brett Holman (holmanb) wrote : Fixed in cloud-init version 22.3.

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.