disk-image-create broken in virtualenvs created with `python3 -m venv ...`
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
diskimage-builder |
Confirmed
|
Medium
|
Unassigned |
Bug Description
Python 3.3+ ships the "venv" module in the standard library. Among other things, one can create a virtualenv with it. Unfortunately, diskimage-builder doesn't work in these virtualenvs. Here's a simple reproducer:
python3 -m venv ~/.venvs/
source ~/.venvs/
pip install diskimage-builder
disk-
Here's a sample traceback:
$ disk-image-create
Traceback (most recent call last):
File "/home/
File "/home/
File "/home/
globs = runpy.run_
File "/usr/lib/
code, fname = _get_code_
File "/usr/lib/
with open(fname, "rb") as f:
FileNotFoun
The issue is easy enough to track down. Here's a snippet from `disk_image_
def main():
# If we are called directly from a venv install
# (/path/
# virtualenv bin/ dir to $PATH. the exec'd script below will be
# unable to find call other dib tools like dib-run-parts.
#
# One solution is to say that you should only ever run
# disk-image-create in a shell that has already sourced
# bin/activate.sh (all this really does is add /path/venv/bin to
# $PATH). That's not a great interface as resulting errors will
# be very non-obvious.
#
# We can detect if we are running in a virtualenv and use
# virtualenv's "activate_this.py" script to activate it ourselves
# before we call the script. This ensures we have the path setting
In other words, according to this comment, both of these two workflows are supported:
# typical workflow
source ~/.venvs/
disk-
# workflow supported by activate_venv()
~/.
Unfortunately, activate_venv() makes assumptions about the internal structure about virtualenvs, and those assumptions aren't valid. Specifically, activate_venv() assumes that an executable called `activate_this.py` exists. `activate_this.py` does exist in virtualenvs created by `virtualenv`. It doesn't exist in virtualenvs created by the standard library's venv module.
Let's look a bit closer. Here's the full source of activate_venv():
def activate_venv():
if running_
globs = runpy.run_
del globs
Importantly, running_
python3 -m venv ~/.venvs/
source ~/.venvs/
pip install diskimage-builder
disk-
virtualenv ~/.venvs/
source ~/.venvs/
pip install diskimage-builder
disk-
There's a couple possible fixes.
1. Drop the logic that automagically adds path-to-venv/bin/ to $PATH.
2. Make activate_venv() execute only when disk-image-create is in a venv *and* the venv hasn't been activated.
I like the first solution, because I dislike automagic things that break user expectations. My expectation is that virtualenvs are only active when I make them active, and this is the only Python application I'm aware of that tries to be clever about this. But that's a regression in functionality, and is therefore off the table (at least until a major version change). So, the second option is the more feasible one.
description: | updated |
I'd be willing to consider any patches on this. I don't particularly like this either but people run the binary straight out of the virtualenv and get just as confused when it all breaks with weird tracebacks because it can't find the other binaries it needs. I remember at the time we looked around for other solutions, such as pkgutil but didn't really find anything that worked quite the same.
I think dib-run-parts is actually solved because we have diskimage_ builder. paths.get_ path('lib' ) where we run it from now. if there's some way we can know the canonical path to dib-block-device and others without this that would be good