"Basically you can very easily get yourself in a situation where *just
enough* of the distro package is left behind that pip thinks its there,
so won't install it, but the python loader doesn't, so won't work.
<dhellmann> fungi: the tl;dr is that when oslo.config is installed globally with "pip install -e" and oslo.sphinx is installed in a virtualenv with "pip install" python in the virtualenv can't find oslo.config
...
<dhellmann> sdague: I think bnemec worked out that it's caused by a combination of pip install -e and something else in the same namespace package not being installed in that mode
<dhellmann> sdague: alternatives to fix it seem to be install oslo.sphinx with pip install -e or change its package name
<dhellmann> sdague: or don't install it at all, of course
<bnemec> dhellmann: sdague: Right, the problem is that we pip install -e oslo.config in the system site packages, then pip install oslo.sphinx in the venv.
<bnemec> That combination results in oslo.config being unavailable in the venv.
<dhellmann> fungi: common namespace, split, *and* installed with .egg-link in some cases (I think)
<dstufft> namespaces don't play well with -e
<dhellmann> so for normal production oslo libs, we'd just install them from devstack with pip install -e and be done
<dhellmann> but because oslo.sphinx is not a production lib, that's not necessarily the best answer
<dhellmann> we should probably just reserve the oslo namespace for production libraries, and rename the theme package
<dhellmann> well, it could also be said that if devstack didn't use that -e option this wouldn't be an issue, so it's a combination of all of it
The problem seems to be when we pip install -e oslo.config on the
system, then pip install oslo.sphinx in a venv. oslo.config is
unavailable in the venv, apparently because the namespace package for
o.s causes the egg-link for o.c to be ignored. Pretty much every other
combination I've tried (regular pip install of both, or pip install -e
of both, regardless of where they are) works fine, but there seem to be
other issues with all of the other options we've explored so far.
We can't remove the pip install -e of oslo.config because it has to be
used for gating, and we can't pip install -e oslo.sphinx because it's
not a runtime dep so it doesn't belong in the gate. Changing the
toplevel package for oslo.sphinx was also mentioned, but has obvious
drawbacks too.
Uggh, digging up the details behind this was pretty terrible ...
There is clearly some underlying issue we're papering over here ... but nowhere do I see any analysis of that underlying issue recorded for posterity
For example, in https:/ /review. openstack. org/71589
"Having the sphinx theme in the same namespace package
as the production code has been causing issues with
devstack installations."
The thread linked above ends with http:// lists.openstack .org/pipermail/ openstack- dev/2013- December/ 023002. html
"Basically you can very easily get yourself in a situation where *just
enough* of the distro package is left behind that pip thinks its there,
so won't install it, but the python loader doesn't, so won't work.
Then much sadness."
Or in https:/ /review. openstack. org/#/c/ 65414/1/ /COMMIT_ MSG
"After merging https:/ /review. openstack. org/64226 devstack gate started failing."
Oh, now I'm getting somewhere http:// eavesdrop. openstack. org/irclogs/ %23openstack- infra/% 23openstack- infra.2014- 01-08.log
<dhellmann> fungi: the tl;dr is that when oslo.config is installed globally with "pip install -e" and oslo.sphinx is installed in a virtualenv with "pip install" python in the virtualenv can't find oslo.config
...
<dhellmann> sdague: I think bnemec worked out that it's caused by a combination of pip install -e and something else in the same namespace package not being installed in that mode
<dhellmann> sdague: alternatives to fix it seem to be install oslo.sphinx with pip install -e or change its package name
<dhellmann> sdague: or don't install it at all, of course
<bnemec> dhellmann: sdague: Right, the problem is that we pip install -e oslo.config in the system site packages, then pip install oslo.sphinx in the venv.
<bnemec> That combination results in oslo.config being unavailable in the venv.
<dhellmann> fungi: common namespace, split, *and* installed with .egg-link in some cases (I think)
<dstufft> namespaces don't play well with -e
<dhellmann> so for normal production oslo libs, we'd just install them from devstack with pip install -e and be done
<dhellmann> but because oslo.sphinx is not a production lib, that's not necessarily the best answer
<dhellmann> we should probably just reserve the oslo namespace for production libraries, and rename the theme package
<dhellmann> well, it could also be said that if devstack didn't use that -e option this wouldn't be an issue, so it's a combination of all of it
And, at last, I find Ben's email ! http:// lists.openstack .org/pipermail/ openstack- dev/2014- January/ 023759. html
The problem seems to be when we pip install -e oslo.config on the
system, then pip install oslo.sphinx in a venv. oslo.config is
unavailable in the venv, apparently because the namespace package for
o.s causes the egg-link for o.c to be ignored. Pretty much every other
combination I've tried (regular pip install of both, or pip install -e
of both, regardless of where they are) works fine, but there seem to be
other issues with all of the other options we've explored so far.
We can't remove the pip install -e of oslo.config because it has to be
used for gating, and we can't pip install -e oslo.sphinx because it's
not a runtime dep so it doesn't belong in the gate. Changing the
toplevel package for oslo.sphinx was also mentioned, but has obvious
drawbacks too.
And then this pip bug: https:/ /github. com/pypa/ pip/issues/ 3