oslo.config egg doesn't contain namespace_packages.txt

[Mark McLoughlin]

Credit to Alan Pevec for noticing this

If you do 'python setup.py egg_info', notice that there is no namespace_packages.txt file written out

This is despite setup.cfg containing:

 [files]
 packages =
     oslo
 namespace_packages =
     oslo

Seems pretty clear that d2to1 doesn't actually support this:

  https://github.com/iguananaut/d2to1/blob/5a1c80b2/d2to1/util.py#L47

This seems like it's a pretty serious issue, but I don't actually know what the implications are ... is it that without this namespace_packages.txt, we'd get oslo/__init__.py installed and this would conflict with any other packages which want to install into this namespace?

[Alan Pevec (apevec)]

> we'd get oslo/__init__.py installed and this would conflict with any other packages which want to install into this namespace?

yes:

* python-oslo-config-1:1.1.0-1.fc20
 http://koji.fedoraproject.org/koji/rpminfo?rpmID=3818867

* python-oslo-config-1:1.2.0-0.1.a2.fc20
 http://koji.fedoraproject.org/koji/rpminfo?rpmID=4101838

[Mark McLoughlin (markmc)]

Monty says he's on it

[Mark McLoughlin (markmc)]

Monty has proposed this fix to d2to1 : https://github.com/iguananaut/d2to1/pull/32

[Mark McLoughlin (markmc)]

2013-06-26T15:52:42 <markmc> mordred, I see you've some pull requests sitting there for some months :(
2013-06-26T15:53:02 <mordred> markmc: yeah, that's why I pinged him directly
2013-06-26T15:56:31 <mordred> markmc: just got a response, he's been on vacation for a few weeks and is working through post-vacation email backlog
2013-06-26T15:56:43 <mordred> markmc: but he's going to bump that patch to the top of the list

[Monty Taylor (mordred)]

Workaround until the above fix is landed and released:

namespace_packages = ['oslo'],

into setup.py directly.

[Mark McLoughlin (markmc)]

See also bug #1194807 ... as we said, they could be related

[Mark McLoughlin (markmc)]

Here's Monty's attempt to fix this - https://review.openstack.org/34586

It fails with: error in setup command: Distribution contains no modules or packages for namespace package 'oslo'

There's a good chance neither Monty nor I will be able to look at this much today, so any help figuring out the issue would be welcome.

[Doug Hellmann (doug-hellmann)]

I updated the patch this morning. It looks like specifying the packages in setup.py fixes the problem. There's probably a check being done to the settings before d2to1 has a chance to inject the values from setup.cfg, which should be taken care of when we get the namespace_package setting injection working properly.

[Mark McLoughlin (markmc)]

Much appreciated Doug, you're the man

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/35280

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/35280
Committed: http://github.com/openstack/oslo-incubator/commit/9f09474c5f8f591453e1dc5e3c1ddc3c7efd4985
Submitter: Jenkins
Branch: master

commit 9f09474c5f8f591453e1dc5e3c1ddc3c7efd4985
Author: Mark McLoughlin <email address hidden>
Date: Tue Jul 2 13:01:26 2013 +0100

    Fix issue with pip installing oslo.config-1.2.0

    Fixes bug #1194807

    Firstly, we update the oslo.config dep to 1.2.0a3 because of the issue
    with namespace packages (bug #1194742).

    But the main issue here is that if we ever added a dependency to our
    requirements.txt that had a dependency on an older version of
    oslo.config we would see the same issue thatt Nova and Quantum recently
    experienced where you end up with the oslo.config 1.1.1 code installed.
    This is because oslo.config>=1.1.0 gets pulled in as a transitive dep
    and pip gets confused. You can reproduce with e.g.

      $> pip install \
           http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
           python-keystoneclient
      $> pip freeze | grep oslo.config
      oslo.config-1.2.0a3
      $> python -c 'from oslo.config.cfg import DeprecatedOpt'
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
      ImportError: cannot import name DeprecatedOpt

    This is because of a bug with pip where it sees oslo.config-1.2.0a3 and
    oslo.config as two unrelated things. It should strip the version part of
    the egg= fragment before using it as a package name, but it doesn't.

    However, we can simply use the -f/--find-links pip option in our
    requirements.txt to add the tarball URL to the list of URLs considered
    and also add the oslo.config>=1.2.0a3 dependency:

      $> pip install \
           -f http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
           'oslo.config>=1.2.0a3' \
           python-keystoneclient
      $> pip freeze | grep oslo.config
      oslo.config-1.2.0a3
      $> python -c 'from oslo.config.cfg import DeprecatedOpt'

    This is actually exactly the semantics we want and we go to great
    lengths in pbr to get these semantics while using a single tarball URL.
    The only downside to this --find-links strategy is that we gain an extra
    line in our requirements.txt ... but it does work around the pip bug.

    Change-Id: I6f3eb5fd2c75615d9a1cae172aed859b36b27d4c