pkg_resources.DistributionNotFound: The 'argparse' distribution was not found and is required by oslo.config, python-keystoneclient, pysaml2

[Victor Stinner]

Hi,

When trying to install a fresh DevStack, I got issues with pip 6.1. First issue:
https://bugs.launchpad.net/tempest/+bug/1440984

I worked around the first issue, but then I got this issue:

2015-04-07 10:08:34.084 | + /usr/bin/keystone-manage db_sync
2015-04-07 10:08:34.239 | Traceback (most recent call last):
2015-04-07 10:08:34.239 | File "/usr/bin/keystone-manage", line 4, in <module>
2015-04-07 10:08:34.239 | __import__('pkg_resources').require('keystone==2015.1.dev143')
2015-04-07 10:08:34.239 | File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3057, in <module>
2015-04-07 10:08:34.239 | working_set = WorkingSet._build_master()
2015-04-07 10:08:34.240 | File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 639, in _build_master
2015-04-07 10:08:34.240 | ws.require(__requires__)
2015-04-07 10:08:34.240 | File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 940, in require
2015-04-07 10:08:34.240 | needed = self.resolve(parse_requirements(requirements))
2015-04-07 10:08:34.240 | File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 827, in resolve
2015-04-07 10:08:34.240 | raise DistributionNotFound(req, requirers)
2015-04-07 10:08:34.241 | pkg_resources.DistributionNotFound: The 'argparse' distribution was not found and is required by oslo.config, python-keystoneclient, pysaml2

The problem is that newly released pip 6.1 doesn't want to install argparse because argparse is part of the Python standard library:

fedora@myhost$ pip install argparse
Skipping requirement: argparse because argparse is a stdlib package
You must give at least one requirement to install (see "pip help install")

Workaround: downgrade pip to 6.0.8 and install argparse using pip (pip install argparse).

A better fix is maybe to make argparse optional in keystone requirements? It's now possible to add environment markers to dependencies. Example:

futures; python_version < '2.7'

See https://github.com/pypa/pip/pull/1472

[Victor Stinner (vstinner)]

It looks like the issue comes from pysaml2: https://pypi.python.org/pypi/pysaml2

setup.py of pysaml2-2.4.0 calls setup() with install_requirements=[..., 'argparse', ...].

--

Donald Stufft already sent a pull request to revert the change skipping modules part of the standard library:
https://github.com/pypa/pip/pull/2650

The issue asking to skip modules part of the stdlib: https://github.com/pypa/pip/pull/2636

[Dolph Mathews (dolph)]

Removed this from keystone since argparse does not appear as an explicit dep of keystone in any requirements file.

[Dolph Mathews (dolph)]

The same applies to openstackclient.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/171128

[Dolph Mathews (dolph)]

Related reviews: https://review.openstack.org/#/q/Iabc1779cf0e6e347d58da739946d3ea91482caa1,n,z

[Victor Stinner (vstinner)]

Donald Stufft just released pip 6.1.1 which fixes this issue:
https://github.com/pypa/pip/commit/368294be59445592cc50551c9a658248912ec10b

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dolph Mathews (<email address hidden>) on branch: master
Review: https://review.openstack.org/171124

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dolph Mathews (<email address hidden>) on branch: master
Review: https://review.openstack.org/171128

[Dolph Mathews (dolph)]

Abandoning this as invalid since pip 6.1.1 handles argparse correctly now.